File structure and naming

A single site/logical group of flight lines/sortie should be contained in a single directory. If there are multiple sites from different projects, significantly different times of overflight or large time gaps, each should have its own directory.

The directory name should look like: PROJECTCODE-YYYY_JJJxx_SITENAME, e.g. GB07_07-2007_102a_Inverclyde, boresight-2007_198, etc.

There should be:

• no spaces, brackets or other Unix-upsetting characters in filenames
• execute permission removed from anything non-executable
• no empty directories? (need to think about this, maybe only do after checking it's not a stub for later data)

The structure under the project directory should look like:

Top level dir	2nd level dir	Comments
admin		contains the flight logsheet and any miscellaneous information (readme's, copies of trac tickets, etc)
applanix		raw and processed Applanix navigation (raw files in applanix/raw/DCALM*, final processed data in applanix/*sbet or applanix/Proc/sbet*.out)
	auto_applanix	If navigation was processed using auto_applanix script then this dir will contain additional output from that script
	base (deprecated)	?
	eo	?
	extract	plane gps data after extracted from raw files by applanix
	gps	?
	proc	processed applanix data
	raw	raw gps data from plane
atm		raw ATM data files (az???LLL.JJJ, where ??? = disk id, LLL = 3 digit file number (000 typically contains navigation), JJJ = Julian day of capture), pre-2009 only
calibration		typically a symlink to the appropriate calibration directory
casi		raw CASI data files (ca??LLL.JJJ, where ?? = disk id, LLL = 3 digit file number, JJJ = Julian day of capture), pre-2008 only
delivery		contains copies of data delivered to end users
	(see below)	Structure described below
dem		contains azgcorr compatible DEMs
eagle		raw Eagle data files (.raw = image data, .hdr = ENVI compatible header, .nav = navigation synchronisation data)
grid_logs (deprecated)		logs from hyperspectral script runs on gridnode (one log per sct run)
hawk		raw Hawk data files (.raw = image data, .hdr = ENVI compatible header, .nav = navigation synchronisation data)
leica		Leica LIDAR data
	(see below)	Structure described below
lev1		processed hyperspectral level 1 data (note this will be the most current, not necessarily what was delivered), see below for filename info
lev3		processed hyperspectral level 3 data (note this will be the most current, not necessarily what was delivered), see below for filename info
lidar (deprecated)		raw and processed ULM LIDAR (raw LIDAR looks like "surveyXXX.range", processed is typically in a strips/ directory)
logs		logs from hyperspectral script runs on gridnode (one log per flightline)
photos (deprecated)		RC10 scanned aerial photographs
rinex		GPS basestation data appropriate to the site
runa		ATM processing scripts
runc		CASI processing scripts
rune		Eagle processing scripts
runh		Hawk processing scripts
tabi		raw TABI data
vectors		(2007/8 only) - symlink to appropriate vectors

The lev1 and lev3 data filenames are structured as "sdddfnn1b.{hdf|tif}", where:

• s is the sensor id (a=ATM, c=CASI, e=Eagle, h=Hawk)
• ddd is the Julian day of the data acquisition
• f distinguishes between multiple flights on the same day (“a” being the first flight)
• nn is the flight line number (note this is not necessarily the same as the flight line identifier as specified by the PI - these numbers are the number of the flight in order of when it was taken, as on the logsheet)

Leica LIDAR data

subdir of "leica"	2nd level dir	Comments
aux_files		contains a copy of the reg file used and any logs produced that we may conceivably want to keep - also if different boresight values used for different lines then report in a text file and store here for future reference)
ipas
	base
	eo
	extract
	gps
	proc	processed GPS/IMU data (SBETs)
	raw	raw GPS/IMU data
logs		contains the flight/ops log files
proc_laser		contains the finalised processed LAS files only
raw_laser		contains the raw laser data
RawWFD (new)		raw waveform data
rcd
	raw_images
	proc_images
	proc_thumbnails
	logs
webcam

Delivered data

As data may be delivered multiple times (in the event of a problem requiring reprocessing) or in stages (if particular sensors are completed first and others look like they'll be delayed), we take copies of any data sent to the end user for future reference (e.g. in support). The lev1/ and lev3/ directories will typically contain the latest processed version.

Delivered data is stored delivery/YYYYMMDD/PROJECTCODE/, where YYYYMMDD is the date the delivery was created. The directory structure of a delivery looks like:

Hyperspectral data

bin

latest azgcorr and azexhdf at the time of delivery

doc

azgcorr manual and other documentation

lev1

processed level 1 data

logsheet

logsheet as stored in admin/ above

misc

any additional files needed for level 3 processing (e.g. OS National Grid correction file, geoid-spheroid separation grid file, etc)

screenshots

images of the level 3 output we managed when testing the data

COPYRIGHT.txt

copyright notice!

Read_Me.txt

Instructions for use and description of dataset, plus any problems encountered.

The image filenames in the lev1 directory are structured as sdddfnnLL.hdf, where:

• s is the sensor id (a=ATM, c=CASI, e=Eagle, h=Hawk)
• ddd is the Julian day of the data acquisition
• f distinguishes between multiple flights on the same day ("a" being the first flight)
• nn is the flight line number
• LL is the processing level
  • 1b = ungeocorrected data
  • 3a = geocorrected without a DEM
  • 3b = geocorrected with a DEM

Lidar data

ascii_laser

classified ascii lidar point clouds

bin

latest versions of the pt_cloud_filter for linux and windows

dem

dem created from the lidar data

doc

lidar quality report and processing information

logsheet

logsheet as stored in admin/ above

screenshots

images of the dem and intensity mosaic

COPYRIGHT.txt

copyright notice!

Read_Me.txt

Instructions for use and description of dataset, plus any problems encountered.