= File structure and naming = [[PageOutline]] A single site/logical group of flight lines/sortie should be contained in a single directory. If there are multiple sites from different projects, significantly different times of overflight or large time gaps, each should have its own directory. The directory name should look like: PROJECTCODE-YYYY_JJJxx_SITENAME, e.g. GB07_07-2007_102a_Inverclyde, boresight-2007_198, etc. There should be: * no spaces, brackets or other Unix-upsetting characters in filenames * execute permission removed from anything non-executable * no empty directories? (need to think about this, maybe only do after checking it's not a stub for later data) The structure under the project directory should look like: ||'''Top level dir'''|| '''2nd level dir''' || '''Files''' || '''Comments'''|| ||admin|| || || Contains the flight logsheet and any miscellaneous information (readme's, copies of trac tickets, etc) || || || || || || || applanix || || *.ppc ||Raw and processed Applanix navigation (raw files in applanix/raw/DCALM*, final processed data in applanix/*sbet or applanix/Proc/sbet*.out) || || || || || || || || auto_applanix (optional) || auto.bat ||If navigation was processed using auto_applanix script then this dir will contain additional output from that script|| || || || combine_seperation_bestguess_values || || || || || normal_first_convertbase.log || || || || || normal_first_convertflight.log || || || || || *.bat || || || || || *GNSS.cmb || || || || || *GNSS.dat || || || || || *GNSS_gnuplot || || || || || *GNSS.gnv || || || || || *GNSS.log || || || || || *GNSS.png || || || || || *GNSS.sum || || || || || || || || || base (deprecated) || ||? || || || || || || || || eo (possibly optional and often empty) || || ? || || || || || || || || extract || dephem_*.dat || Plane gps data after extracted from raw files by applanix || || || || extract_*.log || || || || || gps_prim_*.dat || || || || || hwconf_*.out || || || || ||iinr_*.out || || || || || iinz_*.out || || || || || imu_*.dat || || || || || mgps_*.epp || || || || || mgps_*.gpb || || || || || mgps_*.nov|| || || || || mgps_*.sta || || || || || navclk_prim_*.dat || || || || || obs_prim_*.dat || || || || || rers_*.out || || || || || rinv_*.out || || || || || rmrs_*.out || || || || || rrms_*.out || || || || || rtstat_*.txt || || || || || tm_*.dat || || || || || vnav_*.out || || || || || vrms_*.out || || || || || *GNSS.dat || || || || || || || || || gps || *.cmb || ? || || || || *.fbv || || || || || *.fml || || || || || *.fss || || || || || *.fwd || || || || || *.his || || || || || *.rbv || || || || || *.rev || || || || || *.rml || || || || || *.rss || || || || || || || || || proc || ers_*.out || Processed applanix data || || || || idx_*.txt || || || || || inv_*.out || || || || || iin_*.out || || || || || mrs_*.out || || || || || rms_*.out || || || || || rmsg_*.out || || || || || sbet_*.out || || || || || smers_*.out || || || || || smhkm_*.dat || || || || || smindex_*.dat || || || || ||smout_*.out || || || || || smparm_*.txt|| || || || || smphi_*.dat|| || || || ||smpx_*.dat || || || || ||smreset_*.dat || || || || ||smrms_*.out || || || || ||smrmsg_*.out || || || || || || || || || raw || DCALM.??? || Raw gps data from plane || || || || || || || atm || || || Raw ATM data files (az???LLL.JJJ, where ??? = disk id, LLL = 3 digit file number (000 typically contains navigation), JJJ = Julian day of capture), pre-2009 only || || calibration || || || Rypically a symlink to the appropriate calibration directory || || casi || || || Raw CASI data files (ca??LLL.JJJ, where ?? = disk id, LLL = 3 digit file number, JJJ = Julian day of capture), pre-2008 only || || delivery || || || Contains copies of data delivered to end users || || || ''(see below)'' || || Structure described below || || dem || || || Contains azgcorr compatible DEMs || || eagle || || || Raw Eagle data files (.raw = image data, .hdr = ENVI compatible header, .nav = navigation synchronisation data) || || || orignal_headers (optional) || || If the header files have been altered for any reason, the originals should be in here || || grid_logs (deprecated) || || || Logs from hyperspectral script runs on gridnode (one log per sct run) || || hawk || || || Raw Hawk data files (.raw = image data, .hdr = ENVI compatible header, .nav = navigation synchronisation data) || || || original_headers (optional) || || If the header files have been altered for any reason, the originals should be in here || || leica || || || Leica LIDAR data || || || ''(see below)'' || || Structure described below || || lev1 || || || Processed hyperspectral level 1 data (note this will be the most current, not necessarily what was delivered), see below for filename info || || lev3 || || || Processed hyperspectral level 3 data (note this will be the most current, not necessarily what was delivered), see below for filename info || || lidar (deprecated) || || || Raw and processed ULM LIDAR (raw LIDAR looks like "surveyXXX.range", processed is typically in a strips/ directory) || || logs || || *.logs || Logs from hyperspectral script runs on gridnode (one log per flightline) || || photos (deprecated) || || || RC10 scanned aerial photographs || || rinex || || || GPS basestation data appropriate to the site || || runa || || || ATM processing scripts || || runc || || || CASI processing scripts || || rune || || || Eagle processing scripts || || runh || || || Hawk processing scripts || || tabi || || || Raw TABI data || || vectors || || ||(2007/8 only) - symlink to appropriate vectors || The lev1 and lev3 data filenames are structured as "sdddfnn1b.{hdf|tif}", where: * s is the sensor id (a=ATM, c=CASI, e=Eagle, h=Hawk) * ddd is the Julian day of the data acquisition * f distinguishes between multiple flights on the same day (“a” being the first flight) * nn is the flight line number (note this is not necessarily the same as the flight line identifier as specified by the PI - these numbers are the number of the flight in order of when it was taken, as on the logsheet) === Leica LIDAR data === || '''subdir of "leica"''' || '''2nd level dir''' || '''Files''' || '''Comments''' || || aux_files || || *.reg || Reg file used, any logs produced, intensity mosaics, boresight values) || || || || *.tif || || || || || *.txt || || || ipas || || || || || || base || || || || || eo || || || || || extract || *.fbv || || || || || *.rbv || || || || || *.RTG || || || || gps || *.cfg || || || || || *.fml || || || || || *.fss || || || || || *.fwd || || || || || *.gpj || || || || || *.his || || || || || *.lat || || || || || *.lat.bin || || || || || *.rev || || || || || *.rml || || || || || *.rss || || || || proc || || Processed GPS/IMU data (SBETs)|| || || raw || || Raw GPS/IMU data || || logs || || *.csv || Contains the flight/ops log files || || || || *.mdb || || || || || *.txt || || || || || *.xml || || || proc_laser || || || Contains the finalised processed LAS files only || || raw_laser || || || Contains the raw laser data || || rawWFD || || || Raw waveform data || || rcd || || || || || || raw_images || || || || || logs || || || || webcam || || || || === Delivered data === As data may be delivered multiple times (in the event of a problem requiring reprocessing) or in stages (if particular sensors are completed first and others look like they'll be delayed), we take copies of any data sent to the end user for future reference (e.g. in support). The lev1/ and lev3/ directories will typically contain the latest processed version. Delivered data is stored delivery/YYYYMMDD/PROJECTCODE/, where YYYYMMDD is the date the delivery was created. The directory structure of a delivery looks like: ==== Hyperspectral data ==== dem (optional):: a dem to help the user with processing to lev3. Can be from their lidar data, from SRTM or a combination. doc:: azgcorr manual and other documentation lev1:: processed level 1 data logsheet:: logsheet as stored in admin/ above misc:: any additional files needed for level 3 processing (e.g. OS National Grid correction file, geoid-spheroid separation grid file, etc) screenshots:: images of the level 3 output we managed when testing the data Read_Me.pdf:: Instructions for use and description of dataset, plus any problems encountered. The image filenames in the lev1 directory are structured as `sdddfnnLL.hdf`, where: * `s` is the sensor id (a=ATM, c=CASI, e=Eagle, h=Hawk) * `ddd` is the Julian day of the data acquisition * `f` distinguishes between multiple flights on the same day ("a" being the first flight) * `nn` is the flight line number * `LL` is the processing level * 1b = ungeocorrected data * 3a = geocorrected without a DEM * 3b = geocorrected with a DEM ==== Lidar data ==== ascii_laser:: classified ascii lidar point clouds bin:: latest versions of the pt_cloud_filter for linux and windows dem:: dem created from the lidar data doc:: lidar quality report and processing information logsheet:: logsheet as stored in admin/ above screenshots:: images of the dem and intensity mosaic COPYRIGHT.txt:: copyright notice! Read_Me.txt:: Instructions for use and description of dataset, plus any problems encountered. ==== Photographic data ==== doc:: camera data quality report eventfile (may be missing if camera crashed):: csv file containing pos/att info per photograph event photographs:: tagged tif files of each photograph in area of interest thumbnails:: jpeg thumbnails of each photograph in area of interest proj_code.kml (unavailable if no eventfile):: google earth file containing eagle/hawk/photo positions Read_Me.pdf:: Instructions for use and description of dataset, plus any problems encountered.