Changes between Initial Version and Version 1 of Processing/FilenameConventions


Ignore:
Timestamp:
Nov 5, 2009 10:31:10 AM (10 years ago)
Author:
mggr
Comment:

--

Legend:

Unmodified
Added
Removed
Modified
  • Processing/FilenameConventions

    v1 v1  
     1= File structure and naming =
     2
     3A single site/logical group of flight lines/sortie should be contained in a single directory.  If there are multiple sites from different projects, significantly different times of overflight or large time gaps, each should have its own directory.
     4
     5The directory name should look like: PROJECTCODE-YYYY_JJJxx_SITENAME, e.g. GB07_07-2007_102a_Inverclyde, boresight-2007_198, etc.
     6
     7There should be:
     8 * no spaces, brackets or other Unix-upsetting characters in filenames
     9 * execute permission removed from anything non-executable
     10 * no empty directories? (need to think about this, maybe only do after checking it's not a stub for later data)
     11
     12The structure under the project directory should look like:
     13 admin::
     14  contains the flight logsheet and any miscellaneous information (readme's, copies of trac tickets, etc)
     15 applanix::
     16  raw and processed Applanix navigation (raw files in applanix/raw/DCALM*, final processed data in applanix/*sbet or applanix/Proc/sbet*.out)
     17 atm::
     18  raw ATM data files (az???LLL.JJJ, where ??? = disk id, LLL = 3 digit file number (000 typically contains navigation), JJJ = Julian day of capture)
     19 calibration::
     20  typically a symlink to the appropriate calibration directory
     21 casi::
     22  raw CASI data files (ca??LLL.JJJ, where ?? = disk id, LLL = 3 digit file number, JJJ = Julian day of capture)
     23 delivery::
     24  contains copies of data delivered to end users (see below)
     25 dem::
     26  contains azgcorr compatible DEMs
     27 eagle::
     28  raw Eagle data files (.raw = image data, .hdr = ENVI compatible header, .nav = navigation synchronisation data)
     29 hawk::
     30  raw Hawk data files (.raw = image data, .hdr = ENVI compatible header, .nav = navigation synchronisation data)
     31 leica::
     32  raw and processed Leica LIDAR
     33 lev1::
     34  processed level 1 data (note this will be the most current, not necessarily what was delivered), see below for filename info
     35 lev3::
     36  processed level 3 data (note this will be the most current, not necessarily what was delivered), see below for filename info
     37 lidar::
     38  raw and processed ULM LIDAR (raw LIDAR looks like "surveyXXX.range", processed is typically in a strips/ directory)
     39 rinex::
     40  GPS basestation data appropriate to the site
     41 runa::
     42  ATM processing scripts
     43 runc::
     44  CASI processing scripts
     45 rune::
     46  Eagle processing scripts
     47 runh::
     48  Hawk processing scripts
     49 vectors::
     50  may or may not appear - symlink to appropriate vectors
     51
     52The lev1 and lev3 data filenames are structured as "sdddfnn1b.{hdf|tif}", where:
     53 * s is the sensor id (a=ATM, c=CASI, e=Eagle, h=Hawk)
     54 * ddd is the Julian day of the data acquisition
     55 * f distinguishes between multiple flights on the same day (“a” being the first flight)
     56 * nn is the flight line number (note this is not necessarily the same as the flight line identifier as specified by the PI - these numbers are the number of the flight in order of when it was taken, as on the logsheet)
     57
     58
     59=== Delivered data ===
     60
     61As data may be delivered multiple times (in the event of a problem requiring reprocessing) or in stages (if particular sensors are completed first and others look like they'll be delayed), we take copies of any data sent to the end user for future reference (e.g. in support).  The lev1/ and lev3/ directories will typically contain the latest processed version.
     62
     63Delivered data is stored delivery/YYYYMMDD/PROJECTCODE/, where YYYYMMDD is the date the delivery was created.  The directory structure of a delivery looks like:
     64
     65 bin::
     66  latest azgcorr and azexhdf at the time of delivery
     67 doc::
     68  azgcorr manual and other documentation
     69 lev1::
     70  processed level 1 data
     71 logsheet::
     72  logsheet as stored in admin/ above
     73 misc::
     74  any additional files needed for level 3 processing (e.g. OS National Grid correction file, geoid-spheroid separation grid file, etc)
     75 screenshots::
     76  images of the level 3 output we managed when testing the data
     77 COPYRIGHT.txt::
     78  copyright notice!
     79 Read_Me.txt::
     80  Instructions for use and description of dataset, plus any problems encountered.