Changes between Version 26 and Version 27 of Procedures/OwlProcessing

Timestamp:

Nov 9, 2016, 10:52:09 AM (7 years ago)

Author:

lah

Comment:

--

Procedures/OwlProcessing

@@ -21 +21 @@
 3) Check for calibration data:
 
-ls thermal/owl/*/capture/T1*
-ls thermal/owl/*/capture/T2*
+ls thermal/owl/*/capture/T1* [[BR]]
+ls thermal/owl/*/capture/T2* [[BR]]
 
 Make sure that there is at least 1 T1 and 1 T2 file present for the flight and that it is located in the same directory as the flight data. If there are double the number of expected flightlines, then the calibration data is probably stored in separate directories from the flight data. The unpacking scripts should move the raw data into ordered number flightline directories and the T1 and T2 files into directories labelled by time.
@@ -42 +42 @@
 Original Owl data should not be modified, nor scripts ran by ARSF in the raw directories. Use the batch script stitch_all_owl.py with a config file to automatically stitch desired files into the processing/owl/flightlines/stitched directory. The config files should be a text file with each flight line and each file to be stitched on a new line separated by a comma e.g:
 
-1, T1_OWL274-15_0844.raw
-2, T1_OWL274-15_0844.raw
-3, T1_OWL274-15_0853.raw
+1, T1_OWL274-15_0844.raw [[BR]]
+2, T1_OWL274-15_0844.raw [[BR]]
+3, T1_OWL274-15_0853.raw [[BR]]
 
 The script stitchOwl.py may be used for individual lines.
@@ -96 +96 @@
 Create a dem in exactly the same way as for fenix.  This should be saved in the processing/hyperspectral/dem directory, not the owl directories, so only 1 copy is required.  e.g. from the project directory type create_apl_dem.py --aster.
 
-Geocorrection is conducted in the same way as for fenix data, using APL. Generate a config file using generate_apl_config.py -o processing/owl/o2014219b.cfg (make sure it begins with an '''o''') to keep processing separate from fenix. You may need to add the boresight values to the config file ( e.g. owl_boresight = 0.12 -0.32 0.32). You may also want to delete or set to false the fenix lines if only processing owl data.  Make sure the project directory structure is complete (same as fenix). Submit to the grid using specim_qsub.py <config_file> and calculate the SCTs. Once you have done this you can generate the mapped data (102 bands) by setting owl_bandlist = ALL and setting the SCT value for each line.
+Geocorrection is conducted in the same way as for fenix data, using APL. However, if lines need to be split an extra step is required - see "Splitting files" section below. Generate a config file using generate_apl_config.py -o processing/owl/o2014219b.cfg (make sure it begins with an '''o''') to keep processing separate from fenix. You may need to add the boresight values to the config file ( e.g. owl_boresight = 0.12 -0.32 0.32). You may also want to delete or set to false the fenix lines if only processing owl data.  Make sure the project directory structure is complete (same as fenix). Submit to the grid using specim_qsub.py <config_file> and calculate the SCTs. Once you have done this you can generate the mapped data (102 bands) by setting owl_bandlist = ALL and setting the SCT value for each line.
 
 Early owl files have corrupt nav files, so if you get this message in the log or 'Requested sync time index is out of bounds in GetSyncDelay()' set the config file to use_nav = false.
 
 If there appears to be striping across the mapped files, bright and dark lines, or sections of a flight line don't match up the sensor has probably dropped frames. These can be checked in the .log file in the capture directory. These empty lines need to be inserted into the level 1b data file before geocorrection to map the flight line correctly.
+
+generate_apl_config will check the processing/owl/stitched directory for modified raw files to use before defaulting to the original raw directory, so make sure this directory is present only if you intend to use it.
+
+=== Splitting files ===
+
+APL splits files during the radiometric calibration stage, so doesn't actually split owl files as it doesn't run this stage. Files can be split using the splitOwl.py script line by line. You can either specify how to split the lines on the command line or use the apl config file e.g.
+
+splitOwl.py -f (full path)/OWL174-15-2_proc.bil -l 1 25400 25300 50700 50600 74930  or [[BR]]
+splitOwl.py -f (full path)/OWL174-15-2_proc.bil -c processing/owl/o2015174.cfg
+
+Then proceed with specim_qsub as normal. If you skip splitting the lines this way the whole line will be processed for every split, resulting in duplicate large files and a very slow network.
+
+To modify the split produced by generate_apl_config use the section_size flag. As a guide, you should divide the owl fps by the fenix fps and multiply by the default (10000). e.g. 100/46 * 10000 = 21740.
+
 
 == Making a delivery ==