Eagle/Hawk Processing Guide

This guide describes how to process hyperspectral data using the apl suite. This should be used for 2011 data onwards. If you are processing earlier data, see here for instructions for processing with the az suite.

All processing in the ~arsf/arsf_data directories should be done as airborne user:

If no config file exists (in <proj_dir>/processing/hyperspectral) then, in top project directory, run:

generate_apl_runscripts.py -s s -n <numlines> -j <jday> -y <year>

This should generate a config file based on the raw data and applanix files, and output it to the processing/hyperspectral directory. The file may need editing, the main parts to check:

• project_code
• dem and dem_origin
• transform_projection is correct for the data

Scripts to process the data:

• To submit jobs to the grid, from the top level directory use: specim_qsub.py <config_file>
• The actual script which does the processing of each job is: process_specim_apl_line.py
• If using SBETs from IPAS to process the hyperspectral make sure to use these lever arm values (referenced from pav80 not GPS antenna):

Eagle : 0.415 -0.014 -0.129

Hawk : 0.585 -0.014 -0.129

And use these boresight values (PRH):

Eagle : -0.322 0.175 0.38

Hawk : -0.345 0.29 0.35

Making a Delivery

Use the make_hyper_delivery.py script to make the delivery directory. Run it from within the main project directory. By default it runs in dry run mode.

Use --final if happy with what it says it will do. Use -m <config> to generate screenshots and mosaics

To make the readme file use the script: create_latex_hyperspectral_apl_readme.py

Individual processing stages:

Stage 1: Radiometric Calibration

The software that performs this is called aplcal. It uses the cal files to calibrate the raw data to level 1b. It strips off the FODIS region (also calibrated to level 1b), "flips" the data such that:

• eagle runs low->high wavelengths
• hawk mirrored so that it is comparable to eagle (i.e. things on the left of the aircraft are on the left of the data file)

It also applies smear correction to the eagle data. Rather than mask out bad pixels it creates a separate mask file that contains flags for the following:

• overflows
• underflows
• smear affected
• bad ccd pixel
• dropped scans

Example command to calibrate a file:

aplcal -input VNIR11092-1.raw -output e09201b.bil -calfile calibration/2011/eagle/SN001

Example command to calibrate first 2501 lines of file:

aplcal -input VNIR11092-1.raw -output e09201b.bil -calfile calibration/2011/eagle/SN001 -lines 0 2500

Stage 2: Navigation syncing

This is performed using the aplnav software. It can use navigation data either from an SBET or specim nav file, depending on whether you want to use post-processed or real-time data. It is possible to add a timing offset to shift the navgation data w.r.t the scan lines, or to shift the position w.r.t to attitude. It would be rare to require the latter as this suggests a problem with the SBET file.

Lever arms and boresight values are entered here to offset the position and attitude to the eagle/hawk sensor view direction. It is possible to smooth the data (using a triangle filter) and interpolate the scan time data using either linear or spline methods. BIL files containing quality flags for the navigation data are output also.

IMPORTANT NOTE:

• Lever arms are entered as X Y Z in the aircraft coordinates (X +ve forward, Y +ve to starboard, Z +ve down). This is different to using aznav, which used X +ve forward, Y +ve port, Z +ve up.
• Specim have previously said that there is a -0.055 second offset to apply to the sync message. Rather than hard code this into aplnav, you should add it onto the scantimeoffset parameter. This is automatically done using the processing scripts by the sct_global_offset parameter.

Example command to sync post-processed navigation data:

aplnav -nav VNIR092-11-1.nav -sbet sbet_2011092.out -lev1 e092011b.bil -interp Spline -leverarm 0.559 0.015 1.543 -boresight -0.28 0.16 0.38 -scantimeoffset 0.045 -output e092011b_p_sct0.1_nav_post_processed.bil -qualityfile e092011b_nav_post_processed_quality.bil

Stage 3: Geocorrection

The first step of geocorrection is to generate the IGM file using aplcorr. This takes an optional 1-band BIL / BSQ DEM (in WGS-84 Lat/Lon) and the instrument FOV vector file.

Example command:

aplcorr -lev1file e092011b.bil -vvfile eagle_fov_fullccd_vectors.bil -navfile e092011b_p_sct0.1_nav_post_processed.bil -dem ASTER_resampled_0.1667_arcsec.dem -igmfile e092013b_p_sct0.1.igm

The second step of geocorrection is to re-project the IGM file into the chosen map projection, using apltran.

Example commands:

apltran -inproj latlong WGS84 -igm e092013b_p_sct0.1.igm -outproj utm_wgs84N 31 -output e092013b_p_sct0.1_utm31n.igm

apltran -inproj latlong WGS84 -igm e092013b_p_sct0.1.igm -outproj osng /users/rsg/arsf/dems/ostn02/OSTN02_NTv2.gsb -output e092013b_p_sct0.1_utm31n.igm

The third step is then to map it.

Example commands:

aplmap -igm e092013b_p_sct0.1_utm31n.igm -lev1 e092011b.bil -mapname e092013b_utm31.mapped -bandlist 30 15 7

aplmap -igm e092013b_p_sct0.1_utm31n.igm -lev1 e092011b.bil -mapname e092013b_utm31.mapped -bandlist 30 15 7 -pixelsize 2 2

aplmap -igm e092013b_p_sct0.1_utm31n.igm -lev1 e092011b.bil -mapname e092013b_utm31.mapped -bandlist 30 15 7 -oversample 3 3