Opened 4 months ago

Last modified 6 days ago

#642 new flight processing

CA18/208, flight day 229/2018, Ontario, Canada

Reported by: dac Owned by:
Priority: immediate Milestone:
Component: Processing: general Keywords:
Cc: Other processors:

Description (last modified by dac)

Data location: /users/rsg/arsf/arsf_data/2018/flight_data/canada/CA18_207-2018_229_Ontario

Data arrived from NERC-ARF via hard driver on 12/09/2018

Scientific objective: Canadian Wildfire Observations with SLSTR & Aircraft: Directional Effects of Thermally Emitted Radiation & Exploration of Flaming/Smouldering Partitioning and Plume Trace Gas Emission Ratios. Fenix and Owl data are being processed to aid geocorrection of other thermal sensors flown on the aircraft.

PI: Martin Wooster

BAS Project Code: D207

  • Only two owl lines – one marked as T2 test. Haven't renamed until checking logsheets
  • Navigation data (.sol format) and level1b (unmapped) data to be delivered first. The level1b data will be used to determine which lines map.


  • Fenix (requested, flown)
  • Owl (requested, flown)

Change History (22)

comment:1 Changed 4 months ago by dac

  • Description modified (diff)

comment:2 Changed 4 months ago by wja

Navigation Processing
Basestation PPP:

Ellipsoid Height346.745

There are two sequential sets of aircraft GNSS/IMU data. The smaller (*dat.00[0-3]) is a relatively straight flight and is unlikely to contain a flightline. The larger set of *dat.000 files have been processed.
All produced files put in their relevant project subdirectories.

Last edited 4 months ago by wja (previous) (diff)

comment:3 Changed 4 months ago by asm

Owl processing

Have moved the dir called "T2_test" outside this project as it looks it was a calibration test. Have renamed the owl lines from the original names to our standard naming format for raw data.
There are a total of 16 flightlines with T2 only for lines: 1, 2, 6 and 7. Will symlink those files into the other directories and process to level1b.

Last edited 4 months ago by asm (previous) (diff)

comment:4 Changed 4 months ago by asm

Hyperspectral processing

Started geocorrecting the files to find the SCT values. The generated ASTER DEM was invalid and giving wrong errors. Have created a SRTM DEM that works fine.

comment:5 Changed 4 months ago by asm

Owl Processing

Started. Found SCTs:

Line SCT
1 2
2 2
3 2
4 2
5 2
6 2
7 2
8 2
9 2
10 2
11 2
12 2
13 2
14 2
15 2
16 Not in log and outside the range of nav files

The boresight is not done yet and the lines barely overlap (on going). Have started Fenix processing to see how good is the accuracy in the overlaping regions. The sensors should have not moved since the last calibrations as operations tried to make sure the sensors were not moved during the journey to Canada. Will hold on a bit the creation of the files with all bands until the boresight checks are done.

comment:6 Changed 4 months ago by asm

Fenix Processing

Started. Found SCTs:

Line SCT
1 2
2 2
3 1
4 2
5 2
6 1
7 2
8 2
9 2
10 1
11 3
12 2
13 2
14 1
15 2

Flightline 11 needed to force aplnav because the navigation was going backwards in time at some point.

Last edited 4 months ago by asm (previous) (diff)

comment:7 Changed 4 months ago by asm

Owl Processing

The lines aligned well with the Fenix. Went ahead and mapped all bands. Later on the boresight concluded that the sensors did not moved so created a full delivery

Have added a new directory called "mosaic" and added a 3 bands composite GeoTiff mosaic that is georreferenced. Needed to make changes to the scripts.

comment:8 Changed 4 months ago by asm

Owl Processing

Have created a readme file. With the exception of the Logsheet, the delivery is ready so will mark as "Ready for delivery check" as PI needs it ASAP.

comment:9 Changed 4 months ago by asm

Owl Processing

Created a logsheet but some of the information such as pilot or observer is not present.

comment:10 follow-up: Changed 3 months ago by wja

Fenix/Owl Delivery Check
Manual checks of the level1b data is still to be undtertaken, updates so far:
Fenix & Owl:

  • Some metadata missing in logsheets (not on origional hard copies).
  • Naming convention of the screenshots need updating.
  • No XML files in open (in both project and line information directories).


  • ASTER & SRTM DEMs are present. I believe ASTER was not used and needs removing.
  • Extra level 1b data: level1b/*bil_mask*, potential duplicates with the wrong names. BIL sizes need checking afterwards.
  • /mosaic/*.tiff should be *.tif for consistency.
  • failed, Grass is unable to find UTM15N.


  • There is an extra flightline in Level1b (16).
  • Example gdal_translate command in ReadMe has no band numbers or coordinates input and the wrong UTM zone.
  • failed. Error complained about being unable to find mapped BILs in tmp processing directory.

comment:11 Changed 3 months ago by asm

Fenix processing

Did not have time to write that the Fenix delivery was also created.

comment:12 Changed 3 months ago by asm

Owl processing

-Deleted level1b flightline 16 as it is outside the navigation file
-Solved xml files problem. Changed character "&" for "And" as it was causing problems. Now xml looking OK
-There was a typo in the apl example comands with the uncorrect UTM (copy pasted from other projects). Solved that and rerun the checks. The lines have created fine (except for the tif step but that is OK because there is a problem in the script)

The delivery should be ready for a second check. And hopefully will be ready to deliver.

comment:13 in reply to: ↑ 10 Changed 3 months ago by asm

Replying to wja:

  • failed, Grass is unable to find UTM15N.

This is OK, there is a problem with the script. Please see the latest issue I opened for more info.

comment:14 Changed 3 months ago by wja

Owl Delivery
Level1b and mapped data has been checked. First ~25 bands are saturated, response is low with the rest of the bands.
Mapped data have been zipped.
Screenshots have been renamed.
However ReadMe is now only one page.

comment:15 Changed 3 months ago by asm

Owl Delivery

Not sure what was wrong with the readme to be one page only, solved that. Also added a note about why the bands look like that (T2 was set to 75C and T1 to 30C to fulfill purpose of the project so that is what is causing the bands to be noisy or featureless). The owl however should be enough for them to geocorrect the other sensors they flew.

comment:16 Changed 3 months ago by wja

Fenix Delivery Check
Line 54 contains overflowing values and is 'bad' in every band and line.
Underflows are frequent greater than band 370 due to all the waterbodies in the flightline.
Spectra check with Py6S shows good observed spectra.

comment:17 Changed 3 months ago by asm

Owl and Fenix deliveries

Fenix mapped files finally zipped (everything OK). Delivered both datasets via FTP slot 1 and sent notification to PI on 11th October 2018. Finalised deliveries.

comment:18 Changed 3 months ago by wja

Fenix/Owl Processing
SRTM DEM generated.

comment:19 Changed 3 months ago by dac

Fenix and Owl

Moved delivered data to 'previous_deliveries' and created new copy with renamed mosaic. Once readme has been updated with correct SRTM information ready to re-deliver.

comment:20 Changed 3 months ago by dac

Fenix and Owl

Updated readme with SRTM info. Can be re-delivered with 224b.

comment:21 Changed 9 days ago by wja

JSONs created and added to PostGIS. In the queue to rsync to CEDA.

comment:22 Changed 6 days ago by wja

Upload to CEDA complete (18/01/2019)

Note: See TracTickets for help on using tickets.