Opened 11 months ago

Last modified 7 months ago

#647 new flight processing

CA18/208, flight day 234/2018, Ontario, Canada

Reported by: dac Owned by:
Priority: immediate Milestone:
Component: Processing: general Keywords:
Cc: Other processors:

Description

Data location: /users/rsg/arsf/arsf_data/2018/flight_data/canada/CA18_207-2018_234_Ontario

Data arrived from NERC-ARF via hard driver on 12/09/2018

Scientific objective: Canadian Wildfire Observations with SLSTR & Aircraft: Directional Effects of Thermally Emitted Radiation & Exploration of Flaming/Smouldering Partitioning and Plume Trace Gas Emission Ratios. Fenix and Owl data are being processed to aid geocorrection of other thermal sensors flown on the aircraft.

PI: Martin Wooster

BAS Project Code: D207

  • Missing Owl calibration data
  • Owl line 234-18_234-18_2018-08-22_16-53-50 contained no header data so removed.
  • Navigation data (.sol format) and level1b (unmapped) data to be delivered first. The level1b data will be used to determine which lines map.

Sensors:

  • Fenix (requested, flown)
  • Owl (requested, flown)

Change History (27)

comment:1 Changed 11 months ago by dac

Fenix

Quickly processed Fenix data to 1b to test script.

Version 0, edited 11 months ago by dac (next)

comment:2 Changed 11 months ago by asm

Unpacking
-There are 11 owl lines and 11 fenix.
-All owl lines have T1 and T2 calibration files

comment:3 Changed 11 months ago by asm

Owl Processing

Started in order to create the level1b files.

comment:4 Changed 11 months ago by dac

Navigation Processing

Subset basestation data to single day using:

teqc -st 20180822000000 -e 20180822235959 +nav Default_9488_0509_114526.18n \
           Default_9488_0509_114526.m00 > Default_9488_0509_114526.18o

Got antenna info from file (LEIAX1202GG) and used height of 0.889 m provided by Gary.

Basestation PPP result

Latitude 51 04 01.40114
Longitude -93 48 14.26959
Ellipsoid Height 345.856 m

comment:5 Changed 11 months ago by dac

Navigation Processing

Processed navigation data. Reduced elevation mask to 8 degrees to get better result. Position separation is higher than normal but within 10 cm for most of flight.

comment:6 Changed 10 months ago by wja

Hyperspectral Processing
SRTM DEM created.

comment:7 Changed 10 months ago by wja

Owl Processing
Starting Owl processing.

comment:8 Changed 10 months ago by dac

Fenix Processing

Needed to increase buffer around DEM to 0.5 degrees so generate_apl_config.py would run through.

comment:9 Changed 10 months ago by dac

Fenix Processing

Found SCT values

Flightline FENIX
1 2
2 2
3 2
4 2.9
5 2
6 2
7 2
8 2
9 2
10 2
11 1

Changed pixel size from 5 m to 3 m. Given altitude is 3.5 m at nadir and 3.76 at the edge of swath. Went for 3 rather than 4 as high resolution is required to match other cameras to.

Starting mapping all bands.

Last edited 10 months ago by dac (previous) (diff)

comment:10 Changed 10 months ago by wja

Owl Processing

FlightlineSCT
1N/A
22.00
32.00
42.00
52.00
62.00
72.00
82.00
92.00
102.00
112.00

Flightline 1 header file reports 1024 lines, however, level1b *BIL shows ~520 lines (no errors in log). This is too small to be useful and will be omitted from further processing.
Three band BIL of flightline 10 is noisy.

Last edited 10 months ago by wja (previous) (diff)

comment:11 Changed 10 months ago by dac

Fenix Processing

All bands mapped. Generating delivery.

comment:12 Changed 10 months ago by wja

Owl Processing
Flightline 10 T1 calibration file has some strange features resulting in poor quality. Reprocessed with calibration files (T1 & T2) from flightline 11 (close and only flight with same integration time) and looks acceptable.
All mapped flightlines have been processed.
Generating delivery.

comment:13 Changed 10 months ago by dac

Fenix Processing

Delivery created, ready for checking.

comment:14 Changed 10 months ago by wja

Owl Delivery
Delivery created, ready for delivery check.

comment:15 Changed 10 months ago by dac

Owl

Starting delivery check.

comment:16 Changed 10 months ago by wja

Fenix Delivery Check
Started Fenix delivery check.

comment:17 Changed 10 months ago by dac

Owl delivery check
Generally looks good, couple of comments

  • In data quality section of readme non-latex character '11aZa' needs to be replaced.
  • No underflows or overflows listed in readme - I'm not sure this is correct, suggest re-running autoqc.py script and checking for errors.
  • check_ap_cmd ran through OK, three band for line 9 looks a bit odd though.
  • Spectral for line 9 has some weird features, should add a note to the readme about it.

Once these are addressed ready to deliver.

comment:18 Changed 10 months ago by wja

Fenix Delivery Check

  • Logsheet line 1 has very short duration (looks like from Owl header).
  • Flightline 4 is missing from navigation/, line_information/ & mapped/
  • flightline/fodis needs removing.
  • Logsheet metadata: weather = '?', probably best if left blank.
  • Read_Me: 'logsheet file name' table is incomplete. Also picture of Dornier.
  • XML files: '&' in project discription and title needs changing to '&' so that XMLs can be opened.
  • Py6S looks okay, although perhaps not as precise as normal in some peaks? Can sheck this in wja's machine /tmp/fenix_vs_py6s
  • check_apl_cmd ran through and looks good.
  • Spectra of flightline 11 should be looked at, very little reflectance in higher wavelengths. i notice it has a shorter integration time than the others.
Last edited 10 months ago by wja (previous) (diff)

comment:19 Changed 10 months ago by dac

Fenix processing

  • Updated logsheet to use times from Fenix headers.
  • Removed fodis dir
  • Updated logsheet names in readme
  • Replaced '&' in XML files
  • Spectra look OK to me compared to simulations in Py6S
  • Added note about line 11 in readme. The integration time is very short so I assume it was intentional given project aims and earlier discussions with PI.

Line 4 failed due to navigation data going backwards in time, will investigate and try to process.

comment:20 Changed 10 months ago by wja

Owl Delivery

  • Read_Me corrections:
    • non-LaTex charecter removed
    • auto.qc script gave no under-/over-flows; section has been removed with a mention that they should be expected due to the low integration times of this flight.
    • Description of /mosaic/ directory edited to match that of Fenix delivery.
  • Flightline 9 reprocessed. T1 calibration file contain what looks like errorneous data starting after line 274, a new T1 calibration was created and used up until this line. Spectra looks as expected.

comment:21 Changed 10 months ago by wja

Owl/Fenix Processing
Flightline number reference:

Logsheet (hardcopy)OwlFenix
-1 (Failed)-
121
232
343
454
5-5
6.166
777
8.188
999
8.21010
6.21111

comment:22 Changed 10 months ago by dac

Owl Delivery Check

Problems have been fixed and reprocessed line 9 looks good. Zipping mapped files and marking as ready to deliver.

comment:23 Changed 10 months ago by dac

Fenix processing

Processed line 4 with -force flag and manually synced navigation using SCT value of 2.9. Added to existing delivery and recreated mosaic. Added note about line 4 to readme. Ran check_apl_command to check all files where there for line 4.

comment:24 Changed 10 months ago by wja

Fenix Delivery
Ready to deliver.

comment:25 Changed 10 months ago by wja

Fenix/Owl Delivery
Deliveries hardlinked to FTP slot 1, notification e-mail sent to PI. Delivery finalised.

comment:26 Changed 7 months ago by wja

Archiving
JSONs created and added to PostGIS. In the queue to rsync to CEDA.

comment:27 Changed 7 months ago by wja

Archiving
Upload to CEDA complete (18/01/2019)

Note: See TracTickets for help on using tickets.