Opened 3 years ago

Last modified 14 months ago

#679 new flight processing

Magic, flight day 233a/2021, Kiruna, Sweden

Reported by: wja Owned by:
Priority: immediate Milestone:
Component: Processing: general Keywords:
Cc: Other processors:

Description (last modified by wja)

Data location: /users/rsg/arsf/arsf_data/2021/flight_data/sweden/Magic-2021_233a_Kiruna/
Data arrived from Kings College London via hardrive from Sweden on September 20th 2021.

Scientific objective: International project (includes NASA, CNRS, ESA, DLR, Eumetsat) to monitor and identify methane inventories in the arctic. JPL's HyTES instrument was utilised. A total of three aircraft were used to collect data, along with deployed balloons and ground measurments.

PI: Martin Wooster

Change History (18)

comment:1 Changed 3 years ago by wja

Unpacking

  • Issues with navigation timestamps reported when using run_unpacked_checks.py:
Running check: arsf_check_library.checks.check_nav_timestamps                                                         
No date record ($GPZDA) found in nav file - will assume it is binary if cannot find a line starting with '$GPGGA' or '$PRDID'.                                                                                                              
Error in /users/rsg/arsf/usr/lib/python/site-packages/arsf_unpacking/unpacking_utilities.py : write_message: Nav file does not have time stamps or failed to read the file using readAscii: /users/rsg/arsf/arsf_data/2021/flight_data/swede
n/SwathS-2021_233a_Kiruna/hyperspectral/fenix/FENIX233a-21-1/capture/FENIX233a-21-1.nav 

for all lines, along with:

Running check: arsf_check_library.checks.check_nav_last_frame
No date record ($GPZDA) found in nav file - will assume it is binary if cannot find a line starting with '$GPGGA' or '
$PRDID'.
Error in /users/rsg/arsf/usr/lib/python/site-packages/arsf_unpacking/unpacking_utilities.py : write_message: Nav file 
does not have time stamps or failed to read the file using readAscii: /users/rsg/arsf/arsf_data/2021/flight_data/swede
n/SwathS-2021_233a_Kiruna/hyperspectral/fenix/FENIX233a-21-1/capture/FENIX233a-21-1.nav
  • No fps_set in Fenix1k headers (expected)
  • Found empty file: /users/rsg/arsf/arsf_data/2021/flight_data/sweden/SwathS-2021_233a_Kiruna/posatt/basestation/Data/logfile.txt

comment:2 Changed 3 years ago by wja

  • Description modified (diff)

comment:3 Changed 3 years ago by wja

Nav Processing

233a nav processing

Basestation in Leica format and has intermittent coverage of the days 199, 200, 203, 210, 228, 229 & 233.

Day 233 comprises:

  • Start: 21/8 12:04
  • Issue at 12:07 (apparently gap of 330908 seconds? or change in recording rate?)
  • End: 21/8 17:03

Have extracted the period of day 223 from 12:08 to 17:03 with the following command:

teqc -st 210821120800 -e 210821170300 +nav rinex/2021233a.21n Default_9488_0718_111526.m00 > rinex/2021233a.21o

The following warning was raised by POSPac when importing basestation data:

:\data\scratch\arsf\2021\flight_data\sweden\Magic-2021_233a_Kiruna\processing\basestation_proc\rinex\2021233a.21o

Date created: 10/15/2021 2:21:40 PM
Date imported: 10/15/2021 2:22:50 PM
Errors: 1
1.	Unrecognized receiver with serial number: '469488'. Unknown receiver created.

Continued doing single base station processing chain:

1) Compute RTX Coordiantes:

  • Horizontal sigma: 0.012 m
  • Vertical sigma: 0.013 m

2) Set base station
3) IN-Fusion Single Base processing

Position Error RMS is low according the exported POSPac QC report:

  • Vertical < 3 cm apart from two spikes < 4 cm
  • Horizontal error < 3cm

comment:4 Changed 3 years ago by wja

Hyperspectral Processing

Processing 3 RGB band flightlines to check geo-accuracy.

Issue with aplcal:

########## ERROR #########
Error on line: 0
A number of bands in the raw file disagree with the calibration file wavelengths. Number that agree: 0 Number of bands: 87
This probably means that the calibration file and raw file are not compatible. Maybe the wrong bandset / config file has been used at data collection.
########## ERROR #########

Will try updating wavelengths in Fenix1k header files.

All these flightlines have a VNIR binning factor of 8.

The following command was used:

replace_wavscale.py -w /users/rsg/arsf/calibration/2021/fenix1k/wlcal8b_vnir_1b_swir.txt hyperspectral/fenix/F*/capture/*hdr --final

Majority of 3 band RGB mapped data is predominantly green or red. Could be related to an underflow but I am unsure why it would only affect a single section of a flightline. Will investigate. These bands also look noisey.

comment:5 Changed 3 years ago by wja

Hyperspectral Processing

3 band data is fine. The green nad red was just a result of contrast stretching done by Tuiview.

comment:6 Changed 3 years ago by dac

Digital Camera Processing
Started converting raw PhaseOne images to Tiff.

comment:7 Changed 3 years ago by dac

Digital Camera Processing

All raw images converted to tiff.

comment:8 Changed 3 years ago by wja

No darkframes for line 11. Cannot calibrate.

Version 0, edited 3 years ago by wja (next)

comment:9 Changed 3 years ago by wja

Hyperspectral Processing

All lines processed with all bands (except line 11). Creating delivery

comment:10 Changed 3 years ago by wja

Ready for Checking

Created Fenix1K delivery.

comment:11 Changed 2 years ago by mark1

Hyperspectral delivery check:

  • Missing project_information XML
  • Add a note to the logsheet that line 11 data were unable to be processed

Otherwise looked good. Not zipped the mapped data.

comment:12 Changed 2 years ago by wja

Delivery Check

Apologies for the delay on this.

  • Have added project XML.
  • Have added note in ReadMe about line 11 not being processed. Should we add a note on logsheet also?
  • Mapped data zipping

comment:13 Changed 2 years ago by wja

Delivery

Have updated logsheet with note about line 11 failing.

Should be ready to deliver

comment:14 Changed 2 years ago by wja

Delivery

Finalised, put on FTP slot 5

comment:15 Changed 2 years ago by wja

Delivery Sent

PI notifified

comment:16 Changed 15 months ago by wja

Archiving

Created JSONs, committed them to database.

Ready to sync to CEDA.

comment:17 Changed 15 months ago by wja

Archiving

Started transfer to CEDA. 08/09/2023 16:20

comment:18 Changed 14 months ago by wja

Transfer complete

Note: See TracTickets for help on using tickets.