Opened 6 years ago

Closed 3 years ago

#518 closed flight processing (fixed)

GB12/05, flight day 075/2014, Monks Wood

Reported by: knpa Owned by:
Priority: alpha 4 medium Milestone: 2014 data processing completion
Component: Processing: general Keywords:
Cc: Other processors:

Description (last modified by dap)

Data location: /users/rsg/arsf/arsf_data/2014/flight_data/uk/GB12_05-2014_075_Monks_Wood/

Data arrived from ARSF via network transfer on 17/03/2014

GB12/05 Scientific objective: unknown

GB12/05 Priority: 8

PI: Ross Hill

Sensors:

Camera (Requested)
LiDAR (Requested), delivered 25/09/2014
FW LiDAR (Requested but lower priority), delivered 25/09/2014
Fenix (Requested)

Change History (47)

comment:1 Changed 6 years ago by knpa

  • Description modified (diff)

comment:2 Changed 6 years ago by knpa

There is a problem with the Fenix data, the .nav and .f98 files are blank.

From Tom:

Okay there was a problem with the Fenix PPS cable (bad solder joint) hence the reason that the .f98 and .nav files are empty. You will have to talk to Gary as we have been informed that there are enough ground control points on the Monks Wood site for the data to be corrected. Also the navigation data was collected and is there for the Fenix, it just wasn't included in the sensor generated files.

comment:3 Changed 5 years ago by tipo

Started navigation processing.

comment:4 Changed 5 years ago by tipo

Finished navigation processing, started RCD processing.

comment:5 Changed 5 years ago by tipo

Started LiDAR processing.

comment:6 Changed 5 years ago by tipo

Created hyperspectral delivery.

comment:7 Changed 5 years ago by tipo

LiDAR has lines flown at different heights which need slightly different pitch and roll values.
High lines: p=0.00118716 r=-0.003968414

comment:8 Changed 5 years ago by stgo

RCD D/c in progress

comment:9 Changed 5 years ago by stgo

RCD d/c complete, no problems found. Will include in delivery this afternoon.

comment:10 Changed 5 years ago by stgo

RCD data dispatched on 04/06/2014

comment:11 Changed 5 years ago by tipo

LiDAR is taking a long time, as I'm having to find the roll error seperately for each line.

comment:12 Changed 5 years ago by tipo

Higher altitude lines:

line pitch roll
150700 0.00113716 -0.00426684
151651 0.00113716 -0.00376841
152408 0.00113716 -0.00416841
153326 0.00113716 -0.00376841
153957 0.00108716 -0.00399841
154845 0.00108716 -0.00374841
155537 0.00108716 -0.00428414
160352 0.00108716 -0.00374841
161021 0.00118716 -0.00404841
165550 0.00118716 -0.00404841
164909 0.00118716 -0.00404841
164120 0.00118716 -0.00404841


The lower altitude lines are all:
0.00108716 -0.003968414
Last edited 5 years ago by tipo (previous) (diff)

comment:13 Changed 5 years ago by tipo

Lines 151651 and 154845 have areas where they narrow or taper off. The log shows that it was close to the maximum range gate, so its possible it moved slightly out of range, causing the gaps in the data.

Last edited 5 years ago by tipo (previous) (diff)

comment:14 Changed 5 years ago by tec

Beginning hyperspectral processing

comment:15 Changed 5 years ago by tec

Hyperspectial processing has stoped due to lack of suitable darkframes. Speaking to ops to get data with an integration time of 20.000

Fenix data has integration times of

Int 1 Int2
20.0000 10.0000
29.3000 2.2000
25.0000 12.0000
30.0000 18.0000
Last edited 5 years ago by tec (previous) (diff)

comment:16 Changed 5 years ago by stgo

Low altitude full waveform created.

comment:17 Changed 5 years ago by stgo

High altitude waveform has been created. Lidar ready for delivery check.

comment:18 Changed 5 years ago by dap

Beginning LiDAR delivery check.

comment:19 Changed 5 years ago by dap

LiDAR Delivery Check
Complete. Issues found as follows:

  • Contents page of readme does not have full waveform files listed.
  • Link in "Data Details" section is a LAS 1.0 link, but LAS 1.2 data was processed.
  • "Per flight line statistics" section is on next page from heading. I'd suggest you put the heading on the next page.
  • HAWK data was collected, but typed up logsheet does not reflect this.
  • Navigation directory is empty - should contain the .sol, .sup and .trj files.
  • Flightline screenshots have ".txt.jpg" file extension - should be just .jpg.
  • No vector screenshots - PI has not replied to knpa's email regarding this.
  • Multiple LiDAR deliveries found in delivery directory - please delete one.
  • Line 18 is oddly shaped - please comment on this in the data quality remarks.
  • ASCII files should have had DOS line terminators - I've fixed this.

comment:20 Changed 5 years ago by stgo

I've recreated the readme for the lidar delivery following the comments above.

The logsheet is at fault here, we can't have collected hawk data as the hawk instrument has not been installed in the plane. I think Ops meant the SWIR part of Fenix.

comment:21 Changed 5 years ago by dap

LiDAR Delivery Check

Rechecking delivery.

comment:22 Changed 5 years ago by dap

LiDAR Delivery Check
Completed. Issues found were:

  • Two LiDAR delivery directories - please remove irrelevant one
  • .sup file is missing from navigation directory
  • Screenshots named incorrectly - the LDR need not precede the project code in the file names.
  • .sol file is named incorrectly - the Julian day should be preceded by a '_' character, not '-'.

comment:23 Changed 5 years ago by dap

LiDAR Delivery Check
Complete. proj_tidy complained at screenshot of each flightline - script should be updated to not do this.

LiDAR data ready for delivery.

Last edited 5 years ago by dap (previous) (diff)

comment:24 Changed 5 years ago by dap

LiDAR data delivered to Dr Ross Hill, 25/09/2014.

comment:25 Changed 5 years ago by dap

  • Description modified (diff)

comment:26 Changed 4 years ago by tec

Archiving
Started

Last edited 4 years ago by tec (previous) (diff)

comment:27 Changed 4 years ago by dac

Fenix

Ops have now collected darkframes - resuming processing.

comment:28 Changed 4 years ago by dac

Fenix Processing

Found SCT values. Will process VNIR bands but can't deliver SWIR.

Line SCT
-1 No data
-2 0.92 Short and not on logsheet (removed)
-3 1
-4 0.92
-5 0.96
-6 0.91
-7 0.95
-8 0.9
-9 1.84
-10 0.9
-11 0.99
-12 0.96
-13 1
-14 1.02
-15 0.93
-16 0.98
-17 1.01
-18 1.03
-19 0.94
-20 0.9
-21 0.96
-22 1.02
-23 0.96
-24 0.93
-25 0.94
-26 0.99
-27 1.05
-28 1.01
-29 0.93
-30 1
-31 0.9
-32 0.91
-33 1.02
-34 0.96
Last edited 4 years ago by dac (previous) (diff)

comment:29 Changed 4 years ago by dac

Fenix Processing

Mapped all bands - creating delivery.

comment:30 Changed 4 years ago by dac

Fenix Processing

Delivery created, splitting VNIR and SWIR bands.

comment:31 Changed 4 years ago by dac

Fenix Processing

After checking, determined SWIR data is OK so will deliver. Ready for DC.

comment:32 Changed 4 years ago by lah

Fenix DC

  • line info & proj info xml files are missing.
  • Have copied in missing logsheet and removed eagle & hawk pages (and renamed 075).
  • renamed screenshots.
  • check_bil_size reports: f075133b_mapped_osng.bil: BIL Filesize incorrect. Has correct no. bands and maps ok, so not sure what's wrong with it.
  • under/overflows seem reasonable, but table goes off page. Readme needs recreating.
  • scts look ok.
  • spectra could do with double checking. SWIR seems rather high compared to VNIR. Our favourite spike is also present (bands 346-348 the cause).

Can't finish delivery check until xml files are created. Looking into fixing scripts.

comment:33 Changed 4 years ago by dac

Fenix Processing

  • Have generated line info and project info XML files.
  • Tried reprocessing f075133b_mapped_osng.bil, still have error. Actual size 1559516964, calculated size 1558215740. Not sure why this is.
  • Generated plots comparing to 6S simulated spectra, for some pixels looks OK for others SWIR is too high. Possibly integration time issue, will investigate further before sending.

comment:34 Changed 4 years ago by dac

Fenix Processing

After more investigation determined SWIR is not usable (as originally though). Splitting off bands.

comment:35 Changed 4 years ago by dac

Fenix Processing

  • Removed SWIR bands and updated Readme
  • Trimmed off additional bytes from end of f075133b_mapped_osng.bil
  • Ready for DC

comment:36 Changed 4 years ago by gej

Starting fenix delivery Check

comment:37 Changed 4 years ago by gej

Fenix delivery check

  • Since SWIR bands have been removed, we should remove SWIR overflows and underflows from read me
  • f075333b_mapped_osng.bil is reporting incorrect filesize.

comment:38 Changed 4 years ago by dac

Fenix Processing

  • Have removed SWIR bands from readme.
  • f075333b_mapped_osng.bil has lost most of the information from the header file. Needs to be recreated.

comment:39 Changed 4 years ago by asm

Fenix Processing
-f075333b_mapped_osng.bi recreated, removed SWIR bands and replaced the .bil and .hdr files on delivery directory. Still complaining about size. I believe is the same error that f075133b_mapped_osng.bil and probably needs to be trimmed.

comment:40 Changed 4 years ago by asm

Fenix Processing

-Flightline f075333b_mapped_osng.bil recreated a second time using apl and repleaced. Not complaining about size any more..

-In the process discover some missing info in the .hdr files. Corrected manually all *1b.bil.hdr

comment:41 Changed 4 years ago by asm

Fenix Processing

-All .hdr files corrected. Should be ready to continue DC.

comment:42 Changed 4 years ago by gej

Fenix Delivery Check

Everything okay now.
Zipping mapped files then ready to deliver

comment:43 Changed 4 years ago by gej

Fenix Delivery Check

Mapped Files zipped
Ready for delivery

comment:44 Changed 4 years ago by dac

Fenix Delivery

Dispatched Fenix data on USB hard drive to Ross Hill 15/09/2015

comment:45 Changed 4 years ago by dac

Ross Hill confirmed receipt of hard drive (17/09/2015). Has mailed back.

comment:46 Changed 3 years ago by dac

Archiving

Tidied up and started uploading to NEODC (running on gridmaster4).

comment:47 Changed 3 years ago by asm

  • Resolution set to fixed
  • Status changed from new to closed

Archiving

Data is available from NEODC

Note: See TracTickets for help on using tickets.