Opened 8 years ago

Closed 6 years ago

#610 closed flight processing (fixed)

EU15/18, flight day 274, Toulouse

Reported by: asm Owned by:
Priority: immediate Milestone:
Component: Processing: general Keywords:
Cc: Other processors:

Description

Data location: /users/rsg/arsf/arsf_data/2015/flight_data/france/EUFAR15_18-2015_274_Toulouse

Data arrived from ARSF via SATA disk on 26/10/2015.

Scientific objective: Hyperspectral measurements over agro-forestry areas for landscape assessment of agricultural health, ecophysiology, and satellite product validation.

Priority: Unknown

PI: Jean-Louis Roujean
EUFAR Project ID: AHSPECT

Sensors:

  • Fenix (requested)
  • Leica LIDAR (not requested)
  • Leica FW LIDAR (not requested but flown anyway)
  • Owl (not requested but flown anyway)

Change History (52)

comment:1 Changed 8 years ago by dac

From Gary 05/01/2016 "all LiDAR, RCD to be processed. Speak to Gary re Fenix but both altitudes are very likely to be required . . depending on illumination / signal. One altitude will certainly be required. "

comment:2 Changed 8 years ago by mark1

Base station verification

Latitude 43 37 18.97144
Longitude 1 22 43.90890
Ell. Height 200.478 metres
AntHgt 0.085m (LEIAX1202GG, MeasDist 0.0)

comment:3 Changed 8 years ago by mark1

Have moved line 1 for fenix into separate directory (hyperspectral/fenix/line_01) as it is not covered by navigation, line appears short and noisy when viewed, and is not on logsheet. Have moved it to try and generate the processing configuration file. Have also ignored problems with owl in generating config file.

comment:4 Changed 8 years ago by mark1

Have replaced wavescales.

comment:5 Changed 8 years ago by lah

RCD Processing

Images converted to Tiffs.

comment:6 Changed 8 years ago by lah

Navigation

Created camera sol file with saved settings (all good).

comment:7 Changed 8 years ago by mark1

Fenix Processing

Going through the sct corrections. Have processed (with default settings) the LiDAR data to create tiffs to help with sct correcting the fenix.

comment:8 Changed 8 years ago by lah

RCD ready for DC

comment:9 Changed 8 years ago by dac

RCD Delivery Check

Starting delivery check

comment:10 Changed 8 years ago by dac

RCD Delivery check

Everything OK - ready to deliver

comment:11 Changed 8 years ago by mark1

Fenix processing

Lines 2 - 10 process fine and after SCT correction appear to line up OK. Lines 11, 12, 13 all look to suffer from incorrect frame rate being recorded (assume all remaining lines suffer). Will try and correct these 3 lines before proceeding with other lines.

comment:12 Changed 8 years ago by dac

RCD Delivery

Sent to PI on hard drive.

comment:13 Changed 8 years ago by mark1

Fenix processing

Frame rate for line 11 appears to be approx. 23.042. Will try the same value for line 12 onwards.

comment:14 Changed 8 years ago by mark1

Fenix processing

Frame rate of 23.042 is suitable for lines 11 - 18.

Line 18 has no dark frames.

comment:15 Changed 8 years ago by mark1

Fenix processing

Have used darkframes from line 17 to process line 18. Same fps, tint1, tint2 and closest in time.

comment:16 Changed 8 years ago by mark1

Fenix processing

SCT values per line.

Flightline FENIX
2 1.00
3 0.97
4 -0.06
5 1.00
6 1.00
7 0.97
8 1.00
9 0.95
10 0.97
11 3.01
12 2.00
13 13.02
14 2.00
15 15.00
16 13.00
17 3.00
18 3.00

comment:17 Changed 8 years ago by mark1

Fenix processing

Delivery has been created. Ready for check.

comment:18 Changed 8 years ago by asm

Fenix Delivery Check

Started.

comment:19 Changed 8 years ago by asm

Fenix Delivery Check

Everything looks fine. I suggest a couple of changes on the readme but it is up to mark1 to change them:
1-Add a note about flightline UL001 flown and recorded on logsheet but not recorded on sensors. (Will be better than to delete it from logsheet as it shows it was flown)
2-Not sure if we have a how-to procedure but other projects has on the flightline name table the number of parts for every flightline. If you want to change it, can have a look at 242 2015 for example.

Spectra checked using py6s, looks fine except for a peak on the overlapping region that will be masked out.

Only changes to the readme has been suggested; will start zipping mapped files.

comment:20 Changed 8 years ago by asm

Fenix Delivery Check

All flightlines have been zipped correctly as shown on logs.

comment:21 Changed 8 years ago by mark1

Fenix

Have made suggested changes to Read me

comment:22 Changed 8 years ago by stgo

Lidar

The lines make up low/high altitude pairs. These are noted in the table below, there are some areas of additional overlap but these are generally at the corners.

4343 4621
2700 2913
4747 5116
5309 5733
3148 3603
2755 2645
2215 1914
3756 4405

The majority of these lines line up nicely with the following pitch and roll corrections:

roll -0.00101
pitch -0.00226

However the following three pairs suffer from variable pitch/roll:

3148 3603
2755 2645
2215 1914

I've found values between 20cm and 60cm varying along track, can't seem to get them to line up.

comment:23 Changed 8 years ago by asm

Lidar Processing

Taking over this project.

comment:24 Changed 8 years ago by lah

Fenix delivered by hard drive 05/04/16.

comment:25 Changed 8 years ago by dac

Archiving

Started uploading processed Fenix data to NEODC.

comment:26 Changed 8 years ago by asm

Lidar Processing

Output settings were incorrect, changed from OSGB to UTM31N.

comment:27 Changed 8 years ago by dac

Archiving

Fenix data uploaded to NEODC.

comment:28 Changed 8 years ago by asm

Lidar Processing

Flightline high_9 does not overlap with any other flightline and UL001 only overlaps with high_9. Found pitch and roll values, they are consistent trough all lines:

roll: -0.0009
pitch: -0.0024

comment:29 Changed 8 years ago by asm

Lidar Processing

Classification of discrete LiDAR files is finally complete (phew!).

comment:30 Changed 8 years ago by asm

Lidar Processing

Delivery created. Have already measured elevation difference. Will create readme and will be ready for delivery check (big size, may take time).

comment:31 Changed 8 years ago by asm

Lidar Processing

Readme created. Ready for Delivery Check.

comment:32 follow-up: Changed 8 years ago by lah

Lidar DC

  • Weird flight pattern, but overlaps look ok
  • changed dem name in utm hdr
  • Logsheet text goes off page - really ought to be redone. (Was delivered with Fenix, so DCs need to be better as well!)
  • FW look fine in waveviewer
  • Readme text has some errors: should be "flightlines have", not "flightlines has", "same value as", not "same value than" and "screenshot reolution" rather than "screenshots resolution"
  • classification ok, only a few misclassified points
  • demcompare masked non-absolute mean: -7.92

Everything ok apart from some minor changes to the readme and logsheet required

comment:33 in reply to: ↑ 32 Changed 8 years ago by asm

Replying to lah:

Lidar DC

  • Weird flight pattern, but overlaps look ok
  • changed dem name in utm hdr
  • Logsheet text goes off page - really ought to be redone. (Was delivered with Fenix, so DCs need to be better as well!)
  • FW look fine in waveviewer
  • Readme text has some errors: should be "flightlines have", not "flightlines has", "same value as", not "same value than" and "screenshot reolution" rather than "screenshots resolution"
  • classification ok, only a few misclassified points
  • demcompare masked non-absolute mean: -7.92

Everything ok apart from some minor changes to the readme and logsheet required

Quick question, why it has to be "screenshot resolution" instead of "screenshots", some of them are good on details, others are not.

comment:34 Changed 8 years ago by lah

"screenshots resolution" just sounds really wrong. "Screenshot resolution" is a thing, so can be used as a general term to apply to all the screenshots. You could also say:

  • Resolution of (some of) the screenshots - seems most accurate to what you mean
  • screenshots' resolution - resolution belonging to the screenshots

comment:35 Changed 8 years ago by asm

Lidar Processing

All changes made.

comment:36 Changed 8 years ago by dac

Archiving

LiDAR data delivered to PI on hard drive (11/04/2016) - started uploading to NEODC.

comment:37 Changed 8 years ago by dac

Archiving

All processed data uploaded to NEODC.

comment:39 Changed 8 years ago by dac

Archiving

Started uploading raw data to NEODC (running on gridmaster4).

Last edited 8 years ago by dac (previous) (diff)

comment:40 Changed 7 years ago by lah

Owl Processing

  • Manually added GPS stop times to files still missing them (lines 13-16).
  • Manually changed all GPS times to an hour earlier to stop generate_apl_config complaining that the navigation data does not cover the header file. This is much closer to the Fenix times so should help with the scts.
  • Reprocessed level 1b with updated scripts (correct mask boundaries and dropped frame positions).
  • generate_apl_config.py updated to read from modified headers in processing/owl/flightlines/stitched (agreed to keep modified raw data separate but this is first data processed this way)

comment:41 Changed 7 years ago by asm

Owl Processing

Found a problem trying to split level1b files. lah will have a look at it.

comment:42 Changed 7 years ago by asm

Owl Processing

Level 1b files were split. Found sct values

Flightline OWL
1 -780.58
2 -97.49
3 -123.44
4 -295.06
5 -206.58
6 -117.79
7 -1697.29
8 -318.67
9 -770.34
10 -89.39
11 -101.30
12 -301.41
13 -209.41
14 -114.67
15 -146.59
16 -1774.29

Please note that the synchronisation system failed and those values were found in accordance with the creation time of the file. The SCT values will be different if using starting time and ending time of the Fenix files.
Will create readme and delivery.

Last edited 7 years ago by asm (previous) (diff)

comment:43 Changed 7 years ago by asm

Owl Processing

Synchronisation system failed. Have been decided to be consistent to use as starting time and ending time the times taken from Fenix files. With that, the new SCTs are:

Flightline OWL
1 -3.58
2 -4.43
3 10.56
4 20.94
5 0.41
6 27.21
7 -3.29
8 -2.68
9 4.66
10 -4.39
11 6.60
12 -3.41
13 8.59
14 30.33
15 -3.62
16 -2.29

Will create readme and delivery.

Last edited 7 years ago by asm (previous) (diff)

comment:44 Changed 7 years ago by asm

Owl Processing

Delivery and readme created. Ready for delivery check.

comment:45 Changed 7 years ago by lah

OWL DC

  • logsheet doesn't have Owl FPS & IT. Also don't need Eagle & Hawk pages.
  • There are a mix of renamed and not renamed files in the level 1b directory. Looks like they are duplicates that can be removed.
  • There shouldn't be any masked files.
  • Readme title and footer need to be changed to Thermal Hyperspectral. Could add files to delivery contents table.
  • sensor FOV vectos text file is missing. Can just copy this from 2015_174. They won't be different as there is no binning option for the owl.
  • Everything else looks ok, but still checking mapped files.

comment:46 Changed 7 years ago by asm

Owl

  • All changes suggested have been addressed.

comment:47 Changed 7 years ago by lah

OWL DC

Fixed logsheet and finished checking mapped files. Starting zipping.

comment:48 Changed 7 years ago by lah

OWL

Finished zipping, ready to deliver.

comment:49 Changed 7 years ago by asm

OWL Delivered

By postal mail including a hard drive. Dispatched on 2nd December 2016 to PI. A notification email has been sent as well.

comment:50 Changed 7 years ago by lah

Sent hard disk to Philippe Crebassol with Owl data (and 174) on 24/01/17.

comment:51 Changed 7 years ago by asm

Archiving

Started archiving Owl data not archived before. Plymouth 12th May 2017.

comment:52 Changed 6 years ago by dac

  • Resolution set to fixed
  • Status changed from new to closed

Archiving

Owl data available from CEDA. Closing ticket.

Note: See TracTickets for help on using tickets.