EU15/18, flight day 174, Toulouse

[lah]

Data location: /users/rsg/arsf/arsf_data/2015/flight_data/france/EUFAR15_18-2015_174_Toulouse

Data arrived from ARSF via SATA disk on 09/07/2015.

Scientific objective: Hyperspectral measurements over agro-forestry areas for landscape assessment of agricultural health, ecophysiology, and satellite product validation.

Priority: Unknown

PI: Jean-Louis Roujean
EUFAR Project ID: AHSPECT

Sensors:

• Fenix (requested)
• Leica LIDAR (not requested)
• Leica FW LIDAR (not requested but flown anyway)
• Owl (not requested but flown anyway)

[gej]

Navigation Processing complete

Basestation:
Lat: 43 38 13.01189
Long: 1 21 58.49814
Ell. Height: 199.663

Position Seperation: Slight spike to -0.06 towards end of flight line, rest below -0.04
Position Accuracy: Lat and long all within +-0.035 Height within +-0.05

[lah]

Unpacking
Removed corrupt webcam images:

• 150623_095016_000.jpg
• 150623_123907_000.jpg
• 150623_124117_000.jpg

[asm]

Lidar Processing
Started.

[gej]

Hyperspectral Processing
Started.

[dac]

Sent notification email to PI with links to KML

[asm]

Lidar Processing

Found pitch and roll consistent with all lines

Roll	-0.0010109
Pitch	-0.0022489

[asm]

Lidar Processing

Classification started.

[asm]

Lidar Processing

Classification for discrete lidar finished.

[asm]

Lidar Processing

Classification for lidar full wave form started.

[asm]

Lidar Processing
Two flightlines are bigger than 50gbs. Process is killed due to that, stgo is splitting the lines on the spare machine.

[gej]

Hyperspectral Processing
found SCT values

Long flightlines, some had a varying SCT value, have currently gone for best throughout line, but if it looks like they may be have too much variance in the line will change

Flightline	FENIX
1	23.45
2	3.2
3	2.90
4	2.25
5	3.675
6	2.75
7	2.5
8	1.45

[asm]

Lidar Processing
Classification for full wave form finished. Elevation difference measured, delivery created including Readme file. Ready for Delivery Check.

[gej]

Lidar Delivery Check

Read Me
Need to add N/S to UTM projection,

Data qaulity remarks needs reworkding. instead of:
"This might have as a result that the elevation difference on overlapping areas could be not precise",
try:
"This may have resulted in the elevation difference on overlapping areas being less precise"

Navigation
There are only 6 trj files, but 8 flightlines. Not sure if this is due to the flightlines being split or not, if it is should maybe make a comment about it somewhere.

Screenshots
Don't think anything can be done, but maybe make a comment in readme that the screenshots aren't of the best quality due to the length of the flightlines.

[asm]

Lidar Processing

All comments has been changed to trj files and Readme.

Please, ready for re-check.

[gej]

Lidar Delivery Check

All looks good now that changes have been made.
All trj files are now there and comments made in readme on screenshots, and previous comment reworded.

Will be delivered with Fenix Hyperspectral data

[gej]

Lidar delivered to Jean-Louis Roujean via FTP on 12 October 2015

[gej]

Fenix Processing
Started creating mapped using files with new calibration

[dac]

Fenix

Hyperspectral lines are very large producing mapped files 0.5 - 1.5 TB. Emailed PI to check preferred method for splitting data.

[dac]

Fenix

PI happy to receive data split by scanline.

[dac]

Fenix

Started testing splitting by scanline by adding cal_args_extra = -lines 0 100 to config file.

[gej]

Fenix Processing

Lines have now been split, using cal_args_extra. Split line functionality has been added to generate_apl_config.py for future projects.

There was a problem with this flight where the SCT values varied through the flight by about -0.4 for every 10000 scan lines.
This has been fixed by increasing the fps by 0.08 so I am now finding the new scts

[gej]

Fenix Processing

Found new SCT values, SCT values are consistent through sections so have only put full line values.

Flightline	FENIX
1	27.5
2	4.3
3	3.34
4	2.58
5	3.92
6	3.11
7	4.28
8	1.85

Number of sections per line:

Flightline	Number Sections
1	11
2	3
3	4
4	9
5	1
6	1
7	5
8	1
total	35

Files now being mapped with all bands.

[gej]

Fenix Processing

Have split line 6 as final filesize was at 200GB.
Has been split into 3 sections. SCT hasn't changed.

[gej]

Fenix processing

Mapped files processed, creating delivery.

[stgo]

Fenix processing
Have taken over from gej for this week, some files have been included in the delivery creation which resulted in bad naming of all files. Irritating but fixable, if I can't identify the correct ones (should be easy, they will be the largest files) I will reprocess for simplicity's sake.

[stgo]

Fenix processing

Have resubmitted to the grid, will recreate delivery once complete.

[stgo]

All lines still had their dark frames included, have updated end lines to remove these. Also fixed the sct value for line 1 and will be recreating delivery this afternoon.

[stgo]

Fenix processing

Created readme and placed in the delivery folder. A pre release version of the data has already been provided to the PI via ftp however the size of the delivery means it is not very feasible for them to download it on their own connection. Mapped files are already zipped.

Needs final DC before delivery.

[dac]

Fenix Delivery Check

• Created ASCII view vectors and converted to Windows line endings.
• Added header to the table at the top of page 3
• Compared spectra to Py6S simulations - look good

Ready to deliver.

Need to remove old deliveries from processing/deliveries.

[dac]

Archiving

Have started uploading delivered LiDAR data to NEODC - large delivery so will take a while.

[stgo]

Fenix delivery

Fenix data sent on hard drive 10/12/2015

[dac]

Archiving

Started archiving

[dac]

Archiving

Finished upload to NEODC - Notified Wendy.

[dac]

Delivery

Notification from PI: Received hard drive with Fenix data (22/12/2015)

[dac]

LiDAR'

Started reprocessing with full range of scan angles.

[lah]

OWL

found sct for sample flight line 2 = 3.93

Created delivery, ready for checking.

[asm]

Owl Delivery Check

-check_apl_cmd only created the masked files but I tested the instructions written on readme and everthing looks good.
-proj_tidy do not recognise the owl delivery folder and list everything as unrecognised files. Looks fine however.
-There is only one flightline so both screenshots were the same. Removed mosaic.
-Tried to run py6S with the igm created from apl commands for a given pixel x,y (did not work for ndvi pixel finder). Did not work, error: Unexpected radiance units. May be worth implementing this on the future.
-Spectra looks OK for mapped file on Tuiview.
-As dac pointed out on other project: XML metadata files list Carl as the pilot and Tom as the instrument operator (different to logsheet). This is the same as the Fenix data so not worth blocking the project for but will need fixing in templates.
No major problems found. Owl should be ready to deliver.

[dac]

Archiving

Processed data available from NEODC (​http://browse.ceda.ac.uk/browse/neodc/arsf/2015/EUFAR15_18/EUFAR15_18-2015_174_Toulouse)

[dac]

Archiving

Raw data also available from NEODC (​http://browse.ceda.ac.uk/browse/neodc/arsf/raw_data/2015/EUFAR15_18-2015_174_Toulouse_raw).

Leaving ticket open for Owl processing (outside EUFAR)

[lah]

Owl

Uploaded sample delivery to NEODC.

[lah]

Owl Processing

Found scts with a combination of wobble minimisation and alignment to fenix. There isn't much overlap between flight lines in this flight. The same sct is good for each split of an entire line, though there is some increased misalignment with the fenix, which does appear to wobble in these regions.

Flightline	OWL
1	2.67
2	3.89
3	2.44
4	3.14
5	2.60
6	2.96
7	2.51

Now mapping full band files.

[lah]

Owl Processing

Created delivery. Scripts deleted all level 1b split lines so recreating. Some of these appear to have been renamed as the full files but the number of lines between these and the mask files are inconsistent so reprocessing level 1 files.

[lah]

Owl Delivery

Created readme from hyp_config. Manually edited owl sections and removed under/overflow table as there were none. Added in missing lines (files) to delivery content table.

Manually added Owl details to logsheet (FPS & IT).

[lah]

Owl Delivery

Copied in new level 1b files and renamed. Ready to delivery check.

[asm]

Owl Delivery Check

Started.
*No xml files for flightline 4.
*Screenshots are currently being generated and will recheck later.
*Project tidy does not like owl names but everything seems fine.
*Everything else looks fine, including checks for apl comands

Once xml for flightline 4 has been created and the screenshorts sorted out, should be ready to deliver.

[lah]

Owl

Recreated xml files for line 4. These were missing because line 4 had no black body files so used line 5 calibration file. Manually added in software info.

New screenshots have finished. Have renamed these and removed old ones.

[asm]

Owl Delivery Check

Everything looks fine now. Have removed the tmp_cmd_check and will zip mapped files. Once done, project will be ready to be delivered.

[asm]

Owl Delivery Check

Mapped files have been zipped. Checked logfiles and everything looks correct. Will mark as Ready to be delivered.

[asm]

OWL Delivered

By postal mail including a hard drive. Dispatched on 2nd December 2016 to PI. A notification email has been sent as well.

[lah]

Sent hard disk to Philippe Crebassol with Owl data (and 274) on 24/01/17.

[dac]

Archiving

Owl data available from CEDA. Closing ticket.