Opened 22 months ago

Last modified 14 months ago

#623 new flight processing

ES17/126, flight day 172/2017, Alto Tajo

Reported by: asm Owned by:
Priority: immediate Milestone:
Component: Processing: general Keywords:
Cc: Other processors:

Description

Data location: ~arsf/arsf_data/2017/flight_data/spain/ES17_126-2017_172_Alto_Tajo

Data arrived from NERC-ARF via hard drive 2017-06-30

Scientific objective: Assess changes in "Alto Tajo" Natural Park
Priority: Urgent

PI: David Coomes

Sensors:

  • Fenix (flown, requested)
  • Leica LIDAR (flown, requested, failed during acquisition)
  • OWL (flown, requested)
  • PhaseOne (flown, not requested)

Change History (31)

comment:1 Changed 22 months ago by asm

Unpacking

Project has been unpacked. There was no basestation data. LiDAR failed and data is not usable for this flight season. There are 22 fenix flightlines and 22 owl lines.

Have run unpacked checks:
-Fenix nav files 1, 2, 3, 4, 5, 14, 20, 21 and 22 timestamps stopped being recorded before the end of the files. (Also last timestamps seems incorrect)
-2 incidents with a total of 38 dropped frames for fenix line 14.
-Owl nav files present the same problems
-Dropped frames for owl files: 1, 2, 3, 4, 5, 13, 14, 18, 21. Dropped frames range between 1 and 11.

comment:2 Changed 21 months ago by dac

GPS basestation data provided by PI and copied to: /users/rsg/arsf/arsf_data/2017/misc/basestation_data/ES17_126_Basestation_Data

comment:3 Changed 21 months ago by dac

Digital camera processing

Images converted from raw to tiff.

comment:4 Changed 19 months ago by asm

Navigation Processing

Started. Converted files using teqc and created as well navigation files
teqc +nav MIRADOR_2_9500_0621_081454.17n 'MIRADOR 2_9500_0621_081454.m00' > 'MIRADOR 2_9500_0621_081454.17o'

Basestation verification done
Latitude : 40 47 11.51415
Longitude -2 09 30.82658
Ell. Height: 1196.367 m
Ant. Height: 1.408

comment:5 Changed 19 months ago by asm

Navigation Processing

Done. There were errors in the data (Doppler LSQ solution had high RMS), could not compute so had to recalculate Doppler for Static Data using Ephemeris. Used satellite elevation mask 13.5. Could not get ambiguity fix 2 (got 1), separation is bigger than usually (lat, lon and height separation around 0.5) but the accuracy is enough as not having Lidar to process.

comment:6 Changed 19 months ago by asm

Hyperspectral Processing

Found SCT values:

Flightline FENIX
1 2
2 2
3 2
4 2
5 1
6 2
7 2
8 2
9 1
10 2
11 2
12 2
13 1
14 2
15 2
16 1
17 1
18 1
19 2
20 2
21 2
22 1

comment:7 Changed 19 months ago by asm

Hyperspectral Processing

Finished. Created delivery and readme. Manually edit the readme to account for flight lines that have been split. Manually edited the Lidar log file to include a missing flightline (and saved the original as backup). Ready for delivery check.

comment:8 Changed 19 months ago by dac

Hyperspectral delivery check

Starting delivery check.

comment:9 Changed 19 months ago by dac

  • Removed fodis directory
  • Changed logsheets names from Impacts_of_woodland_densification to Alto_Tajo for consistency with project folder name. Also removed 'wrong_logsheet' directory in project admin folder.
  • Removed ARF as aircraft base and left blank (don't know where they flew from), also changed GPS Basestation to NERC GEF as it wasn't the ARF one for this flight.
  • Removed image of plane from logsheet
  • Checked spectra against Py6S modelled spectral and look fine.
  • No project or flight information XML files found. Needs fixing before delivery
  • Wasn't able to check APL commands due to lack of XML files.

Will finish checking once XML metadata files have been generated.

Version 0, edited 19 months ago by dac (next)

comment:10 Changed 19 months ago by asm

Hyperspectral Fenix

Have recreated xml files and named them accordingly. Ready to check.

comment:11 Changed 19 months ago by dac

Ran through APL commands and checkout outputs.

SCT values look off for the following files: 6, 7, 10, 15, 21 part 1

comment:12 Changed 19 months ago by asm

Thermal Processing

Started. Have created level 1 files. Flightline 22 did not have calibration files so used files from flightline 21.

comment:13 Changed 19 months ago by asm

Hyperspectral Fenix

Errors on the SCT values were caused because the navigation was not being used for creating the final mapped files. Previous SCT values were correct, have created a new delivery correcting the error (this time using the navigation). Ready for delivery check.

comment:14 Changed 19 months ago by asm

Thermal Processing

Found the SCT for the Owl:

Flightline OWL
1 2
2 3
3 2
4 2
5 2
6 2
7 2
8 2
9 2
10 3
11 2
12 2
13 2
14 2
15 2
16 2
17 2
18 2
19 3
20 2
21 2
22 2

Apl config file is suggesting to split files into many parts. Will have a look to fix the script before continue.

comment:15 Changed 19 months ago by dac

Hyperspectral delivery check

  • Removed *.aux.xml and *.ovo files from mapped directory (generated with gdalcalc)
  • Ran apl command checker again - SCT values look fine

Marking as ready to deliver, started zipping mapped files.

Last edited 19 months ago by dac (previous) (diff)

comment:16 Changed 19 months ago by asm

Hyperspectral delivery

Fenix delivered via FTP slot 9 on 21/09/2017

comment:17 Changed 19 months ago by asm

Thermal Processing

Fixed problems when splitting files. Large files have been split into 3 parts (same than Fenix to have the same coverage) instead of the suggested 7-8 parts. Created mapped files with all bands and will create delivery.

comment:18 Changed 19 months ago by asm

Thermal Processing

When creating the delivery, there were also problems for level1b files that were split and those were overwritten. Recreated them, solved issues with scripts and sorted the errors. Created the readme file. Project is finally ready to be checked.

comment:19 Changed 19 months ago by dac

Thermal Delivery Check

All looks fine apart from check_apl_cmd failed for line 21 (all parts) due to different number of lines in level1 and IGM file - not sure why this is the case.

Once APL commands run through for all lines ready to deliver

comment:20 Changed 19 months ago by asm

Thermal Processing

Have recreated the split level1b files. Now the apl commands run OK as there are the same number of lines in level1b and navigation. Will zip the mapped files and once done, mark the project as ready to be delivered.

comment:21 Changed 19 months ago by asm

Thermal Processing

Mapped files zipped successfully. Project is ready to be delivered.

comment:22 Changed 19 months ago by asm

Thermal delivery

Owl delivered via FTP slot 9 on 26/09/2017

comment:23 Changed 18 months ago by dac

Digital Camera Processing

Started tagging images.

comment:24 Changed 18 months ago by dac

Digital Camera Processing

Created delivery - ready for checking.

comment:25 Changed 18 months ago by asm

Digital camera Delivery check

Started.

comment:26 Changed 18 months ago by asm

Digital Camera Delivery Check

All the checks made succesfully. There is bit of saturation but useful data for quite a few photographs so it is good that they have not been removed (for example: 00827-00833, 00890 – 00897, 01034, 01057-10059, 01200-01212, 01224, 1368)

As noted on readme, photograph number 1 has not been tagged but it is not present in kml and eventfile.
Will mark the project as Project is ready to be delivered.

comment:27 Changed 18 months ago by asm

Camera Delivery

PhaseOne delivered via FTP slot 9 on 18/10/2017

comment:28 Changed 17 months ago by asm

Archiving

Tidied up for archiving.

comment:29 Changed 17 months ago by dac

Archiving

Data available to download from CEDA: http://data.ceda.ac.uk/neodc/arsf/2017/ES17_126/ES17_126-2017_172_Alto_Tajo/

Leaving ticket open as still need to generate orthomosaic from digital camera data.

Last edited 17 months ago by dac (previous) (diff)

comment:30 Changed 14 months ago by wja

Photogrammetry Processing

Additional processing using Agisoft Photoscan has been carried out to generate DEMs, orthomosaics and point clouds for four sites. This deliver is found under S17_126-172-camera-20180222.
Numerous issues with the data was encounter during processing (mainly, lack of photograph overlap, regions of dense vegetation and lack of ground control points) causing the products to be limited to the four sites.
In order to increase the quality of the products, some photographs from 2017 flights 170 and 171 have been used (these are listed in the Read_Me.
Read_Me describes the data and associated issues to consider, software specific processing parameters are documented in supplied reports in docs/.

comment:31 Changed 14 months ago by wja

Photogrammetry Delivery

Delivery placed in FTP slot9 under 'orthomosaics' directory, notification sent to PI on 28/02/2018.

Note: See TracTickets for help on using tickets.