Opened 2 years ago

Closed 2 years ago

#618 closed flight processing (fixed)

ES17/136, flight day 167a/2017, Albacete

Reported by: asm Owned by:
Priority: immediate Milestone: The Glorious Future
Component: Processing: general Keywords:
Cc: Other processors:

Description

Data location: ~arsf/arsf_data/2017/flight_data/spain/ES17_136-2017_167a_Albacete

Data arrived from NERC-ARF via hard drive 2017-06-30

Scientific objective: Develop LLT/SSE using AisaOwl data
Priority: Urgent

PI: Martin Wooster

Sensors:

  • Fenix (flown, requested)
  • Leica LIDAR (flown, not requested)
  • OWL (flown, requested)
  • PhaseOne (flown, requested)

Change History (27)

comment:1 Changed 2 years ago by asm

Unpacking

Project has been unpacked. There are 29 fenix flightlines and 29 owl lines. Missing 2 fenix log files for flightlines 18 and 12. Quite a few owl directories did not have the same name than files inside so had to rename them manually (wrote changes into the log file -> unpacking/unpacking_2017167a.log)

Checks have been run:
-> 2 incidents = 23 dropped frames in Fenix 11
-> owl timestamps are less than usual
-> quite a few owl files have between 1 and 3 dropped frames incidents
-> 2 lidar files are empty: RawLaser/170616_115528/LDR170616_115528_0000000008.scn and RawLaser/170616_094238/LDR170616_094238_0000000042.scn

Last edited 2 years ago by asm (previous) (diff)

comment:2 Changed 2 years ago by dac

dGPS data collected at the same time as flight provided by PI and copied to: /users/rsg/arsf/arsf_data/2017/misc/basestation_data/ES17_136_Basestation_Data

comment:3 Changed 2 years ago by lah

Navigation Processing - Basestation Precise Position

Lat: 39 03 37.37069
Lon: -2 05 53.41131
Height: 748.623

Note a basestation height of 1.6m has been used as an estimate as there was none in the file. Redo if there is a better estimate from Gary.

Last edited 2 years ago by lah (previous) (diff)

comment:4 Changed 2 years ago by lah

Navigation Processing

Cannot achieve a position separation less than 0.05 for the entire line. Configuration options have little affect and adding an additional basestation only makes things worse. Most of the line is within 0.1 so could continue with this.

comment:5 Changed 2 years ago by lah

Fenix processing

Found scts and proceeding to map final files.

Flightline Fenix SCT
1 1
2 2
3 2
4 1
5 2
6 2
7 1
8 2
9 2
10 2
11 1
12 2
13 2
14 1
15 2
16 2
17 1
18 2
19 2
20 2
21 1
22 2
23 2
24 1
25 2
26 2
27 2
28 1
29 2

comment:6 Changed 2 years ago by lah

Fenix Processing

Files have been mapped. Waiting on data quality report to continue.

comment:7 Changed 2 years ago by dac

Digital camera

Raw images have been converted to tif.

comment:8 Changed 2 years ago by lah

Owl Processing

Founds scts:

Flightline Owl SCT
1 2
2 2
3 2
4 2
5 2
6 2
7 2
8 3
9 2
10 2
11 2
12 2
13 2
14 2
15 2
16 2
17 2
18 2
19 2
20 2
21 2
22 2
23 2
24 2
25 3
26 2
27 2
28 2
29 2

mapped final files, but new problem creating xml files "aplXML has failed to complete because: 'NoneType' object has no attribute 'strftime'". Think this is because the new script (https://gitlab.rsg.pml.ac.uk/arsf/internal-code/merge_requests/182) does not add the creation time to the level 1b header.

comment:9 Changed 2 years ago by lah

Owl extra processing

Have also ran the owl temperature script on all level 1 files for 167a, 167b and 167c for interest. Given that the Owl sct problem appears to be fixed, it may be worth doing something extra with the time (though they appear to overestimating quite a bit). Copied to processing/owl/flightlines/temperature

comment:10 Changed 2 years ago by lah

Owl Processing

Created delivery, ready for checking. Note that the logsheet has ARF as the base, which probably should be changed.

comment:11 Changed 2 years ago by dac

Basestation

NERC GEF retrieved basestation height using Leica GeoOffice. Total height 1.328 m. Will reprocess navigation data with correct height.

comment:12 Changed 2 years ago by dac

Reprocessed basestation data with correct height. Started Fenix and Owl reprocessing.

comment:13 Changed 2 years ago by dac

Digital Camera

Tagged images and generated thumbnails

comment:14 Changed 2 years ago by lah

Fenix Processing

Reprocessed with bad pixel file.

comment:15 Changed 2 years ago by lah

Owl reprocessing

Looks like Owl was sent to reprocess without the level 1b files, so files were not mapped. Have sent level 1b files to reprocess as it's easier than renaming the delivery files.

comment:16 Changed 2 years ago by lah

Fenix ready for Delivery Check

Created Fenix delivery.

comment:17 Changed 2 years ago by dac

Hyperspectral Delivery Check

  • Readme still contains image of Dornier - have removed.
  • Logfile had wrong directory for project (ES17 136-2017 167a-LST from AisaOwl) - fixed
  • Logsheet listed LiDAR data although instrument failed - removed
  • check_apl_cmd ran through OK.
  • FastQC didn't recognise sensor due to different ID.
  • Checked spectral using Py6S - all OK.

Marking as ready to deliver. Started mapping zipped files.

comment:18 Changed 2 years ago by lah

Owl ready for Delivery Check

Created new Owl delivery, ready for checking.

comment:19 Changed 2 years ago by dac

Owl delivery check

  • Updated operator in XML from 'Johnson' to 'James Johnson'
  • check_apl_cmd ran through without any problems

Ready to deliver. Started zipping mapped files.

comment:20 Changed 2 years ago by dac

Data delivery

Fenix and Owl data placed on FTP server (slot 4). Notification sent to PI.

comment:21 Changed 2 years ago by asm

Camera Delivery

Created delivery and readme with a note for blue tinge. Project is ready for DC.

comment:22 Changed 2 years ago by dac

Digital Camera Delivery Check

Removed LiDAR from KML file. Everything else looks good - ready to deliver.

comment:23 Changed 2 years ago by asm

Camera Delivery

Placed on FTP server (slot 4). Notification sent to PI on 18/10/2017.

comment:24 Changed 2 years ago by asm

Archiving

Tidying up before archive. Will delete Owl temperature directory as the owl sct problem was fixed and files can be recreated.

comment:25 Changed 2 years ago by asm

Archiving

Started uploading data to CEDA.

comment:26 Changed 2 years ago by asm

Archiving

Paused the upload of data to CEDA.

comment:27 Changed 2 years ago by dac

  • Resolution set to fixed
  • Status changed from new to closed
Note: See TracTickets for help on using tickets.