Opened 7 years ago
Closed 7 years ago
#618 closed flight processing (fixed)
ES17/136, flight day 167a/2017, Albacete
Reported by: | asm | Owned by: | |
---|---|---|---|
Priority: | immediate | Milestone: | The Glorious Future |
Component: | Processing: general | Keywords: | |
Cc: | Other processors: |
Description
Data location: ~arsf/arsf_data/2017/flight_data/spain/ES17_136-2017_167a_Albacete
Data arrived from NERC-ARF via hard drive 2017-06-30
Scientific objective: Develop LLT/SSE using AisaOwl data
Priority: Urgent
PI: Martin Wooster
Sensors:
- Fenix (flown, requested)
- Leica LIDAR (flown, not requested)
- OWL (flown, requested)
- PhaseOne (flown, requested)
Change History (27)
comment:1 Changed 7 years ago by asm
comment:2 Changed 7 years ago by dac
dGPS data collected at the same time as flight provided by PI and copied to: /users/rsg/arsf/arsf_data/2017/misc/basestation_data/ES17_136_Basestation_Data
comment:3 Changed 7 years ago by lah
Navigation Processing - Basestation Precise Position
Lat: 39 03 37.37069
Lon: -2 05 53.41131
Height: 748.623
Note a basestation height of 1.6m has been used as an estimate as there was none in the file. Redo if there is a better estimate from Gary.
comment:4 Changed 7 years ago by lah
Navigation Processing
Cannot achieve a position separation less than 0.05 for the entire line. Configuration options have little affect and adding an additional basestation only makes things worse. Most of the line is within 0.1 so could continue with this.
comment:5 Changed 7 years ago by lah
Fenix processing
Found scts and proceeding to map final files.
Flightline | Fenix SCT |
1 | 1 |
2 | 2 |
3 | 2 |
4 | 1 |
5 | 2 |
6 | 2 |
7 | 1 |
8 | 2 |
9 | 2 |
10 | 2 |
11 | 1 |
12 | 2 |
13 | 2 |
14 | 1 |
15 | 2 |
16 | 2 |
17 | 1 |
18 | 2 |
19 | 2 |
20 | 2 |
21 | 1 |
22 | 2 |
23 | 2 |
24 | 1 |
25 | 2 |
26 | 2 |
27 | 2 |
28 | 1 |
29 | 2 |
comment:6 Changed 7 years ago by lah
Fenix Processing
Files have been mapped. Waiting on data quality report to continue.
comment:7 Changed 7 years ago by dac
Digital camera
Raw images have been converted to tif.
comment:8 Changed 7 years ago by lah
Owl Processing
Founds scts:
Flightline | Owl SCT |
1 | 2 |
2 | 2 |
3 | 2 |
4 | 2 |
5 | 2 |
6 | 2 |
7 | 2 |
8 | 3 |
9 | 2 |
10 | 2 |
11 | 2 |
12 | 2 |
13 | 2 |
14 | 2 |
15 | 2 |
16 | 2 |
17 | 2 |
18 | 2 |
19 | 2 |
20 | 2 |
21 | 2 |
22 | 2 |
23 | 2 |
24 | 2 |
25 | 3 |
26 | 2 |
27 | 2 |
28 | 2 |
29 | 2 |
mapped final files, but new problem creating xml files "aplXML has failed to complete because: 'NoneType' object has no attribute 'strftime'". Think this is because the new script (https://gitlab.rsg.pml.ac.uk/arsf/internal-code/merge_requests/182) does not add the creation time to the level 1b header.
comment:9 Changed 7 years ago by lah
Owl extra processing
Have also ran the owl temperature script on all level 1 files for 167a, 167b and 167c for interest. Given that the Owl sct problem appears to be fixed, it may be worth doing something extra with the time (though they appear to overestimating quite a bit). Copied to processing/owl/flightlines/temperature
comment:10 Changed 7 years ago by lah
Owl Processing
Created delivery, ready for checking. Note that the logsheet has ARF as the base, which probably should be changed.
comment:11 Changed 7 years ago by dac
Basestation
NERC GEF retrieved basestation height using Leica GeoOffice. Total height 1.328 m. Will reprocess navigation data with correct height.
comment:12 Changed 7 years ago by dac
Reprocessed basestation data with correct height. Started Fenix and Owl reprocessing.
comment:13 Changed 7 years ago by dac
Digital Camera
Tagged images and generated thumbnails
comment:14 Changed 7 years ago by lah
Fenix Processing
Reprocessed with bad pixel file.
comment:15 Changed 7 years ago by lah
Owl reprocessing
Looks like Owl was sent to reprocess without the level 1b files, so files were not mapped. Have sent level 1b files to reprocess as it's easier than renaming the delivery files.
comment:16 Changed 7 years ago by lah
Fenix ready for Delivery Check
Created Fenix delivery.
comment:17 Changed 7 years ago by dac
Hyperspectral Delivery Check
- Readme still contains image of Dornier - have removed.
- Logfile had wrong directory for project (ES17 136-2017 167a-LST from AisaOwl) - fixed
- Logsheet listed LiDAR data although instrument failed - removed
- check_apl_cmd ran through OK.
- FastQC didn't recognise sensor due to different ID.
- Checked spectral using Py6S - all OK.
Marking as ready to deliver. Started mapping zipped files.
comment:18 Changed 7 years ago by lah
Owl ready for Delivery Check
Created new Owl delivery, ready for checking.
comment:19 Changed 7 years ago by dac
Owl delivery check
- Updated operator in XML from 'Johnson' to 'James Johnson'
- check_apl_cmd ran through without any problems
Ready to deliver. Started zipping mapped files.
comment:20 Changed 7 years ago by dac
Data delivery
Fenix and Owl data placed on FTP server (slot 4). Notification sent to PI.
comment:21 Changed 7 years ago by asm
Camera Delivery
Created delivery and readme with a note for blue tinge. Project is ready for DC.
comment:22 Changed 7 years ago by dac
Digital Camera Delivery Check
Removed LiDAR from KML file. Everything else looks good - ready to deliver.
comment:23 Changed 7 years ago by asm
Camera Delivery
Placed on FTP server (slot 4). Notification sent to PI on 18/10/2017.
comment:24 Changed 7 years ago by asm
Archiving
Tidying up before archive. Will delete Owl temperature directory as the owl sct problem was fixed and files can be recreated.
comment:25 Changed 7 years ago by asm
Archiving
Started uploading data to CEDA.
comment:26 Changed 7 years ago by asm
Archiving
Paused the upload of data to CEDA.
comment:27 Changed 7 years ago by dac
- Resolution set to fixed
- Status changed from new to closed
Archiving
Flight archived at CEDA: http://data.ceda.ac.uk/neodc/arsf/2017/ES17_136/ES17_136-2017_167a_Albacete/
Unpacking
Project has been unpacked. There are 29 fenix flightlines and 29 owl lines. Missing 2 fenix log files for flightlines 18 and 12. Quite a few owl directories did not have the same name than files inside so had to rename them manually (wrote changes into the log file -> unpacking/unpacking_2017167a.log)
Checks have been run:
-> 23 dropped frames in Fenix 11
-> owl timestamps are less than usual
-> quite a few owl files have between 1 and 3 dropped frames incidents
-> 2 lidar files are empty: RawLaser/170616_115528/LDR170616_115528_0000000008.scn and RawLaser/170616_094238/LDR170616_094238_0000000042.scn