Opened 10 years ago
Closed 8 years ago
#548 closed flight processing (fixed)
RG13/06, flight day 291/2014, Danum Valley
Reported by: | dap | Owned by: | |
---|---|---|---|
Priority: | alpha 5 | Milestone: | |
Component: | Processing: general | Keywords: | |
Cc: | Other processors: |
Description (last modified by tec)
Data location: /users/rsg/arsf/arsf_data/2014/flight_data/malaysia/RG13_06-2014_291_Danum_Valley
Data arrived by network transfer on 17/12/2014.
Scientific objective: Investigating the impact of humans on tropical forests.
Priority: alpha 5
PI: David Coomes
Full Waveform data present
No Owl data present
proj_tidy indicates that there are differences between some of the fps_set and fps_qpf values in the following Fenix header files:
- FENIX290-14-10.hdr (difference of 11.451)
- FENIX290-14-11.hdr (difference of 0.332)
- FENIX290-14-13.hdr (difference of 0.311)
- FENIX290-14-21.hdr (difference of 0.528)
Sensors:
- Fenix (requested)
- Leica FW LIDAR (delivered 27/04/2015)
- Leica LIDAR (requested, delivered 27/04/2015)
- RCD (requested, delivered 23/03/2015)
Duplicate lines
Process only one for Fenix and RCD - check for which has least cloud cover, failing that choose closest to local noon
- 1244-1249
- 1120
- 1131-1138 (day 293)
Change History (47)
comment:1 Changed 10 years ago by dap
- Description modified (diff)
comment:2 Changed 10 years ago by tec
comment:3 Changed 10 years ago by tec
Navigation Processing
Started navigation processing
SAFE_6th_ Basestation Data | |
---|---|
Latitude | 4 43 30.56081 |
Longitude | 117 36 19.18785 |
El Height | 479.827m |
comment:4 Changed 10 years ago by tec
Navigation Processing
Some lines have a second of two of quality 3 gps data but other than that the nav is sound.
comment:5 Changed 10 years ago by tec
Navigation Processing
Processed with IPAS Pro, settings below:
Elevation Mask: 14.0 degrees
KAR Maximum dual frequency distance: 80.00 km
Code: 1.20m
Nav finished
comment:6 Changed 10 years ago by tec
RCD Processing
Started RCD Processing
comment:8 Changed 10 years ago by dac
Navigation Processing
Reprocessed base station data
Latitude | 4 43 30.56073 |
Longitude | 117 36 19.18705 |
El Height | 480.162 m |
comment:9 Changed 10 years ago by dap
RCD Processing
Moving the following images to to_remove, as they were affected by significant cloud coverage:
- 128
- 129
- 130
- 167
- 261
- 280
comment:10 Changed 10 years ago by dac
Navigation Processing
Reprocessed navigation data in IPAS Pro. Apart from a noisy patch between 519500 - 520500 s quality is good.
comment:11 Changed 10 years ago by dap
RCD Processing Complete
Ready for delivery check
comment:12 Changed 10 years ago by dac
RCD Delivery Check
Starting delivery check
comment:13 Changed 10 years ago by dac
RCD Delivery Check
Ready to deliver. Two minor comments:
- Needed to copy Readme into delivery directory.
- A lot of the images are cloudy (as noted in the Readme). I would say 106 and 291 fall into the 'significantly affected' by cloud coverage but they still have some usable pixels so OK to keep.
comment:14 Changed 10 years ago by tec
Hyperspectrial Processing
Have some time so thought I'd do this
Scratch this, now have no time again
comment:15 Changed 10 years ago by dap
Beginning Fenix Processing. Renamed Fenix raw files so that they are named FENIX291* instead of FENIX290*.
comment:16 Changed 10 years ago by dap
RCD Data Delivery
Rsyncing RCD delivery to ftp server
comment:17 Changed 10 years ago by dap
Fenix Processing
Processed all lines in 'fast mode' to start identifying SCT values. File number 10 is not usable as it is tiny, this line wasn't recorded in the log sheet anyway so will ignore this file.
comment:18 Changed 10 years ago by dap
Fenix Processing
apl was returning the following error when processing line 12:
No navigation file found to match times in /users/rsg/arsf/arsf_data/2014/flight_data/malaysia/RG13_06-2014_291_Danum_Valley/hyperspectral/fenix/ for FENIX291-14-12 Closest navigation file found /users/rsg/arsf/arsf_data/2014/flight_data/malaysia/RG13_06-2014_291_Danum_Valley/hyperspectral/fenix/FENIX291-14-11.nav. Does not cover last 192.0000 seconds
Solved this by adding
nav2_args_extra = -nav /users/rsg/arsf/arsf_data/2014/flight_data/malaysia/RG13_06-2014_291_Danum_Valley/hyperspectral/fenix/FENIX291-14-12.nav
to the flight line's section in the config file.
comment:19 Changed 10 years ago by dac
LiDAR Processing
Handing project over to tec. Best pitch and roll values found so far for first two lines:
Pitch: -0.00063
Roll: -0.006095
comment:20 Changed 10 years ago by dap
Fenix Processing
Have looked at the raw flight lines (lines 1-7, 1131-1138) duplicated in flight 293 in FastQC in both flights. The ones collected in flight 293 are less affected by cloud so will not process lines 1 - 7 for flight 291.
comment:21 Changed 10 years ago by tec
LiDAR Processing
Line | Roll | Pitch | Done FW | Classified |
---|---|---|---|---|
LDR141018_000915_1.LAS | -0.006315 | -0.00025 | Y | Y |
LDR141018_001308_1.LAS | -0.006315 | -0.00025 | Y | Y |
LDR141018_001706_1.LAS | -0.006185 | -0.00025 | Y | Y |
LDR141018_002109_1.LAS | -0.006185 | -0.00025 | Y | Y |
LDR141018_002455_1.LAS | -0.006185 | -0.00025 | Y | Y |
LDR141018_002841_1.LAS | -0.006185 | -0.00025 | Y | Y |
LDR141018_003305_1.LAS | -0.006185 | -0.00025 | Y | Y |
LDR141018_003659_1.LAS | -0.006185 | -0.00025 | Y | Y |
LDR141018_004332_1.LAS | -0.006185 | -0.00025 | Y | Y |
LDR141018_004611_1.LAS | -0.006185 | -0.00025 | Y | Y |
LDR141018_004920_1.LAS | -0.006185 | -0.00025 | Y | Y |
LDR141018_005158_1.LAS | -0.006185 | -0.00025 | Y | Y |
LDR141018_005447_1.LAS | -0.006185 | -0.00025 | Y | Y |
LDR141018_005735_1.LAS | -0.006185 | -0.00025 | Y | Y |
LDR141018_010239_1.LAS | -0.006185 | -0.00025 | Y | Y |
LDR141018_010520_1.LAS | -0.006185 | -0.00025 | Y | Y |
LDR141018_010809_1.LAS | -0.006185 | -0.00025 | Y | Y |
LDR141018_011038_1.LAS | -0.006185 | -0.00025 | Y | Y |
LDR141018_011322_1.LAS | -0.006185 | -0.00025 | Y | Y |
LDR141018_011630_1.LAS | -0.006185 | -0.00025 | Y | Y |
LDR141018_012256_1.LAS | -0.006185 | -0.00025 | Y | Y |
LDR141018_013134_1.LAS | -0.006185 | -0.00025 | Y | Y |
LDR141018_013726_1.LAS | -0.006185 | -0.00025 | Y | Y |
LDR141018_014315_1.LAS | -0.006185 | -0.00025 | Y | Y |
LDR141018_015135_1.LAS | -0.006185 | -0.00025 | Y | Y |
LDR141018_015428_1.LAS | -0.006185 | -0.00025 | Y | Y |
LDR141018_015711_1.LAS | -0.006185 | -0.00025 | Y | Y |
comment:22 Changed 10 years ago by tec
LiDAR Processing
Finished finding roll/pitch, was quite difficult, as there was no houses or decent structures but apart from the first two lines, the rest of the flight was rather consistent.
Processing FW now.
comment:23 Changed 10 years ago by tec
LiDAR Processing
Finished creating LiDAR files, now proceeding with classification.
comment:24 Changed 10 years ago by dap
Fenix Processing
Found all SCTs
SAFE Site Letter | Flightline | Fenix SCT |
---|---|---|
D | 1 | Use 293 duplicate |
2 | Use 293 duplicate | |
3 | Use 293 duplicate | |
4 | Use 293 duplicate | |
5 | Use 293 duplicate | |
6 | Use 293 duplicate | |
7 | Use 293 duplicate | |
8 | Use 293 duplicate | |
M | 9 | Use line 13 |
10 | No usable data contained by file | |
11 | Use line 14 | |
12 | Use line 15 | |
13 | 0.92 | |
14 | 0.94 | |
15 | 0.95 | |
L | 16 | 1.05 |
17 | 0.89 | |
18 | 1.05 | |
19 | Use line 16 | |
20 | Use line 17 | |
21 | Use line 18 | |
U | 22 | Use line 25 |
23 | 0.75 | |
24 | 1.03 | |
25 | 0.70 | |
S | 26 | -0.05 |
27 | 0.91 | |
28 | 0.99 |
comment:25 Changed 10 years ago by tec
LiDAR Processing
Finished Classification, shall move onto creating a delivery tomorrow
comment:26 Changed 10 years ago by dap
Fenix Processing
Created delivery and read me, Fenix data now ready for delivery check.
comment:27 Changed 10 years ago by tec
LiDAR Processing
Ready for delivery check.
comment:28 Changed 10 years ago by dap
LiDAR Delivery Check
Beginning delivery check.
comment:29 Changed 10 years ago by dap
LiDAR Delivery Check
- Roll/pitch offsets seem good.
Part of flight line 21 seems to have a kink in the end of the line. Dan suggested that this is to be trimmed off.I tried to process the hyperspectral flight line 21 and it didn't process (not a problem since it was a duplicate anyway). You'll also need to mention this in the data quality remarks.- Trim was done during delivery check, the new flight line looks fine.
- Classifications are good.
- Ran unix2dos on the ascii flight lines.
- Add cloud, lack of buildings and kink trimming (mentioned above) to data quality remarks.
- DEMs will need recreating due to the trimmed flight line (don't forget to also trim the resolution in the new DEM header).
- demcompare.py was not run as the DEM will need re-producing.
- Screenshots look fine, but I would suggest regenerating flight line 21's screenshot.
- laszip caused no errors on the full waveform files and they opened fine in Wave Viewer with no errors.
I'll recheck once the above issues have been seen to.
comment:30 Changed 10 years ago by dap
LiDAR Delivery Check
- Renamed new read me so that it follows proj_tidy regex.
- New data quality remarks are fine.
- Trimmed the resolution in the UTM DEM header to 2.0
Looks fine, ready for delivery.
comment:31 Changed 10 years ago by tec
- Description modified (diff)
Delivery
LiDAR -> FTP server 27/04/2015
comment:32 Changed 10 years ago by dap
Fenix Processing
Final files have been created with new calibration files and they've been moved into the delivery directory. Now ready for delivery check.
comment:33 Changed 10 years ago by lah
Started Fenix DC
Says BALI-SAFE in logsheet, not Danum Valley...
comment:34 Changed 10 years ago by lah
Fenix DC
- Mapped files only have 3 bands so need reprocessing.
- Only the readme has Danum_Valley, so that just needs altering.
- Most other things were ok, but will need rechecking when all bands created.
As an aside, proj_tidy is looking at the wrong calibration file when doing checks.
comment:35 Changed 10 years ago by dap
Fenix Processing
Seems I had either forgotten to set the correct options in the config file or not saved the config file before putting the processing on the grid when I changed the options. I've reprocessed the mapped files and moved them to the delivery directory.
comment:36 Changed 10 years ago by lah
Fenix DC
- Just needs site location changing in readme then good to go.
- I have added the 2015 quality report.
- Under/overflows look ok, but could consider reducing the threshold, particularly for short flight lines. Only a couple of lines have noticeable missed bands; the threshold still works well for the majority of lines.
- Some bad pixels, remain, but can't get rid of all all the time.
comment:37 Changed 10 years ago by dap
Fenix Processing
Changed readme site location to 'SAFE Area, Malaysia'.
Fenix delivery ready to go.
comment:38 Changed 10 years ago by lah
Fenix
Uploaded delivery to ftp. (arsf1)
comment:39 Changed 9 years ago by lah
PI notified of delivery 05/05/2015 (Fenix, Lidar & RCD)
comment:40 Changed 9 years ago by stgo
Started reprocessing hyperspectral
comment:41 Changed 9 years ago by gej
Created hyperspectral delivery
creating readme
comment:42 Changed 9 years ago by gej
Ready for Delivery Check
comment:43 Changed 9 years ago by lah
Fenix DC
- renamed screenshots.
- under/overflows look ok. Just a couple of bands missed in the shorter flight lines.
- spectra all good.
Ready to deliver when data quality report is finished
comment:44 Changed 9 years ago by dac
Fenix
- Updated data quality report
- Copied to ftp (arsf1) - likely to have more projects ready today so will notify PI with these.
comment:45 Changed 9 years ago by dac
Fenix
Delivered to PI via FTP (arsf1) 21/10/2015
comment:46 Changed 9 years ago by lah
Archiving
Tidied up and transferred to NEODC.
comment:47 Changed 8 years ago by dac
- Resolution set to fixed
- Status changed from new to closed
Archiving
Raw and processed data available from NEODC: http://browse.ceda.ac.uk/browse/neodc/arsf/2014/RG13_06/RG13_06-2014_291_Danum_Valley
RCD Processing
Finished converting tifs