RG13/06, flight day 291/2014, Danum Valley

[dap]

Data location: /users/rsg/arsf/arsf_data/2014/flight_data/malaysia/RG13_06-2014_291_Danum_Valley

Data arrived by network transfer on 17/12/2014.

Scientific objective: Investigating the impact of humans on tropical forests.

Priority: alpha 5

PI: David Coomes

Full Waveform data present
No Owl data present

proj_tidy indicates that there are differences between some of the fps_set and fps_qpf values in the following Fenix header files:

• FENIX290-14-10.hdr (difference of 11.451)
• FENIX290-14-11.hdr (difference of 0.332)
• FENIX290-14-13.hdr (difference of 0.311)
• FENIX290-14-21.hdr (difference of 0.528)

Sensors:

• Fenix (requested)
• Leica FW LIDAR (delivered 27/04/2015)
• Leica LIDAR (requested, delivered 27/04/2015)
• RCD (requested, delivered 23/03/2015)

Duplicate lines

Process only one for Fenix and RCD - check for which has least cloud cover, failing that choose closest to local noon

• 1244-1249
• 1120
• 1131-1138 (day 293)

[tec]

RCD Processing
Finished converting tifs

[tec]

Navigation Processing
Started navigation processing

SAFE_6th_ Basestation Data
Latitude	4 43 30.56081
Longitude	117 36 19.18785
El Height	479.827m

[tec]

Navigation Processing
Some lines have a second of two of quality 3 gps data but other than that the nav is sound.

[tec]

Navigation Processing
Processed with IPAS Pro, settings below:
Elevation Mask: 14.0 degrees
KAR Maximum dual frequency distance: 80.00 km
Code: 1.20m

Nav finished

[tec]

RCD Processing
Started RCD Processing

[benj]

Added duplicate line details

[dac]

Navigation Processing

Reprocessed base station data

Latitude	4 43 30.56073
Longitude	117 36 19.18705
El Height	480.162 m

[dap]

RCD Processing

Moving the following images to to_remove, as they were affected by significant cloud coverage:

• 128
• 129
• 130
• 167
• 261
• 280

[dac]

Navigation Processing

Reprocessed navigation data in IPAS Pro. Apart from a noisy patch between 519500 - 520500 s quality is good.

[dap]

RCD Processing Complete

Ready for delivery check

[dac]

RCD Delivery Check
Starting delivery check

[dac]

RCD Delivery Check

Ready to deliver. Two minor comments:

• Needed to copy Readme into delivery directory.
• A lot of the images are cloudy (as noted in the Readme). I would say 106 and 291 fall into the 'significantly affected' by cloud coverage but they still have some usable pixels so OK to keep.

[tec]

Hyperspectrial Processing
Have some time so thought I'd do this

[dap]

Beginning Fenix Processing. Renamed Fenix raw files so that they are named FENIX291* instead of FENIX290*.

[dap]

RCD Data Delivery

Rsyncing RCD delivery to ftp server

[dap]

Fenix Processing

Processed all lines in 'fast mode' to start identifying SCT values. File number 10 is not usable as it is tiny, this line wasn't recorded in the log sheet anyway so will ignore this file.

[dap]

Fenix Processing

apl was returning the following error when processing line 12:

No navigation file found to match times in /users/rsg/arsf/arsf_data/2014/flight_data/malaysia/RG13_06-2014_291_Danum_Valley/hyperspectral/fenix/ for FENIX291-14-12
Closest navigation file found /users/rsg/arsf/arsf_data/2014/flight_data/malaysia/RG13_06-2014_291_Danum_Valley/hyperspectral/fenix/FENIX291-14-11.nav. Does not cover last 192.0000 seconds

Solved this by adding

nav2_args_extra = -nav /users/rsg/arsf/arsf_data/2014/flight_data/malaysia/RG13_06-2014_291_Danum_Valley/hyperspectral/fenix/FENIX291-14-12.nav

to the flight line's section in the config file.

[dac]

LiDAR Processing

Handing project over to tec. Best pitch and roll values found so far for first two lines:

Pitch: -0.00063
Roll: -0.006095

[dap]

Fenix Processing

Have looked at the raw flight lines (lines 1-7, 1131-1138) duplicated in flight 293 in FastQC in both flights. The ones collected in flight 293 are less affected by cloud so will not process lines 1 - 7 for flight 291.

[tec]

LiDAR Processing

Line	Roll	Pitch	Done FW	Classified
LDR141018_000915_1.LAS	-0.006315	-0.00025	Y	Y
LDR141018_001308_1.LAS	-0.006315	-0.00025	Y	Y
LDR141018_001706_1.LAS	-0.006185	-0.00025	Y	Y
LDR141018_002109_1.LAS	-0.006185	-0.00025	Y	Y
LDR141018_002455_1.LAS	-0.006185	-0.00025	Y	Y
LDR141018_002841_1.LAS	-0.006185	-0.00025	Y	Y
LDR141018_003305_1.LAS	-0.006185	-0.00025	Y	Y
LDR141018_003659_1.LAS	-0.006185	-0.00025	Y	Y
LDR141018_004332_1.LAS	-0.006185	-0.00025	Y	Y
LDR141018_004611_1.LAS	-0.006185	-0.00025	Y	Y
LDR141018_004920_1.LAS	-0.006185	-0.00025	Y	Y
LDR141018_005158_1.LAS	-0.006185	-0.00025	Y	Y
LDR141018_005447_1.LAS	-0.006185	-0.00025	Y	Y
LDR141018_005735_1.LAS	-0.006185	-0.00025	Y	Y
LDR141018_010239_1.LAS	-0.006185	-0.00025	Y	Y
LDR141018_010520_1.LAS	-0.006185	-0.00025	Y	Y
LDR141018_010809_1.LAS	-0.006185	-0.00025	Y	Y
LDR141018_011038_1.LAS	-0.006185	-0.00025	Y	Y
LDR141018_011322_1.LAS	-0.006185	-0.00025	Y	Y
LDR141018_011630_1.LAS	-0.006185	-0.00025	Y	Y
LDR141018_012256_1.LAS	-0.006185	-0.00025	Y	Y
LDR141018_013134_1.LAS	-0.006185	-0.00025	Y	Y
LDR141018_013726_1.LAS	-0.006185	-0.00025	Y	Y
LDR141018_014315_1.LAS	-0.006185	-0.00025	Y	Y
LDR141018_015135_1.LAS	-0.006185	-0.00025	Y	Y
LDR141018_015428_1.LAS	-0.006185	-0.00025	Y	Y
LDR141018_015711_1.LAS	-0.006185	-0.00025	Y	Y

[tec]

LiDAR Processing

Finished finding roll/pitch, was quite difficult, as there was no houses or decent structures but apart from the first two lines, the rest of the flight was rather consistent.

Processing FW now.

[tec]

LiDAR Processing
Finished creating LiDAR files, now proceeding with classification.

[dap]

Fenix Processing

Found all SCTs

SAFE Site Letter	Flightline	Fenix SCT
D	1	Use 293 duplicate
	2	Use 293 duplicate
	3	Use 293 duplicate
	4	Use 293 duplicate
	5	Use 293 duplicate
	6	Use 293 duplicate
	7	Use 293 duplicate
	8	Use 293 duplicate
M	9	Use line 13
	10	No usable data contained by file
	11	Use line 14
	12	Use line 15
	13	0.92
	14	0.94
	15	0.95
L	16	1.05
	17	0.89
	18	1.05
	19	Use line 16
	20	Use line 17
	21	Use line 18
U	22	Use line 25
	23	0.75
	24	1.03
	25	0.70
S	26	-0.05
	27	0.91
	28	0.99

[tec]

LiDAR Processing
Finished Classification, shall move onto creating a delivery tomorrow

[dap]

Fenix Processing

Created delivery and read me, Fenix data now ready for delivery check.

[tec]

LiDAR Processing
Ready for delivery check.

[dap]

LiDAR Delivery Check

Beginning delivery check.

[dap]

LiDAR Delivery Check

• Roll/pitch offsets seem good.
• Part of flight line 21 seems to have a kink in the end of the line. Dan suggested that this is to be trimmed off. I tried to process the hyperspectral flight line 21 and it didn't process (not a problem since it was a duplicate anyway). You'll also need to mention this in the data quality remarks.
  • Trim was done during delivery check, the new flight line looks fine.
• Classifications are good.
• Ran unix2dos on the ascii flight lines.
• Add cloud, lack of buildings and kink trimming (mentioned above) to data quality remarks.
• DEMs will need recreating due to the trimmed flight line (don't forget to also trim the resolution in the new DEM header).
• demcompare.py was not run as the DEM will need re-producing.
• Screenshots look fine, but I would suggest regenerating flight line 21's screenshot.
• laszip caused no errors on the full waveform files and they opened fine in Wave Viewer with no errors.

I'll recheck once the above issues have been seen to.

[dap]

LiDAR Delivery Check

• Renamed new read me so that it follows proj_tidy regex.
• New data quality remarks are fine.
• Trimmed the resolution in the UTM DEM header to 2.0

Looks fine, ready for delivery.

[tec]

Delivery
LiDAR -> FTP server 27/04/2015

[dap]

Fenix Processing

Final files have been created with new calibration files and they've been moved into the delivery directory. Now ready for delivery check.

[lah]

Started Fenix DC
Says BALI-SAFE in logsheet, not Danum Valley...

[lah]

Fenix DC

• Mapped files only have 3 bands so need reprocessing.
• Only the readme has Danum_Valley, so that just needs altering.
• Most other things were ok, but will need rechecking when all bands created.

As an aside, proj_tidy is looking at the wrong calibration file when doing checks.

[dap]

Fenix Processing

Seems I had either forgotten to set the correct options in the config file or not saved the config file before putting the processing on the grid when I changed the options. I've reprocessed the mapped files and moved them to the delivery directory.

[lah]

Fenix DC

• Just needs site location changing in readme then good to go.
• I have added the 2015 quality report.
• Under/overflows look ok, but could consider reducing the threshold, particularly for short flight lines. Only a couple of lines have noticeable missed bands; the threshold still works well for the majority of lines.
• Some bad pixels, remain, but can't get rid of all all the time.

[dap]

Fenix Processing

Changed readme site location to 'SAFE Area, Malaysia'.

Fenix delivery ready to go.

[lah]

Fenix
Uploaded delivery to ftp. (arsf1)

[lah]

PI notified of delivery 05/05/2015 (Fenix, Lidar & RCD)

[stgo]

Started reprocessing hyperspectral

[gej]

Created hyperspectral delivery
creating readme

[gej]

Ready for Delivery Check

[lah]

Fenix DC

• renamed screenshots.
• under/overflows look ok. Just a couple of bands missed in the shorter flight lines.
• spectra all good.

Ready to deliver when data quality report is finished

[dac]

Fenix

• Updated data quality report
• Copied to ftp (arsf1) - likely to have more projects ready today so will notify PI with these.

[dac]

Fenix

Delivered to PI via FTP (arsf1) 21/10/2015

[lah]

Archiving
Tidied up and transferred to NEODC.

[dac]

Archiving

Raw and processed data available from NEODC: ​http://browse.ceda.ac.uk/browse/neodc/arsf/2014/RG13_06/RG13_06-2014_291_Danum_Valley