BAS13/01, flight day 068a/2014, Tamar

[knpa]

Data location: /users/rsg/arsf/arsf_data/2014/flight_data/uk/BAS13_01-2014_068a_Tamar/

Data arrived from ARSF via network transfer on 10/03/2014

BAS13/01 Scientific objective: unknown

BAS13/01 Priority: ?

PI: France Gerard

Sensors:

Camera (Requested)
LiDAR (Requested)

[knpa]

France would like the flightlines divided into manageable chunks, 3 per flightline was suggested.

[tipo]

RCD images are all somewhat to very overexposed. According to the log, exposure was fixed at 2 for the entire flight, rather than varying.

[benj]

Gary has confirmed that the PI would like FW LiDAR data for this flight (if we have any).

[tipo]

Processed LiDAR, will get steve to check it before creating the delivery, as they take a long time to create.

[tipo]

Created LiDAR delivery, but apl fails to create the wgs84 dem. Possibly due to the large size of the flight? Trying lower resolution to see if that helps.

[tipo]

Created RCD delivery. Given the poor quality of the images, I'm unsure if it's worth delivering, but it's there in case it's wanted.

[tipo]

Created LiDAR delivery. Flightline files have been split into 3 parts with overlap as requested.

[stgo]

Beginning RCD D/c

[stgo]

D/c complete, photographs don't have full exif tagging. Also any explanation for the overexposure available for the readme?

[tipo]

Images are now tagged, readme updated to give more info on the overexposure.

[stgo]

Beginning RCD D/c

[stgo]

RCD delivery check finished, no problems found.

[stgo]

Lidar DC in progress, couple of initial comments:

• need a note somewhere about the format of file splitting (a b c)
• need to make sure it's clear the elevation difference is between lines and not along lines.

[stgo]

LiDAR DC (tentatively) complete.

Roll errors present between lines 5, 6 7 and 8. Will check others for more, seems to be about 16 cm in size. Particularly noticeable in part a/c of these lines in the town.

[stgo]

RCD data dispatched on 04/06/2014

[mark1]

Edited KML file in-place in RCD delivery to correct data paths for rcd thumbnails.

[stgo]

Lines 105248 through to 121418 match up with the following roll values:

roll-correct = -0.004034393
pitch-correct = -0.000880469

1012 and 1026 are seperate from the main data body. I will check for any lines that overlap but they appear to have a considerable roll variation (40 cm).

Line 122811 has a roll variation of 15 cm from 121418. I have not checked the lines beyond this in the dataset.

[dac]

Found correct roll offset values.

Line	Roll	Pitch	Heading
140309_101258	-0.003904393	-0.000880469	0.00045631
140309_102632	-0.003904393	-0.000880469	0.00045631
140309_103944	-0.003904393	-0.000880469	0.00045631
140309_105248	-0.003804393	-0.000880469	0.00045631
140309_110610	-0.003804393	-0.000880469	0.00045631
140309_111932	-0.003904393	-0.000880469	0.00045631
140309_113312	-0.003904393	-0.000880469	0.00045631
140309_114615	-0.003904393	-0.000880469	0.00045631
140309_115952	-0.004004393	-0.000880469	0.00045631
140309_121418	-0.004004393	-0.000880469	0.00045631
140309_122811	-0.004004393	-0.000880469	0.00045631
140309_124117	-0.004004393	-0.000880469	0.00045631
140309_125519	-0.004104393	-0.000880469	0.00045631
140309_130917	-0.004104393	-0.000880469	0.00045631

[dac]

Only a single raw full-waveform file is available (140309_092407) which could not be processed.

AGL(m) given as -1 in ALSPP. Not possible to process full-waveform or corresponding discrete file.

Likely acquired prior to rest of data as all 14 lines of discrete return data on log sheet accounted for.

[dac]

Classifying data

[dac]

LiDAR Processing
Created delivery, ready for delivery check.

[lah]

starting lidar DC

[lah]

Lidar DC complete
Have removed .LAS from screenshot names and suggest rechecking a couple of things:

• Roll values seem slightly off for a few lines, particularly 4-6. From 8 onwards are fine. Pitch is off too generally.

Roll errors

2-3	0.2
3-4	0.3
4-5	0.4
5-6	0.6
6-7	0.2
7-8	0.3

• There is some missed noise in the classification. Above ground is probably acceptable, but there is a lot more below ground missed. Some points are classified as noise within the cluster if you want a way of finding these manually. Can you write a script to find the local ground level to remove these? Also can you check lines 7, 24 and 36 for haze. There are a lot of points that don't look like vegetation in a small group in each of these lines. Lines 25-30, 31-33 and 35 are fine as is.

The dem is so big it took me a while to find the flight data in it and ages for demcompare to run (4+ hours) giving mean unmasked -0.2.

[dac]

LiDAR Processing

Refining roll values and finding pitch values for lines 1 - 8.

[dac]

LiDAR Processing
Found pitch and roll value for first two lines:

roll-correct = -0.0039081257
pitch-correct = -0.0006873354

Different value required for third line:

[dac]

Found pitch and roll values for third and fourth lines:

roll-correct = -0.0040081257
pitch-correct = -0.0006273354

[dac]

Updated pitch and roll vales:

Line	Roll	Pitch
140309_101258	-0.0039081257	-0.0007273354
140309_102632	-0.0039081257	-0.0007073354
140309_103944	-0.0040081257	-0.0007073354
140309_105248	-0.0040081257	-0.0007073354
140309_110610	-0.0040081257	-0.0007073354
140309_111932	-0.0040581257	-0.0007073354
140309_113312	-0.0040581257	-0.0007473354
140309_114615	-0.0040081257	-0.0007473354
140309_115952	-0.0040081257	-0.0007473354
140309_121418	-0.0040081257	-0.0007673354
140309_122811	-0.0040081257	-0.0007673354
140309_124117	-0.0040081257	-0.0007673354
140309_125519	-0.0040081257	-0.0007673354
140309_130917	-0.0041081257	-0.0007673354

Will get lah to check before splitting, classifying and redoing delivery.

[lah]

Checked pitch and roll values for each pair. Roll is within tolerance for all lines, though is slightly off at the bottom of lines 4-5. Pitch is approx 0.7m of for lines 1-5 and about 0.4m off for lines 5-9. Lines 9-14 are fine.

[dac]

Updated pitch and roll values, starting classification.

[dac]

LiDAR Processing
Finished classification, creating delivery.

[dac]

LiDAR Processing

Delivery created, ready for checking.

[dap]

Beginning LiDAR Delivery Check

[dap]

LiDAR Delivery Check

Renamed logsheet using rename.sh -f 68a 068a in logsheet directory.

DEM compare results:

Difference statistics:
Min:		-53.1125415039062
Max:		104.707923583984
Sum:		-1692823.31105974
Mean:		-5.36858845318959
Median:		-5.00869
Absolute mean:	7.24126409432035
Std deviation:	7.67325032796051
Total cells:	2106484
Non-null cells:	315320


Unmasked statistics:
Min:		-53.1125415039062
Max:		104.707923583984
Sum:		-1690463.24793075
Mean:		-0.864203041227932
Median:		2.0752e-06
Absolute mean:	1.44709831096314
Std deviation:	3.86373121379511
Total cells:	2106484
Non-null cells:	1956095

Roll and pitch offsets:

• Haven't checked all pairs of lines but roll and pitch seem to be offset for at least the following pairs
  • 49 and 48
  • 48 and 47
  • 47 and 46
  • 46 and 45

Classification hasn't yet been checked.

Everything checked up to now except roll and pitch errors seems fine.

[dap]

Taken over LiDAR processing from dac, correcting roll and pitch error values. Will put new values in a new comment.

[dap]

LiDAR Processing

New roll and pitch error values have now been found, and are as follows:

Line	Roll (rad)	Pitch (rad)
140309_101258	-0.003978125	-0.000997335
140309_102632	-0.003868125	-0.000707335
140309_103944	-0.004018125	-0.000987335
140309_105248	-0.003858125	-0.000707335
140309_110610	-0.004008125	-0.000947335
140309_111932	-0.003958125	-0.000687335
140309_113312	-0.004008125	-0.000907335
140309_114615	-0.004008125	-0.000747335
140309_115952	-0.004008125	-0.000807335
140309_121418	-0.004008125	-0.000797335
140309_122811	-0.004008125	-0.000787335
140309_124117	-0.004008125	-0.000847335
140309_125519	-0.004038125	-0.000777335
140309_130917	-0.004108125	-0.000817335

Note that for some flight lines, the best correction that could be found was ~15cm. Beginning classification.

[dap]

LiDAR Processing

Classification complete. Flight lines have also been split using

split_las.py -d processing/als50/las-classified/ -o processing/als50/las-classified/split -p 3 --time 7

[dap]

LiDAR Processing

Dan found problems with the LAS files when they were sorted (y values all 0). Will re-split the files and move them in to the delivery. Will then re-sort them.

[dap]

LiDAR Processing

A delivery has now been created for these data, now generating a read me document.

[dap]

LiDAR Processing

Read me document generated, delivery at BAS13_01-068a-lidar-20150114 ready for checking.

[lah]

Lidar DC

• Changed txt files to windows compatible
• DEM results:

Difference statistics:
Min: -54.8722155761719
Max: 104.695563964844
Sum: -1700968.53721365
Mean: -5.39529270944385
Median: -5.02067
Absolute mean: 7.29369607734545
Std deviation: 7.74696294199489
Total cells: 2107794
Non-null cells: 315269

Unmasked statistics:
Min: -54.8722155761719
Max: 227.498000549316
Sum: -870256.564774547
Mean: -0.444394575886368
Median: -0.269921
Absolute mean: 3.22244584094512
Std deviation: 8.90760847517729
Total cells: 2107794
Non-null cells: 1958297

• All other checks fine, just need to check offsets and classification when get LAG running.

[lah]

Lidar DC

• Residual roll and pitch offsets are small (at most 0.1m except for lines 11 & 14).
• Weird dip appears in 8 & 11, but could be real.

[lah]

Lidar DC

• Just looking at lines 1-6 there seems to be quite a bit of noise that isn't classified. Obvious bits are below ground and on lake. Might need to allow for bigger clusters of noise.

[dap]

Replying to lah:

• Weird dip appears in 8 & 11, but could be real.

I had noticed the dip while processing. Having looked at the kml overview, it looks as though it should be there - I think it's the overflow tower in the south-west corner of Roadford Lake.

[dap]

LiDAR Processing

Giving files another look over, classifying missed noise.

[lah]

Lidar DC

Classification now fine - most noise has been classified.

[dap]

LiDAR Processing

Delivery has now been created with re-classified data. Ready for delivery check if necessary.

[dac]

LiDAR DC

• In readme states roll error has been corrected to around 15 cm but in the table no mean difference between lines exceeds 10 cm - suggest rewording.

• In latlong DEM ASTER and lidar weren't patched correctly, have fixed bug in dem scripts and recreated DEM.

Once readme has been reworded ready to deliver.

[dap]

LiDAR

Read me has been reworded, now dispatching the dataset by priority archiving at NEODC.

[dap]

All raw and processed data have now been uploaded to NEODC, 05/02/2015.