GB18/66, flight day 178/2018, Sefton Coast

[dac]

Data location: ~arsf/arsf_data/2018/flight_data/uk/GB18_66-2018_178_Sefton_Coast

Data arrived from NERC-ARF via hard drive on 13/07/2018

Scientific objective: Mapping and monitoring Dune fields on the Sefton Coast.

PI: Paul Aplin
BAS Project Code: D066

Due to lack of LiDAR PI requested SfM processing to generate DSM.

The following issues were identified during unpacking:

• Missing darkframes for owl capture data.

Sensors:

• Fenix (requested, flown)
• OWL (not requested, flown)
• Phase One (requested, flown)

[dac]

Fenix

GPS stop time for line 1 was listed as xx:xx:xx.x, replaced with 13:43:43, calculated using FPS and number of lines.

[dac]

Digital Camera

Converted raw images to tiffs.

[asm]

Navigation Processing

Started. Downoladed basestation BLAP from ​OS National GPS Network. Used navigation files downloaded by IpasTC from closest basestation. Basestation verification:
Lat: 53 46 36.94825
Lon: -3 02 05.82891
Ell. Height 66.193 m

Good results on navigation with position separation smaller than 0.2 m (good for Fenix Pixel size) and ambiguity fix 2. Files copied acros. Navigation Processing finished.

[asm]

Processing

Wrote PI on 27/07/2018 to ask him if they deployed their own basestation data as they have done in the past so we can use it for processing. Further processing is for now on hold until we have a reply.

[dac]

Hyperspectral Processing

Ran through with existing .sol file to check navigation sync. Doesn't look to be any problems. SCT value of 2.0 works for most lines apart from 3 where 1.0 looks good. Match well with OS vector data.

Holding off further processing until we hear back about GPS basestation.

[asm]

Hyperspectral Processing

Found SCT values:

Flightline	FENIX
1	2
2	2
3	1
4	2
5	2
6	2
7	2
8	2
9	2
10	2
11	1
12	1
13	2

Navigation was going backwards for line 11, using -force flag in aplnav solved the issue. This is likely to be related to the persistent problem we are having with the fps of the instrument this season. The lines overlap OK with very small offsets so can't correct the fps manually as it is within acceptance values. Will create a delivery and add a note on readme.

[dac]

Navigation processing

Heard back from PI on 06/08/2018. GPS basestation wasn't set up for flight, continuing processing using OS RINEX data.

[asm]

Hyperspectral Processing

Delivery and readme created. Hyperspectral procesing finished, ready for DC.

[dac]

Hyperspectral Delivery Check

Starting delivery check.

[dac]

Hyperspectral Delivery Check

• There was 'logsheet.pdf' in the root directory as well as the correctly named one in logsheet. I checked the one in the directory logsheet which was newer. logsheet.pdf needs to be removed (unless this is the correct one)
• Logsheet is missing PI
• In readme, quality and issues with data section suggest following changes to text:
  • During this season, the framerate of the instrument has been not responding accordingly and could not be set correctly. This led to a small "stretching" of the mapped data and inaccurate geocorrection of the Fenix data in other flights > During this season the framerate of the instrument has sometimes being recording incorrectly, this has led to a small stretching of the mapped data and inaccurage coecorrection for these flights.
  • via QGIS (delete).
• removed fodis directory
• Registration accuracy looks good, it could be improved with a better DEM (which is what we normally recommend).
• Colours in the screenshots look a bit off, and there are lots of black patches in the middle around the dunes, data look OK though so should be fine.
• Spectral look good when compared to Py6S simulations

After minor updates have been made to logsheet and readme ready to deliver.

[asm]

Hyperspectral Delivery

Made changes suggested to the logsheet and README files. Will zip mapped files.

[asm]

Hyperspectral Delivery

Mapped files were zipped correctly. PI selected hard drive as method of delivery so started copying files across.

[asm]

Hyperspectral Delivery

Delivered via FTP slot 5 and hard drive sent to PI. Notification sent on 17/08/2018. Finished

[dac]

Digital Camera

By comparing to Fenix lines and satellite imagery through Google Earth estimated difference between file creation time and GPS time to be 7287 seconds. Processing using this.

[dac]

Digital Camera

Created delivery. Ready for checking.

[wja]

Digital Camera Delivery Check

• PI's e-mail is @nottingham, although their institute is Edge Hill. Worth checking?
• The following photos look identical, although EXIF shows they are 3" apart:
  • 13 & 14
  • 20 & 21
  • 35 & 36
  • 55 & 56
• Photos 1 - 7 are in the Midlands, far from the project site.

[dac]

Removed 1 - 7, 14, 21, 36 and 56. I'm not sure why the duplicates have different times but I've had a quick look and the photos after 56 look like they still line up OK.

I have correct (Edge Hill) email for PI.

[dac]

Digital Camera

Sent data to PI on harddrive (2018-08-31). Also copied to FTP slot 5 for use by CoIs.

[wja]

Orthomosaic/DEM Processing
Aligning photos.
The last two photographs (*1082.tif & *1083.tif) are not present in the eventfile. I have made a directory of symlinks in /processing/photoscan/photographs which link to all photos apart from the two mentioned. This is the directory input into the processing script.

[wja]

Orthomosaic/DEM Processing
Troubleshooting issues...
GCPs coordinates and screenshots of their photograph locations have been stored in /processing/phaseone/gcps/

[wja]

Orthomosaic/DEM Processing
Generating point cloud. Using 'high' settings, anticipating processing time of >1 day.

[wja]

Orthomosaic/DEM Processing
Processing killed, ran out of memory on VM (!).
Generating point cloud, using 'medium' settings.
GCP fit error of the 8 points reportedly 0.156m.

[wja]

Orthomosaic/DEM Processing
Point cloud, DEM and Orthomosaic produced. Orthomosaic resolution = 8cm, DEM resolution = 32cm.
Assessing X, Y & Z accuracy.

[wja]

Orthomosaic/DEM Processing
X & Y accuracy of DEM and Orthomosaic is has good X & Y accuracy in UTM30N (the reference system of the GCPs). However the Z (vertical) has a significant error. There is a large differnce (~20m) to the WGS84, there is a ~+3m bias to OS Datum Newlyn.
Exploring different datums.

[wja]

Orthomosaic/DEM Processing
19 GCPs received from Daniel Knight. Only one was not able to be used (car was parked on top of the area in the photos).
Processing using MetaShape completed, delivery created (have done an accuracy assessment comparing the generated DSM by comparing to EA LIDAR derived 2m DSM).

[dac]

Delivery Check

I've just checked the new part of the delivery, not the parts included with the digital photographs.

• I think the error section in the readme needs more explanation.
• No need to include exif output from orthomosaic. Also the text you pasted in includes the commands run as well as the output (including the cd command).
• Orthomosaic is a 16 bit image, would be better stored as 8 bit.
• 32 cm is a strange size for the output DEM, I'd round to the nearest 5, why not 30 cm, then at least it is a multiple of the orthomosaic.
• I'd create a tiff version of the DEM as well (gdal_translate -a_nodata -9999 -of GTiff -co "COMPRESS=LZW" -co "TILES=YES" input.dem output.tif) as I think this will be easier for users.

[wja]

Orthomosaic/DEM Processing

• EXIF output in ReadMe removed.
  
• New orthomosaic exported in 8-bit (GeoTIFF with JPEG compression) with 10cm resolution.
  
• GeoTIFF DEM with 30m resolution exported. Also copied in ENVI format *dem. Can delete the latter if needed.

[wja]

Orthomosaic/DEM Processing

• New orthomosaic created as GeoTIFF with 8-bit values (no JPEG compression) at 10cm resolution.
  
• ReadMe updated.

[dac]

Delivery Check

• Readme is much better
• I set the no data value as 255 in the orthomosaic geotiff and redid overviews to ignore this value

Ready to deliver.

[wja]

Orthomosaic/DEM Processing

• Delivery placed on FTP slot 5 (same as previous deliveries for this project).
  
• PI and Daniel Knight notified.
  
• Delivery finalised.
  
• Cleaning up processing directory.

[wja]

Orthomosaic/DEM Processing
PI had trouble accessing data form FTP so it has been dispatched via two hardisks.