GB14/04, flight day 219b/2014, Aberfoyle

[knpa]

Data location: /users/rsg/arsf/arsf_data/2014/flight_data/uk/GB14_04-2014_219b_Aberfoyle/

Data arrived from ARSF via network transfer on 14/08/2014

GB14/04 Scientific objective: unknown

GB14/04 Priority: 10

PI: Rachel Gaulton

Notes: FW is present Owl is present

Sensors:

Camera (Delivered on 14/10/2014)
Fenix (Delivered on 14/10/2014)
LiDAR
FW LiDAR
Owl

[tec]

RCD

Started converting RAW images to TIFF

[tec]

RCD
No images need removing.

Waiting on NAV.

[dac]

Started lidar processing.

[dac]

PPP Result:

Station	Latitude	Longitude	Elevation
GLAS	55 51 14.41473	-4 17 47.34487	71.839
KILN	56 27 17.79080	-4 19 14.64440	232.357

[tec]

RCD
Tagging TIFF images

[tec]

RCD
Awaiting delivery check.

[lah]

Fenix
Processed using existing boresight with SCTs:

1	0.95
2	0.97
3	0.98
4	0.98
5	0.98
6	0.98
7	0.98
8	0.98
9	0.96
10	0.98

Looks ok in mosaic if ignore blurry edges. Ready for DC.

[tec]

Fenix DC
Starting delivery check

[tec]

Fenix DC

• In readme move table of flightlines to next page
• I removed the empty fodis dir
• fenix_viewvector_list.txt has CRLF nice ;)
• Can't remember if we can fix blurry edges, ask mark1, if not then screenshots look fine
• I would of used a pixel size of 1.9 for the fenix not 2. Should be fine though.
• Level 1 Files (add stuff about noise and underflows to readme)
  • Line 1-5 high noise and underflows at end of line, bands 405-447
  • Line 6-10 high noise and underflows over the whole line, bands 405-447
  • All lines underflows and noise from around band 310-390

[dac]

Starting RCD Delivery Check

[dac]

RCD Delivery Check complete. Corrected spelling of my name in readme (templates had already been corrected).

Ready to deliver

[lah]

Fenix

• Have made suggested changes to logsheet.
• mark1 says there's not currently a way of fixing pixel stretches at boundaries.
• just used default fenix pixel size

[tec]

Fenix DC
Ready for delivery. Currently zipping mapped files.

[tec]

Fenix DC
Ready for delivery

[tec]

Emailed PI about vectors 09/10/2014 18:15:00

[tec]

Sent off RCD and Hyperspectral to Rachel Gaulton on 14/10/2014

[dac]

Default pitch roll and heading values from new boresight produce good results, although few buildings on the overlap of parallel strips.

Roll: -0.006375
Pitch: 0.00016318
Heading: 8e-005

Starting classification.

[dac]

Classified data, default parameters classified lots of ground and within-canopy returns as noise. Changed to 4 3 which produced better results. There are powerlines going across the strips - haven't classified these as noise.

Full waveform data available for four lines, line 160350 contains full-waveform data for only a small section of the line. Copied classification from discrete data.

Created DSM and checked registration against Fenix data - looked good.

Making delivery.

[dac]

Created delivery, waiting for delivery check.

[dap]

Beginning LiDAR Delivery Check

[dap]

LiDAR Delivery Check

Complete with the following issues found:

• knpa still in the contacts section of the read me file - have changed the template but you will need to change the source tex file.
• Only 4 FW files (and 10 discrete files) with no mention of it in the data quality remarks - are some missing?
• The number of trj files isn't the same as the number of full waveform files.
• Last page of logsheet blank - I've removed this.
• Think we deliver a screenshot for each line now - none were included in this delivery.
• Flightline names table in read me document goes off page
• bng dem looks significantly smaller than the latlong dem - not sure if this is a problem or not

Also checked full waveform files with laszip, no errors returned and two files produced for each las1.3 file in delivery. Checked waveform files in Wave Viewer, no problems seem to persist.

[dac]

LiDAR Delivery Check

Have addressed issues found by dap in delivery:

• Have added comment about only having 4 FW files to Readme (only 4 were available).
• Have removed trj files for lines without FW data.
• Had moved screenshots of individual lines back (had moved out of delivery folder as hasn't realised they needed to be included).
• Have forced a new page before flightline names table so it is all on a single page.
• Have recreated latlong DEM, size is now similar to BNG DEM.

Ready to deliver.

[dap]

LiDAR Delivery Check

• Renamed screenshots so that they have .jpg extension (as opposed to .txt.jpg) with rename.sh -f .txt.jpg .jpg
• Ran proj_tidy.sh to check naming conventions and for missing files - output was fine.

LiDAR data delivery GB14_04-219b-lidar-20141015 ready for delivery.

[dac]

LiDAR Delivery
Sent LiDAR data on 16/10/2014 to

Rachel Gaulton
School of Civil Engineering and Geosciences
Room G.16
Cassie Building
Newcastle University
Newcastle upon Tyne
NE1 7RU
UK

[lah]

OWL

SCTs:

1	1.95
2	1.98
3	1.95
4	1.95
5	1.96
6	1.97
7	1.96
8	0.96
9	1.95
10	0.98

[lah]

Experimental delivery created with Owl l3 files (no mask files, no quality reports) GB14_04-219b-owl-20150501. Uploaded to ftp and PI notified 05/05/2015.

[lah]

Fenix re-DC

• New delivery looks fine
• Spectra look good except for the odd pixel with a drop in VNIR e.g line 5 sample 490 line 3539
• Underflows look reasonable. Still can't see overflows, even raw files in fastQC using 8191

Not zipped files until we've checked ground spectra and made some comparisons.

[dac]

Fenix

Have compared spectra to 6S simulations - look fine. Waiting to hear back from PI on ground spectra before sending.

Have started zipping mapped files on molucca - will use the level1b files to check the spectra.

[dac]

Fenix

Received field spectra and have copied to /users/rsg/arsf/arsf_data/2014/misc/GB14_04_ground_truth.

Will check against before dispatching data.

[dac]

Fenix - Spectra check

Checked Fenix against the following spectra:

Target	Lon	Lat	Line
T1 (car park)	-4.380651	56.177175	10
T2 (pond)	-4.390611	56.193553	8
T3 (tarmac)	-4.391132	56.193688	8
T4 (quarry)	-4.408332	56.198332	7
T5 (water)	-4.448856	56.183071	1

Radiance values match well for T4, for other targets radiance values vary but similar shapes. However, still seeing a small discontinuity between two detectors. Will investigate before sending.

[dac]

Fenix Spectra

Calibration still doesn't seem right for this flight - trying to atmospherically correct data and compare to field spectra (rather than adding atmospheric effects to field spectra).

[lah]

Fenix
Created a new delivery without the SWIR bands (as these are too unstable and cannot be dark frame corrected).

/data/molucca2/scratch/arsf-temporary/GB14_04-2014_219b_Aberfoyle/processing/delivery/GB14_04-219b-hyperspectral-20150924

Needs thorough checking

[dac]

Fenix Delivery Check

Starting delivery check.

[dac]

Fenix Delivery Check

Compared to field data with atmosphere simulated using 6S:

• Matches well to T1
• Much lower than T2
• Much lower than T3
• Slightly lower than T4
• Much lower than T5. Distance increases as wavelength reduces

Given potential errors in parametrisation of 6S and uncertainty with ground spectra (geolocation etc.,) not sufficient proof there is a problem with the spectra.

Other points:

• Might be worth mentioning in the Readme why us and Specim think the VNIR bands are OK
• check_apl_cmd fails on aplcorr, error: Spatial binning is missing from the level-1 file header. Please add a line in the header file containing: binning = VALUE where VALUE is the correct spatial and spectral binning of the data in the form, e.g., {1,1} If the data is Fenix data then add binning_VNIR = VALUE and binning_SWIR = VALUE
• XML has old address for ops (probably not a major problem as readme has correct address).

Ready to go once these are addressed.

[lah]

Fenix
Addressed all problems in delivery. Added binning_SWIR to all level 1 headers has removed printed error in check_apl_cmd. Added some more words to readme, which are probably worth double checking before delivering.

[dac]

Fenix

New text looks fine but would remove:

To check the validity of the VNIR bands, ground spectra with atmosphere
simulated using 6S have been compared to the fenix data. The results of this comparison indicate
that the detector instability has not noticeably affected the VNIR bands.

As I don't think the comparison with 6S simulations are sufficient to indicate there are not problems with the VNIR bands. Neither are they sufficient to indicate there are problems.

Ready to deliver but add note in delivery email about problems.

[lah]

Fenix delivery

Have uploaded delivery to ftp (arsf7) and emailed PI 02/10/15 (asking to be contacted about SWIR bands if they are still required).

[dac]

Fenix

Copied delivered data to visuyan and finalised delivery.

[asm]

Fenix

Recent in depth analysis included exhaustive inspection of the ground spectra using py6s for fligtlines 4-10.

Target	Lon	Lat	Line
T1 (car park)	-4.380651	56.177175	10
T2 (pond)	-4.390611	56.193553	8
T3 (tarmac)	-4.391132	56.193688	8
T4 (quarry)	-4.408332	56.198332	7, 6, 5
T5 (water)	-4.448856	56.183071	10

Radiance values match well for T4, for other targets radiance values vary but similar shapes. For T5 there were cloud shadows over flightline 10 and T1 matches fine. T2 and T3 have a shape that matches but the spectra recorded is much lower because they were recoded at the edge of the swath.

[dac]

Archiving

Started archiving processed data to NEODC. Includes Owl but not Fenix SWIR bands (as unstable). Have left processing directory as might still need files for SWIR tests. Will archive raw data later.

[lah]

Owl tidying

• Moved pixstabi results to my scratch area for now
• Tidied up owl processing directory (removed copies of delivered data, post processed nav, igm etc) - should have been removed before archiving!

[dac]

Tidied up owl processing directory (removed copies of delivered data, post processed nav, igm etc) - should have been removed before archiving!

Raw data hasn't been archived yet (tests with SWIR haven't been completed yet so I left processing files for now).

[dac]

Generated atmospherically corrected versions of lines 4 and 5 using ATCOR.

Copied to ~arsf/arsf_data/2014/misc/GB14_04_atmospheric_correction and delivered via FTP.

[dac]

Tidied up delivery directory:

• GB14_04-219b-hyperspectral-20150924 is the 'official' delivery. SWIR bands were dropped due to instability
• GB14_04-219b-fenix-swirbands-reprocessing-20160422 is the investigation and SWIR reprocessing carried out for the PI. Doesn't have hyperspectral in the name so 'official' delivery is picked up by our scripts not this one.

[asm]

Archiving

All data present at CEDA. Processed data at: ​http://data.ceda.ac.uk/neodc/arsf/2014/GB14_04/GB14_04-2014_219b_Aberfoyle/