EUFAR15/48, flight day 249/2015, Hidhaz

[asm]

Data location: ~arsf/arsf_data/2015/flight_data/iceland/EUFAR15_58-2015_249_Hidhaz

Project acronym: HIDHAZ_N_ICELAND

Data arrived from ARSF via SATA disk 8 on 28/09/2015.

Priority: Unknown

PI: Armelle Decaulne

EUFAR code: E048

Sensors:

• Fenix (requested)
• Owl (requested)
• Leica LIDAR (requested)
• RCD (requested)

[asm]

General Processing
-Logsheet generated.
-Send a copy of the notifications to: Dr. Susan Conway (s.j.conway.00@…)
-Landing and taking off times are approximate.

[asm]

General Processing
Notification of ARSF data arrival has been sent to PI.

[asm]

Navigation Processing

Started.
Needed to download base station information for this project. Used IpasTC to download basestation REYK

Basestation verification

Lat: 64 08 19.63049
Lon: -21 57 19.7659
Elipsoidal Height: 93.066 (metres)
Antenna Height: 0.232 (metres)

Could not use Ipas Pro, used best solution from Ipas TC with downloaded basestation.
Settings:
-Switched data to static (default kinematic)
-Computed Position via P3 using Ipas TC
-Needed to specify antena name Leiar25.r4
-Re-calculated all positioning using C/A code
-Elevation Mask: 13.5 degrees
-L2C phase correction: 0

Solution has value 1 on ambiguity fix.

Navigation processing finished.

[dac]

Marking as blocked - still waiting on GPS data from IMO.

[asm]

Navigation processing

Navigation reprocessed with GPS data provided by Icelandic Met-office.
Basestation verification for SIFJ:
Latitude 66 08 16.94583
Lon -18 53 57.66368
Ell Height 81.499
Ant Height 1.098
Which corresponds on decimal degrees to the values sent by Met-office: -18.89935061 66.13804033 81.49616202

Basestation verification for AKUR:
Latitude 65 41 07.53592
Lon -18 07 20.93094
Ell Height 134.169
Ant Height 0.165 m, to L1-PC (TRM29659.00, MeasDist 0.055)
Also match data recieved: -18.12248256 65.68542672 134.20199055

Best solution, only using both basestation. Copying files and have renamed old basestation and ipas_honeywell to old_nav_basestation (Please clean after processing, before archiving).

[stgo]

Lidar processing

started processing

[asm]

RCD processing

Started. Converting raw files to tif.

[dac]

Fenix Processing

Started processing - file 1 (not on logsheet) moved to 'bad_lines'

[stgo]

Lidar processing

Pitch and roll errors seem consistent across all lines. Currently classifying.

pitch	-0.00076
roll	-0.00266

[stgo]

Lidar processing

There is only one file of waveform data which appears to be corrupt. There will be no waveform in the lidar delivery.

[asm]

RCD processing

Tagging pictures.

[stgo]

Lidar processing

There are two sites, 122822 - 134338 and 135102 - 151115.

122822 - 134338

pitch	-0.00256
roll	-0.00066

135102 - 151115

pitch	-0.00076
roll	-0.00266

Recreating delivery

[dac]

Found SCT values using autoSCT. A couple of lines were out of range and needed to be calculated manually. All lines match well against each other and LiDAR intensity data.

Flightline	SCT
2	0.99
3	1.01
4	1.01
5	1.01
6	1.03
7	1.03
8	1.02
9	1.08
10	1.05
11	-0.05
12	1.00
13	1.01
14	1.01
15	0.99
16	-0.06
17	1.00
18	0.99
19	0.97
20	0.99
21	1.09
22	0.99
23	0.99
24	1.0
25	0.04
26	0.99
27	1.05
28	1.0
29	0.99

[asm]

RCD

Delivery created. Readme file includes this explanation: The RCD camera is set automatically to an specific gain value depending on light conditions for the recording area. Photos taken over snow areas are brighter than standards but have been included on this delivery if they were not overexposed as they have usable data.

This effect is clearly visible in many pictures but specially on 101 (a bit overexposed) that still have been included as it may be useful (closest pictures were not usable and this one covers that area showing snow).

Ready for delivery check.

[stgo]

Lidar processing

The first site seems to suffer from range gate issues due to the data being outside the minimum range gate set on the instrument. This can be seen in the large data holes and tapering of the flightlines.

The second site suffers from quite a lot of noisey/hazey points in the first lines. This has been classified as needed.

[dac]

Fenix Processing

Created delivery - ready for delivery check

[stgo]

Lidar processing

Created lidar delivery. Ready for check.

[asm]

Hyperspectral Delivery Check

All checks done, only a few minor comments.
-I suggest to add a note on readme about data quality for fenix saturation over snow. Also would be useful a note explaining that the first flightline is named 02 and not 01.
-Spectra checked with py6s. Looks fine except for some peaks a bit higher than usual but within expecting limits on the overlapping region and they will be masked out.
-Checked apl commands, will remove folder once ready to deliver.
-Have removed last page from logsheet about Eagle and Fenix. Also changed "*_cross" flightline for two different lines for avoiding overwriting next cell.

[lah]

RCD DC

• I've rephrased the readme text, as many of the snowy images are overexposed in the snow regions, but are still useful:

"The RCD camera is set automatically to a specific gain value depending on light conditions for
the entire recording area. Some photos have snow pixels that appear overexposed, but have been
included if they have usable data."

Everything else is fine, so ready for delivery.

[dac]

Replying to lah:

"The RCD camera is set automatically to a specific gain value depending on light conditions for
the entire recording area.

It will be exposure rather than gain for a camera. In reality just exposure time as the aperture is fixed.

[dac]

Fenix

I suggest to add a note on readme about data quality for fenix saturation over snow.

Have added:
"Pixels over bright snow-covered areas are overexposed in some wavelengths. These bands have been masked out in the level 3b files."

Also would be useful a note explaining that the first flightline is named 02 and not 01.

Have added:
"Note file numbering starts at 2 as the first file was saved prior to acquisition and contains no usable data."

Started zipping mapped files.

If asm is happy will changes to readme will mark as ready to deliver.

[lah]

Replying to dac:

Replying to lah:

"The RCD camera is set automatically to a specific gain value depending on light conditions for
the entire recording area.

It will be exposure rather than gain for a camera. In reality just exposure time as the aperture is fixed.

changed gain value to exposure time in readme

[asm]

Hyperspectral Delivery Check

Happy with the changes made on readme. Mapped files have been zipped and everything is correct on their logfiles. Removed an aux file from the mapped directory. Fenix is ready to deliver.

[dac]

Delivery

To be delivered via hard drive to Costanza Morino - waiting on LiDAR. Have uploaded Fenix and RCD to FTP server (arsf14) for PI and other collaborators.

Also copy VM for training course to hard drive.

[lah]

Lidar DC

• Roll & pitch look fine
• Classification fine
• changed las1.0 to 1.2
• removed eagle & hawk pages from logsheet (actually copied from HS delivery, then removed fodis folder from delivery!)
• changed the readme name to exclude _01
• removed extra sol file
• demcompare:

Min: -105.999249023437
Max: 128.388448730469
Sum: -2256662.91263792
Mean: -7.19989443460396
Median: -6.14898
Absolute mean: 10.1892045178341
Std deviation: 11.7393376564784
Total cells: 4776398
Non-null cells: 313430

• only problem is in readme - you've spelt noisy and hazy wrong. Apart from those typos it's ready for delivery

[stgo]

Lidar delivery

Fixed readme, always get confused by ey's!

[dac]

Delivery and Archiving

Data copied to hard drive ready to deliver tomorrow.

Started uploading processed data to NEODC.

[dac]

Delivery

Sent on hard drive to Costanza Morino (Open University) 04/03/2016

Processed data uploaded to NEODC. Raw Owl data needs to be unpacked before raw data are uploaded.

[lah]

OWL

Found sct = -2.78 for Owl line 007 (identified from time on Owl folder name). Only unpacked (copied and renamed) this line as only processing a single line for now. GPS timestamps copied from Fenix data, and used T1 calibration file for dark frames.

Now mapping all bands.

[dac]

Archiving

Processed Camera, LiDAR and Fenix data have been archived to ​http://browse.ceda.ac.uk/browse/neodc/arsf/2015/EUFAR15_48/EUFAR15_48-2015_249_Hidhaz

[lah]

OWL

Created delivery with 1 line. Strangely, the level 1b was overwritten during delivery creation with a mask file of more lines than the created mask file, so no idea what happened there. Have recreated the level 1b file again and copied across.

Again, proj_inf xml is not created. I believe this is because we decided to stop it being created automatically if there are missing items (calibration stuff for owl?).

Also ascii FOV vectors failed.

[lah]

OWL

Reprocessed with corrected processing chain as file sizes were incorrect and couldn't run autoqc.

Created new complete delivery. Ready for checking.

[dac]

Owl delivery check

Looks fine apart from would change figure caption in readme from 'mosaic' as it is only a single line. Also can remove mosaic screenshot.

Other than that ready to go.

[lah]

OWL

Made suggested changes and zipped mapped file.

[lah]

Owl delivery uploaded to ftp (arsf 14) and email sent to Susan Conway (requested data).

[lah]

Owl renaming

• Ran new script to rename original raw files.
• Overwrote line 9 with modified raw data used to process sample data (labelled line 7). Calibration data used to process this line now named 1404.
• There is still an originally named folder with no data in which can probably be deleted.

[lah]

Owl Processing

Appended T1 files acquired immediately after each flight line to raw data for lines 1-16 (requested South site) to allow processing with Specim's tool (no dark frames for this flight).

[lah]

Owl Processing

Have processed to level 1b (radiometric calibration) and checked files can be mapped (Had to add gps stop times to raw header files). Apart from a few lines being outside the dem area and the sct offsets very large, everything look ok.

Will wait until finances approved before proceeding with manual sct corrections. (sct offsets too large for automated scripts to speed up process)

[dac]

Archiving

Archiving of processed data on-hold until Owl processing is complete.

[lah]

Owl Processing

Found scts. Still some across track deviation between different flightlines. Lines 4 and 8 have not been included as they are failed lines (errors mentioned in the logsheet). Large offset is due to GPS timestamp being automatically added during unpacking.

Flightline	OWL
1	-3767.15
2	-3758.59
3	-3763.04
5	-3749.18
6	-3759.54
7	-3749.80
9	-2.86
10	-3755.24
11	-3757.75
12	-3703.51
13	-3760.89
14	-3740.81
15	-3772.50
16	-3749.00
17	-3819.20

[lah]

Owl Processing

Delivery created and ready for checking. Added comment to end of page 2 in the readme that only the South site data is included.

[mark1]

Owl delivery check

Have all level1 files but:

• No XML files for lines: 04, 08
• No navigation for lines: 04 08
• No mapped for lines: 04 08
• No screenshots for lines: 04 08

Level 1 files for 04 and 08 are small.

No Owl information in the logsheet - we should enter FPS and integration time (manually if need be).

Table in read me does not list the matching owl line names.

Will check example commands and view data next

[lah]

Owl delivery

Have removed level 1b lines 4 and 8 as they shouldn't be there (no usable data). This is probably what messed up the readme table. Have added in the lines manually and made a comment about lines 4 and 8 not being there, but all lines being captured in subsequent files.

Looks like missing Owl info in the logsheet is a recurring problem so probably needs the logsheet generation altering. I'll raise an issue, but might not be a quick fix, so manual input might be best for this project.

[mark1]

Owl delivery check

Have tested all level1b and level3 images and they open and show good data.

All example commands worked-

[dac]

Owl

Manually added Owl information to logsheet. Updated readme to NERC-ARF (with note facility was NERC ARSF when data were flown).

Zipped mapped files.

Delivered via FTP (arsf14).

[lah]

Owl Archiving

Uploaded Owl delivery to NEODC 21/09/16.

[asm]

Archiving

Archived and it everything available from CEDA. Processed data at ​http://browse.ceda.ac.uk/browse/neodc/arsf/2015/EUFAR15_48/EUFAR15_48-2015_249_Hidhaz

Plymouth 12th May 2017