RANNIS15/28, flight day 241b/2015, Hekla

[asm]

Data location: ~arsf/arsf_data/2015/flight_data/iceland/RANNIS15_28-2015_241b_Hekla

Data arrived from ARSF via SATA disk 8 on 28/09/2015.

Priority: Unknown

PI: Gro Pedersen

CAFAM code: R028

Sensors:

• Fenix (requested)
• Owl (requested - currently unfunded)
• Leica LIDAR (requested)
• RCD (requested)

[asm]

Flight Details
Logsheet has been created

-The logsheet shows that flightline 009a has Fenix VNIR with Binning to 4:2
-Also says that flightlines 008, 009 were aborted (but there is Fenix data from those, please check when processing.)
-There is only Full-waveform data for flightlines: 005, 006, 008, 009a.
-Landing taking off and engine times are approximate.

[asm]

General Processing
Sent to PI ARSF data arrival notification for RANNIS15/28.

[asm]

Navigation Processing, Basestation verification.

Needed to download base station information for this project. Used REYK

Lat: 64 08 19.62955
Lon: -21 57 19.75549
Elipsoidal Height: 93.238 (metres)
Antenna Height: 0.057 (metres)

[asm]

Navigation Processing

Finished. Everything looks fine.

[asm]

Navigation Processing

Also created sol file for camera.

[lah]

RCD Processing
Missing ImageEvents file. Copied in extra from 241a and now processing fine.

[lah]

RCD Processing
removed overexposed images:
29072835100098G3.jpg, 29072835100099G3.jpg, 29072835100107G3.jpg, 29072835100108G3.jpg,
29072835100228G3.jpg, 29072835100229G3.jpg, 29072835100291G3.jpg, 29072835100292G3.jpg,
29072835100301G3.jpg, 29072835100357G3.jpg, 29072835100358G3.jpg

and "not found" images:
29072835100204G3.tif, 29072835100205G3.tif, 29072835100206G3.tif

[asm]

Lidar Processing

Started.

[asm]

Lidar Processing

There is a lack of usable features and almost no overlap between flightlines except for two of them and a flightline that goes across the others. Used the overlapping flightlines to find the best pitch and roll value. Offset error seems to be consistent across flightlines and the error is within limits but the lack of features prevent us from finding finer tuning.

Will create the delivery with this values:
Roll: - 0.0014
Pitch: -0.0022458

[asm]

Lidar Processing

Classification started.

[asm]

Lidar Processing

Classification finished. Will not process FW data since it was not requested and only 3 flightlines were recorded since the sensor failed and was turned off.

[asm]

Lidar Processing

Meassuring elevation difference when possible.

[asm]

Lidar Processing

Delivery created. Ready for Delivery check.

[gej]

Lidar DC

Roll and pitch for overlapping lines 5-6 is wrong, there is a house in the middle of the flightlines that you can check on. Other lines can't really tell so I would say get 5-6 right and then check the other ones are correct with the new roll and pitch.

Everything else looks okay to me.

[lah]

RCD Ready for DC

Had to create a new delivery as kml file was missing RCD entries. Event file created ok this time, though had to change permissions to view it.

[asm]

Lidar Processing

Found better offset values with the feature gej pointed out. All lines seem consistent:
Roll: - 0.000642
Pitch: -0.002741

Will start classification.

[asm]

Lidar Processing

Finished classification and created delivery. Measured elevation difference and created readme file. Ready for Delivery Check.

[gej]

Lidar delivery check

Readme states that elevation difference could be calculated for 01-02, 03-04, 05-06 and 11-12. There are only 3 elevation differences on the table.

[dac]

RCD Delivery

Starting delivery check

[dac]

RCD Delivery

All looks fine. Would be good to add a note in the readme about the cloud shadow in some images. Other than that ready to go.

[lah]

RCD

Have mentioned cloud shadowing in readme, on bottom of page 5.

[dac]

PI prefers delivery by hard drive

[gej]

Fenix processing

Have started processing Fenix up to finding SCT offsets.

[gej]

Fenix Processing

Found SCTs

Flightline	FENIX
1	0.97
2	1.00
3	1.00
4	1.00
5	1.00
6	0.00
7	1.00
8	1.00
9	1.00
10	1.00
11	1.00
12	0.00
13	1.00
14	1.00
15	1.00

Waiting for new nav before mapping.

[dac]

Marking as blocked - still waiting on GPS data from IMO.

[dac]

RCD Processing

Converted raw files to tif.

[asm]

Navigation processing

Navigation reprocessed with GPS data provided by Icelandic Met-office to compare data quality. Basestations are the same that 241a 2015

Best solution found using MJSK. Copying files and will rename old basestation and ipas_honeywell to old_nav_basestation (Please clean after processing, before archiving).

[asm]

Replying to gej:

Lidar delivery check

Readme states that elevation difference could be calculated for 01-02, 03-04, 05-06 and 11-12. There are only 3 elevation differences on the table.

That was caused because there was a aborted line I did not notice. Will solve that for the new delivery.

[gej]

Fenix Processing

SCTs for new navigation same as old.
Mapping files.

[asm]

Lidar Processing

Found new offset using new navigation. All lines seem consistent:
Roll: -0.0006405
Pitch: -0.00253

Will start classification and create delivery.

[gej]

Fenix Processing

Have checked SCTs with new navigation and all are the same.
Mapped files generated, creating delivery now.

[asm]

Lidar Processing

Have created delivery and measured elevation difference. Ready for delivery check.

[dac]

LiDAR Delivery Check

Started delivery check.

[dac]

LiDAR Delivery Check

Putting on hold until processed with full range of scan angles.

[asm]

LiDAR

Redone with full range of scan angles. Will classify and create delivery.

[asm]

LiDAR

Re-created delivery, measured elevation difference and creating readme file. Awaitting delivery check

[stgo]

LiDAR Delivery check

Something has happened to the ASCII Point Filter functions on the readme, the command is on a separate page to the text. This should be fixed and checked for other projects.

The same has happened for the copyright notice, shifting over on to the last page, we should try and make sure this stays together.

I disagree with the data quality remark "Flightlines on this project have a
lack of useful features which prevented from finding a more precise value for pitch and roll offsets." - this shouldn't be included.

There are still pitch and roll errors in the overlap regions between between the following lines:

01-02
02-05
03,04,07,12 (especially visible here)

[gej]

Fenix delivery created, ready for delivery check.

[gej]

RCD processing
Have re-tagged using new navigation, creating delivery now.

[gej]

RCD processing

Delivery and read-me created, ready for delivery check

[asm]

RCD delivery check

DC started.
-Need to remove old delivery
-Eventfile did not have read permissions for group.(-rw-------. !) Could not open it (less) with other user apart form airborne.

[asm]

RCD delivery check

Anything else looks fine. Ready to deliver.

[dac]

Fenix Delivery Check

Starting delivery check.

[dac]

Fenix Delivery Check

All looks fine. However, according to the handwritten logsheet some of the lines (8a, 9a, 12a and 10a) were reflown due to problems during the first acquisition. Only the second (correct) versions should be delivered and the other files removed.

Suggest just remove repeat lines and add note to readme.

[asm]

Lidar Processing

Did not realise that with wider angle range, there were usable features to find better pitch and roll values. Found offsets and also checked their value with stgo. Consistent trough all lines.
Roll: -0.000805
Pitch: -0.00251

Will classify and create delivery.

[gej]

Fenix processing

Have removed flightlines 8a,9a,12a and 10a from delivery. Recreated mosaic jpeg and added a note to readme that 8a,9a,10a and 12a have been removed due to being reflown.

Ready for DC

[dac]

Changed readme text to make it clear lines were repeated due to a problem with the sensor:

"Due to problems with the Fenix sensor lines 008a 009a 012a and 010a were repeated for this flight, only the second acquisition has been included in the delivery"

Have started zipping files, marking as ready to deliver.

[asm]

Lidar Processing

Pitch and roll errors are consistent trough all lines except flighlitne 01 and 02. Will solve that before creating delivery.

[asm]

Lidar Processing

Flightline 01:
Roll: -0.000805
Pitch: -0.0027

Flightline 02:
Roll: -0.000805
Pitch: -0.0020

Remaining flightlines:
Roll: -0.000805
Pitch: -0.00251

Will classify and create delivery.

[asm]

Lidar Processing

Delivery and readme created. Ready for delivery check

[stgo]

Lidar Delivery check
Starting check

[stgo]

Lidar delivery check

Pagination errors still existed in the readme, we need to either move sections on to their own pages or reduce the smallskips between sections to fix this.

I can't run dem_compare at the moment, will look into this and hopefully have a figure this afternoon. Otherwise this delivery is ready to go.

[stgo]

Lidar delivery check

demcompare results:
Mean: -0.418515014075755
Median: -0.111104
Absolute mean: 2.12190412497265
Std deviation: 3.78019477066741

These are very positive. Once the readme is fixed the lidar is ready to go!

[stgo]

Lidar delivery check

Erm, actually no. The overlap measurements table is definitely out of date, you should update it to contain measurements between all lines that have an overlap region (which is all of them).

[asm]

Lidar

It is not out of date, they were not consecutive. As dac wrote on 242 ticket but was intended for this project: Should get overlap between more lines. The readme does state only sequential flightlines were used but could go by line ID rather than filename and get more - from the screenshot it is clear there are a lot of overlapping lines.

Will measure some more elevation difference.

[dac]

Also checked (and wrote on 242 ticket).

• Should get overlap between more lines. The readme does state only sequential flightlines were used but could go by line ID rather than filename and get more - from the screenshot it is clear there are a lot of overlapping lines.
• Pitch and roll look good.

[asm]

Lidar

Have measured more elevation differences even for not consecutive flightlines and replaced old readme file. Will mark as ready to deliver and will deliver via hard drive as soon as possible.

[asm]

Delivery

RCD, Lidar and Fenix data dispatched via hard drive on 23/02/2016.

[dac]

Archiving

Data uploaded to CEDA at request of PI.

[dac]

Archiving

Data available from CEDA. Marking as closed.

[wja]

Please ignore previous two comments. They belong (and have been moved to) ticket 595.

[wja]

Owl Processing

• 14 flightlines
• 13 calibration files
• Calibration files in separate directories to raw data
• Calibration files with the closest timestamps to raw data have been symlinked into raw data directories
• All lines have been stitched
• All lines only contain one timestamp - start time at the bottom of the header

Level1b data has been processed. Continuing to process mapped lines.

[wja]

Owl Processing
'GPS Start Time' in stitched line headers more likely resemble stop times. APL config has been generated using start and stop times from Fenix lines.
Currently processing without using real time navigation.

[wja]

Owl Processing
Level1b lines have been split in sections of 10000 lines. An APL config file has been created with split lines
Start and stop times have been calculated from the original headers.
Some lines were failing due to the DEM being smaller than the nav files. A new ASTER DEM has been created with a larger extent to resolve this

• Flightlines which are sucessfully processing: 1 (part 3), 2, 3 ,4 ,5 ,6, 7, 8 (parts 0, 1 & 2)
• Flightlines which have not processed: 1 (parts 1 & 2), 5 (parts 0 & 1), 8 (part 3), 9, 10, 11, 12, 13, 14
  

Unprocessed flightlines all are producing an error because times are outside of the nav file range. Investigating...

[wja]

Owl Processing
APL seems to have taken the start times from the processed level1b lines. These start times are 1 hour and 1 second ahead of the raw data end times (which explains why the later lines failed to process and the SCT values of the other lines were huge). This is now resolved.
Finding SCT values of all lines.

[wja]

Owl Processing
Flightline 1 contains no features in any bands and therefore will be omitted from further processing. Processing has been attempted with calibration files T1637 and T1645.
Owl SCT values:

Flightline	SCT
1	N/A
2	2.96
3	3.36
4	2.90
5	2.34
6	3.08
7	3.82
8	3.00
9	2.74
10	3.06
11	2.54
12	2.70
13	2.78
14	2.96

Creating delivery.

[wja]

Owl Processing
Delivery created, currently editing ReadMe.

[wja]

Owl Delivery
ReadMe edited. Awaiting delivery check.

[asm]

Owl Delivery Check

Started.

[asm]

Owl Delivery Check

-Just looking through the screenshots, it looks like there are problems with SCTs:

-owl 06
-owl 07
-owl 10
-owl 14 part 0?

-No xml files (wja already on it)
-Logsheet:

-Line Ids do not match with the previous delivered logsheet. This complicates a lot to match Fenix already delivered and new owl data. I suggest to deliver the old style logsheet and just add the owl data (File, FPS, IT) on it for consistency

-Readme:

-Logsheet vs flightline name: This is not a table and does not match with the fenix logsheet. Again, I think the logsheet names should follow old style logsheet format
-gdal_translate command: The bands and areas are all 0. Needs an working command as example.
-aplcorr command: Goes out of the page
-aplmask command: Wrong filenames
-After aplmap: also goes to another line

[wja]

Owl Delivery
Reprocessing with the following SCT values:

Flightline	SCT
1	N/A
2	2.96
3	2.32
4	2.90
5	2.34
6	2.15
7	2.84
8	3.00
9	2.74
10	2.07
11	2.54
12	2.70
13	2.78
14	2.96

[wja]

Owl Delivery
Delivery has been updated to remain consistant with hyperspectral delivery.

• Flightlines have not been split.
• Screenshots have been updated.
• All nessessary level1, mapped, navigation and line information is present and named appropriately.
• New logsheet and ReadMes have been generated with flightline IDs that match hyperspectral delivery.
• APL commands have been updated.
  
• project_information *xml had to be generated manually, stop times were need to be input into level1 headers.

Awaiting Delivery Check

[asm]

Owl Delivery Check

Changes look good:
-xml files created correctly
-No SCT errors. Lines are not split but they are not too heavy so there is no need
-Readme problems have been fixed
-APL commands work fine.
-All others checks were made with positive results

Will zip the files and mark project as ready to be delivered once done.

[asm]

Owl Delivery Check

All files have been zipped correctly, marking project as ready to be delivered.

[wja]

Owl Delivered
Data placed on FTP slot 11.
Notification e-mail sent.
Delivery finalised.

[wja]

Owl Archiving
Started archiving.

[wja]

Owl Archiving
Added to PostGIS.
Starting rsync to CEDA (15:18 31/05/2019).

[wja]

Owl Archiving
rsync to CEDA complete (3/6/19).
E-mail sent to confirm it has been received.

[wja]

Owl Archiving
CEDA has confirmed recepit of the reprocessed Owl data.
All data is now archived.