Opened 5 years ago

Closed 2 years ago

#534 closed flight processing (fixed)

GB12/03, flight day 219a/2014, Loch Leven

Reported by: knpa Owned by: benj
Priority: alpha 4 medium Milestone: 2014 data processing completion
Component: Archiving Keywords:
Cc: Other processors:

Description (last modified by tec)

Data location: /users/rsg/arsf/arsf_data/2014/flight_data/uk/GB12_03-2014_219a_Loch_Leven/

Data arrived from ARSF via SATA disk on 14/08/2014

GB12/03 Scientific objective: Improve lake remote sensing algorithms

GB12/03 Priority: 9

PI: Peter Hunter

Notes: There is OWL data There is FW LiDAR

Sensors:

Camera (Not Requested) Delivered 11/11/2014
Fenix (Requested) Delivered 11/11/2014
LiDAR (Not Requested) Delivered 11/11/2014
FW LiDAR (Not Requested) Delivered 11/11/2014
Owl (Not Requested)

Change History (44)

comment:1 Changed 5 years ago by tec

RCD

Started converting RAW images to TIFF

Last edited 5 years ago by tec (previous) (diff)

comment:2 Changed 5 years ago by tec

NAV

Started navigation processing

DUDE Basestation Data
Latitude56 27 50.14853
Longitude-2 52 38.33568
El Height57.514m


EDIN Basestation Data
Latitude55 55 29.23389
Longitude-3 17 41.23162
El Height119.064m


KILN Basestation Data
Latitude56 27 17.79089
Longitude-4 19 14.64556
El Height232.141m
Last edited 5 years ago by tec (previous) (diff)

comment:3 Changed 5 years ago by tec

RCD

Removed RCD Images
07083510100001G4.jpg
07083510100002G3.jpg
07083510100003G3.jpg
07083510100013G3.jpg
07083510100023G3.jpg
07083510100074G3.jpg

Waiting on Nav to continue

Last edited 5 years ago by tec (previous) (diff)

comment:4 Changed 5 years ago by tec

NAV
Elevation Mask set to 16

Not great solution over lines 3, 2 and 1, Latitude and Longitude separation over 0.05

Last edited 5 years ago by tec (previous) (diff)

comment:5 Changed 5 years ago by tec

NAV
Nav processing finished, just copying the files back now.

comment:6 Changed 5 years ago by knpa

  • Description modified (diff)

comment:7 Changed 4 years ago by tec

RCD
RCD Finished, awaiting to be delivery checked.

comment:8 Changed 4 years ago by tec

FENIX
Starting fenix processing.

comment:9 Changed 4 years ago by dap

RCD

Quality checking delivery.

comment:10 Changed 4 years ago by dap

RCD Delivery Check

Found the following minor issues, which need to be addressed:

  • Data quality remarks "Some cloud cover...": Doesn't follow on from previous paragraph. I would recommend changing this to something like "Some images are affected by cloud coverage but were not removed if there was less than approximately XX% cloud coverage. Some images were also affected by shadows due to this cloud coverage."
  • Sample output from exiftool in Readme could start on page 4. Fix this by inserting the \newpage command before the \rule command.
  • As above with "Image Collection" section.
  • Images 1, 14 and 15, in my opinion, look overexposed - I would comment on these in the data quality remarks or remove the images and recreate the delivery.

Please address these issues, then resubmit for delivery checking.

comment:11 Changed 4 years ago by tec

RCD
Fixed issues raised in delivery check. Ready for delivery check again.

comment:12 Changed 4 years ago by dap

Rechecking RCD Delivery Quality

comment:13 Changed 4 years ago by dap

RCD Delivery Check

Just a couple of issues:

  • Images were overexposed as opposed to underexposed. Please change data quality remarks to reflect this.
  • Please put latest .tex version of the readme in the processing/rcd delivery so that it can easily found and edited at a later date if necessary.

Once the above issues have been addressed, camera data will be ready for delivery.

comment:14 Changed 4 years ago by tec

RCD
Done.

comment:15 Changed 4 years ago by dac

Started processing Fenix data to test boresight values. Fenix wavelength scale in header files replaced (originals moved).

SCT values for lines 2, 3 and 4 ~ 0.94.

Last edited 4 years ago by dac (previous) (diff)

comment:16 Changed 4 years ago by tec

Fenix
Starting from scratch testing new database scripts

comment:17 Changed 4 years ago by lah

Fenix
Started processing with initial set of SCTs using initial boresight:

1 0.94
2 0.96
3 0.94
4 0.94
5 0.96
6 0.96*
7 0.96
9 0.95
10 0.94*

Lines 6 and 10 are still wobbly so need SCTs outside initial range. Others also don't look great in mosaic, so looking for new set.

comment:18 Changed 4 years ago by lah

Fenix
Found final SCTs, making delivery.

1 0.98
2 0.94
3 0.94
4 0.94
5 0.94
6 0.04
7 0.96
8 0.96
9 0.95
10 -0.06

comment:19 Changed 4 years ago by lah

Fenix delivery created, ready for delivery check.

comment:20 Changed 4 years ago by tec

Hyperspectral Delivery Check
Starting DC

comment:21 Changed 4 years ago by tec

Hyperspectral Delivery Check

  • Logsheet notes fields are empty.
  • Fixed a typo in readme.
  • Fenix pixel size is too high in my opinion, can you send it though with a pixel size of 2.2

comment:22 Changed 4 years ago by lah

Fenix

  • Removed empty page from logsheet.
  • Reprocessed with pixel size of 2.2 and recreated delivery.

comment:23 Changed 4 years ago by tec

Hyperspectral Delivery Check

  • flightlines/line_information missing f219a071b.xml
  • logsheet notes fields are empty, (it needs regenerating run generateLogsheet)
  • Readme missing, if it is the 20141013 one then the table listing the logsheet name vs file identifier, are you sure its in the right order?

Other than thats its done

comment:24 Changed 4 years ago by lah

Fenix again

  • Reprocessed line 7 to get xml file, no idea where that went to.
  • Added notes to loghseet.
  • Copied across ReadMe. Table is correct, lines nos. are backwards / line files start from the right.

comment:25 Changed 4 years ago by tec

Hyperspectral Delivery Check
You didn't copy the new logsheet into the delivery, did that for you.

Delivery check complete. zip_mapped.sh currently running, once thats done it will be ready to be delivered to peter hunter.

comment:26 Changed 4 years ago by tec

Hyperspectral Delivery Check
zip_mapped.sh finished. Ready for delivery

comment:27 Changed 4 years ago by tec

LiDAR
Starting LiDAR

comment:28 Changed 4 years ago by tec

LiDAR
Roll Values

Flight line Roll Pitch
102119 -0.0064250.000163180
102714 -0.0064250.000163180
103316 -0.0064250.000163180
103944 -0.0063250.000163180
104645 -0.0062750.000163180
105542 -0.0062750.000163180
110214 -0.0062750.000163180
111031 -0.0062750.000163180
111835 -0.0062750.000163180
112610 -0.0064250.000480180
Last edited 4 years ago by tec (previous) (diff)

comment:29 Changed 4 years ago by tec

RCD Processing
Ready for delivery.

comment:30 Changed 4 years ago by dap

Beginning LiDAR Delivery Check

comment:31 Changed 4 years ago by dap

LiDAR Delivery Check

  • ASCII files in ascii/ need to be unix2dos'd
  • Elevation differences seem to be very large and looks like there could still be roll/pitch errors between lines 09 and 10 especially. Speak to dac about this.
  • There seems to be a lack of features usually used to check roll errors, which isn't mentioned in read me. Please add some remarks to that effect in the read me.
  • In data quality remarks please make it clear what files you mean by "3, 4 and 5" by putting the leading zeros in, as the logsheet flight line names and the processed file names are similar.

Note, full waveform data has not yet been checked. Will run laszip overnight and check with Wave Viewer tomorrow.

Last edited 4 years ago by dap (previous) (diff)

comment:32 Changed 4 years ago by dap

LiDAR Delivery Check

laszip produced 20 files, as expected. Wave Viewer showed that some waveforms in some of the flightlines had abnormally high peaks, possibly due to cloud coverage.

Delivery check complete. Please address issues in previous comment and resubmit for delivery check when finished.

comment:33 Changed 4 years ago by tec

LiDAR
Ready for delivery check, delivery dir ending in 30

comment:34 Changed 4 years ago by dap

Checking LiDAR data delivery

comment:35 Changed 4 years ago by dap

LiDAR Delivery Check

Found the following problems:

  • las1.2 ascii files need to have CRLF line endings - fixed with
    find -name "*.txt" | xargs unix2dos from the delivery directory.
  • Roll/pitch errors look better but these data seem to have a considerably large height offset and should probably be commented on in the read me.
  • No vectors screenshots - check with PI that delivery is ok to send before vector delivery if this hasn't been done already for this project.
  • Multiple deliveries found in processing/delivery/, please delete old one when finished with.
  • las1.3 files have some waveforms missing in the middle records (not at start or end) and some of the waveforms have abnormally high peaks. Assuming this is due to the fact that the flight lines were flown at low altitudes (c.f. 175b LiDAR 537) and large amounts of noise. May be worth noting in the data quality remarks in the read me. (Note, I looked at pre-classified las1.3 files in Wave Viewer and these showed the same issues).
Last edited 4 years ago by dap (previous) (diff)

comment:36 Changed 4 years ago by tec

LiDAR
Done, all 3 datasets can be delivered but need to ask PI if they want vectors.

comment:37 Changed 4 years ago by tec

Delivery
RCD, Fenix and LiDAR delivered to Pete on 11/11/2014

comment:38 Changed 4 years ago by tec

  • Component changed from Processing: general to Archiving
  • Description modified (diff)
  • Owner set to benj

comment:39 Changed 4 years ago by mark1

Fenix
Fenix data needs reprocessing due to data being binned spectrally where processing used an incorrect algorithm.

Data have been reprocessed.

comment:40 Changed 4 years ago by stgo

Fenix re-DC
Have checked the spectra against the example in the wiki, looks correct.

No issues with delivery, ready to deliver

comment:41 Changed 4 years ago by dac

Fenix

Sent reprocessed data on USB hard drive to Peter Hunter on 31/07/2015

comment:42 Changed 3 years ago by stgo

Archiving

Starting archiving

comment:43 Changed 3 years ago by dac

Archiving

Processed data available from NEODC (http://browse.ceda.ac.uk/browse/neodc/arsf/2014/GB12_03/GB12_03-2014_219a_Loch_Leven). Raw data not available - uploading now.

comment:44 Changed 2 years ago by dac

  • Resolution set to fixed
  • Status changed from new to closed

Archiving

Raw data now available from NEODC.

Marking as closed.

Note: See TracTickets for help on using tickets.