Opened 5 years ago

Closed 4 years ago

Last modified 17 months ago

#522 closed flight processing (fixed)

GB12/05, flight day 152a/2014, Monks Wood

Reported by: knpa Owned by: mark1
Priority: alpha 4 medium Milestone: 2014 data processing completion
Component: Archiving Keywords:
Cc: dap Other processors:

Description (last modified by tec)

Data location: /users/rsg/arsf/arsf_data/2014/flight_data/uk/GB12_05-2014_152a_Monks_Wood

Data arrived from ARSF via network transfer on 02/06/2014

GB12/05 Scientific objective: unknown

GB12/05 Priority: 8

PI: Ross Hill

Sensors:

Fenix (Requested), delivered 25/09/2014, Archvied 13/05/2015
LiDAR (Requested), delivered 29/10/2014, Archvied 13/05/2015
FW LiDAR (Requested but lower priority), Archvied 13/05/2015
RCD (Not requested?), delivered 17/11/2014, Archvied 13/05/2015

Change History (52)

comment:1 Changed 5 years ago by knpa

This flight can not be processed until we have calibration data (day 141 looks like it's not usable).

comment:2 Changed 5 years ago by mark1

Basestation PPP result:

Latitude = 52 18 26.95697
Longitude = -0 37 28.68639
Ell Height = 97.448

comment:3 Changed 5 years ago by mark1

Navigation processing completed.

comment:4 Changed 5 years ago by mark1

LiDAR processing started.

comment:5 Changed 5 years ago by mark1

Stopped processing as we don't have an up to date lidar calibration for this project yet.

comment:6 Changed 5 years ago by mark1

Renamed fenix raw files to have 'a' sortie in filename, also renamed files 6 sets of files from 'FENIX152a-14-1-1' to 'FENIX152a-14-12', 'FENIX152a-14-1-2' to 'FENIX152a-14-13' etc to be consistent and allow scripts to pick up file names.

comment:7 Changed 5 years ago by mark1

SCT correcting of 17 flightlines has taken ~3.5 hrs (excluding the processing time). Processing final files for further checks and delivery creation.

comment:8 Changed 5 years ago by knpa

Created HS delivery.

comment:9 Changed 5 years ago by stgo

HS delivery check complete:

  • I've changed the aplcorr command to point to the view vector bil file as it was segfaulting, haven't recreated the readme yet though.
  • Line 1 has a bad pixel from band 400 to band 621 in the form of a white line at sample 145 which needs mentioning in the readme. This doesn't effect other lines.
Last edited 5 years ago by stgo (previous) (diff)

comment:10 Changed 5 years ago by knpa

Fixed above issues. Delivering.

comment:11 Changed 5 years ago by knpa

This area is still waiting for vectors.

Have emailed Ross Hill if he wants a pre-vector delivery.

Autoreply says he is away until 4th August.

comment:12 Changed 5 years ago by mark1

Made some slight corrections to read me file

comment:13 Changed 5 years ago by mark1

Fenix timing corrections:

file sct
-1 0.94
-2 0.95
-3 0.95
-4 0.95
-5 0.97
-6 0.95
-7 0.93
-8 0.95
-9 0.94
-10 0.95
-11 0.96
-12 0.95
-13 0.98
-14 0.96
-15 0.96
-16 0.97
-17 0.97

comment:14 Changed 5 years ago by knpa

The HS has been blocked waiting for vectors since 25/07/14.

We've sent an email asking if Dr Hill would like delivery but have not had a reply.

Suggest sending another one when the LiDAR is ready.

comment:15 Changed 5 years ago by knpa

  • Owner set to mark1

comment:16 Changed 5 years ago by lah

Started lidar processing with variable roll and pitch offsets:

File Roll Pitch
-1 -0.006022596-0.00095719
-2 -0.005822596-0.00080719
-3 -0.005822596-0.00080719
-4 -0.005922596-0.00085719
-5 -0.005672596-0.00083219
-6 -0.005972596-0.00090719
-7 -0.005522596-0.00080719
-8 -0.006022596-0.00095719
-9 -0.005422596-0.00075719
-10 -0.006072596-0.00105719
-11 -0.005422596-0.00065719
-12 -0.006122596-0.00125719
-13 -0.005222596-0.00055719
-14 -0.006222596-0.00125719
-15 -0.005122596-0.00055719
-16 -0.006222596-0.00135719
-17 -0.005822596-0.00055719

comment:17 Changed 5 years ago by lah

Lidar ready for delivery check.

comment:18 Changed 5 years ago by dap

Hyperspectral data delivered to Dr Ross Hill, 25/09/2014

comment:19 Changed 5 years ago by dap

  • Description modified (diff)

comment:20 Changed 5 years ago by dap

Beginning LiDAR delivery check

comment:21 Changed 5 years ago by dap

LiDAR Delivery Check

Complete. Found the following problems with the delivery located at GB12_05-152a-lidar-20140917:

  • las1.2 files were produced but directory structure and read me states las1.0 files were produced.
  • Hyperspectral section (page 4) in logsheet is blank - not sure if there is point in keeping it.
  • Screenshots incorrectly named - I've renamed these so that they follow the convention using rename.sh .txt.jpg .jpg
  • Flightline no. 10 is oddly shaped at the end, looks like it could be a range gate issue. This should be commented on in the data quality remarks section of the read me.
  • proj_tidy.sh is complaining about there being no vectors intensity screenshot but, as noted above, PI was sent the pre-vector delivery enquiry and we did not get a response. Will therefore ignore this problem.
  • Navigation files were incorrectly named, I've renamed these from BG12_05-2014-152a.* to BG12_05-2014_152a.*

I'm unsure of how to use demcompare.py so am unsure as to whether the output was good. I'll speak to mark1 about this upon his return.

Please address the above issues and resubmit for delivery check when finished.

comment:22 Changed 5 years ago by lah

  • Renamed to las1.2 to in the directory structure and in read me.
  • Removed the extra page in logsheet. Script really needs changing to avoid doing this every time.
  • Made comment about line 10 in read me.
  • Ran dem compare (demcompare.py -d ./dem/GB12_05-2014_152a-lidar_ASTER-bng.dem --lidar flightlines/discrete_laser/ascii/ -l UKBNG -n ./navigation/BG12_05-2014_152a.sol) with means of -1.67 and -0.22.

comment:23 Changed 5 years ago by dap

LiDAR Delivery Check

Checked the delivery again and found some minor problems:

Edit: demcompare results weren't actually a problem, human error on my behalf!

Last edited 5 years ago by dap (previous) (diff)

comment:24 Changed 5 years ago by dap

LiDAR Delivery Check

Re-checked the results from demcompare against acceptable values according to the delivery checking procedure and noticed that there was no problem with the results. Previous ticket amended accordingly.

Once the issues with the readme and logsheet (noted in previous ticket) are fixed, the dataset can be delivered.

comment:25 Changed 5 years ago by lah

Have changed the link in logsheet.
Every previous logsheet has this issue, so the script needs changing. Not going to do this now.
Still looking into dem issue.

comment:26 Changed 5 years ago by dap

LiDAR data at GB12_05-152a-lidar-20140917/ now ready for delivery.

comment:27 Changed 5 years ago by dap

  • Cc dap added
  • Component changed from Processing: general to Archiving
  • Description modified (diff)

LiDAR data delivered on 29 October 2014 to Dr Hill at Bournemouth University.

comment:28 Changed 5 years ago by benj

  • Description modified (diff)

comment:29 Changed 5 years ago by dap

  • Component changed from Archiving to Processing: general

comment:30 Changed 5 years ago by dap

Beginning RCD Processing

comment:31 Changed 5 years ago by dap

RCD Processing

Had to create camera sol file as one had not been created. Navigation data for line 001 is poor.

comment:32 Changed 5 years ago by dap

RCD Processing at GB12_05-152a-camera-20141113/ now ready for delivery check.

comment:33 Changed 5 years ago by tec

RCD Delivery Check
Started

comment:34 Changed 5 years ago by tec

RCD Delivery Check
Delivery check finished.

No errors, but check with mark1 about the RCD smudge as it appears to be worse.

comment:35 Changed 5 years ago by dap

RCD

Have updated data quality report to include information about further smudges and updated symlink in ~arsf/doc/data_quality_reports/.

Now copying data to hard drive for delivery.

comment:36 Changed 5 years ago by dap

  • Component changed from Processing: general to Archiving
  • Description modified (diff)

RCD data delivered to Dr Hill at Bournemouth University, 17/11/2014

comment:37 Changed 4 years ago by tec

Archiving
Starting Archiving
Transferring to NEODC

Last edited 4 years ago by tec (previous) (diff)

comment:38 Changed 4 years ago by tec

Archiving
Done confirmed as of 13/05/2015 12:10

comment:39 Changed 4 years ago by tec

  • Description modified (diff)
  • Resolution set to fixed
  • Status changed from new to closed

comment:40 Changed 4 years ago by mark1

Fenix data needs reprocessing due to fault with sensor. Incorrect SWIR times recorded in raw files. Have changed tint2 value in hdr files for lines 1-11 from 1.0 & 23 to 2.15 as recommended by Specim.

comment:41 Changed 4 years ago by mark1

Fenix data have been reprocessed.

comment:42 Changed 4 years ago by lah

Fenix re-DC

  • spectra now look fine
  • rest of delivery checks fine
  • screenshots are very pretty, especially those pink trees. What happened to true colour with fenix?

haven't zipped files so can compare reprocessed projects after ground measurements have been looked at

comment:43 Changed 4 years ago by dac

Fenix

Didn't have ground spectra so compared to library green veg spectra using Py6S. Plots saved to processing/fenix_spectra_vs_6s. Looks good, given unknowns in atmospheric parameters and differences between ground spectra. Zipping files and marking as ready to deliver.

comment:44 Changed 4 years ago by asm

Dispatching Data to PI
As requested, had a quick look to the folders, flightlines, screenshots and readme; everything looks OK as DC said.
Started. Delivery method via hard drive.

comment:45 Changed 4 years ago by asm

Dispatching Data to PI
Done: Re-processed data sent to Ross Hill via hard drive

comment:46 Changed 4 years ago by dac

Fenix Delivery

PI confirmed receipt of hard drive (2015-08-25).

comment:47 Changed 4 years ago by dac

Fenix

Hard drive received back from PI (2015-08-27).

comment:48 Changed 4 years ago by lah

Archiving

Archived 07/10/15. Needs email to Wendy.

comment:49 Changed 3 years ago by dac

Archiving

Reprocessed hyperspectral data available from NEODC: http://browse.ceda.ac.uk/browse/neodc/arsf/2014/GB12_05/GB12_05-2014_152a_Monks_Wood/

comment:50 Changed 19 months ago by dac

Atmospheric Correction

Ran through unmapped files in ATCOR using our standard parameters and mapped files. Zipping up mapped data.

comment:51 Changed 19 months ago by dac

Atmospheric Correction

Sent via FTP (slot 12). Noted in email noise in SWIR bands.

comment:52 Changed 17 months ago by dac

Atmospheric Correction

Moved atmospheric correction files to delivery directory and hardlinked files from previous delivery to create a single delivery.

Note: See TracTickets for help on using tickets.