Opened 5 years ago

Closed 3 years ago

#542 closed flight processing (fixed)

MA14/16, flight day 241/2014, Bosnia

Reported by: dac Owned by:
Priority: alpha 4 medium Milestone: 2014 data processing completion
Component: Processing: general Keywords:
Cc: Other processors:

Description (last modified by dap)

Data location: /users/rsg/arsf/arsf_data/2014/flight_data/malaysia/MA14_16-2014_241_Sarajavo

Data arrived from ARSF via network transfer on 19/12/2014

MA14/16 Scientific objective: Unknown

MA14/16 Priority: a4m

PI: Andy Ford

Notes: No RCD data Large difference between fps_set and fps_qpf

Sensors:

Fenix (Requested)
LiDAR (Requested)
RCD (Requested)
FW LiDAR (Requested)

Change History (37)

comment:1 Changed 5 years ago by dac

Basestation position, through PPP is:

Lat 43 49 33.277701
Lon 18 19 59.53932
Height 554.971

comment:2 Changed 5 years ago by dac

Processed Nav data. Solution is acceptable for most of the flightlines (+/- 5 cm lat, +/- 10 cm lon). Ambiguity fix is 1 (should be 2).

Will need refining later but sufficient for now to process hyperspectral data in order to check for potential frame rate problems (high priority).

comment:3 Changed 5 years ago by dac

  • Processed 4 fenix flightlines.
  • Changed fps to fps_qpf in header.
  • Needed to set realtime_nav to false in header, with true flightlines were curved and half had missing lines.
  • Checked overlapping lines (10 and 11), line up well with SCT values of 0.8 and 2 respectively.
  • Confirmed location using Landsat 8 panchromatic scene (15 m res) and GoogleEarth
  • Emailed Ops to notify of frame rate issue.

comment:4 Changed 5 years ago by dac

Processed 4 lidar lines to check they were OK.
No problems observed. Waiting for remaining lines to be downloaded and lidar boresight before proceeding.

comment:5 Changed 4 years ago by dap

  • Description modified (diff)

comment:6 Changed 4 years ago by dap

LiDAR Processing

Started processing all flight lines, outputting them into processing/als50/las, using boresight roll and pitch error values.

comment:7 Changed 4 years ago by dap

LiDAR Processing

The pitch error value 0.000163180 and roll error value -0.006285 seem consistent for all flight lines so will continue with processing the full waveform data with these values.

There were five flight lines which wouldn't process with ALSPP (115939, 124251, 120309, 125209 and 120334). These weren't recorded in the log sheet, so am assuming these contain no useful data.

Additionally, flight line 125234 processed with ALSPP but the resultant file seems to only contain 66 points and the file size is only ~25kB, so I won't include the files associated with this flight line in the data delivery.

comment:8 Changed 4 years ago by dap

  • Description modified (diff)

comment:9 Changed 4 years ago by lah

Fenix Processing
File 1 is not covered by the navigation, but is marked as aborted in the logsheet, so won't be processed.

comment:10 Changed 4 years ago by tec

RCD Processing
Taking a look at the RCD

comment:11 Changed 4 years ago by tec

Navigation Reprocessing
Lat 43 49 33.27693
Lon 18 19 59.53931
Ell 555.057
Antenna Type is LEIAX1202GG apparently.

First flightline 13 03 - 478920
Last flightline 14 32 - 484380
There is a gap in nav though its before start of 1st line
Nav gets rather good after 13:16:40 (479800) but is acceptable from the start.

Would not process in TC, moaning about a.gpb did not exist, no clue why it wanted that specific file. Processed in Grafnav and Pro, defaults were fine.

Nav Done

Last edited 4 years ago by tec (previous) (diff)

comment:12 follow-ups: Changed 4 years ago by lah

Fenix
Dem creating using create_apl_dem.py does not cover most flightlines, so needs reproducing with different boundaries.

comment:13 Changed 4 years ago by tec

RCD Processing
Ready for delivery check

comment:14 in reply to: ↑ 12 Changed 4 years ago by lah

Replying to lah:

Fenix
Dem creating using create_apl_dem.py does not cover most flightlines, so needs reproducing with different boundaries.

Previously generated dem "MA14_16-2014_241_Sarajavo_aster.dem" seems fine, so processing with that.

No idea why create_apl_dem.py doesn't produce a dem covering all lines.

comment:15 in reply to: ↑ 12 Changed 4 years ago by dac

Have looked into this - DEM created from create_apl_dem.py looks like it covers all the lines (opened up DEM and mapped files to check). Could be a related to the frame rate problem.

Will leave for now as old DEM works. If the same problem occurs for future projects could increase the buffer around the navigation data used to calculate the DEM size.

Should also check DEM supplied with lidar data is sufficient size for processing.

Replying to lah:

Fenix
Dem creating using create_apl_dem.py does not cover most flightlines, so needs reproducing with different boundaries.

comment:16 Changed 4 years ago by lah

Fenix
Set fps to fps_set in header files as fps was set to ~4 fps. Also set use_nav to false in config file to prevent disjointed lines stretched beyond flight line (this stretch was probably the reason the lines weren't covered by the dem). SCT correction is not good across whole lines - will be distorted at one end. They also don't match up very well with each other. So far SCTs could be:

-2: 12.02
-3: 3.79
-4: 1.76
-5: 1.74
-6: 16.41

comment:17 Changed 4 years ago by lah

Fenix
dac suggests that data could be improved by estimating frame rates from lidar data, so will not find any more SCTs until we've looked into this.

comment:18 Changed 4 years ago by dap

LiDAR Processing

Reprocessed LAS files with new sol file, similar results were observed. Have also classified the data. Now creating delivery.

comment:19 Changed 4 years ago by dap

LiDAR Processing

Delivery created. Tested automated las elevation differences script on the LAS 1.2 files (still currently in development), measurements seem accurate. Now creating read me.

comment:20 Changed 4 years ago by dap

LiDAR Processing

Complete and ready for delivery check.

comment:21 Changed 4 years ago by dap

RCD Delivery Check complete. Ready for delivery.

comment:22 Changed 4 years ago by tec

LiDAR DC
Is ok. demcompare gave -2.7
Ready for delivery

comment:23 Changed 4 years ago by dac

Fenix

Emailed PI (23/06/2015) to request bounding box of study site so we can make sure SCT values are correct within this. Fenix processing on hold until we hear back,

comment:24 Changed 4 years ago by dac

LiDAR and RCD Delivery

LiDAR and RCD data sent to Andrew Ford on hard drive 26/06/2015.

Delivery via BluRay was requested on application form but we don't have this capability,

comment:25 Changed 4 years ago by lah

Fenix Processing
Study site covers most of flight line, so SCTs can't be focused on a specific region.

Have tried a few frame rates and found that setting the frame rate for lines 2-7 (no line 1) to 23.005 works very well, with very little distortion remaining and flight lines now lining up with each other. Will carry on refining SCTs and processing remaining lines with this estimated fps value.

comment:26 Changed 4 years ago by lah

Found scts for fps = 23.005:

Flightline FENIX
2 12.14
3 4.17
4 1.96
5 1.99
6 16.89
7 1.94
8 3.05
9 1.03
10 1.13
11 1.96
12 6.99
13 2.98
14 25.08
15 7.99
16 1.87
17 3.94
18 93.06
19 3.06
20 2.07

Not continuing with processing until calibration has been sorted. Worth checking scts now though, but given that the frame rate is estimated the geolocation accuracy probably can't get much better without significant effort. Have compared with lidar intensity tif and most of the features are in the right place. Quality doesn't seem too be too far off normal really, for the regions specified by the PI and all but the extreme edges of some flight lines.

Last edited 4 years ago by lah (previous) (diff)

comment:27 Changed 4 years ago by dac

Found a further problem in the SWIR, was recorded at a courser resolution than the VNIR (although not recorded in header). As binning is unknown radiometric calibration is not possible using standard method. Have emailed PI (Andrew Ford) informing him and notified Specim of problem.

comment:28 Changed 4 years ago by dac

Following discussion with Specim VNIR bands are OK to use so will deliver these.

Will try default integration time for SWIR to see if this fixes problems and check sensor was stable along line.

Still no response from PI if only delivering VNIR bands is acceptable.

comment:29 Changed 4 years ago by lah

Fenix Processing
Have created a delivery with just VNIR bands. Spectra for all lines look reasonable. Ready for delivery checking.

comment:30 Changed 4 years ago by dac

Fenix Delivery Check

Starting delivery check.

comment:31 Changed 4 years ago by lah

Fenix SWIR processing
Looking at bands 336 (1883.14 nm) and 347 (1944.25 nm) in the water absorption region of line 2 there is a trend of decreasing signal along the flight line, so it looks like the SWIR detector has not had time to stabilise. Will check other lines with subtract_darkframes_raw.py.

comment:32 Changed 4 years ago by dac

Fenix Delivery Check

All looks fine.

Compared to Py6S simulations spectra seems slightly shifted for an absorption feature around 780 nm but other features look OK. Also compared to 2014/169 data and good match so OK to deliver.

(Edited after rechecking spectra).

Last edited 4 years ago by dac (previous) (diff)

comment:33 Changed 4 years ago by lah

Fenix Delivery
Updated readme with minor clarification in data quality section.

comment:34 Changed 4 years ago by lah

Fenix
Delivered VNIR bands to PI via ftp (arsf3) 30/10/15

comment:35 Changed 3 years ago by dac

Archiving

Checked data. Ready to be uploaded to NEODC.

comment:36 Changed 3 years ago by dac

Archiving

Data uploaded to NEODC.

comment:37 Changed 3 years ago by asm

  • Resolution set to fixed
  • Status changed from new to closed
Note: See TracTickets for help on using tickets.