Opened 4 years ago

Closed 3 years ago

#547 closed flight processing (fixed)

RG13/06, flight day 290/2014, Danum Valley

Reported by: dap Owned by:
Priority: alpha 5 Milestone:
Component: Processing: general Keywords:
Cc: Other processors: tec

Description (last modified by benj)

Data location: /users/rsg/arsf/arsf_data/2014/flight_data/malaysia/RG13_06-2014_290_Danum_Valley

Data arrived from network transfer on 16/12/2014.

Scientific objective: Investigating the impact of humans on tropical forests.

Priority: alpha 5

PI: David Coomes

Some corrupt webcam images were deleted:
141017_025820_000.jpg

Sensors:

  • Fenix (requested)
  • Leica FW LIDAR (?)
  • Leica LIDAR (requested)
  • RCD (requested)

Duplicate lines

Process only one for Fenix and RCD - check for which has least cloud cover, failing that choose closest to local noon

  • 1220-1221
  • 1159-1163 (also day 295)
  • 1237-1243

May be three copies of 1239, logsheet says aborted.

Change History (51)

comment:1 Changed 4 years ago by tec

  • Other processors set to tec

Started navigation processing

Latitude4 43 30.55910
Longitude117 36 19.18721
El Height479.808m
Last edited 4 years ago by tec (previous) (diff)

comment:2 Changed 4 years ago by dap

  • Description modified (diff)

comment:3 Changed 4 years ago by tec

RCD Processing
Finished converting tifs

comment:4 Changed 4 years ago by tec

RCD Processing
Started RCD processing

comment:5 Changed 4 years ago by tec

Navigation Processing
Line 001g has quality 3 nav at the time range 02:13:00.00 -> 02:13:10.50
Line 003c has quality 4 nav at the time range 02:19:16.00 -> 02:16:25.50
Other than that the nav has small patches of quality 2 nav, in general this nav data is pretty poor.

comment:6 Changed 4 years ago by tec

RCD Processing
Error when running create_latex_camera_readme.py.

Shall look into this later.

comment:7 Changed 4 years ago by benj

  • Description modified (diff)

Added duplicate line details

comment:8 Changed 4 years ago by tec

RCD Processing
Have a little bit of time so decided to finish this RCD.
Am reconverting raw's to tiffs as the tiff's got messed up somehow.

comment:9 Changed 4 years ago by dap

Taking over RCD Processing

comment:10 Changed 4 years ago by dap

RCD Processing

Removing images 150, 151, 156, 157, 166, 189 - 191, 206, 207, 242, 243, 262, 263, 286, 287, 296, 297 and 300 due to excessive cloud coverage.

comment:11 Changed 4 years ago by dap

RCD Processing

Delivery created, now ready for delivery check (processing/delivery/RG13_06-290-camera-20150112/)

comment:12 Changed 4 years ago by lah

started Fenix processing

  • dem created fine
  • header wavelengths replaced fine

comment:13 Changed 4 years ago by lah

Fenix processing
Not processing inferior duplicates 6(1220), 9(1221), 14 & 17(1159), 15 & 18(1160), 16 & 19(1161), 10(1162), 13(1163), 20(1237), 21(1238), 22 & 27(1239), 28(1240), 29(1241), 30(1242), 31(1243).

1159, 1160 & 1161 best lines are from day 295.

comment:14 Changed 4 years ago by dac

Navigation Processing

Reprocessed base station data

Latitude4 43 30.55909
Longitude117 36 19.18668
El Height480.155 m
Last edited 4 years ago by dac (previous) (diff)

comment:15 Changed 4 years ago by dac

Navigation Processing

Reprocessed navigation data using IPAS Pro.

comment:16 Changed 4 years ago by dap

RCD Processing

RCD has been re-tagged with the new positional information created as a result of the reprocessing of the navigation and basestation data.

RG13_06-290-camera-20150112 is now ready for delivery check

comment:17 Changed 4 years ago by lah

Lidar processing
Roll of -0.0062 and pitch of -0.0004 seems to work for all flightlines, though most lines only have trees to correct with.

comment:18 Changed 4 years ago by lah

Fenix ready for delivery check. Everything in delivery processed with reprocessed nav.

comment:19 Changed 4 years ago by lah

Lidar ready for delivery check.

comment:20 Changed 4 years ago by lah

Fenix

  • Have removed surplus Eagle & Hawk pages from logsheet
  • Have changed Under/overflow from subsubsection to subsection in readme

comment:21 Changed 4 years ago by dac

LiDAR delivery Check

Overall looks good, couple of minor points but ready to deliver once they have been resolved.

  • DEM should be capitalised in Readme figure caption. Have corrected and fixed template
  • Should note in readme if flat terrain wasn't available for calculating height offsets
  • There was a large block of noise in line 17 not classified as noise. I have fixed this but will need copying to FW data / ASCII files.
  • dem_compare gave a difference of 15 m between ASTER data. I suspect there are likely to be some errors associated with the ASTER DEM so this isn't a problem.
  • The AGC value hasn't been exported the the LAS files (which could cause problems if the intensity data need to be normalised but not enough to justify reprocessing.)

comment:22 Changed 4 years ago by dac

RCD delivery check

Starting delivery check

comment:23 Changed 4 years ago by dac

RCD delivery check

Completed delivery check, generally OK. A couple of points:

  • Suggest removing 'Some of the navigation data for this flight are of a low quality. This means that the orthorectification of the images may not be as accurate as we would normally be able to achieve.' The part about ortorectification is incorrect (as we don't othorectify just geotag) and also the quality of the reprocessed navigation data was OK.
  • 207 has more than 50 % cloud cover
  • 238 has more than 50 % cloud cover
  • 247 has more than 50 % cloud cover
  • Remove unneeded delivery folders.

comment:24 Changed 4 years ago by dap

RCD

Rewrote data quality remarks in the read me and removed the part about bad navigation data. Also removed the previous deliveries.

RCD dataset now ready for delivery.

comment:25 Changed 4 years ago by lah

Lidar Processing

  • Have transferred classification in line 17 to fw & remade ascii (+made windows compatible).
  • Have added extra text to readme.
  • Have reprocessed the fw to get the AGC value anyway, and copied in classification.
  • Have not recreated dem without line 17 noise.

comment:26 Changed 4 years ago by dac

LiDAR Delivery Check

Updates look good. Ready to deliver.

comment:27 Changed 4 years ago by dac

RCD and LiDAR Delivery

RCD and LiDAR data sent to PI via FTP 03/03/2015

comment:28 Changed 4 years ago by lah

Fenix
Reprocessed with new bad pixel file, but bad pixels still remain. Will ask Mark what's going on.

comment:29 Changed 4 years ago by lah

Fenix
Reprocessing again with bad pixels list from all flight lines.

comment:30 Changed 4 years ago by lah

Fenix Ready for DC
Had to use original screenshot in readme as latex couldn't use thumbnail at any sensible resolution.

comment:31 Changed 4 years ago by dac

Fenix Delivery Check

Starting delivery check.

comment:32 Changed 4 years ago by dac

Fenix Delivery Check

Looks fine, couple of small points:

  • Should note in the readme that not all lines acquired have been processed (due to duplicate coverage).
  • The data quality report is for the 2014 calibration, I'm not sure if this needs to be updated as the 2015 calibration was used.
  • Fixed thumbnail and remade readme.

Other than these ready to deliver.

comment:33 Changed 4 years ago by lah

Fenix
Have added note to readme. Just need new data quality report now.

comment:34 Changed 4 years ago by lah

Fenix ready for DC
Have recreated with newest calibration. GDAL env variable doesn't seem to have affected screenshots.

comment:35 Changed 4 years ago by dac

Fenix DC

Rechecking Fenix delivery.

comment:36 Changed 4 years ago by dac

Fenix DC

Looks good. check_apl_cmd is still running but outputs so far look OK and everything else is fine so suggest start zipping files.

Proj tidy printed error about wavelengths not matching 2014 cal (which isn't surprising as the 2015 cal was used).

Don't forget to remove old delivery.

comment:37 Changed 4 years ago by dac

Fenix DC

Output from check_apl_cmd looks fine. Files zipped and ready to deliver.

comment:38 Changed 4 years ago by lah

Fenix
Have changed Danum Valley to BALI SAFE to be consistent with the logsheet. (In readme and loghseet name)

comment:39 Changed 4 years ago by lah

PI notified of delivery 05/05/2015 (Fenix, Lidar & RCD)

comment:40 Changed 4 years ago by dac

Fenix Processing

Replaced wavelengths in header with those from updated calibration. Started reprocessing.

comment:41 Changed 3 years ago by stgo

Fenix Processing

Have processed flightlines with new calibration

comment:42 Changed 3 years ago by gej

Fenix Delivery Check

Data Quality report needs changing to 2015 along with the project_metadata.
fenix_viewvector_list.txt is missing according to proj_tidy.sh

Spectra looks okay to me, a few high bits compared to py6s, and also a few low bits but not too low.

Have removed fodis directory

comment:43 Changed 3 years ago by stgo

Fenix

Updated data quality and project metadata

created fenix vectors, weird that the delivery creator skipped over this.

comment:44 Changed 3 years ago by lah

Fenix DC - spectra check
Have checked all lines and no major problems. There is a spike at the end of the VNIR and a hump at the start of the SWIR, both of which are within the mask region. Comparing spectra to Py6S simulations, some fenix spectra are lower than predicted, but not by an alarming amount and never all from the same line. There is cloud in most flight lines that could account for the lower signal level.

The spectral shift is within 5 nm compared to previous data (2014 169). As this is a Malaysian flight, the wavelength shift may be slightly off the 2015 calibration wavelengths, most likely due to environmental conditions. This issue is currently under investigation.

comment:45 Changed 3 years ago by gej

Fenix DC

Everything looks okay now. proj_tidy is complaining about the logsheet name (BALI_SAFE not Danum_Valley). But I don't think that's a problem since on the processing page it's down as SAFE area.

comment:46 Changed 3 years ago by stgo

Fenix DC

The logsheet is named as it was in the original delivery. Is there anything else holding this back?

comment:47 Changed 3 years ago by gej

Fenix DC

All good now, zipping mapped files then ready for delivery.

comment:48 Changed 3 years ago by gej

Fenix DC

Mapped files zipped, ready for delivery

comment:49 Changed 3 years ago by gej

Fenix Delivered to David Coomes via FTP on arsf1, November 18th 2015

comment:50 Changed 3 years ago by gej

Started Archiving

comment:51 Changed 3 years ago by dac

  • Resolution set to fixed
  • Status changed from new to closed

Archiving

Raw and archived data available from NEODC: http://browse.ceda.ac.uk/browse/neodc/arsf/raw_data/2014b/RG13_06-2014_290_Danum_Valley_raw

Marking as closed.

Note: See TracTickets for help on using tickets.