Opened 10 months ago

Last modified 5 months ago

#703 new flight processing

TreeScapes, flight day 193/2023, York

Reported by: wja Owned by:
Priority: immediate Milestone:
Component: Processing: general Keywords:
Cc: Other processors:

Description

Flight over York for the TreeScapes project. Fenix1k, PhaseOne, LiDAR data received.

Change History (33)

comment:1 Changed 10 months ago by wja

Navigation Processing

Downloaded OS network basestations:

  • HOOB
  • SCAU
  • SWAN
  • YEAL

Starting navigation processing.

Last edited 10 months ago by wja (previous) (diff)

comment:2 Changed 10 months ago by wja

Navigation Processing

Processing complete. Quality report metrics look fine.

SBET nav file exported.

comment:3 Changed 9 months ago by wja

Phase One

Started converting images from IIQ to TIF.

There are 1061 photographs in total.

comment:4 Changed 9 months ago by wja

PhaseOne

Conversion to TIF complete.

comment:5 Changed 9 months ago by wja

Hyperspectral Fenix1k Processing

Starting processing.

comment:6 Changed 9 months ago by wja

Using boresight from flight 143 Siena (Fenix1k install had not changed). See ticket 702 for boresight processing information.


Fenix1k boresight:

Pitch0.63
Roll-1.10
Heading1.90

comment:7 Changed 9 months ago by wja

Fenix1k Processing

  • Have needed to flip the data (left to right). This is as if the sensor is fitted backwards (which is shouldn't be). Possible that IMU is backwards. Have contacted Callum for his insight
  • View vectors also need flipping with Fenix1k
  • Siena boresight is not suitable
  • Field of view mask needs extending. Some of right side of the sensor is still obscured by the plane aircraft hatch

comment:8 Changed 9 months ago by wja

PhaseOne Processing

Nav sync is not working properly. Need to investigate further to produce a eventfile of all the image locations and orientations with the post-processed nav.

Real time navigation data is already tagged in each image and look appropriate for Structure from Motion (need to specify that longitude is W not E).

comment:9 Changed 9 months ago by wja

Summary of relevant notes from ticket 704 (198, Cardiff).

  • Confirmed that Fenix1k is not fitted backward, but the IMU was.
  • Need to reprocess navigation data.
  • Boresight likely needs redoing, appears sensor has moved. There is a boresight flight on 190 - have submitted a request to BIGF for basestation data.
  • Possible frame delay or small timing offset - investigating.

comment:10 Changed 9 months ago by wja

PhaseOne Processing

Have run the structure from motion processing on all the images on the lowest quality settings.

Orthomosaic exported. 7cm spatial resolution.

Geoaccuracy looks very good. Real time navigation data tags are fine for this process in this case.

Will start creating GCPs from an elevation data source.

comment:11 Changed 9 months ago by wja

Fenix boresight underway, see ticket #706

comment:12 Changed 8 months ago by wja

PhaseOne Processing

Have created 17 ground control points using features visible in both the 2021 National LiDAR program 1m DSM and the preliminary (uncalibrated) orthomosaic.

Am running metashape on a machine from the grid with a huge amount of RAM. Will share results with PI and proceed to follow the process of creating a DTM if they're happy with the results.

comment:13 Changed 8 months ago by wja

PhaseOne Processing

Processing complete for DSM with ground control points.

Three points were not used for the processing and can be used to assess the accuracy.

Processing starting with the dense cloud generation step took 2.5 days (48 cores, 377G RAM, GPU not utilised).

comment:14 Changed 8 months ago by wja

PhaseOne Processing

DTM produced, some larger buildings still visible but suits science needs (met with Joe this morning).

Need to check accuracy, reproduce DEMs with interpolation on to fill holes and export in UTM 30N.

comment:15 Changed 8 months ago by wja

Fenix1k Processing

Reprocessing with corrected (flipped) navigation and Alconbury (flight 190, ticket #706) boresight values:

Pitch0.45
Roll0.10
Yaw1.50

Timing error still present. Currently finding the best offset.

comment:16 Changed 8 months ago by wja

Navigation Processing

Navigation solution looked bad... have reprocessed using the traditional single basestation method and excluded alot of the additional non-science flying (i.e transit to and from Duxford). Results look good.

SCT offset appears to be +0.04 seconds (like 2023 198 Cardiff).

comment:17 Changed 8 months ago by wja

Fenix1k Processing

Processing all bands

Summary of parameters:

  • Nav data 'flipped' 180
  • Timing offset +0.04 seconds for all lines
  • 65cm resolution pixels when mapped
  • -2 frame delay
  • Swath is masked to account for obstruction from aircraft
  • Using Alconbury (2023 190) boresight

comment:18 Changed 8 months ago by wja

Fenix1k Processing

Level1b processing taking too long. Half of the flightlines remain to be processed. Have increased aplmap memory usage and split processing across two servers to decrease I/O.

comment:19 Changed 8 months ago by wja

PhaseOne Processing

Filling holes in DSM and Orthomosaic.

Fenix1k Processing

Two jobs failed to processing (time out). Reprocessing.

Then will make delivery

comment:20 Changed 8 months ago by wja

Fenix1k Processing

All lines processed to level1b.

Creating delivery

comment:21 Changed 8 months ago by wja

Delivery Create

Delivery created for Fenix1k Level1b data.

Ready for checking.

comment:22 Changed 8 months ago by asm

Fenix1k DC

Started.

comment:23 Changed 8 months ago by wja

PhaseOne

Delivery created for photogrammetry products. Called TreeScapes_23-193-dem-20231005

Ready for checking.

comment:24 Changed 7 months ago by asm

Fenix1k DC

All test passed. Data had lots of underflows and cloud shadows. There were a few flightlines with sunglint over buildings but no overflows.

Zipped all files correctly, I will mark it now as ready to be delivered.

comment:25 Changed 7 months ago by asm

Fenix1k Delivery

Delivered via FTP (slot 6) and notified PI on 13/Oct/2023

comment:26 Changed 6 months ago by wja

Atmospheric Correction

Comparing ATCOR outputs of lines 1 and 22 (same flying direction, overlapping, no significant cloud).

comment:27 Changed 6 months ago by wja

Atmocspheric Correction

Data quality is has been inconsistent but is largely due to the changeable lighting conditions on the cloudy day this data was collected. Unfortunately we do not have AOT or in situ reluctance data to do any validations.

The obstructed field of fiew of the Fenix1k has also presented problems when processing within ATCOR4, as the sensor has been calibrated and charecterised using the entire field of view. These values have been masked, which ATCOR recognised. But the outputs have required a further masking step.

The following bands are being dropped due to noisey values throughout:

  • 1-49
  • 463-467
  • 508-594

These are not unexpected regions, the edge of the sensor is expected to be more noisy and the small section in the SWIR corresponds with water absorption.

The final processing is currently running on the grid: masking, dropping bands and mapping.

PI has also requested to subset the mapped data into 4 bands. I have code ready to do this. Spectral binning information below:

WavelengthFWHM
53010
70510
74010
80010
244520

comment:28 Changed 6 months ago by wja

Atmospheric Correction

New delivery comprising l2 unmapped, l2 mapped has been created. Just waiting for the spectral binning product to finish being generated.

No bands have been removed like previously mentioned. Will need to let users know to take care with wavelengths at either extreme.

comment:29 Changed 6 months ago by wja

Atmospheric Correction

Ready for checking

comment:30 Changed 6 months ago by wja

Delivery Check

@emsu has starting checking the data.

comment:31 Changed 5 months ago by wja

Have updated documentation as suggest by reviewer.

There's an issue with mapping 6 files which has appeared to even influence spectra in limited regions. This has affected lines 3.00, 3.01, 7.00, 9.00, 9.01, 12.00, which have been removed from the delivery.

Zipping files. Then will start transfer to CEDA / JASMIN

comment:32 Changed 5 months ago by wja

Started transfer to JASMIN workspace and FTP slot 6.

PI notified.

comment:33 Changed 5 months ago by wja

Transfer complete.

Delivery has been finalised.

Note: See TracTickets for help on using tickets.