Opened 15 months ago
Last modified 10 months ago
#703 new flight processing
TreeScapes, flight day 193/2023, York
Reported by: | wja | Owned by: | |
---|---|---|---|
Priority: | immediate | Milestone: | |
Component: | Processing: general | Keywords: | |
Cc: | Other processors: |
Description
Flight over York for the TreeScapes project. Fenix1k, PhaseOne, LiDAR data received.
Change History (33)
comment:1 Changed 15 months ago by wja
comment:2 Changed 15 months ago by wja
Navigation Processing
Processing complete. Quality report metrics look fine.
SBET nav file exported.
comment:3 Changed 14 months ago by wja
Phase One
Started converting images from IIQ to TIF.
There are 1061 photographs in total.
comment:4 Changed 14 months ago by wja
PhaseOne
Conversion to TIF complete.
comment:5 Changed 14 months ago by wja
Hyperspectral Fenix1k Processing
Starting processing.
comment:6 Changed 14 months ago by wja
Using boresight from flight 143 Siena (Fenix1k install had not changed). See ticket 702 for boresight processing information.
Fenix1k boresight:
Pitch | 0.63 |
Roll | -1.10 |
Heading | 1.90 |
comment:7 Changed 14 months ago by wja
Fenix1k Processing
- Have needed to flip the data (left to right). This is as if the sensor is fitted backwards (which is shouldn't be). Possible that IMU is backwards. Have contacted Callum for his insight
- View vectors also need flipping with Fenix1k
- Siena boresight is not suitable
- Field of view mask needs extending. Some of right side of the sensor is still obscured by the plane aircraft hatch
comment:8 Changed 14 months ago by wja
PhaseOne Processing
Nav sync is not working properly. Need to investigate further to produce a eventfile of all the image locations and orientations with the post-processed nav.
Real time navigation data is already tagged in each image and look appropriate for Structure from Motion (need to specify that longitude is W not E).
comment:9 Changed 14 months ago by wja
Summary of relevant notes from ticket 704 (198, Cardiff).
- Confirmed that Fenix1k is not fitted backward, but the IMU was.
- Need to reprocess navigation data.
- Boresight likely needs redoing, appears sensor has moved. There is a boresight flight on 190 - have submitted a request to BIGF for basestation data.
- Possible frame delay or small timing offset - investigating.
comment:10 Changed 13 months ago by wja
PhaseOne Processing
Have run the structure from motion processing on all the images on the lowest quality settings.
Orthomosaic exported. 7cm spatial resolution.
Geoaccuracy looks very good. Real time navigation data tags are fine for this process in this case.
Will start creating GCPs from an elevation data source.
comment:11 Changed 13 months ago by wja
Fenix boresight underway, see ticket #706
comment:12 Changed 13 months ago by wja
PhaseOne Processing
Have created 17 ground control points using features visible in both the 2021 National LiDAR program 1m DSM and the preliminary (uncalibrated) orthomosaic.
Am running metashape on a machine from the grid with a huge amount of RAM. Will share results with PI and proceed to follow the process of creating a DTM if they're happy with the results.
comment:13 Changed 13 months ago by wja
PhaseOne Processing
Processing complete for DSM with ground control points.
Three points were not used for the processing and can be used to assess the accuracy.
Processing starting with the dense cloud generation step took 2.5 days (48 cores, 377G RAM, GPU not utilised).
comment:14 Changed 13 months ago by wja
PhaseOne Processing
DTM produced, some larger buildings still visible but suits science needs (met with Joe this morning).
Need to check accuracy, reproduce DEMs with interpolation on to fill holes and export in UTM 30N.
comment:15 Changed 13 months ago by wja
Fenix1k Processing
Reprocessing with corrected (flipped) navigation and Alconbury (flight 190, ticket #706) boresight values:
Pitch | 0.45 |
Roll | 0.10 |
Yaw | 1.50 |
Timing error still present. Currently finding the best offset.
comment:16 Changed 13 months ago by wja
Navigation Processing
Navigation solution looked bad... have reprocessed using the traditional single basestation method and excluded alot of the additional non-science flying (i.e transit to and from Duxford). Results look good.
SCT offset appears to be +0.04 seconds (like 2023 198 Cardiff).
comment:17 Changed 13 months ago by wja
Fenix1k Processing
Processing all bands
Summary of parameters:
- Nav data 'flipped' 180
- Timing offset +0.04 seconds for all lines
- 65cm resolution pixels when mapped
- -2 frame delay
- Swath is masked to account for obstruction from aircraft
- Using Alconbury (2023 190) boresight
comment:18 Changed 13 months ago by wja
Fenix1k Processing
Level1b processing taking too long. Half of the flightlines remain to be processed. Have increased aplmap memory usage and split processing across two servers to decrease I/O.
comment:19 Changed 13 months ago by wja
PhaseOne Processing
Filling holes in DSM and Orthomosaic.
Fenix1k Processing
Two jobs failed to processing (time out). Reprocessing.
Then will make delivery
comment:20 Changed 13 months ago by wja
Fenix1k Processing
All lines processed to level1b.
Creating delivery
comment:21 Changed 13 months ago by wja
Delivery Create
Delivery created for Fenix1k Level1b data.
Ready for checking.
comment:22 Changed 13 months ago by asm
Fenix1k DC
Started.
comment:23 Changed 12 months ago by wja
PhaseOne
Delivery created for photogrammetry products. Called TreeScapes_23-193-dem-20231005
Ready for checking.
comment:24 Changed 12 months ago by asm
Fenix1k DC
All test passed. Data had lots of underflows and cloud shadows. There were a few flightlines with sunglint over buildings but no overflows.
Zipped all files correctly, I will mark it now as ready to be delivered.
comment:25 Changed 12 months ago by asm
Fenix1k Delivery
Delivered via FTP (slot 6) and notified PI on 13/Oct/2023
comment:26 Changed 11 months ago by wja
Atmospheric Correction
Comparing ATCOR outputs of lines 1 and 22 (same flying direction, overlapping, no significant cloud).
comment:27 Changed 10 months ago by wja
Atmocspheric Correction
Data quality is has been inconsistent but is largely due to the changeable lighting conditions on the cloudy day this data was collected. Unfortunately we do not have AOT or in situ reluctance data to do any validations.
The obstructed field of fiew of the Fenix1k has also presented problems when processing within ATCOR4, as the sensor has been calibrated and charecterised using the entire field of view. These values have been masked, which ATCOR recognised. But the outputs have required a further masking step.
The following bands are being dropped due to noisey values throughout:
- 1-49
- 463-467
- 508-594
These are not unexpected regions, the edge of the sensor is expected to be more noisy and the small section in the SWIR corresponds with water absorption.
The final processing is currently running on the grid: masking, dropping bands and mapping.
PI has also requested to subset the mapped data into 4 bands. I have code ready to do this. Spectral binning information below:
Wavelength | FWHM |
530 | 10 |
705 | 10 |
740 | 10 |
800 | 10 |
2445 | 20 |
comment:28 Changed 10 months ago by wja
Atmospheric Correction
New delivery comprising l2 unmapped, l2 mapped has been created. Just waiting for the spectral binning product to finish being generated.
No bands have been removed like previously mentioned. Will need to let users know to take care with wavelengths at either extreme.
comment:29 Changed 10 months ago by wja
Atmospheric Correction
Ready for checking
comment:30 Changed 10 months ago by wja
Delivery Check
@emsu has starting checking the data.
comment:31 Changed 10 months ago by wja
Have updated documentation as suggest by reviewer.
There's an issue with mapping 6 files which has appeared to even influence spectra in limited regions. This has affected lines 3.00, 3.01, 7.00, 9.00, 9.01, 12.00, which have been removed from the delivery.
Zipping files. Then will start transfer to CEDA / JASMIN
comment:32 Changed 10 months ago by wja
Started transfer to JASMIN workspace and FTP slot 6.
PI notified.
comment:33 Changed 10 months ago by wja
Transfer complete.
Delivery has been finalised.
Navigation Processing
Downloaded OS network basestations:
Starting navigation processing.