Opened 10 years ago
Last modified 9 years ago
#551 new flight processing
RG13/06, flight day 295/2014, SAFE Area
Reported by: | dap | Owned by: | |
---|---|---|---|
Priority: | alpha 5 | Milestone: | |
Component: | Processing: general | Keywords: | |
Cc: | Other processors: | tec, dap |
Description (last modified by dap)
Data location: /users/rsg/arsf/arsf_data/2014/flight_data/malaysia/RG13_06-2014_295_Danum_Valley
Data arrived from network transfer on 17/12/2014.
Scientific objective: Investigating the impact of humans on tropical forests.
Priority: alpha 5
PI: David Coomes
No owl data present
Sensors:
- Fenix (requested)
- Leica FW LIDAR (delivered 22/05/2015)
- Leica LIDAR (delivered 22/05/2015)
- RCD (requested)
Duplicate lines
Process only one for Fenix and RCD - check for which has least cloud cover, failing that choose closest to local noon
- 1159-1161 (may be 3? Check vs. day 290)
- 1250-1252
- 1185-1189 (day 294)
- 1188-1200 (day 298, 1188-1189 repeated 3 times - also day 294)
Change History (52)
comment:1 Changed 10 years ago by tec
comment:2 Changed 10 years ago by tec
- Other processors set to tec
comment:3 Changed 10 years ago by tec
Processed in IPAS Pro
C/A 1.20
ell 12.5
max freq distance 80km
after 1:28:20 the lat/long positional separation goes from between 0±0.05 to 0±0.1
Nav done
comment:5 Changed 10 years ago by lah
Fenix duplicates of 290
1159 - line 26 is best
1160 - line 24 is best
1161 - line 25 is best
These have not been processed as part of 290, so need to be processed as 295.
comment:6 Changed 10 years ago by tec
Nav
Data in ~/arsf_data/2014/flight_data/malaysia/RG13_06-Basestation_data/RINEX DATA SABAH/Rinex/temporary_gps_dataz/295, move out into project folder once processed.
comment:7 Changed 10 years ago by dap
RCD Processing started
comment:8 Changed 10 years ago by tec
Navigation Processing
Antenna Height 1.31
Lat 4 43 30.56119
Lon 117 36 19.18667
Ell 480.179
comment:9 Changed 10 years ago by tec
Navigation Processing
From first flight line until CROSS_004 navigation data is horrible, nothing I can do about that.
comment:10 Changed 10 years ago by tec
RCD Processing
Starting
comment:11 Changed 10 years ago by tec
RCD Processing
Removing images 352, 386, 387, 406, 425.
comment:12 Changed 10 years ago by tec
RCD Processing
Ready for DC
comment:13 Changed 10 years ago by dap
RCD Delivery Check
Complete. Looks fine, just had to change the data quality remarks to make them read a little better.
Marking as ready for delivery.
comment:14 Changed 10 years ago by tec
Delivery
Rsyncing to FTP:arsf1 DONE
comment:15 Changed 10 years ago by lah
PI notified of RCD delivery 05/05/2015 (RCD only)
comment:16 Changed 10 years ago by dap
LiDAR Processing
Roll and pitch offsets found. The values seem to alternate between two numbers for the majority of the flight lines.
Roll and pitch values as follows:
Flight line | Roll error | Pitch error |
230824 | -0.00163180 | -0.006375 |
231103 | +0.00063180 | -0.005975 |
231339 | -0.00163180 | -0.006375 |
231654 | +0.00063180 | -0.005975 |
232347 | -0.00163180 | -0.006375 |
232655 | +0.00073180 | -0.005975 |
233006 | -0.00163180 | -0.006375 |
233313 | +0.00073180 | -0.005975 |
233616 | -0.00163180 | -0.006375 |
233934 | +0.00073180 | -0.005975 |
234230 | -0.00163180 | -0.006375 |
234547 | +0.00073180 | -0.005975 |
234914 | -0.00163180 | -0.006375 |
235208 | +0.00073180 | -0.005975 |
235520 | -0.00163180 | -0.006375 |
235840 | +0.00073180 | -0.005975 |
000142 | -0.00163180 | -0.006375 |
000451 | +0.00073180 | -0.005975 |
000815 | -0.00163180 | -0.006375 |
001128 | +0.00073180 | -0.006675 |
001455 | -0.00163180 | -0.006375 |
002256 | -0.00163180 | -0.006375 |
002553 | +0.00073180 | -0.005975 |
002828 | -0.00163180 | -0.006375 |
003138 | +0.00073180 | -0.005975 |
003527 | -0.00163180 | -0.006375 |
003904 | -0.00163180 | -0.006375 |
004150 | +0.00073180 | -0.005975 |
004444 | -0.00163180 | -0.006375 |
004918 | +0.00073180 | -0.005975 |
005220 | -0.00163180 | -0.006375 |
005452 | +0.00073180 | -0.005975 |
010247 | -0.00163180 | -0.006375 |
010555 | +0.00073180 | -0.005975 |
010826 | -0.00163180 | -0.006375 |
011115 | +0.00073180 | -0.005975 |
011350 | -0.00163180 | -0.006375 |
011623 | +0.00073180 | -0.005975 |
011903 | -0.00163180 | -0.006375 |
012330 | +0.00073180 | -0.005975 |
012628 | -0.00163180 | -0.006375 |
012937 | +0.00073180 | -0.005975 |
013220 | -0.00163180 | -0.006375 |
013529 | +0.00073180 | -0.005975 |
013824 | -0.00163180 | -0.006375 |
014130 | +0.00073180 | -0.005975 |
014441 | -0.00163180 | -0.006375 |
014732 | +0.00073180 | -0.005975 |
015038 | -0.00163180 | -0.006375 |
015605 | -0.00163180 | -0.006375 |
comment:17 Changed 10 years ago by dap
LiDAR Processing
Corrected files (incl. full waveform) have been produced and moved LAS1.2 files to las/corrected and LAS1.3 files to als50/las-fw. Now classifying LAS1.2 files automatically using the isolated 10 5 algorithm (outputting to als50/las-classified).
comment:18 Changed 10 years ago by dap
LiDAR Processing
All files have been classified automatically and manually and classifications have been copied to the LAS1.3 files. Now creating delivery.
comment:19 Changed 10 years ago by lah
Fenix duplicates
Please process 1185-1189 as these are not in 294. 1188-1200 are also not in 294.
comment:20 Changed 10 years ago by tec
Fenix
Starting processing
comment:21 Changed 10 years ago by dap
LiDAR Processing
Delivery and read me have been created. Now ready for delivery check.
comment:22 Changed 10 years ago by lah
Fenix Duplicates
Lines 42-51 (1191-1200) do not need to be processed as day 298 has less cloudy duplicates.
comment:23 Changed 10 years ago by dap
Fenix Duplicates
Lines 6-20 (1205-1219) do not need to be processed as day 297 (ticket:552) has less cloudy duplicates.
comment:24 follow-up: ↓ 25 Changed 10 years ago by lah
Lidar DC
- Pitch & roll look ok.
- Classification:
- Most lines ok with occasional lone points not classified as noise.
- Lines 17, 28, 29 & 41 have missed some rain. Worth reclassifying these?
- Any idea why the elevation offsets are so consistently high?
- Changed las1.0 folder to las1.2
- Changed text files to crlf
- Test delivery folder needs to be deleted
- Looks like AGC value has not been saved
- demcompare: masked mean = 17.3114
- no errors with laszip on fw
- fws look fine in waveviewer
- removed an odd flie from logsheet directory (.nfs000...)
comment:25 in reply to: ↑ 24 Changed 10 years ago by dap
Replying to lah:
Lidar DC
- Pitch & roll look ok.
- Classification:
- Most lines ok with occasional lone points not classified as noise.
- Lines 17, 28, 29 & 41 have missed some rain. Worth reclassifying these?
I've had another look at the classifications and reclassified the missed rain. The number of missed points was quite small so I haven't regenerated the DEMs but have copied the new classifications to the full waveform files.
- Any idea why the elevation offsets are so consistently high?
I did notice this but wasn't sure why. dac is investigating this.
- Changed las1.0 folder to las1.2
- Changed text files to crlf
- Test delivery folder needs to be deleted
Moving this to my network scratch space (was using it for testing script changes)
- Looks like AGC value has not been saved
I've ran lasinfo on some of the LAS 1.2 files, the AGC values are in 'point_source_ID' field in them and the AGC values are present in both the 'user_data' and 'point_source_ID' fields of the LAS1.3 files. dac said this is OK.
- demcompare: masked mean = 17.3114
dac said this is fine.
- no errors with laszip on fw
- fws look fine in waveviewer
- removed an odd flie from logsheet directory (.nfs000...)
comment:26 Changed 10 years ago by dap
LiDAR
dac has had a look at the elevation differences and we suspect this is due to the poor quality of the navigation data (as mentioned in comment:9), but there is nothing we can do about it.
comment:27 Changed 10 years ago by dap
LiDAR
I've added a sentence in the read me about the high elevation offsets.
LiDAR dataset ready for a recheck if necessary or delivery if not.
comment:28 follow-up: ↓ 29 Changed 10 years ago by lah
LiDAR
Just noticed the screenshot names are 2014-295 rather than 2014_295, but we have delivered both forms, so probably nothing to worry about.
comment:29 in reply to: ↑ 28 Changed 10 years ago by lah
Replying to lah:
LiDAR
Just noticed the screenshot names are 2014-295 rather than 2014_295, but we have delivered both forms, so probably nothing to worry about.
actually they should be like that, so never mind. reaady to deliver
comment:30 Changed 10 years ago by dap
rsyncing LiDAR delivery to FTP server (arsf1)
LiDAR delivery rsync complete and PI has been notified of LiDAR delivery (22/05/2015).
comment:31 Changed 10 years ago by dap
- Description modified (diff)
- Summary changed from RG13/06, flight day 295/2014, Danum Valley to RG13/06, flight day 295/2014, SAFE Area
comment:32 Changed 10 years ago by tec
- Description modified (diff)
- Summary changed from RG13/06, flight day 295/2014, SAFE Area to RG13/06, flight day 295/2014, Danum Valley
Fenix Processing
Fenix ready for delivery check.
comment:33 Changed 10 years ago by tec
- Description modified (diff)
- Other processors changed from tec to tec, dap
- Summary changed from RG13/06, flight day 295/2014, Danum Valley to RG13/06, flight day 295/2014, SAFE Area to RG13/06
comment:34 Changed 10 years ago by dap
- Description modified (diff)
comment:35 Changed 10 years ago by dap
- Summary changed from RG13/06, flight day 295/2014, SAFE Area to RG13/06 to RG13/06, flight day 295/2014, SAFE Area
comment:36 Changed 10 years ago by dap
Starting Fenix Delivery Check
comment:37 Changed 10 years ago by dap
Fenix Delivery Check
Just a couple of issues, which I've fixed:
- Removed fodis directory
- Renamed the screenshots
- Copied logsheet to delivery
I've also zipped the mapped files up, so ready for delivery.
comment:38 Changed 9 years ago by lah
Fenix uploaded to ftp and email sent 12/06/15.
comment:39 Changed 9 years ago by stgo
Starting Fenix reprocessing
comment:40 Changed 9 years ago by stgo
Fenix reprocessing
Ready for delivery check.
This delivery needs to be thoroughly checked for:
- correct lines included
- correct linenames from logsheet
- data quality comments
- spectra shifts
comment:41 follow-up: ↓ 45 Changed 9 years ago by asm
Fenix reprocessing DC
Started
-Removed fodis directory.
-Added logsheet.
-All flightlines has correct name compared with logsheet.
-Please, add some extra information on Readme about data quility for flightline 52 which has an inverse 'C' shape.
-Some extra information about flightlines not included should be added (both readme and ticket). The ticket only mentions duplicated lines from number 6 to 20 and 42-51. However number 6 is included and number 5 is not. Also missing flightline 23.
-Need to change data quality report to the latest one.
comment:42 Changed 9 years ago by asm
Fenix reprocessing DC
Checked green vegetation spectra with Py6S. Good match for all flightlines.
comment:43 Changed 9 years ago by asm
Fenix reprocessing DC
Tested all tif files created from apl commands.
comment:44 Changed 9 years ago by lah
Fenix DC - spectra
Checked spectra of a few lines and no major problems. There is a persistent spike in the mask region for this flight, not unlike many other flights. The spectral shift is within 5 nm compared to previous data (2014 169).
comment:45 in reply to: ↑ 41 Changed 9 years ago by stgo
Replying to asm:
Fenix reprocessing DC
-Please, add some extra information on Readme about data quility for flightline 52 which has an inverse 'C' shape.
Will say something similar to 298, I'm assuming it is the same "F2" error in that the sensor doesn't terminate a flightline.
-Some extra information about flightlines not included should be added (both readme and ticket). The ticket only mentions duplicated lines from number 6 to 20 and 42-51. However number 6 is included and number 5 is not. Also missing flightline 23.
Missing flightlines weren't included in the original delivery, no explanation was given for the reasoning. I'll put a comment that flightlines are missing but tec hasn't left me a great deal of information to go on, I won't include flightlines that weren't in the original delivery though.
-Need to change data quality report to the latest one.
True of all flights
comment:46 Changed 9 years ago by stgo
Fenix reprocessing
Made changes outlined above, new readme in the delivery folder.
comment:47 Changed 9 years ago by asm
Fenix DC
-Need to change flightline 006a (wrong name being 005a) on readme to have the same flightlines than the last delivery.
comment:48 Changed 9 years ago by stgo
fenix dc
Fixed readme flightline numbering
comment:49 Changed 9 years ago by asm
Fenix DC:
All comments addressed and solved. Creating zipped files now. Would be ready to go once they have been created.
comment:50 Changed 9 years ago by asm
Fenix delivery
Delivery placed on FTP server (arsf1). Notification mail to PI still pending remaining deliveries within the same project.
comment:51 Changed 9 years ago by asm
Fenix delivery
Notification sent to PI on 3rd November 2015. Delivery finalised.
comment:52 Changed 9 years ago by dac
Archiving
Raw and archived data available from NEODC: http://browse.ceda.ac.uk/browse/neodc/arsf/2014/RG13_06/RG13_06-2014_295_Danum_Valley
Marking as closed.
Navigation Processing