Opened 5 years ago

Closed 3 years ago

#550 closed flight processing (fixed)

RG13/06, flight day 294/2014, SAFE Area

Reported by: dap Owned by:
Priority: alpha 5 Milestone:
Component: Processing: general Keywords:
Cc: Other processors: tec

Description (last modified by dap)

Data location: /users/rsg/arsf/arsf_data/2014/flight_data/malaysia/RG13_06-2014_294_Danum_Valley

Data arrived from network transfer on 17/12/2014.

Scientific objective: Investigating the impact of humans on tropical forests.

Priority: alpha 5

PI: David Coomes

Some corrupt webcam images were deleted:
141021_015352_000.jpg

No owl data present

Sensors:

  • Fenix (delivered 22/05/2015)
  • Leica FW LIDAR (?)
  • Leica LIDAR (requested)
  • RCD (requested)

Duplicate lines

Process only one for Fenix and RCD - check for which has least cloud cover, failing that choose closest to local noon

  • 1222-1236
  • 1253-1256
  • 1185-1189 (day 295)
  • 1137-1138 (3 times, days 293/297); 1188-1189 (3 times, days 297/298)

Change History (49)

comment:1 Changed 5 years ago by tec

Navigation Processing

SAFE_10th Basestation Data
Latitude4 43 30.56083
Longitude117 36 19.18643
El Height479.789m

comment:2 Changed 5 years ago by tec

  • Other processors set to tec

comment:3 Changed 5 years ago by tec

Navigation Processing
Used IPAS pro, few spots of quality 2 nav
C/A 1.30
Ell Mask 15
Max Dual Freq Distance 80

Nav done

Last edited 5 years ago by tec (previous) (diff)

comment:4 Changed 4 years ago by benj

  • Description modified (diff)

Added duplicate line details

comment:5 Changed 4 years ago by tec

Last edited 4 years ago by tec (previous) (diff)

comment:6 Changed 4 years ago by tec

RCD
Converting Tiffs

comment:7 Changed 4 years ago by tec

Nav
Data in ~/arsf_data/2014/flight_data/malaysia/RG13_06-Basestation_data/RINEX DATA SABAH/Rinex/temporary_gps_dataz/294, move out into project folder once processed.

comment:8 Changed 4 years ago by tec

Navigation Processing
Starting

comment:9 Changed 4 years ago by tec

Navigation Processing
Antenna Height 1.29
Basestation SAFE_9th_
Latitude: 4 43 30.56143
Longitude: 117 36 19.18599
Ell: 480.155m

Processing with IPAS Pro, no decent results from TC
C/A 1.20
Ell Mask 16
Max Dual Freq Distance 80

seperation is ±0.05 for the most part but does peak to ±0.10 a couple of times, this is the best I can get it.

Nav Done

Last edited 4 years ago by tec (previous) (diff)

comment:10 Changed 4 years ago by tec

RCD Processing
Starting RCD

Last edited 4 years ago by tec (previous) (diff)

comment:11 Changed 4 years ago by tec

RCD Processing
Removing images
20223721100010G3
20223721100011G3
20223721100019G3
20223721100020G3
20223721100021G3
20223721100025G3
20223721100026G3
20223721100028G3
20223721100029G3
20223721100030G3
20223721100031G3
20223721100348G3
20223721100464G3
20223721100465G3
20223721100466G3
20223721100467G3
20223721100482G3
20223721100483G3

comment:12 Changed 4 years ago by tec

RCD Processing
Tagging Tiffs

comment:13 Changed 4 years ago by dap

LiDAR Processing

Started. First two directories in RawLaser (141020_221602 and 141020_221627) won't process. These flightlines aren't likely to contain useful data as the directories only contain one file and there was no flightline recorded in the logsheet around this time. ALSPP also shows that the two flight lines have AGL values of -1.

posatt/ipas_honeywell/proc doesn't have a .sup file for LiDAR processing so pausing processing for now.

comment:14 Changed 4 years ago by dap

RCD Delivery Check

Looks fine, no problems found. Ready for delivery.

comment:15 Changed 4 years ago by tec

Delivery
Copying to FTP now, will say its done.

comment:16 Changed 4 years ago by lah

PI notified of RCD delivery 05/05/2015 (RCD only)

comment:17 Changed 4 years ago by lah

Fenix Duplicates
Processing lines:
1222 17
1223 18
1224 4
1225 19
1226 6
1227 21
1228 8
1229 9
1230 10
1231 11
1232 12
1233 13
1234 14
1235 15
1236 16
1253 31
1254 32
1255 33
1256 34
Cross 39
1130 40

(Problem with line 7 - time problem remains)

Last edited 4 years ago by lah (previous) (diff)

comment:18 Changed 4 years ago by lah

Fenix
Found SCTs:

Flightline FENIX
4 0.99
6 0.98
8 0.96
9 0.96
10 1.00
11 0.97
12 0.99
13 0.99
14 1.00
15 0.94
16 1.04
17 0.98
18 0.92
19 0.94
21 -0.04
31 1.00
32 0.94
33 0.97
34 0.99
39 1.00
41 0.97

comment:19 Changed 4 years ago by lah

Fenix ready for DC

comment:20 Changed 4 years ago by dap

Fenix Delivery Check

Everything looks OK apart from the logsheet, which has 293 as the Julian day and has the wrong date.

check_apl_cmd is still running so will inspect the files generated when complete and comment when the data are ready for delivery check.

comment:21 Changed 4 years ago by dap

Fenix Delivery Check

check_apl_cmd complete, results seem good and no error messages were produced. Just the minor alteration mentioned in comment:20 needs to be made then ready for delivery.

comment:22 Changed 4 years ago by lah

Fenix
Remade logsheet with correct date & Julian day. Started zipping files.

comment:23 Changed 4 years ago by lah

Fenix Delivery
Uploaded to ftp (arsf1). Not yet sent email notification (but PI has password from other deliveries).

comment:24 Changed 4 years ago by dap

LiDAR Processing

All corrected roll and pitch values have been found, and are as follows:

Flight line Pitch Roll
232527-0.00163180-0.006375
232846-0.00063180-0.006075
233111-0.00033180-0.006375
233352-0.00063180-0.006075
233646-0.00003180-0.006375
234214-0.00163180-0.006375
2345380.00063180-0.006075
234859-0.00163180-0.006375
2352230.00063180-0.006075
235536-0.00163180-0.006375
2358550.00063180-0.006075
000224-0.00163180-0.006375
0005440.00063180-0.006075
000916-0.00163180-0.006375
0012310.00063180-0.006075
001533-0.00163180-0.006375
0018450.00063180-0.006075
002142-0.00163180-0.006375
0024380.00063180-0.006075
002738-0.00163180-0.006375
003354-0.00163180-0.006375
0037230.00063180-0.006075
004144-0.00163180-0.006375
0046160.00063180-0.006075
004946-0.00163180-0.006375
0053040.00063480-0.006075
005653-0.00163180-0.006375
0100260.00063180-0.006075
010350-0.00160180-0.006375
0107150.00063180-0.006075
011033-0.00163180-0.006375
0113410.00063180-0.006075
011642-0.00163180-0.006375
0119360.00063180-0.006075
012217-0.00163180-0.006375
012725-0.00163180-0.006375
0130070.00063180-0.006075
013315-0.00163180-0.006375
0136300.00063180-0.006075
013902-0.00163180-0.006375
0141500.00063180-0.006075
014458-0.00163180-0.006075
0147540.00063180-0.006075
0151130.00063180-0.006075
015816-0.00163180-0.006375

(45 lines in total)

Now processing final files.

Last edited 4 years ago by dap (previous) (diff)

comment:25 Changed 4 years ago by dap

LiDAR Processing

Roll- and pitch-corrected files have finished processing and files have been classified automatically using the isolated points algorithm (less than 5 points within 10 metres). Now going through the files checking the classifications.

comment:26 Changed 4 years ago by dap

LiDAR Processing

Classification complete, now copying classifications to full waveform files before creating delivery.

comment:27 Changed 4 years ago by dap

rsyncing Fenix delivery to ftp server (arsf1)

Just realised this has already been done but no email was sent.

Last edited 4 years ago by dap (previous) (diff)

comment:28 Changed 4 years ago by dap

PI has been notified of Fenix delivery (22/05/2015)

Last edited 4 years ago by dap (previous) (diff)

comment:29 Changed 4 years ago by dap

  • Description modified (diff)
  • Summary changed from RG13/06, flight day 294/2014, Danum Valley to RG13/06, flight day 294/2014, SAFE Area

comment:30 Changed 4 years ago by dap

LiDAR Processing

Complete. Delivery and read me has been generated. Will update comment:24 with the updated roll and pitch values.

comment:31 Changed 4 years ago by tec

LiDAR Delivery Check
Started

comment:32 Changed 4 years ago by tec

Checked pitch and roll, they look ok.

comment:33 Changed 4 years ago by tec

checked up to line 20 for class

comment:34 Changed 4 years ago by tec

LiDAR DC
Demcompare is 18, dont think anything can be done so...
Line 33 needs regenerating as I accidently overwrote it, the fw and ascii should be fine

comment:35 Changed 4 years ago by dap

LiDAR Processing

Fortunately I had kept the unclassified ALSPP output in processing/als50/las, so was able to copy the file (33) from there into the delivery and then copy the classifications from the full waveform data to the discrete data. Have also renamed the new file accordingly and checked that the classifications copied over correctly.

comment:36 Changed 4 years ago by tec

LiDAR DC
Delivery check done, all is good

comment:37 Changed 4 years ago by lah

Lidar uploaded to ftp and email sent 12/06/15.

comment:38 Changed 4 years ago by stgo

Starting reprocessing

comment:39 Changed 4 years ago by stgo

Fenix reprocessing

Ready for delivery check.

This delivery needs to be thoroughly checked for:

  • correct lines included
  • correct linenames from logsheet
  • data quality comments
  • spectra shifts

comment:40 Changed 4 years ago by asm

Fenix reprocessing DC

Started
-Removed fodis directory.
-Added logsheet.
-Added readme.
-Need to change data quality report to the latest one.
-All flightlines has correct name compared with logsheet.
-Flightline not included on the delivery that are recorded on the logsheet and not include because those lines are duplicated on other flights according to this ticket. A note on the readme that mentions that fact should be added.

comment:41 Changed 4 years ago by asm

Fenix reprocessing DC

-Checked green vegetation spectra with Py6S. Good match for all flightlines.
-Tested all tif files created from apl commands.

comment:42 Changed 4 years ago by lah

Fenix DC - spectra
Checked spectra of a few lines and no major problems. There is a persistent spike in the mask region for this flight, not unlike many other flights. The VNIR-SWIR ratio is also fairly variable, but this does not appear to correlate to time of flight, so is probably normal variation. The spectral shift is within 5 nm compared to previous data (2014 169).

comment:43 Changed 4 years ago by stgo

Fenix reprocessing delivery

I've added a comment "Some flightlines in this dataset have not been included in this delivery due to better quality duplicates from either the same flight or another flight day on the same project." to the readme and replaced the old one.

If this flight is now ready for delivery please hardlink it to the PIs active FTP, wait on notification as hopefully there will be more to deliver today!

comment:44 Changed 4 years ago by asm

Fenix DC

Zipped files created. Waiting another DC before being delivered.

Last edited 4 years ago by asm (previous) (diff)

comment:45 Changed 4 years ago by stgo

Fenix reprocessing
Lah emailed with these issues:

"There are still a few problems with this delivery. Have we stopped using
proj_tidy now as this would pick these up?

1) Screenshots are incorrectly named
2) aux files are present
3) mapped files have not been zipped"

I have renamed the screenshots and removed aux files. Ready for re DC.

comment:46 Changed 4 years ago by asm

Fenix delivery

Delivery placed on FTP server (arsf1). Notification mail to PI still pending remaining deliveries within the same project.

comment:47 Changed 4 years ago by asm

Fenix delivery

Notification sent to PI on 3rd November 2015. Delivery finalised.

comment:48 Changed 3 years ago by dac

Archiving

Started uploading data to NEODC.

comment:49 Changed 3 years ago by asm

  • Resolution set to fixed
  • Status changed from new to closed
Note: See TracTickets for help on using tickets.