Data Processing at NERC-ARF-DAN

These pages provide an overview of data processing applied at NERC-ARF-DAN after data are received from operations and act as internal documentation. For a single page overview of the processing see the following ​poster from RSPSoc 2016 (PDF).

Project processing procedure

This page contains links to detailed step-by-step instructions on how to deal with data from arrival to dispatch to the user. Please keep updated. A simplified overview of the procedures can be seen in the flow chart. Clicking on the relevant section will give a more detailed, technical description.

---

Instrument Calibration

These procedures are carried out once per season or whenever the instruments are removed and refitted to the plane.

• Hyperspectral calibration
• Bad pixel file generation (part of hyperspectral calibration)
• Leica LiDAR
• Leica RCD105 Digital Camera

Instrument Boresighting

• Hyperspectral and Thermal boresighting
• LiDAR boresighting
• Lever arms

Calibration Sites

• Alconbury site details - 2017 onwards
• Little Rissington site details - September 2008 to October 2015
• Monks Wood site details - up to September 2008
• Gloucester

Other procedures

• Summary of 2023 Installations
• Reprocessing Eagle and Hawk data
• Reprocessing CASI data
• Processing Owl data
• Specim's Owl Tool
• Additional photogrammetry processing using PhotoScan
• Information on proj_tidy.sh
• What needs to be done prior to starting a new season
• Procedures to follow for hyperspectral reprocessing
• Guide to splitting VNIR and SWIR from Fenix
• Guide to splitting Lidar and hyperspectral files spatially
• Checking the spectra with Py6S
• Atmospheric correction using ATCOR
• Information on ground surveys for atmospheric correction
• Information on processing the InfraTec thermal data

Guides to using non-standard software and obsolete software processing methodology

• How to update azgcorr and who to notify
• Processing with CaliGeo
• Handling bad ATM scanlines with az software
• Creation of DEMs from Optech LIDAR point clouds (pre-2009 data)
• Tips on using Microstation and TerraScan
• Match 2 images by boresight corrections
• How to generate overflow and low signal masks for specim sensors
• How to use the visualisation machine and software
• (2010 and earlier) r-syncing from the workspace to the repository

Information on data in system

• Filenaming conventions and structure of unpacked data
• Information on the boresights
• Information on the leverarms
• How the NERC-ARF status database is structured
• How we back-up data held at NERC-ARF-DAN