= Data Processing at NERC-ARF-DAN = These pages provide an overview of data processing applied at NERC-ARF-DAN after data are received from operations and act as internal documentation. For a single page overview of the processing see the following [https://nerc-arf-dan.pml.ac.uk/files/rspsoc_2016_nerc-arf-dan_data_processing.pdf poster from RSPSoc 2016 (PDF)]. == Project processing procedure == This page contains links to detailed step-by-step instructions on how to deal with data from arrival to dispatch to the user. Please keep updated. A simplified overview of the procedures can be seen in the flow chart. Clicking on the relevant section will give a more detailed, technical description. [[Image(simple_overview.jpeg, align=center, link=, margin=20, usemap="#flow")]] {{{ #!html Data Arrival Navigation Processing RCD Raw to Tiff RCD Tagging etc Delivery Creation Delivery Checking Archiving Hyperspectral Processing LiDAR Processing LiDAR Classifying Dispatch }}} ----------------------------- == Instrument Calibration == These procedures are carried out once per season or whenever the instruments are removed and refitted to the plane. * [wiki:Procedures/HyperspectralCalibrationCreation Hyperspectral calibration] * [wiki:ProcessingAtDAN/hyper_cal/bad_pixels Bad pixel file generation] (part of hyperspectral calibration) * [wiki:InstrumentCalibration#LeicaLiDAR Leica LiDAR] * [wiki:InstrumentCalibration#LeicaRCD Leica RCD105 Digital Camera] === Instrument Boresighting === * [wiki:Processing/Boresighting#Hyperspectral Hyperspectral and Thermal boresighting] * [wiki:Processing/Boresighting#Lidar LiDAR boresighting] * [wiki:LeverarmInfo Lever arms] === Calibration Sites === * [wiki:Processing/CalibSite#Alconbury Alconbury] site details - 2017 onwards * [wiki:Processing/CalibSite#LittleRiss Little Rissington] site details - September 2008 to October 2015 * [wiki:Processing/CalibSite#MonksWood Monks Wood] site details - up to September 2008 * Gloucester == Other procedures == * [wiki:Procedures/OwlProcessing Processing Owl data] * [wiki:Procedures/OwlProcessing/OwlTool Specim's Owl Tool] * [wiki:Processing/proj_tidy Information on proj_tidy.sh] * [wiki:Procedures/NewSeason What needs to be done prior to starting a new season] * [wiki:Procedures/reprocessing Procedures to follow for hyperspectral reprocessing] * [wiki:Procedures/FenixProcessing/splitVNIRandSWIR Guide to splitting VNIR and SWIR from Fenix] * [wiki:Processing/Splitting Guide to splitting Lidar and hyperspectral files spatially] * [wiki:Processing/Py6S_vs_Hyperspectral Checking the spectra with Py6S] == Guides to using non-standard software and obsolete software processing methodology == * [wiki:Procedures/NewAzgcorr How to update azgcorr and who to notify] * [wiki:Processing/CaliGeo Processing with CaliGeo] * [wiki:Processing/atcor Atmospheric correction using ATCOR] * [wiki:Processing/BadATMScans Handling bad ATM scanlines] with az software * [wiki:Processing/LIDARDEMs Creation of DEMs from Optech LIDAR point clouds (pre-2009 data) ] * [wiki:Processing/TerrascanTips Tips on using Microstation and TerraScan] * [wiki:Processing/MatchTwoLinesByBoresight Match 2 images by boresight corrections] * [wiki:Processing/MaskCreation How to generate overflow and low signal masks for specim sensors] * [wiki:Processing/Visualisation How to use the visualisation machine and software] * (2010 and earlier) [wiki:Procedures/DeliveryCreation/rsync r-syncing from the workspace to the repository] == Information on data in system == * [wiki:Processing/FilenameConventions Filenaming conventions and structure] of unpacked data * [wiki:BoresightInfo Information on the boresights] * [wiki:LeverarmInfo Information on the leverarms] * [wiki:statusdatabase How the NERC-ARF status database is structured] * [wiki:backup How we back-up data held at NERC-ARF-DAN]