Version 155 (modified by dap, 10 years ago) (diff)


ARSF Scripts and Binaries

Please keep this page updated. Also please keep it sorted in a sensible order or we're never going to be able to find anything...


Script Language Use Author
alspp.exe bash Runs wine version of ALSPP bash Configure wine environment for ARSF users
attune bash Runs wine version of Attune (boresight calibration) bash Runs get_class_las on full directories, so it doesn't have to be done one by one dap bash Checks the ASCII LiDAR files in the delivery for erroneous data
check_ascii_lidar Binary (C) Checks the ASCII LiDAR files in the delivery for erroneous data, prints the bounds of the ascii file
classify_las Binary (C) LiDAR classification bash Runs classify_las on a set of LAS files using a specified classification algorithm Python Creates a LaTeX file for generating a PDF read-me for delivery
fugroviewer.exe bash Runs wine version of Fugroviewer python Extracts waveform data for specified area
getDEMbufferpy Python Get the buffer size around LiDAR data for use when creating a DEM for use with apl
get_class_las c++ Copy the classification from one las file to another unknown - adbe? bash Creates new ASCII lidar files with 1st returns only bash Imports lidar ascii file into grass (run from within grass) mark1
lag Binary (C++) LiDAR file viewer harg + chrfi bash Convert a directory of LAS files or a single LAS file into ascii format Python Populates LAS file headers with information that may have been lost: min and max of x,y,z and number of points by return knpa bash Merges sets of LAS files of the form *_000.LAS, *_001.LAS... etc. together. Can also split a LAS file into several. Python Samples a LAS file at a specified interval and creates an output LAS file of these points knpa Python Classifies all points as a user defined number (quite slow) knpa
LASsorter Python sorts Las files by time, x, y, z , ascending and descending
leica_photo_processing bash Runs wine version of Leica's photo processor Bash Create a DEM from LiDAR data either las of ascii format Bash Create a DEM from LiDAR and aster data, wrapper from and Python Extracts statistics from a set of LAS files emca bash Create a LiDAR delivery folder bash Create a lidar intensity image from ascii / las + output jpgs + use vectors
pt_cloud_filter Binary (C) Removes points of a specified classification from ascii lidar point cloud knpa Python Splits las files using lasmerge. tipo


Script Language Use Author Python Generate a bad pixel file in CSV for a delivery, uses the CSV bad pixel file from calibration
aplxml_delivery_wrapper Python Generate an aplxml command for a flight in delivery format Python Spectrally bins (x2) a hyperspectral raw file knpa Python Get the North/South/East/West extent of a number of raw hdr files in bng coords knpa Python Generates a calibration curve in each band given input data in BIL or BSQ format and a known lamp file benj Python Wrapper for
check_apl_cmd bash Runs the apl commands in the config file for all level1 flightlines, output to tmp_cmd_check dir adbe Python Checks quality of bil files in hyperspectral delivery adbe Python Works out the corrected value for frame shift smear at a given band assuming a constant DN value through the spectrum benj Python Test specified Eagle/Hawk framerates against actual framerates calculated from qpfTiming benj Python Matches measured spectral lines to fityk-detected peaks and eagle/hawk pixel numbers for use in spectral calibration benj Python Create a LaTeX file for a PDF Readme for Hyperspectral azgcorr deliveries Python Create a LaTeX file for a PDF Readme for Hyperspectral APL deliveries Python Truncates BIL files so that the file size matches what the header file says it should benj Python Replaces a broken line counter on the first band of eagle and hawk (values 0 to number of lines) knpa Python Checks a BIL or BSQ file to see if it contains a specified overflow value benj
fastQC Binary (C) Program for viewing BIL files (raw data or level 1) harg Python Generates bandset (.prn, .bnd, .wls) files from a wavelength scale file benj Python Generates a synthetic Eagle or Hawk calibration file benj Python Generates a synthetic Eagle or Hawk dataset benj Python Generates a synthetic SBET file benj Python Generates a .prn file from some subset of bands in a given .wls file benj Python Generates config file for APL hyperspectral processing Python Generates an ascii file containing binned and trimmed sensor fov view vectors for level 1 files mark1 Python Extracts boresight values for given day from csv file Python Extracts calibration file names for given day from data file Python Generates example commands for APL for the read me file Python Gets North/South/East/West extent of (part of) an SBET file in lat/lon benj
getposition Python Gets position information at from an SBET file for a specified time (or file limits) benj Python Finds sync records on one or more AISA nav files bash Determines BNG grid squares for flight from Eagle/Hawk header files benj Python Checks if a Specim file has a mismatch between recorded and actual frame rate (Hawk problem) Python Python tools for hyperspectral readme table creation
bash Script to use IGM files to generate a GLT file and map level 1 data Python Convert interleave of a 3D file between BIL/BSQ Python Outputs mean altitude, heading and speed from an SBET file for a number of Eagle or Hawk lines benj
bash Script to convert GeoTIFFs into JPGs and create a mosaic image (with vectors) Python Make a hyperspectral delivery fro APL data Python Checks which of a selection of Eagle/Hawk header files have compatible binning, framerate and integration time settings benj Python Merges a .cfg and a .sync (eagle/hawk line sct offsets)
nav2txt Binary (C) Output contents of an AISA nav file to an ASCII text file Python Script to remove once-per-second sync messages from a eagle/hawk nav file that were introduced with the firmware upgrade in October 2011 and sometimes break aplcorr knpa Python Perform phase correlation on a lidar intensity image and a hyperspectral image Python Process the specim data using APL Python Submit a correlation job to the grid Python Runs az suite on specified Eagle/Hawk flightline for processing benj Python Master script for running calibration scripts benj Python Script to split Eagle or Hawk raw files containing autodark frames into a light file and a dark file. benj Python Script/Library for splitting Eagle/Hawk flightlines, using headers Python Cuts a chunk of an AISA nav file out into a smaller nav file Python Searches Eagle/Hawk raw files for dropped frames by looking at the band 1 frame counter Python Submits an ARSF project for processing through the grid engine benj Python Splits hyperspectral lines using apl processing tipo Python Re-writes the comment in the APL level 1 mask header files so that it refers to re-named level-1 file. Run after delivery has been created Python Generates a new wavelength scale from matching spectral peaks benj


Script Language Use Author Python Creates a LaTeX file for generating a PDF read_me for delivery Python Corrects image event files, mainly by using a sol/sbet file to correct position accuracy Python Create a camera delivery bash Converts RCD TIF files to JPG thumbnails Python Converts RCD TIF files into JPG thumbnails using lidar log files to filter out of bounds images Python generates new file names for tiffs and their entries in the image events file Python Adds tags to tiff files (using image event files) which define position, attitude and other metadata Python Swap pitch and roll in tiff if different from that in csv file


Script Language Use Author(s) bash/grass Adds an ascii or binary grid to a DEM - useful when doing geoid-spheroid seperation knpa bash Tarballs projects for NEODC mggr/knpa Bash Creates a ASTER DEM for use with aplcorr knpa/adbe
atcor bash Runs ATCOR-4 auto_grass_vector_tiff_conversion_control_script.grass Python uses grass to produce tiff images of vectors Python Converts binary files from one data type to another (eg. Float32 -> Int16)
bil2tiff Bash Convert bil files to geotiffs using gdal Python Converts BNG numeric grid references to OS grid square co-ordinates Python Allows easy building of project directory structure from the command line. Python Checks binary data files for presence of a specified overflow value Bash Will convert a delivery to a new project code and PI name. Can also be used on single deliveries to correct PI/proj-code inconsistencies Bash / Grass Convert a non-UK DEM into APL format (may have to be in WGS84 heights?) Bash / Grass Convert a UK DEM into APL format Python Convert file formats and create images Python Performs an arithmetic operation on all of the values from two data files and writes the results to an output data file Python Reads eagle or hawk header files and SBET file to generate a list of the DEM tiles that the SBET covers bash Converts dems from az format header to grass header bash Converts dems from grass format header to az header Python Converts .doc or .odt files to PDF format Python Tests if a project folder is still in embargo (1 year from last delivered data) stgo Python Converts between various document formats using Open Office Python Script to find the baseline value of a list of numbers containing value spikes. Intended for part of Specim cal procedure. Python Finds linux-unfriendly characters in filenames Python Auto-renames (line numbers and removes sct section) level 1, level 3, screenshots or rcd images/thumbnails/raws for a delivery.
freelook bash Runs ENVI Freelook Python Generates a fityk script file that can be used to fit peaks to particular datasets Python Creates a config file for hyperspectral and camera read_me generators Python Generate a KML file containing Eagle, Hawk or RCD information bash Convert GML Vector files to Shapefiles
gpstime Python Converts GPS time-of-week to human-readable format and vice-versa Python Lists contents of an HDF file to the terminal Python Extracts navigation data from hdf files to a bil file chrfi/knpa
julian CShell Converts between julian day+year and day/month/year Python Generate KML Overview for all projects in repository for a given year bash Converts specified lat/lon into BNG co-ordinates (wrapper around appropriate proj command)
lsolder bash Like ls, but can filter list to get files/directories older than a certain time Python Sets defined bad pixels in a BIL or BSQ file to be a mask value Python Automatically produce delivery letters and emails for a PI Python Make the KML file and overview support files - part of unpacking procedure bash Generates a DEM from NextMap data benj/mggr/knpa
projdir bash Automatically CDs to a project directory anch bash Checks that a specified project conforms to the current defined structure and file name formats knpa
Create JPGs of raw eagle/hawk data for KML overview Python Averages a BIL or BSQ file in the spatial direction bash Symlinks all files in a source directory into a target directory
teqc Binary (third-party) Program for slicing/splicing/fiddling with RINEX or other GPS base station data (like raw data) Python Searches E/H headers and gets start/stop times for importing into Grafnav to show on trajectory Python Extracts date from Trimble GPS format file names


Script Language Use Author Python Functions to convert numbers to letters and back (A=1, B=2 etc) Python Functions to access key/value ARSF database (please use Python Provides classes and functions to read from the ARSF database Python Reads AISA band (.prn) files Python Sorts config files into sensible order (can also be executed directly) Python Interpolates an array and/or many arrays wrt. one array. Finds peaks/Derivates/Integrates for 1D arrays. Python Functions for latex readme generator scripts Python Read data in that is required for the correlator Python Reads CSV columns as lists and writes lists to a CSV file Python Reads/writes BIL and BSQ files Python Class to hold info on Eagle/Hawk flight lines (used in Python Holds the current arsf folder structure Python For converting between different geographic coordinate systems knpa Python For working with Envi DEM files in grass stgo Python Set of functions for reading and checking hyperspectral config files Python Creates and writes KML files. Python Reads LAS 1.3 files Python Converts GPS time-of-week to human-readable format and vice-versa Python Gets mean altitude, heading and speed from an SBET file for a number of Eagle or Hawk lines Python Basic Specim nav file parser for ASCII nav files Python Returns basic project info (project code, jday etc) from an available source Python Class for holding RCD image information (used in Python Functions to manipulate SBET files, also limited AISA nav file support Python Sol handling library chrfi and adth


These are not in everyday use but are still useful on occasion

Script Language Use Author Python Part of old processing job submission system Python Parse output from az* programs to remove cruft - not kept up to date with newer versions and possibly too ambitious Python Creates a .txt ReadMe for lidar deliveries bash Script to make a workspace copy of a project directory (ie. avoid copying raw files, etc) & Python Script for generating mask file for use in GRASS for old Optech LiDAR Python Generates AZ runscripts or config files for hyperspectral data processing mark1/benj Python Convert GeoTIFFs to JPEGs
kqsub bash Handy for quick submission of individual flightlines and sct values. Designed for use with old rune/runh scripts Python bash Generates an azgcorr-format ASCII DEM from old Optech lidar point cloud files & Python Reads old Optech LiDAR files and outputs a numeric histogram for use in filtering bash Old script for making deliveries Python Converts azgcorr's ppv (per-pixel view vector) output to an ENVI IGM chrfi bash Cuts UTM zone number off old Optech LiDAR point cloud files Python Unpacking check files script Python Unpacking check the folder structure make_lidardem_or_intensity.grass? bash Create a lidar dem or intensity image from ascii lidar files

Explanation of symlinks in ~arsf/usr/bin

Link name Link target Use
configSorter /users/rsg/arsf/usr/lib/python/site-packages/ Link to configSorter library (see Libraries section) - library is also executable


Note: See also Python resources/libraries on main RSG wiki at