id summary reporter owner description type status priority milestone component resolution keywords cc processors 545 DEM Scripts Rewrite/Restructure dac dac "There are currently a large number of scripts and libraries for DEM related tasks, most commonly used to: * Create DEMs for use in APL * Create DEMs from LiDAR data * Perform general tasks on DEMs (e.g., reprojection) These scripts are a complicated mix of bash and Python which are very difficult to debug. Previous attempts to simplify them have only increased the complexity. There are problems with the existing scripts which are proving difficult to fix given the current state of the code, the biggest one is that the DEMs produced are often much larger than needed. A rewrite/restructure of the existing DEM scripts is therefore proposed with the following aims: * Use Python instead of bash where possible * Store common functions within a Python library * Make use of existing libraries both external (e.g., GDAL, Proj, GRASS Python bindings) and internal where possible. Given these aims the following is proposed: '''New DEM Scripts''' ''create_apl_dem.py'' Create DEM subset to navigation data for use in APL. Locations of existing DEMs (ASTER and NMB) will be hardcoded but will allow any DEM mosaic (real or virtual raster) to be provided. {{{ -o Out DEM, --outdem Out DEM Output name for DEM -n Nav file, --nav Nav file Navigation data (.sol / .sbet file) -p Main project directory, --project Main project directory Main project directory (default=""."") --aster Use ASTER data (/users/rsg/arsf/aster/aster_15m_dem_mosaic.vrt) --nextmap Use Nextmap data (/users/rsg/arsf/nextmap/neodc/nextma p_dsm_mosaic_bng.vrt) --srtm Use SRTM data (/local1/data/basedata/srtm/srtm_global_ mosaic_90m.tif) --demmosaic Input DEM mosaic Input DEM mosaic. For non-standard DEM. Use ""--aster"" or ""--nextmap"" for standard DEMs. --separation_file Seperation file File with Height offset to add if ""--demmosaic"" is used and DEM heights are not relative to WGS-84 elepsoid. Not required if using ""--aster"", ""--nextmap"" or ""--srtm"" for standard DEMs. -b BIL Navigation Files, --bil_navigation BIL Navigation Files Directory containing post-processed navigation files in BIL format. By default raw navigation data will be used for ""--project"". If this is not available (e.g., for ARSF delivered data use this option and point to ""flightlines/navigation"" within delivery directory --keepgrassdb Keep GRASS database (default=False) }}} ''create_dem_from_lidar.py'' Create DEM from lidar data {{{ -o Out DEM, --outdem Out DEM Output name for DEM -s Out Screenshot File or Directory, --screenshot Out Screenshot File or Directory Output directory for screenshots or single file for screenshot of mosaic, in JPEG format. --las Input LiDAR data are in LAS format (default=True) --ascii Input LiDAR data are in ASCII format (default=False) -r Resolution, --resolution Resolution Resolution for output DEM (default=2) --in_projection In Projection Input projection (e.g., UTM30N; default=UKBNG) --out_projection Out Projection Out projection. Default is same as input -n Nav file, --nav Nav file Navigation data (.sbet / .sol file) used if patching with another DEM -p Main project directory, --project Main project directory Main project directory, used if patching with another DEM --demmosaic Input DEM mosaic Input DEM mosaic to patch with in GDAL compatible format. Vertical datum needs to be the same as output projection. Only required for non-standard DEM. Use "" --aster"" or ""--nextmap"" for standard DEMs. --aster Patch with ASTER data (/users/rsg/arsf/aster/aster_15m_dem_mosaic.vrt) --nextmap Patch with Nextmap data (/users/rsg/arsf/nextmap/neodc /nextmap_dsm_mosaic_bng.vrt) --hyperspectral_bounds If patching with another DEM, get extent from hyperspectral navigation data, recommended if DEM is to be used with APL and navigation data are available. --lidar_bounds If patching with another DEM, get extent from lidar data plus default buffer of 2000 m. If DEM is not required to be used with APL this option is recommended. }}} '''New DEM Library''' Putting all the DEM functions in a separate module (arsf_dem) is proposed rather than multiple libraries. {{{ |-- arsf_dem | |-- arsf_dem.cfg | |-- common_functions.py | |-- dem_common.py | |-- dem_lidar | | |-- ascii_lidar.py | | |-- grass_lidar.py | | |-- __init__.py | | |-- laspy_lidar.py | | |-- lastools_lidar.py | | |-- lidar_utilities.py | | |-- spdlib_lidar.py | |-- dem_nav_utilities.py | |-- dem_utilities.py | |-- grass_library.py }}} " task closed immediate The Glorious Future Processing: general fixed