Generating Photogrammetry Products

This page aims to guide a user through photogrammetry processing using Agisoft Metashape (formally PhotoScan).
Refer to ~arsf/doc/photoscan_processing/ for a step by step guide using a Metashape GUI.

Processing using Metashape

Overview: The following files comprise the arf_photoscan library:

arf_photoscan/
	init.py
	ps_export.py
	ps_processing.py
	ps_tools.py
	ps_tools.pyc
ps_ortho_processing_after_input_gcp.py
ps_ortho_processing_before_input_gcp.py
README.md
templates/
	ps_workflow.cfg

The arf_photoscan library currently contains two scripts for automating Metashape processing of NERC-ARF data: ps_ortho_processing_before_input_gcp.py will set up a Metashape project and align photographs. If Ground Control Points (GCPs) are available, then the script will stop and prompt you to manually locate them in the Metashape project. If no GCPs are available, the script will complete the whole Metashape processing workflow without (obviously accuracy of data is significantly reduced). This step is not automated. ps_ortho_processing_after_input_gcp.py is run after the and placement of Ground Control Points.

Ground Control Points
Coordinates of GCPs in CSV format can be added to Metashape. The user will then need to 'drag and drop' the marker locations to their locations visible in the photographs input. This needs to be done for all photographs each GCP is visible in.
Note: once a couple of GCPs have been located in multiple respective photographs; the 'reproject' button (looks like a recycling symbol) will automatically place the remaining photo. These will still need to be accurately places, but will save a lot of time nevertheless.

Running a script:

• Update config file (template is located in /templates/ps_workflow.cfg).
• As airborne, in the rsg-photoscan virtual machine, run:

/opt/photoscan/photoscan.sh -r ps_ortho_processing_before_input_gcp.py -c [CONFIG]

• If no GCPs have been input in config file, (gcp_csv_path = None) or if gcp_csv_path does not exist, then ps_ortho_processing_before_input_gcp.py will run through the whole Metashape workflow without GCPs.
• If a path to GCP CSV has been input into the config file, these GCP coordinates will need to be manually positioned to their correct locations on the aligned photographs. Once this has been achieved, the second script can be run:

/opt/photoscan/photoscan.sh -r ps_ortho_processing_before_after_gcp.py -c [CONFIG]

Config Information
The same config template is used.
The project directory (defined in config file) is the location of the Metashape project file (*.psx) and other data produced whilst processing.
The output directory (also defined in config) is where the Point Cloud, DEM and/or Orthomosaic is exported, if the associated parameters are True in config file. If the output directory does not exist, then it will be created.

Quality Checks
To assess the quality of the exported data, compare the orthomosaic to Fenix/Owl data, or any higher spatial resolution imagery that you have access to. Use this to assess if the X and Y accuracy is suitable.
For the Z (vertical) accuracy, you should compare the generated DEM to any points of known elevation at the time of flight but not the GCPs used in processing. For UK projects, there is good coverage of LiDAR data which may be able to make a good comparison. You can calculate the bias and RMSE between the processed elevations and the expected elevations (inferred from proxy data). Any statistical metrics calculated should be reported in the delivery's Read_Me.
If the accuracy of the data is questionable, investigate whether this is caused by the following:

• GCPs
  • GCPs need to be frequent, well spread, precise and in the correct units. Also consider reprojection errors if the GCPs and/or exported data are in contrasting projections.
• Eventfile
  • Check the values in the eventfile. There may have been a problem when extracted them from the navigation data.
• Processing
  • Check that the GCP placement in Metashape has been undertaken in as many photographs that GCPs are present in as possible. Also look at the residual error of the project's spatial fit. Unusually high error values for a particular GCP marker would suggest that it has been placed in the wrong location.

Creating Delivery

This additional processing service will be delivered as additional items in the otherwise conventional digital camera delivery using the following structure:

Dirs	Sub dir	Example Files	Comments
		ES17_126-2017_170.kml	Google earth file containing Fenix/Owl/camera positions
		Read_Me-20171018.pdf	Instructions for use and description of dataset, plus any problems encountered
		Read_Me-20180222_orthomosaic.pdf	Instructions for use and description of photogrammetry products
dem			Digital Surface Model(s) produced from Metashape.
		PHASEONE-IXU-ES17_126-2017170_dsm-utm30n.dem	Digital Surface Model in ENVI format
		PHASEONE-IXU-ES17_126-2017170_dsm-utm30n.hdr	Header for accompanying Digital Surface Model in ENVI format
doc			Useful documents
		camera_data_quality_report2012.pdf	Camera data quality report
		ES17_126-2017170_ground_gps_points_composite.csv	Ground control point (supplied by PI) used during Metashape Processing
		PHASEONE-IXU-ES17_126-2017170_photoscan_processing_report_20180222.pdf	Agisoft Metashape generated processing report.
		RCD105_SN21_Calibration_Certificate.pdf	Camera Calibration Certificate
eventfile		PHASEONE-IXU-ES17_126-2017170-ImageEvents.csv	CSV file containing pos/att info per photograph event (may be missing if camera crashed)
orthomosaic		PHASEONE-IXU-ES17_126-2017170_mosaic-utm30n.tif	Orthorectified mosaic of photographs in GTiff format
photographs		photographs/PHASEONE-IXU-ES17_126-2017170-00001.tif	Tagged tif files of each photograph in area of interest
point_cloud			Exports of the 'Dense Cloud' generated in Metashape
	ascii	PHASEONE-IXU-ES17_126-2017170_point_cloud.txt	Point cloud in ASCII format
	las1.2	PHASEONE-IXU-ES17_126-2017170_point_cloud.laz	Point cloud in LAS 1.2 format
thumbnails		photographs/PHASEONE-IXU-ES17_126-2017170-00001.jpg	JPEG thumbnails of each photograph in area of interest

==Licenses==

Our MetaShape license is currently activated on pmpc1650 and the 'reshosting' directory is

We only have one license and it is important to keep track of. Please make sure that the license is revoked before nay hardware changes or OS upgrades are performed on the machine it is installed on.

By default, a file with license and system information is create in the following directory of the activated machine:

/var/tmp/agisoft/photoscan-pro/Do-NOT-Touch-Anything-in-This-RLM-Directory/

Where 'agiosoft' is the 'rehosting directory'.

However, it seems this can be automatically deleted by the system. Therefore a rehosting directory needs to be created in a safe space, where the user of MetaShape also has appropriate permissions.

This can be changed by modifying metashape.sh to point to a desired directory using AGISOFT_REHOST_PATH and later exporting it like the following example:

#!/bin/sh
appname=`basename "$0" | sed s,\.sh$,,`

dirname=`dirname "$0"`
tmp="${dirname#?}"


if [ "${dirname%$tmp}" != "/" ]; then
dirname=$PWD/$dirname
fi

AGISOFT_REHOST_PATH=[ENTER DIRECTORY HERE]

TCL_LIBRARY=$dirname/python/lib/tcl8.5
TK_LIBRARY=$dirname/python/lib/tk8.5
export TCL_LIBRARY
export TK_LIBRARY
export AGISOFT_REHOST_PATH

LD_LIBRARY_PATH=$dirname:$dirname/python/lib:$LD_LIBRARY_PATH
export LD_LIBRARY_PATH

"$dirname/$appname" "$@"