Changes between Initial Version and Version 1 of Procedures/DeliveryCreation/pythonlibrary

Timestamp:

Jan 21, 2014, 10:58:05 AM (10 years ago)

Author:

mark1

Comment:

--

Procedures/DeliveryCreation/pythonlibrary

@@ +1 @@
+---------------------------------
+== Make delivery instructions ==
+--------------------------
+
+Delivery creation is now via a python library rather than individual scripts. This should make it easier to perform individual steps to the creation of the deliveries. The library files are located in ''~arsf/usr/lib/python/site-packages/arsf_delivery_library/'' with functions separated into library scripts based on the delivery data type. These files are:
+
+basic_functions.py - functions that are required for multiple delivery types 
+hyperspectral.py - functions for creating hyperspectral deliveries
+lidar.py - functions for creating lidar deliveries
+rcdcamera.py - functions for creating RCD camera deliveries
+
+To use or run any of the functions just import the required module and run the function, e.g.:
+
+{{{
+import arsf_delivery_library.basic_functions
+arsf_delivery_library.basic_functions.CreateDeliveryStructure(deltype='hyperspectral',delpath='/path/to/delivery',maketoplevel=True,onlyaddmissing=True)
+}}}
+
+For 'normal' use there is a wrapper script that is run to generate a default delivery for each sensor type. This is the make_arsf_delivery.py script. It makes assumptions about the layout of the data (i.e. where in the system it is), uses default parameters for certain processes and should replace all previous delivery creation scripts. It can be run more than once if required (e.g. if something breaks the first run) and the steps to be performed (or missed out) can be selected. By default it will run in dry-run mode, although this mode of operation may fail certain tests because directories have not yet been created.
+
+When the script is run it will clearly state which part of the processing it is starting, so as to make it as easy as possible to correct mistakes or bug fix the script.
+
+---------------------------------
+=== Example - create a hyperspectral delivery ===
+-----------------------------------------
+
+{{{
+make_arsf_delivery.py --projectlocation /users/rsg/arsf/arsf_data/2013/flight_data/arsf_internal/GB13_00-2013_308a_Little_Riss/ \
+                      --deliverytype hyperspectral
+}}}
+
+will (assuming everything is 'good' about the project) output information about commands it will run. As the delivery directory does not exist there will appear to be many errors, so if the output of the first stage looks correct (the STRUCTURE section) then run that step only to create the structure.
+
+{{{
+make_arsf_delivery.py --projectlocation /users/rsg/arsf/arsf_data/2013/flight_data/arsf_internal/GB13_00-2013_308a_Little_Riss/ \
+                      --deliverytype hyperspectral --steps STRUCTURE --final
+}}}
+
+This will have now created the empty delivery structure. The script can be run again in dry run mode now but stating that STRUCTURE does not need to be run.
+
+{{{
+make_arsf_delivery.py --projectlocation /users/rsg/arsf/arsf_data/2013/flight_data/arsf_internal/GB13_00-2013_308a_Little_Riss/ \
+                      --deliverytype hyperspectral --notsteps STRUCTURE
+}}}
+
+Hopefully the output will be a list of commands the script will run, separated up into sections based on the step names. If everything looks to be correct then the ''--final'' can be added to the command line to run the delivery creation.
+
+The output text should be inspected for error messages. These should be obvious and appear as yellow text against a red background. If problems occur then individual steps can be run after the problem has been fixed (note that you may need to also run steps after the one that failed). Also note that GRASS outputs a lot of progress messages to standard error - these will appear as errors (in yellow + red on terminal) but are not actually errors. 
+
+---------------------------------
+=== A not-so-straight-forward example ===
+---------------------------------
+
+For cases which are not so straight forward there are some extra options that can be used in ''make_arsf_delivery.py'', or you can use the library functions directly.
+
+For example, to work with projects not in the repository, you will need to specify project details (year, julian day, project code and sortie) using the ''--projectinfo'' keyword. This is because the database can not be queried. 
+
+If multiple configfiles exist the correct one to use can be specified using ''--configfile''.
+
+To create a delivery in a different place to the default (processing/delivery/...) you can use ''--deliverylocation'' to specify the delivery name and file path.
+
+If you feel unsure about anything or want to be cautious then you can run each step at a time by specifiying it with the ''--steps'' keyword. A list of the steps (in the order they're run) can be seen in the script ''--help'' - the ordering is important as the step may rely on something from a previous step.
+
+