= Data arrival by network transfer =

 1. change to arsf user on a safe machine
{{{
ssh arsf@gridmaster2 # (or su - arsf and don't turn your machine off!)
}}}
 1. find the exact data path on Ops server
{{{
ssh arsfdan@thelma.nerc-arsf.ac.uk
 # go find the path to the data to download
 # these will vary..
cd /data/Data_2009/
cd UK
cd 281-09_GB08-02_Delamere
 # check path
pwd
 # returns /data/Data_2009/UK/281-09_GB08-02_Delamere
 #
 # check size isn't crazy big
 # > 150GB and think about a disk transfer
du -hsc .
logout
}}}
 1. create a temporary directory in ~arsf_data/YYYY/flight_data/unpacking/ to download to, e.g.
{{{
DOWNDIR=~/arsf_data/`date +%Y`/flight_data/unpacking/download-`date +%Y%m%d%H%M%S`
mkdir $DOWNDIR
cd $DOWNDIR
}}}
 1. do the download with rsync, using compression (-z flag).  Note there is '''no trailing /''' on the PROJECTNAME part of the path below - this is important!
   * example rsync command follows, with a --dry-run option to prevent any damage occurring on the trial attempt.  If it lists the files you'd expect it to transfer, remove the --dry-run option and repeat.  If not, check your slashes!
{{{
date
rsync --dry-run -avz arsfdan@thelma.nerc-arsf.ac.uk:/PATH/YOU/FOUND/EARLIER/PROJECTNAME . ; date
 # date commands are optional, they just let you see when it started and stopped
}}}


If something goes wrong after an hour or two, don't delete what you've already downloaded.  Instead, repeat the rsync command and it'll pick up where it left off.

== Unpacking data ==

Return to [wiki:Procedures/NewDataArrival]