Changes between Version 9 and Version 10 of Procedures/NewDataArrival/Network


Ignore:
Timestamp:
Aug 7, 2014 9:32:25 AM (5 years ago)
Author:
mark1
Comment:

--

Legend:

Unmodified
Added
Removed
Modified
  • Procedures/NewDataArrival/Network

    v9 v10  
    11= Data arrival by network transfer =
     2
     3== Looking for data on Thelma ==
    24
    35 1. change to arsf user on a safe machine
     
    1012 # go find the path to the data to download
    1113 # these will vary..
    12 cd /data/Data_2009/ ##Also try /mnt/synology1/data/Data\ 2014/ or synology2
     14cd /data/Data_2009/
    1315cd UK
    1416cd 281-09_GB08-02_Delamere
     
    2628logout
    2729}}}
     30
     31== Looking for data on the disk station ==
     32
     33 1. change to arsf user on a safe machine
     34{{{
     35ssh arsf@gridmaster2 # (or su - arsf and don't turn your machine off!)
     36}}}
     37 1. find the exact data path on Ops server
     38{{{
     39ssh arsfdan@thelma.nerc-arsf.ac.uk
     40 # go find the path to the data to download
     41 # these will vary..
     42cd /mnt/synology1/ or /mnt/synology2
     43cd data/Data\ 2014/ or maybe Data/Data\ 2014/ or similar
     44cd 281-09_GB08-02_Delamere
     45 # check path
     46pwd
     47 # returns /mnt/synology1/data/Data 2014/UK/281-09_GB08-02_Delamere
     48 #
     49 # check size isn't crazy big
     50 # > 150GB and think about a disk transfer
     51du -hsc .
     52 # if you get any permission denied errors, fix up permissions then redo the du.
     53 # Not sure how for the disk station
     54logout
     55}}}
     56
     57== Getting the data ==
     58
    2859 1. create a temporary directory in ~arsf_data/YYYY/flight_data/unpacking/ to download to, e.g.
    2960{{{
     
    4374If something goes wrong after an hour or two, don't delete what you've already downloaded.  Instead, repeat the rsync command and it'll pick up where it left off.
    4475
     76=== Spaces ===
     77
     78Spaces in the path name need double escaping in rsync, eg. arsfdan@thelma.nerc-arsf.ac.uk:/mnt/synology1/Data/Data\\\ 2014/216-14_GB14-00_Riss_Cal_Hyperspectral
     79
    4580== Unpacking data ==
    4681