Changes between Version 9 and Version 10 of Procedures/NewDataArrival/Network

Timestamp:

Aug 7, 2014, 9:32:25 AM (10 years ago)

Author:

mark1

Comment:

--

Procedures/NewDataArrival/Network

@@ -1 +1 @@
 = Data arrival by network transfer =
+
+== Looking for data on Thelma ==
 
  1. change to arsf user on a safe machine
@@ -10 +12 @@
  # go find the path to the data to download
  # these will vary..
-cd /data/Data_2009/ ##Also try /mnt/synology1/data/Data\ 2014/ or synology2
+cd /data/Data_2009/
 cd UK
 cd 281-09_GB08-02_Delamere
@@ -26 +28 @@
 logout
 }}}
+
+== Looking for data on the disk station ==
+
+ 1. change to arsf user on a safe machine
+{{{
+ssh arsf@gridmaster2 # (or su - arsf and don't turn your machine off!)
+}}}
+ 1. find the exact data path on Ops server
+{{{
+ssh arsfdan@thelma.nerc-arsf.ac.uk
+ # go find the path to the data to download
+ # these will vary..
+cd /mnt/synology1/ or /mnt/synology2
+cd data/Data\ 2014/ or maybe Data/Data\ 2014/ or similar
+cd 281-09_GB08-02_Delamere
+ # check path
+pwd
+ # returns /mnt/synology1/data/Data 2014/UK/281-09_GB08-02_Delamere
+ #
+ # check size isn't crazy big
+ # > 150GB and think about a disk transfer
+du -hsc .
+ # if you get any permission denied errors, fix up permissions then redo the du.
+ # Not sure how for the disk station
+logout
+}}}
+
+== Getting the data ==
+
  1. create a temporary directory in ~arsf_data/YYYY/flight_data/unpacking/ to download to, e.g.
 {{{
@@ -43 +74 @@
 If something goes wrong after an hour or two, don't delete what you've already downloaded.  Instead, repeat the rsync command and it'll pick up where it left off.
 
+=== Spaces ===
+
+Spaces in the path name need double escaping in rsync, eg. arsfdan@thelma.nerc-arsf.ac.uk:/mnt/synology1/Data/Data\\\ 2014/216-14_GB14-00_Riss_Cal_Hyperspectral
+
 == Unpacking data ==