Version 123 (modified by knpa, 15 years ago) (diff)

--

Data Delivery

Preparation and verification steps are on another page? now.

Preparing hard disks for delivery

Plug in the external hard drive.

You will need to have root permissions before you start ('sudo su' and then type in your password).

Run dmesg to find device name. Listed at the bottom if just plugged in.

e.g 

SSELinux: initialized (dev sdb, type fuseblk), uses genfs_contexts

or 

sd 7:0:0:0: [sdb] 488397168 512-byte hardware sectors (250059 MB)

Device path is then /dev/sdb (not sdb1 etc, this indicates a partition on sdb)

Create a partition:
 run mount to find the name of the partition you need to unmount e.g /dev/sdb1

Unmount the partition with umount /dev/device_name, e.g. umount /dev/sdb1 (if not already unmounted)

Run fdisk /dev/DEVICE_NAME (not partition)

enter 'p' to print partition table and check you have selected the correct disk
enter 'd' to delete the current partition
enter 'n' to create a new partition
enter 'p' to make new partition the primary partition. If it asks you to give a partition number press 1 and enter. Then press enter twice.
enter 'p' to print new partition table – if all seems fine enter 'w' to write

Format the disk:

To be on the safe side, run dmesg again to make sure device name hasn't changed. You should now see the partition listed as device_name1, e.g. sdb1
Unmount the partition.

Run mke2fs -j /dev/partition

Change permissions:

Unplug the disk and plug back in. The disk space should be located under /media/disk.

It is wise to check that this destination is correct: 'df /media/disk' to check that this is where the device is mounted.

Make writable for everyone using chmod a+rwx /media/disk


Copy over your data onto the disk

Copy the delivery folder from the workspace to /media/disk. Top directory should be the one with the project code.


Finalising hard disk

Set permissions and owner:

chmod a+rX,a-w -R /media/disk
chown root.root -R /media/disk

NOTE: Be sure to record the number of the hard disk (or giving it one if it does not yet have one) before packing it.

Cover letter

There should be a cover letter with all deliveries. Templates can be found in ~arsf/arsf_data/YYYY/delivery/

Data quality report

There should be a hard copy of the most recent data quality report included with all deliveries. Print off the most recent one from ~arsf/doc

Trac and website updating

  1. Update trac ticket for this project:
    1. Add a comment that this data has been delivered (describe what was sent, to whom and when)
    2. Add date of delivery next to sensor name in ticket description.
  2. Update the status database to reflect the delivered data (​http://arsf-dan.nerc.ac.uk/status/addflight/editflight.php)
  3. Update the page saying who has our disks? (if sent on disk)
  4. Update the status spreadsheet (and record the time the project took you)

Email the PI

  1. email them that you're putting the disk in the post and to ask for confirmation of receipt
    1. Please CC to arsf-processing@… to keep others informed
    2. Please set a reply-to header to arsf-processing@… to ensure replies don't go to personal mail

Subject:

Notification of ARSF data delivery for <PROJECT CODE>

Reply-to and CC headers:

arsf-processing@…

Body:

(note you need to fill in PI, DATE, NOTES, TICKETNO, also mention if you aren't sending all their data at once) 

Dear <PI>,

--PICK ONE OF THE NEXT THREE SENTENCES (HDD, DVD, FTP)--
This is to notify you that we have just dispatched your ARSF data on a USB hard disk formatted with the Linux ext3 filesystem.  Please let us know when the data arrive so we know not to resend.  We'd appreciate it if you could copy the data onto your system and return the disk to us (see below for address) for reuse.  
This is to notify you that we have just dispatched your ARSF data on DVDs.  Please let us know when the data arrive so we know not to resend.
This is to notify you that we have placed your ARSF data on a FTP server accessible at <FTPADDRESS>.  Please let us know when you have downloaded the data so we can reuse the space.

The delivery comprises:
 - Specim Eagle flight lines taken on <DATE>
 - Specim Hawk flight lines taken on <DATE>
 - LIDAR flight lines taken on <DATE>
 - Digital Photography taken on <DATE>

<NOTES - any other notes, including what data is held back>

Information about the processing of the project is included with the delivery, but our internal notes are also available at:
 http://arsf-dan.nerc.ac.uk/trac/ticket/<TICKETNO>

Regards,


ARSF Data Analysis Node.
Plymouth Marine Laboratory,
Prospect Place,
Plymouth.
PL1 3DH.
UK.

Email: arsf-processing@pml.ac.uk
Tel: +44 (0)1752 633432
Fax: +44 (0)1752 633101
Web (NERC): http://arsf.nerc.ac.uk/
Web (processing wiki): http://arsf-dan.nerc.ac.uk/trac/

Next stage

  1. If this was the final delivery and all sensors have been rsynced back to respository then set ticket to 'archiving'

This section relates to burning data to DVD

Burning the data

  1. create an iso image with mkisofs -R -udf -verbose -o /tmp/dvd.iso -V <PROJECTCODE-JULIANDAY> <delivery_directory>
  2. test the iso with sudo mount -o ro,loop /tmp/dvd.iso /mnt/tmp - mounts to /mnt/tmp, go look at it, then unmount with sudo umount /mnt/tmp (yes, that is umount not unmount). Create /mnt/tmp if it doesn't exist.
  3. if it's ok, burn to CD/DVD with sudo cdrecord dev=/dev/dvd -sao fs=100m driveropts=burnfree -v /tmp/dvd.iso
  4. clean up with rm /tmp/dvd.iso
  5. check the dvd on another machine (ideally Windows)

Lightscribe

  1. open the CD template from ~arsf/arsf_data/2007/delivery/cd_covers/arsf_cd_label_gimp_editable.xcf in gimp
  2. lick on the text button in the toolbox, then click the GB... text and edit to reflect the correct project code
  3. save as a .bmp file somewhere, e.g. ~/ipy0708214ef.bmp
  4. Insert the disk (upside down) on the lightscribe machine (currently pmpc974) and run 4L-cli print /dev/sr0 ~/ipy0708214ef.bmp

If it appears to be stuck on "Starting up", it may still be burning - give it time to complete. For a dense full disk picture, wait at least 45mins!

If the drive seems wrong, try 4L-cli enumerate to list attached lightscribe compatible drives.