Transfers directly to the to Cuppa cluster are a temporary measure while the PBstore is offline. The process uses Tsunami and is mostly similar to the Bonn transfers.
First move to the directory where the data to be copied is located with
cd /exports/bfg_internal8/aust12hb/ and then start the server with
tsunamid --port=52100 aust12*. Start another terminal then ssh into the remote server using the appropriate login. All Hb and Yg data should be transferred onto cuppa24.cira.curtin.edu.au, while Ke data should go to cuppa23.cira.curtin.edu.au.
|Curtin||cuppa23||ssh -i .ssh/id_dsa_observer@bfg firstname.lastname@example.org|
|Curtin||cuppa24||ssh -i .ssh/id_dsa_observer@bfg email@example.com|
Please note that if you're transferring the data from another machine, you'll need to change the identity file (.ssh/id_dsa_observer@bfg) to match. Tab-completion from .ssh/id_dsa should work.
Once logged, create a directory on the remove server for the data to be copied to, and then
cd into it. Cuppa uses a syntax where these commands would be appropriate:
mkdir /mnt/raid/transfers/aust12/Hb/ ; cd /mnt/raid/transfers/aust12/Hb/
Once in the correct directory, start the tsunami client with
tsunami. You will then need to set the data transfer rate in the tsunami program with
set rate 200m and also the UDP port with
set udpport 52100. Bear in mind the current status of the Mt Pleasant network (total throughput can be seen here). With an otherwise free link between Hobart and Bonn, you can get ~ 600 Mbps throughput when reading from an NFS-mounted RAID.
You now need to connect the client to the server with the tsunami command
connect IP 52100, replacing with the address of the relevant server in Hobart. The list is included below. 52100 is the port used, and should always be 52100
You should see a connect message in both the server and client terminals. If you want to test that the transfer rate chosen is sustainable, try transferring a single file using the tsunami command
get filename. Pick a smaller file for test transfers. During the transfer, a lot of debugging information is present - the second last column on the client shows the current number of dropped packets. It's ok to see a small number of these (<10) but larger numbers indicates a connection problem. Try reducing the rate and repeat the tests until the transfer is successful. Use the tsunami command
get * to transfer all files hosted by the server.
If you need to restart a transfer, or re-transfer a subset of files, then start the tsunami server with something like
tsunamid –port=52100 r1599_hb_234-15* r1599_hb_234-16*. This is necessary as tsunami defaults to overwriting any existing files and it starts from the top of the list.
When you start a transfer (or when one completes) please update the Bonn e-transfer website. This is done by copying an empty file with a properly formatted name to Bonn. There is a script on cornucopia, vortex and jupiter called ncBonn.sh which will assist you in this, but the full instructions are included below.
To send a start message, use the following commands:
ncftpput ftp.mpifr-bonn.mpg.de /incoming/geodesy/transfers 20130902033000_aust12hb_Hb209_Curtin_250m_52100_cuppa24_start
The underscore-separated items are respectively a timestamp in UT, the experiment being transferred, the tsunami server (Hb for Hobart and 177 for the last number in the IP address), the destination, the transfer rate, the transfer port, the destination raid and lastly the word start.
When a transfer completes, send a stop message with
ncftpput ftp.mpifr-bonn.mpg.de /incoming/geodesy/transfers 20130902033000_Hb209_stop
You must use the same serial number (timestamp) as in the start message.