User Tools

Site Tools


operations:documentation.ivs.bonnetransfer

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision
Previous revision
Last revision Both sides next revision
operations:documentation.ivs.bonnetransfer [2012/07/09 00:52]
Jamie McCallum
operations:documentation.ivs.bonnetransfer [2012/11/13 23:35]
Jamie McCallum
Line 1: Line 1:
 ====== Transferring Data to Bonn ====== ====== Transferring Data to Bonn ======
 +  * Use a VNC session to run the transfers - the hobart.phys.utas.edu.au:​1 session is normally a good option. ​
   * Open four terminal windows on the Mark5 unit the data will be sent from   * Open four terminal windows on the Mark5 unit the data will be sent from
-  * In the first terminal, make sure dimino ​isn'​t ​running <​code>​ps -ef | grep dim</​code>​ and that another transfer isn't running <​code>​ps -ef | grep tsuna</​code>​.  +  * In the first terminal, make sure that neither fuseMk5 or dimino ​is running <​code>​ps -ef | grep dim</​code>​ and <​code>​ps -ef | grep fuse</​code>​ and that another transfer isn't running <​code>​ps -ef | grep tsuna</​code>​.  
-  * Check no moudules are mounted ​at /​mnt/​diskpack or /​home/​oper/​wash. If they are, dismount with <​code>​fusermount -u <​directory_name></​code>​ +  * If fuseMk5A is running and the wrong module is mounted, dismount with <​code>​fusermount -u <​directory_name></​code>​ 
-  * Then use fuse to mount the module: <​code>​fuseMk5A ​--bank=0 --verbose --cachesize=128000000 /​mnt/​diskpack</​code>​ This will mount Bank A (set bank=1 for Bank B) at /​mnt/​diskpack. Wait for a message to say <​code>​.... Registering fuseMk5A to FUSE with: ....</​code>​. It can take a while.+  * Then use fuseMk5 (or fuseMk5A for unupgraded mark5 units) ​to mount the module: <​code>​fuseMk5 ​--bank=0 --verbose --cachesize=128000000 /​mnt/​diskpack</​code>​ This will mount Bank A (set bank=1 for Bank B) at /​mnt/​diskpack. Wait for a message to say <​code>​.... Registering fuseMk5A to FUSE with: ....</​code>​. It can take a while. Background this process with Ctrl-z and <​code>​ bg </​code>​
   * In a second terminal, change to the mounted directory and make sure the data are there <​code>​cd /​mnt/​diskpack   * In a second terminal, change to the mounted directory and make sure the data are there <​code>​cd /​mnt/​diskpack
 ls ls
Line 11: Line 11:
   * In the third terminal window, log on to the computer at Bonn that will receive the data and change to the directory where the data will be sent <​code>​ssh evlbi@<​machine.address>​   * In the third terminal window, log on to the computer at Bonn that will receive the data and change to the directory where the data will be sent <​code>​ssh evlbi@<​machine.address>​
 cd /​data3/​r1/​hobart12/​r1485/​ cd /​data3/​r1/​hobart12/​r1485/​
-</​code> ​+</​code> ​The [[http://​www3.mpifr-bonn.mpg.de/​cgi-bin/​showtransfers.cgi 
 +  |Bonn web page]] shows available servers and space. 
 +(194.94.199.166 = sneezy2) 
   * Start tsunami and set up for a transfer   * Start tsunami and set up for a transfer
 <​code>​ <​code>​
-~/​australia/​tsunami-udp-r42/​client/​tsunami+tsunami
 set rate 300m set rate 300m
 connect 131.217.63.175 52100 connect 131.217.63.175 52100
 get * get *
 </​code>​ </​code>​
-  * in the fourth terminal window, create an empty file with a name describing the start time of the transfer, rate etc: <​code>​touch ​20110525033000_r1485_Hb_Bonn_300m_52100_start</​code>​ This puts an entry on the [[http://​www3.mpifr-bonn.mpg.de/​cgi-bin/​showtransfers.cgi+  * in the fourth terminal window, create an empty file with a name describing the start time of the transfer, rate etc: <​code>​touch ​20110525033000_r1485_Hb_Bonn_300m_52100_sneezy2_start</​code>​ This puts an entry on the [[http://​www3.mpifr-bonn.mpg.de/​cgi-bin/​showtransfers.cgi
   |Bonn web page]] to say there'​s a transfer under way   |Bonn web page]] to say there'​s a transfer under way
-  * and send it to the web server <​code>​ncftpput ftp.mpifr-bonn.mpg.de /​incoming/​geodesy/​transfers ​0110525033000_r1485_Hb_Bonn_300m_52100_start</​code>​+  * and send it to the web server <​code>​ncftpput ftp.mpifr-bonn.mpg.de /​incoming/​geodesy/​transfers ​0110525033000_r1485hb_Hb177_Bonn_300m_52100_sneezy2_start</​code>​
   * When the transfer finishes, send a stop message with <​code>​touch 20110525033000_Hb_stop   * When the transfer finishes, send a stop message with <​code>​touch 20110525033000_Hb_stop
-ncftpput ftp.mpifr-bonn.mpg.de /​incoming/​geodesy/​transfers ​20110525033000_Hb_stop</​code>​+ncftpput ftp.mpifr-bonn.mpg.de /​incoming/​geodesy/​transfers ​20110525033000_Hb177_stop</​code>​ 
 + 
 +It is also possible (and preferable) to export the data onto a RAID first. Currently, there are three RAIDS which can beused for temporary storage at Hobart - Vortex, Cornucopia and 131.217.63.140 (Currently unnamed and contains two separate RAIDS of 6 and 12 TB). These can be NFS mounted on the mk5 machine with <​code>​mount vortex.phys.utas.edu.au:/​exports/​vortex_internal/​ /​mnt/​vortex</​code>​ (as root). Once mounted the data can be copied with <​code>​cp /​mnt/​diskpack/​r1485* /​mnt/​vortex/​r1485hb/</​code>​. 
 + 
 +Once fully copied, please check the data has not been corrupted in the transfer. ssh into hex0 and NFS mount the RAID if necessary. Then run <​code>​directory2filelist /​mnt/​vortex/​r1485hb Mark5B-256-16-1 > /​tmp/​ok.txt</​code>​ . The output to the screen will be the list of those scans which show problems. It's not uncommon to see up to ~10% of the scans fail the test - these files should be deleted and retransferred to the RAID.  
 + 
 +The same <​code>​directory2filelist</​code>​ procedure should be run after a transfer to Bonn completes to check for errors in transmission. ​
/home/www/auscope/opswiki/data/pages/operations/documentation.ivs.bonnetransfer.txt · Last modified: 2013/07/12 06:30 by Jamie McCallum