User Tools

Site Tools


operations:handover

Handover notes

Please use this page to write up any notes on anything that needs to be passed on to the next observer or should go into the end of experiment message.

Notes should be placed below the horizontal line as a new blog entry.

Continuing problems should be copied to the Current Issues page.

**Template for Handover notes:**

Be sure to add any specific notes about the experiment if required, for example “Special back end configuration for x reason”. Put as much detail as you can - the more the better.

It is ESSENTIAL to put your name after EVERY entry. Copy and paste the template into the new handover notes

====== Experiment Name ======

**Hobart 12m**

Disk VSN: abc-123 | Data volume at beginning: 2.34 GB

   * 1300UT a comment from me (user)

**Katherine 12m**

Disk VSN: | Data volume at beginning:  GB

   * 

**Yarragadee 12m**

Disk VSN: | Data volume at beginning:  GB

   * 1300UT Experiment started okay (JMc).

Format for entering a comment It is important to stick to this formula - there is a good reason we need this information, so please include it!

Time stamp in UT * Comment * (Your Name)

For Example:

20:00UT: Wind stows, Missed scans 012-3456 to 012-3457 (Jim)

20:50UT: Mark5 lost connection, had to restart Mark5 and reconfigure DBBC, called Jamie on call to fix, missed scans 012-3456 to 012-4567 (Jim)

Again - as much detail as you can. It is important to remember that these notes will be sent to the correlator, so please make sure they make sense to others.


RD1810

Hobart 26m

Disk VSN : USN-0102/6000/1024. Data volume at the beginning 0 GB

  • 1800 UT: Experiment started OK (Jay).
  • The clkoff and maserdelay commands not always return a value. Informed to Warren who's having a look at it (Prad).
  • 0607 UT: clkoff, maserdelay fixed. Fixed by typing the commands in hb12 fs and the commands in newsmerd became responsive again. (Atifur)
2018/12/12 23:30 · Tiege McCarthy

AOV030

Hobart 26m

Disk VSN : PKS+0011/8000. Data volume at the beginning 4103.707 GB

  1. 1730 UT: Experiment started OK (mas)

Katherine 12m

Disk VSN: HOB+0067/8000. Data volume at the beginning 0.0 GB.

  1. 1730 UT: Experiment started OK (mas).
  2. Couldn't get value for maserdelay and clkoff. Fmset is OK (mas)

Yarragadee 12m

Disk VSN: HART+104/8000. Data volume at the beginning 0.0 GB.

  1. 1730 UT: Experiment started OK (mas).
2018/12/11 18:06 · Tiege McCarthy
R1872

Katherine 12m

Disk VSN: USN-0095/4000. Data volume at the beginning 0.769 GB.

  • 1700 UT - Experiment started OK (Prad).
  • 0840 UT - Noticed system temperatures had not being recorded since setup so restarted dbbc. First usable scan is 345-0850b (Ross).
  • At 13:16 UT - received error antenna communication software (antcn) problem. Halt the schedule. Received continuous Alarm: error f1 -1 previous source in this schedule not reached before new source was commanded. Reboot the antenna system . Missed scans 345-1320 till 345-1354 (mas)

Yarragadee 12m

Disk VSN: USN-0150/4000. Data volume at the beginning 3.654 GB.

  • 1700 UT - Experiment started OK (Prad).
2018/12/10 16:55 · Tiege McCarthy
R4871

Katherine 12m

Disk VSN: USN-0173/2000. Data volume at the beginning 0.0 GB.

  • 18:30 UT - Experiment started OK. (Atifur)
  • 18:55 UT - Frequently receiving Antenna communication error (mbus error)(every 5-10 mins). Fixed rapidly with antenna=open, antenna=operate. No data lost so far.
  • 10:07 UT - ALARM: error m5 -104 mk5cn: time-out, connection closed. [no action taken, checked mk5 okay and auto-correlations seems okay, onsource is tracking] (Lim)

Yarragadee 12m

Disk VSN: USN-0005/2000. Data volume at the beginning 0.0 GB.

  • 18:30 UT - Experiment started OK. (Atifur)
2018/12/06 13:44 · Atifur R. Khan · 0 Comments
AUA047

Hobart 26m

Disk VSN: HOB+0120 | Data volume at beginning: 12989.266 GB

  • Needed to restart OTTER on observer@rx26m as it crashed? (Lucas).
  • Experiment started late. Antenna drives was in weird state, continually getting 'Error: Allocating memory' for #atcn#. Also telescope thought wind was high and was wind stowed. Monica (desktop 5 newsmerd vnc) reported wind as quite normal.
    • Fixed problem by resetting the drive pc (vdesk would not load sys26m which was a large diagnostic) with command: rem_reboot -r rakbus sys26m. For good measure, also opened vdesk with sys26m and abort, abort-ed. Then reset field system and voila, it worked. note - found this fix on the ra-wiki (not auscope wiki).
    • First scan is 3338-1857, all previous scans missed.
  • sx receiver parameter pres out of range showed up multiple times from 1414UT. However the values on the live page seem to be fine (Prad).

Katherine 12m

Disk VSN: MAT+0008/8000 | Data volume at beginning: 0.0 GB

  • Experiment started one scan late, missed scan 338-1730 (Lucas).
  • 10:30 UT, started getting alarms about:

2018.339.10:52:46.01?ERROR st -5 Error return from antenna, see Mbus error. 2018.339.10:53:01.01?ERROR st -998 reading SystemClock1 2018.339.10:53:01.01?ERROR st -999 TCP/IP connection was closed by remote peer

and antenna=open, antenna=operate did not solve the issue. I first reset the drives on timeke, and when that also failed to work after open/operate, rebooted the system in timeke. Then after antenna=open, antenna=operate this worked. Normally this is not necessary but this time it was - Lucas.

Yarragadee 12m

Disk VSN: | Data volume at beginning: 0.0 GB

  • 17:45 UT Experiment started late, ping to telescope was so bad it took a very long time for the setup commands to successfully and and correctly go through the network (Lucas).
2018/12/04 18:58 · Tiege McCarthy · 0 Comments
R1871

Katherine 12m

Disk VSN: NRAO+144/2000/ | Data volume at beginning: 0.449 GB

  • 1700UT - Experiment started OK (Tiege)

Yarragadee 12m

Disk VSN:NRCAN-05/2000/ | Data volume at beginning: 0.224 GB

  • 1700UT - Experiment started OK (Tiege)
  • 1711UT - Apparently only for one scan though, pps_delay1 value is all over the place (+ve ~2000000000). Setting off alarms. Auto-correlations look very bad. Restarting software on DBBC was not effective. Trying a power-cycle now. (Tiege)
  • 1733UT - Power-cycling DBBC unsuccessful, looks like even after fmset, times are still not synced up (though sync error goes away). Going to try rebooting mk5 for good measure. (Tiege)
  • 1740UT - This didn't work either, currently stumped. (Tiege)
  • 1750UT - reset hwclock on pcfsyg, and rebooted machine. Hasn't fixed issue. mk5 time seems to be consistent wit UT, but field system (and maybe DBBC) is off slightly. (Tiege)
  • 1800UT - fmset clocks being different seems to be unrelated, on second DBBC reboot, clocks now line up. However clkoff value is still far too high (by factor of 10000). It is definitely not a monitoring issue also, as webcam confirms high value, along with pps_delay being far too high. (Tiege)
  • 1815UT - Called Lucas, he could not figure out what was wrong with it. Seems more and more likely its a HW issue. 'bread' command shows a lot of zeroes, and the IFs with zeroes change between DBBC reconfigs/reboots (Tiege)
  • 1830UT - Have contacted Yg on-site person, nothing looks out of the ordinary (but they are new). Getting them to manually power cycle the DBBC. (Tiege)
  • 1852UT - Still getting nonsense negative pps_delay and too large clkoff values. Going to stow dish and get Warren/Brett to take a look in the morning (Tiege)
  • 0423UT - The PCI 7200 card's been replaced (which was expected to be causing the problems) and the experiment appears to be back up however with a warning saying WARNING: 0 : 0x00340003 : 1028 : interrupts error from mark-5b dim or dom (Prad).
  • 04:36 UT - John did hardware fix on DBBC replaced pci-7200 card and necessarily reconnected input cables etc. Did usual setup stuff, reconfig dbbc, restarted field system, fmset, seems good now, bread sensible, getting steady pps_delay 61971, autocorrelation spectra good, syncerr_eq_0, start again from first good scan 338-0436 (warren)
  • 0456UT -DBBC drift and mk5 sync error. Scan 338-0453 to 338-0512a affected. The scan resumed from the 338-0515, however, this was after a momentary anomaly (between channel 14 and 15) in the correlations which Lucas pointed out (Prad).
2018/12/03 14:13 · Tiege McCarthy · 0 Comments
/home/www/auscope/opswiki/data/pages/operations/handover.txt · Last modified: 2018/10/29 16:27 by Jesse Swan