User Tools

Site Tools


operations:documentation_ops2_pcfs9.10.5plus

Operator Documentation for observations

Background

IVS Observation Instructions

Below are the links to a step-by-step procedure for making IVS observations on the 12m and 26mtelescopes. The current version of e-RemoteCtrl has a built-in checklist which can now be used during the setup procedure. So the first task is to start the Observing and monitoring software at each site, then e-RemoteCtrl.

During the AUST CONT session in June 2015, perform station checks in the assigned time using the CONT antenna checkout procedure while setting up, follow the standard instruction for 12 m telescopes:

Monitoring Observations on Parkes 64-m

Observations on the 12m telescopes from the Control Room

These are notes on IVS observations from ops2, the six-monitor PC in the AuScope VLBI Operations room in Hobart.

If you are unfamiliar with the computers and software used at the sites and in the control room, please read this before continuing:

Before the Observations

Prepare the schedule file

Timing PC Software

Check that the software on the timing PCs is running:

Field System Software

System Monitor

Monica

e-Remote Ctrl

The checklist should now be visible. The items in the list are described below and can be ticked off as you go

  • For your convenience you may wish to use Multi Command Input – moprin (written by Dave). The link to moprin is located in the upper left screen (alternatively open a terminal on ops2 and type 'moprin'), together with the other helpful tools like Tsys displays and backup control – ssh-based remote operator input. With moprin you can send the same command to all three telescop operator input simultaneously. Useful for pointing, ckloff, maserdelay, terminate at the end of an experiment and so on. Also you can incert the telescope name by typing $s instead. For example, if you wish to set up relevant log names at each station, you can type
    log=crf86$s

    You can use it for setting up procedures and starting schedules as well. For more information (e.g. how to change the stations, type 'help' into the moprin terminal.

The Before Observation Checklist

The RF and IF signal paths, and DBBC are now configured through the Field System.

RF and IF signal path
DBBC Configuration

The Digital Base-Band Converter (DBBC) takes the IF signals from the telescope and digitises them for the Mark5 recorder. The DBBC needs to be properly configured prior to an observation. Check that the DBBC is healthy and the server software is running:

To configure the DBBC with the correct IF and frequency settings, use the following command

setupsx

setupsx calls ifdsx which sets up the DBBC Conditioning module inputs, filters and power levels, then dbbcsx8 (or dbbcsx4 for a 4 MHz DBBC mode) which sets the BBC frequencies. Then if you type:

iread

You will see what the Conditioning module settings are. Output format is:

<time>/<Module label>/<IF input number>,<Auto or Manual gain control>,<Nyquist filter number>,<Target power level>,<attenuation>,<actual power level>

Check there’s agreement with what appears in ifdsx, as well as the values listed here. The actual power level should agree pretty well with the target level. The attenuation number can be anywhere between 0 (none) and 63 (maximum). If you see it at 0 or 63, it means the Conditioning module is having trouble getting the power to the right level. See the Current Issues page for information on how to do this.

Then type:

bread

You will see what the DBBC has set the BBC freqs to (compare with the .prc file). Output looks like this:

<time>/<bbc name>/<Frequency (MHz)>/<Conditioning module in use>,<Bandwidth (MHz)>,...

the Frequency, Conditioning module label and bandwidth should agree with the listing in dbbcsx8 (or dbbcsx4).

Check the maser
Mark5 Recorder setup

Next the Mark5 recorder needs to be prepared for the observations.

Check that a suitable module is in the Mark 5 recorder.

If there is no specific module allocated to the experiment, the relevant column on the spreadsheet will be left blank.

Clock and delay check
Test recording
System Temperature (Tsys) check
Antenna checks
Weather checks
Fringe check
Skype
  • Start Skype and begin a chat session with all participating sites plus the on-call person

Send a Station Ready email

Lastly, after submitting the checklist form, send a ready message to IVS:

Starting and Monitoring

After the Observations

eVLBI observations

Fringe Checking

Alarms and Errors documentation

New content! Please add! Here is an ever growing list of common errors and alarms that you may come across whilst observing. Some are diagnostics and others are plain alarms. If you come across more while observing please add them to this page and note the data and time of the occurrence and your fix. Or, if you couldn't resolve it leave it blank so observers know that it is a problem that was encountered and this would be a good time to call an on-call person List of alarms and fixes.

Observations on the 26m telescope from the Control Room

Note: these notes are based on the 26m setup notes from the ra-wiki and are still in a terse, draft form - Jim Lovell

Before Setup:

1. Drudg schedule for the experiment (if it has not been done already). Drudg the schedule on newsmerd ( ssh observer@newsmerd ). with

./slogit.ivs.sh

and say yes to the print command.

From ops2, ssh into hobart (ssh oper@hobart) and copy the .sum file with the command

scp /tmp/sched.tmp observer@ops-serv2:/vlbobs/ivs/sched/aug019ho.sum

.

If the .sum file was not written:

a) before the experiment started: drudge the schedule manually. On Hobart, go to the sched directory and type drudg r4705.skd. Then type Ho for station Hobart and then 3 to create the prc file and 5 to write the sum file. The sum file is written in the ./tmp directory as sched.tmp.

b) after the experiment has started: in order not to overwrite the prc file, copy the schedule file to the tmp directory and drudg it there.

2. Check to see if Jim P (or anyone else!) is observing.

CHECK THE CALENDAR!

Go to the 26m Live Page for the 26m and see if the telescope is tracking. Jim P puts a sticky note on the screen 5 where the Field System is saying he is observing also.

Please wait until the other person's experiment has finished until you set up.

If it is urgent, please see the notes on his instructions page on how to cancel the Vela observations:

http://ra-wiki.phys.utas.edu.au/index.php?n=MtPleasant26m.PulsarObserving

DO NOT SHUT DOWN THE FIELD SYSTEM. The field system is used to monitor the S-X temperatures, and if you shut down the field system, it will stop the temperature monitoring which is essential for the observations

Setup of Telescope:

Step 1. Open VNC viewer to newsmerd

vncviewer newsmerd:1

The password is the usual one.

Step 2. Check that the field system is running on newsmerd. Usually, the Geodesy window will be Desktop 5

Step 3. Check receiver position

IS THE S-X RECIEVER ON AXIS? MAKE SURE IT IS!

Check the receiver position with the command rxp. It should be -256, +248, +18 (x,y,z).

Be sure the telescope is stowed BEFORE changing the reciever

To change it go to Desktop 3, bottom right window and enter '1' (OTTER interface). Changing the receiver takes a couple of minutes (check on the live page http://ra-state.phys.utas.edu.au/ until the antenna stops moving - 5 sec update rate).

Check that the receiver is actually moving, and wait until it reaches position before moving the telescope (can take a few minutes if moving from L-Band).

Stuck Receiver

If the receiver is stuck, first try a reset in the OTTER interface on the newsmerd vnc desktop 3 (type r and hit enter).

Sometimes the OTTER interface crashes/fails to communicate with the reciever. If the GUI fails to start or returns error messages, please check that it is running (with ps -ef | grep OTTER as observer@rx26m). You can restart the server with sudo /etc/init.d/OTTER restart if there are any problems. The server is being continuous polled by MoniCA with the data being logged in the hobart database. More information on the OTTER server can be found hereOTTER

Step 4. Check the procedure file The procedure file will be in /usr2/proc and called <experiment name>ho.prc. It may need editing. Check recent similar .prc files, but probably just the ifd and if3 attenuations need changing. In which case, you're looking for a procedure called ifdsx ((instead of sx, it sometimes is 8f, 01 etc). You can edit the file directly with your favourite editor or use pfmed, like this (example is for r1704):

oper@hobart:~$ pfmed
pfmed:pf,r1704ho
pfmed:dl   [this lists all the procedures defined in r1704ho.prc]
pfmed:ed,ifd8f

This will start an editor and you can change the attenuator values:

ifd=4,7,nor,nor 
if3=4,in,2,2,,,on

it is safe to change the procedure during observation. To increase attenuation, put a higher number in. (e.g. if3=5, in,2,2 ….). Hereby, if1 is for vc 1-4, if3 for 5-8 and if2 for 9-14.

Make sure you exit from pfmed by typing exit or the changes won't be propagated.

Note that pfmed can be used to edit procedure files live. i.e. when you change a procedure with pfmed, the field system will use the new version next time it comes along in a command or the schedule (you don’t have to type proc=<whatever> or restart the schedule first). However, if you use pfmed before starting a schedule, the only procedure file it knows about by default is station.prc which contains the default procedures. You have to tell pfmed which procedure file, specific to your experiment, to use if you’re editing a procedure prior to starting a schedule.

Step 5. Mark 5 setup:

The 26mk uses mkv (oper@mkv) for recording.

Log in to oper@mkv. Mark5A is this Mark 5's version of DIMino. Check to see if Mark5A is running with

ps auxww | grep -i mark

If Mark5A is running, it will return an output like this:

root     10832  5.2  0.5  40564 12984 ?        Rl   May08 2666:16 Mark5A -m0
oper     32071  0.0  0.0   2852   544 pts/2    D+   04:14   0:00 grep -i mark

If the first line in this output does not appear, Mark5A is not running. Run it with

Mark5A -m0 &

Check the VSN of the module using

tstDIMino
vsn?

Check if there is any data on the module by using the command DirList. If there is (and there is supposed to be data on there), note the end pointer (as in how much data has been recorded so far on that module) in the Handover Notes

NOTE: If there is no free space on a module in the Mark 5, call your on-call person for assistance in changing modules.

Step 6. Start eRemoteCtrl.

It is availalbe from the drop down menus as it is for the 12m telescopes.

Step 7. Start the Log Monitor as for the 12m telescopes (from the drop down menu). Start a new log file and begin recording to it.

Step 8. Load the experiment procedure file (if you haven't already)

proc=rd1234ho

note the 'ho' on the end for Hobart 26m.

Step 9. Run the setup of the experiment using

 setupsx
  1. Run the setup. The command for an R1 experiment is setupsx but look in the experiment procedure library printout to find the correct setup procedure name. Watch for warnings1 (e.g. WARNING: error ma -829 mk4 fm illegal sample rate is critical!)
  2. Use the remote controllable camera ptzhb to view the MarkIV rack for some of the next steps.
  3. Check all the Video Converters (VCs, or BBCs) are happy, i.e. no red lights and no nasty messages appearing in the log. Command valarm will turn off any remaining alarm lights on the VCs.
  4. Check that if3 (second from the bottom, second from left) is happy with no red lights. You may need if3=alarm
  5. Enter check=*,-v1 and check=*,-ifd into operin in the fs
  6. Check the formatter error light is off (bottom right; rigth from big green display). You may need to do form
  7. Check times are ok with fmset on oper@hobart.
  8. Check that power reading for each VC USB (Upper Side Band) is between 0.1 and 1.0 volt (if not (and telescope is onsource) you will need to adjust attenuations in the proc file. See pfmed above)
  9. Set the cal to be controlled by PCFS.
    observer@newsmerd: calu -m pcfs -r rakbus
  10. Check that caltsys works. Caltsys should return results between 50 and 70K
  11. Check that weather works. wth
  12. Check that fmout-gps works. clkoff
  13. Check that the Mark5 has a blank module big enough for the experiment. Compare the Drudg summary of the schedule with the label on the front of the disk module.
  14. Run disk_pos to check the disk module is empty. Result should be 0/0 otherwise the disk module contains data.
  15. Record some data. disk_record=on
  16. Observe green disk activity lights on module
  17. Stop recording. disk_record=off
  18. Check the recording. scan_check
  19. Observe the time and details in the log window for the test recording.
  20. Start the schedule. schedule=r1186ho,#1
  21. Watch PCFS log for errors.
  22. Check the antenna is on source. onsource

Lastly, send the start message.

Once the schedule starts, watch the log and everything closely for the first scan or so.

Manual Re-starting the drives on the 26-m

Sometimes the drives need to be manually re-started. Documentation on how to do this can be found here.

Monitoring Observations on the 26-m

  • Please note that e-RemoteControl does not adequately show the recorded/free space on a module in mkv. To see how full is the module, please use commands mk5=bank_set (module capacity) and disk_pos (write pointer position).
  • Parameters of the check-list during observation for 26-m are explained here.

After 26m Observations

DO NOT SHUT DOWN THE FIELD SYSTEM. The field system is used to monitor the S-X temperatures, and if you shut down the field system, it will stop the temperature monitoring which is essential for the observations

Send the end message

Park the dish with

 source=stow 

in the Field System

Monitoring Observations on Parkes 64-m

User accounts and email addresses

Gmail accounts have been set up for registration with skype and other services. The usual Observer password applies.

Katherine

gmail: keobserver@gmail.com

Skype: keobserver

Yarragadee

gmail: ygobserver@gmail.com

skype: ygobserver

Hobart 12m

gmail: hbobserver@gmail.com

skype: hbobserver

/home/www/auscope/opswiki/data/pages/operations/documentation_ops2_pcfs9.10.5plus.txt · Last modified: 2017/03/17 00:43 by Jim Lovell