the AUM experiments are designed to investigate and hopefully demonstrate the compatibility of the new wideband VGOS receiver with the existing S/X system. This is important for maintaining a global geodetic network during the transition to a fully global VGOS array.

The Hobart12 antenna now has a functional wideband system, with reasonable sensitivities. The initial observations of the AUM series will be made using the “VGOS”-mode DBBC3 software, recording channels with an (excessive) bandwidth of 32 MHz which will be dealt with in the correlation stage. If the current issues with the DBBC3 and its “Legacy”-mode software are resolved, we sill change over to use this instead, so the instructions may be updated - please check the page carefully each time you are starting or monitoring an AUM experiment.

The major differences for observing an AUM experiment rather than a standard S/X experiment are that there is no FS support for the hardware and that we need to run two DBBCs simultaneously to record the S- and X-band data. The recorder has also changed from the mark5B to a flexbuff which operates in a very different fashion. Preliminary scripts have been written to monitor the relevant parameters but again, these are likely to be changed and improved as needed.

Taking the existing e-remote control checklists as a template, here is a listing of the significant differences.

• Preparing the schedule file: Do not use the existing slogit scripts, but instead download the skd file directly to pcfshb and run drudg to generate the prc, snp and sum files and put them where they're needed.

cd /usr2/sched
wget ftp://cddis.gsfc.nasa.gov/vlbi/ivsdata/aux/2018/aum001/aum001.skd
drudg aum001.skd
   hb
   12
   3
   5
   0
scp /tmp/sched.tmp observer@ops-serv2:/vlbobs/ivs/logs/aum001hb.sum

The next step is to convert the existing snp file into one which will control the flexbuff recordings appropriately. This is done via a script at present - please use the following commands

cp aum001hb.snp aum001hb.orig.snp
/usr2/sched/snp2flexbuff.sh aum001hb.orig.snp aum001hb.snp hb

The newly created aum001hb.snp file retains the timing of the source commands, scan names and times but calls a script to run the recorder and omits many of the typical procedures like preob, etc. NB - do not use the line number entries in the sum file!

• RF and IF paths configured/DBBC settings: The RF/IF/DBBC configuration has been compiled into a single script on pcfshb /usr2/sched/aum.32mhz.py. Running this should set up all the essential settings for both the X-band recorded through the DBBC3 and S-band through the 26m backend and dbbcho.
  • No RF configuration is needed for the X-band data recorded through the DBBC3, but it is necessary to set the inputs on the 26m's remote backend (e.g “the folly” or “PalfreymansBackend”. Go to the hobart26 VNC session (vncviewer newsmerd:1) and go to desktop 3. Refresh the Folly display with 'r' and confirm that S/X SRCP and S/X XRCP are highlighted for channels 1 and 2 respectively.
  • DBBC server: If the aum.32mhz.py script cannot communicate with the DBBC servers, you can restart the software running through the VNC sessions on dbbc3hb (DBBC# Control DDC_V_v121.exe) and dbbcho (DBBC2 Control DDC v105_2.exe and SerialServer.sem.py), respectively.
  • IF selections can be checked with the aum.query.py script on pcfshb. For the DBBC3, all inputs and filters should be 2 and all power levels should have a target level of 42000. For the dbbcho, the input should be 1, the filter 2 and the target power level should be 48000
• Maser ok: Check the Hobart26 maser as per usual
• Mark5: Not used. Instead log in to the Flexbuff recorder (with ssh observer@flexbuffhb) and confirm that the time is correct (with ntpq-np) and that jive5ab is running (with ps -ef | grep jive)
• The clock offsets can be (partially) read through the FS. maserdelay reads the maser-GPS difference as usual while dbbc3delay logs the DBBC3-GPS delay as gps-fmout. The existing clkoff command is measuring the dbbchb-gps difference and logging this as fmout-gps - this is irrelevant to the Hb12 recordings but is needed for the simultaneous Ho26 experiments!. In summary - compare the outputs of the maserdelay and dbbc3delay commands, logged as /maser2gps/ and /gps-fmout/ respectively. You should check the dbbc-gps difference for DBBCHO using the ptzhb camera. The relevant counter is to the left of the mark4 rack at the top of rack 9. It's labeled DBBCHO-TACGPSand is typically ~1 microsecond offset from the maserdelay. You do not need to log this value but not the difference nd confirm that it is steady.
• Test Recordings: To make a test recording, as oper@pcfshb run sxflexbuff.sh test 19 to carry out a 20s test recording. You can check the status of the flexbuff by tailing the log file (e.g tail -100f /usr2/log/$(lognm).flexbuff.log). Confirm that the total number of packets is incrementing after each check and that there are no lost or out-of-order packets - these are symptoms of the flexbuff being overloaded.
• Check for any integer second offsets in the recorded data by running sxtest.sh on pcfshb. This will make a short test recording and then present the decoded time of the three files - all three entries (presented as MJD = 58330/01:09:59.91) should agree to within 0.01 seconds.
• No autocorrelation script has been implemented yet.
• There is no systemp measurewment implemented yet.
• There is no pointing/SEFD check implemented as yet
• No fringe check implemented as yet.

• Antenna checks as per usual
• LO check unavailable
• No autocorrelations
• Check the delays with maserdelay and dbbc3delay, confirm the dbbcho time delay is stable
• Check thew Hb maser as usual.
• Check the weather data is being logged & add in sky conditions as usual.
• No Tsys checks.
• On pcfshb, run ./aum.check.sh to check the dbbc and recorder status. There is a lot of debugging info stored here, so use a large terminal window to run the script and be aware that you will need to press space to go through the list. Included below is the full output of a check, with the relevant parameters to check commented on below.

###############################
Checking dbbc3hb
###############################
dbbcifc/ 2,33,agc,2,41815,42000
dbbcifd/ 2,22,agc,2,42416,42000

This first block is the equivalent of iread - check the inputs and filters are set to 2 and that the power level matches the target

dbbc17/ 2204.990000,a,32,1,agc,55,63,14974,15206,0,0
dbbc18/ 2244.990000,a,32,1,agc,58,69,17035,15254,0,0
dbbc19/ 2344.990000,a,32,1,agc,79,100,17160,14828,0,0
dbbc20/ 2504.990000,a,32,1,agc,124,108,15562,14913,0,0
dbbc21/ 2724.990000,a,32,1,agc,107,121,16908,15844,0,0
dbbc22/ 2844.990000,a,32,1,agc,181,181,16086,16144,0,0
dbbc23/ 2904.990000,a,32,1,agc,188,196,16211,16191,0,0
dbbc24/ 2924.990000,a,32,1,agc,200,202,16298,16046,0,0
dbbc25/ 2204.990000,a,32,1,agc,57,58,13976,17561,0,0
dbbc26/ 2244.990000,a,32,1,agc,68,75,15413,16071,0,0
dbbc27/ 2344.990000,a,32,1,agc,90,92,15310,14988,0,0
dbbc28/ 2504.990000,a,32,1,agc,119,100,16736,16911,0,0
dbbc29/ 2724.990000,a,32,1,agc,104,109,16772,14937,0,0
dbbc30/ 2844.990000,a,32,1,agc,149,172,16204,15428,0,0
dbbc31/ 2904.990000,a,32,1,agc,183,175,15675,16114,0,0
dbbc32/ 2924.990000,a,32,1,agc,171,209,16013,16102,0,0

This is the quivalent of bread confirm all the values are within ~1000 of the 16000 target and that the frequencies match those listed above.

pps_delay/ [1]: 43 ns, [2] 43 ns, [3] 39 ns, [4] 39 ns, [5] 39 ns, [6] 39 ns, [7] 0 ns, [8] 0 ns;

This is the translated equivalent of dbbc=pps_delay, separated by processing board. Check that all the listed values are either 43 or 39 ns, and are stable.

core3hstats/ 
Core3H[3] Power:
Sampler 0: 60781106
Sampler 1: 61694675
Sampler 2: 62199248
Sampler 3: 62160250

The Core3H power readings from each sampler should all agree to within ~3-5%. This is the first of two samplers in use.

Core3H[3] Bstat.
Sampler 0:
11:       44 0.28%
10:     7699 49.27%
01:     7846 50.21%
00:       34 0.22%

Sampler 1:
11:       71 0.45%
10:     7915 50.66%
01:     7604 48.67%
00:       33 0.21%

Sampler 2:
11:       48 0.31%
10:     7768 49.72%
01:     7773 49.75%
00:       33 0.21%

Sampler 3:
11:       45 0.29%
10:     7700 49.28%
01:     7844 50.20%
00:       35 0.22%

The Bstats can be ignored at this time - they're displayed because it's complicated to hide them.

Core3H[3] Corr.
Sampler 0-1: 180656919
Sampler 1-2: 182717105
Sampler 2-3: 182062361

All inter-sampler correlation values should be ~180000000.

core3hstats/ 
Core3H[4] Power:
Sampler 0: 88700574
Sampler 1: 90430690
Sampler 2: 92322148
Sampler 3: 93681776

The Core3H power readings from each sampler should all agree to within ~3-5%. This is the second sampler of two, and the above readings are ok.

Core3H[4] Bstat.
Sampler 0:
11:      170 1.09%
10:     7635 48.86%
01:     7680 49.15%
00:      138 0.88%

Sampler 1:
11:      226 1.45%
10:     7680 49.15%
01:     7582 48.52%
00:      135 0.86%

Sampler 2:
11:      184 1.18%
10:     7638 48.88%
01:     7658 49.01%
00:      142 0.91%

Sampler 3:
11:      181 1.16%
10:     7569 48.44%
01:     7733 49.49%
00:      140 0.90%

The Bstats can be ignored at this time - they're displayed because it's complicated to hide them.

Core3H[4] Corr.
Sampler 0-1: 186212284
Sampler 1-2: 190747558
Sampler 2-3: 187420379

All inter-sampler correlation values should be ~180000000.

###############################
Checking dbbcho
###############################
dbbcifa/1,4,agc,2,47097,48000,1
dbbcifb/1,14,agc,2,47991,48000,1

This is the equivalent of iread. All inputs for dbbcho should be 1, all filters 2 and all target power levels 48000

dbbc01/317.990000,a,32,1,agc,97,89,14803,14952,0,0
dbbc03/337.990000,a,32,1,agc,108,89,14945,15083,0,0
dbbc05/357.990000,a,32,1,agc,103,111,14891,14875,0,0
dbbc07/387.990000,a,32,1,agc,119,100,12784,14783,0,0
dbbc09/317.990000,b,32,1,agc,130,119,16077,16249,0,0
dbbc11/337.990000,b,32,1,agc,139,123,16091,16175,0,0
dbbc13/357.990000,b,32,1,agc,149,138,16248,16228,0,0
dbbc15/387.990000,b,32,1,agc,125,145,14144,16119,0,0

This is the equivalent of bread. Note that only odd-numbered bbcs are used. As this is S-band, power levels may vary significantly so a range of 10000-25000 is acceptable (but notable - please put an entr in the handover notes)

pps_delay/61971

The pps_delay should be stable at 61971, to within 1 count of so.

sysstat

^MSystem status:
^M Selected input      : vsi1
^M Input sample rate   : 64000000 Hz
^M VSI input swapped   : no
^M VSI input bitmask   : 0x0000FFFF
^M VSI input width     : 16 bit
^M PPS count           : 78607

^M TVG mode            : vsi-h

^M MK5B timesync       : yes
^M VDIF timesync       : yes
^M GPS receiver        : installed

^M Output              : started
^M Output 0 format     : vdif
^M Output 0 dest.      : 192.168.1.11:46236
^M Output 1 format     : vdif
^M Output 1 dest.      : none
^M Ethernet ARPs       : off

^M Selected VSI output : vsi1-2

^MFiLa10G % 
^MFiLa10G % 

This last block is the Fila10G status. Please confirm that these settings match the output when performing the checks.

• You can check on the status of the recordings by tailing the flexbuff log (/usr2/log/aum001hb.flexbuff.log on pcfshb - the name will change when a new FS log file is started). This is written to by the recording script and running tail -100f /usr2/log/aum001hb.flexbuff.log will show the same output as in the recording tests, e.g

50
!runtime = 0 : stream46230 ;!evlbi? 0 : total : 782143 : loss : 0 ( 0.00%) : out-of-order : 0 ( 0.00%) : extent : 0seqnr/pkt ;
!runtime = 0 : stream46236 ;!evlbi? 0 : total : 782107 : loss : 0 ( 0.00%) : out-of-order : 0 ( 0.00%) : extent : 0seqnr/pkt ;
!runtime = 0 : stream46229 ;!evlbi? 0 : total : 782064 : loss : 0 ( 0.00%) : out-of-order : 0 ( 0.00%) : extent : 0seqnr/pkt ;

Please note that the output is “laggy” and is usually delayed by ~1 minute. You can see the current status of the recorder using ptzhb - when recording you will see the blue lights of disk activity scattered over the flexbuff (below the DBBC in rack 13).