AuScope Feedback Database


Database set-up by Tiege to contain information scraped from IVS analysis reports, spoolfiles and correlator reports. This database uses MariaDB for the database itself, along with python scripts for retrieving reports as they become available, scraping them for relevant info and then adding into the database.

The current mariaDB database is named auscope_db2 on frenkie, with scripts in the /home/observer/auscopeDB directory. The main script (auscopeDB_daily.py) is run daily as a cron job on frenkie, then every Monday, another cron job runs a shell script (weeklylog.sh) which emails the contents of 'current.log' (the output of newly added experiments for the week) to Tiege before archiving the weekly log and opening a new one.

Due to a lack of strict report creation guidelines within the IVS the scripts have had to be written with lots of exception handling in mind. This makes the scripts fairly hard to read for the uninitiated and overly complicated for 99% of experiment reports. Documentation is currently fairly poor, please email Tiege if you run into any issues.

Note: Every year you will need to update the crontab job to include the new master schedule name (master21.txt etc.), for the start of the year, while experiments from the previous year are still being processed, you can add cron jobs for both master schedules, or just manually run the scripts on the previous year schedule every few weeks when reports have become available.

Current versions of scripts should be available at https://github.com/tiegemccarthy/auscope_database.