Single Dish Radio Astronomy Software Tools

For an overview of SDRAST and the current status please visit https://sdrast.github.io/.

Note

These packages are in a very preliminary state and need lots of work. Some code is obsolete or buggy from not having been used and therefore not maintained a long time.

Base Class Module

Data_Reduction Package

Modules to support data reduction in Python.

The main purpose of the base module Data_Reduction is to provide a suplerclass with a good set of attributes and methods to cover all common needs.

The base module is also able to read data from a text file as a numpy structured array. This is done with a class called DataGetterMixin which must be invoked after the base class has been initiated.

The module function examine_text_data_file() reveals the structure of the file(s) that provide the data..

Examples

Here we initiate a base class after mixing in the data getter. The first line o the file has column names but the first three columns are all under one name UTC so we specify column widths to consider the first three columns to be one column. We use the names from the first line of the file, which could have been done with an open(), readline(), and close():

mixIn(Observation, DataGetterMixin)
obs = Observation(dss=28, date="2012/127", project="SolarPatrol")
obs.open_datafile('t12127.10',
                        delimiter=[17,16,3,11,7,9,8,2,6],
                        skip_header=1,
                    names="UTC Epoch Chan Tsys Int Az El Diode Level".split())

Now the data getter is already mixed in to Observation so we don’t need to do it again. In this case we specify the names of the columns, changing Int to Integr:

obs2 = Observation(dss=28, date="2012/127", project="SolarPatrol")
obs2.open_datafile('t12127.10', skip_header=1,
        names="Year DOY UTC Epoch Chan Tsys Integr Az El Diode Level".split())

The class Map inherits from DataGetterMixin, so no explicit mixin required:

obsmap = Map(dss=84, date="2020/163", project="SolarPatrol")
obsmap.initialize('sim-venus.dat', source="Venus")

Let’s examine obsmap. We have only one signal column:

In [3]: obsmap.channel.keys()
Out[3]: dict_keys(['xl'])
In [4]: obsmap.channel['xl'].keys()
Out[4]: dict_keys(['freq', 'bw', 'pol', 'ifmode', 'atten', 'power'])

Functions

examine_text_data_file(filename)	Examine a file to guide genfromtxt()
get_obs_dirs(project, station, year, DOY[, …])	Returns the directories where data and working files are kept
old_get_obs_session([project, dss, date, path])	Provides project, station, year and DOY, asking as needed.

Classes

DataGetterMixin	Class for getting data from a CSV file.
GriddingMixin	Class for all the data and methods associated with a raster scan map
Map([parent, name, dss, date, project])	Map class without special features for GAVRT and Malargue
Observation([parent, name, dss, date, project])	superclass for a data structure and methods
Recording([session, path, date, dss, name])	Class for raw data
Session([parent, date, project, dss, path])	Base class for an observing session on a given year and DOY
Spectrum()	Class for spectra

Class Inheritance Diagram

Data_Reduction.plotting Module

Provides subclasses with plotting capability for the base classes

Classes

MapPlotter(mapobj)

Class Inheritance Diagram

Context Modules

Data_Reduction.DSN Package

Data_Reduction.DSN

Subclasses for reducing data taken with DSN-like open loop recorders.

Open-loop recorders are raw IF voltage recorders that are not synchronized with the communications between the spacecraft and the ground station. As such, they are the most basic kind of recorder possible in radio astronomy, equivalent to VLBI recorders. Indeed, an early implementation was known as the “VLBI Science Recorder” (VSR), followed by later varieties of VSR and eventually, the OSR.

OLR recordings at different stations are indeed combined for VLBI measurements of spacecraft with respect to distant radio sources, a powerful navigation tool.

Raw IF recordings can be computational converted into any of the standard signal types used in radio astronomy – square-law detected power, spectra, Stokes parameters, VLBI U-V maps, high time ans spectral resolution pulsar data, etc.

Functions

get_file_metadata(project, dss, year, doy, …)
print_file_metadata(project, dss, year, doy, …)

Classes

Map()
Observation([parent, name, dss, date, …])	Class for observations based on open-loop recordings
Recording([session, path, date, dss, name])	Superclass for DSN recordings based on VSR, WVSR, OLR, etc.

Class Inheritance Diagram

Data_Reduction.DSN.database Module

Functions for extracting complicated information from the DSN database.

The methods obtain data which are stored in different tables according to various criteria. They may just be a front end for a MySql query.

Functions

get_scan_times(db, dss, start_time, end_time)	Gets the times when antenna pointing was stable and the data valid.
get_scans(db, dss, scan_times_dict)	Processes a scan times dictionary and returns data about each scan.
report_scans(scan_data)	Pretty print the scan data obtained from get_scans()
test()	Test this module

Data_Reduction.DSN.logs Module

Functions for extracting data from the various DSN logs.

Functions

eac_macro_file_name(datadir, year, DOY)	Returns the name of the EAC macro log file name.
find_EAC_logs(logdir, year, doy)	Adds EAC status, Tsys and five-point logs to ‘log_dict’.
find_EAC_macro_log(year, DOY)	Get the EAC macro processor log file names.
find_log_file(logdir, prefix, year, midfix, …)	Find a generic log file.
find_minical_logs(logdir, year, doy)	Finds the EAC minical log files.
get_EAC_logs(year, DOY, dest_path)	Copy the EAC logs and scripts.to a local directory.
get_EAC_macro_log(year, DOY, dest_path)	Copy the EAC macro processor log
get_RAC_logs(year, DOY, dest_path)	Get the RAC logs
get_RAVI_logs(year, DOY, dest_path)	Get the RAVI logs and scripts and data
get_VSR_logs(year, DOY, dest_path)	Get logs from the VSR.
get_boresights(bore_files)	Extract data from five-point boresight files.
get_cals(cal_file)	Extract the data from a minical file.
get_log_data(logs)	Gets the data from EAC logs of each type.
get_on_off_times(eaclog)	Get the times when the antenna is on-source or off-source.
get_track_status(status_files)	I’ve forgotten what is is supposed to do.
get_tsys(tsys_files)	Get Tsys data.
parse_boresight_line(line)	Parse a line of boresight data.
parse_tsys_log_line(line)	Parse a line in a Tsys log.
report_logs(cal_data, tsys_data, bore_data, …)	Prints out the logs of each type.

Data_Reduction.DSN.Malargue Package

Data_Reduction.Malargue

Subclasses for reducing data taken with DSN-like open loop recorders.

Open-loop recorders are raw IF voltage recorders that are not synchronized with the communications between the spacecraft and the ground station. As such, they are the most basic kind of recorder possible in radio astronomy, equivalent to VLBI recorders. Indeed, an early implementation was known as the “VLBI Science Recorder” (VSR), followed by later varieties of VSR and eventually, the OSR.

OLR recordings at different stations are indeed combined for VLBI measurements of spacecraft with respect to distant radio sources, a powerful navigation tool.

Raw IF recordings can be computational converted into any of the standard signal types used in radio astronomy – square-law detected power, spectra, Stokes parameters, VLBI U-V maps, high time ans spectral resolution pulsar data, etc.

Classes

Map()
Observation([parent, name, dss, date, …])	Class for observations based on open-loop recording made as DSA-3 (DSS-84)
Recording([session, name, dss, date, project])	Metadata and directory for raw data files from DSA-3.
Session([parent, date, project, dss, path])	Class for a Malargue observing session on a given year and DOY

Class Inheritance Diagram

Data_Reduction.DSN.old_init Module

Functions for scripts, logs and data files on the EAC, RAC, RAVI and VSR.

A lot of stuff here is either obsolete or no longer accurate

This package has functions to generate scripts, download and parse DSN logs, and manage data files. The module also initializes path names relative to a root directory whose default is:

root_dir = "/usr/local/projects/PESD/"

The observations are kept in subdirectories with names like YYYY-MM-DD in:

obs_dir  = root_dir + "observations/"

Notes

This module and the others in this package assume that sshfs has been used to mount the relevant Flight Ops radio astronomy hosts as sub-directories of localhost:/usr/local/projects/PESD/rahost. The logs and some post-processed data files will be or have been copied into localhost:/usr/local/projects/PESD/observations/YYYY-MM-DD. This directory will be created when ‘make_scripts’ is run.

Functions

backup_one_line(fd)	Moves the file pointer to right after the previous newline.
check_vsr()	Check the usage of the VSR BLS disk area.
clear_vsr(year, DOY)	Delete the data files on venus-vsr1 for a given date.
get_binary_record(fd, file_header_size, …)	Get the binary record at the specified index.
get_last_line(fd)	Gets the last line in an ASCII file.
get_obs_date(filename)	Parse a filename for the date of observation.
header_report(year, doy, start_sec, freq, …)	Format a report of the observing parameters
path_to_remote(workstation, local_path)
print_header(header)	Prints selected contents from a VSR data record header.
rx_band(freq)	Associate a waveguide band with a frequency
send_ssh_command(local_cmd, remote_cmd)	Sends an ssh comand to a remote host.
set_binary_record_pointer(fd, …)	Position the file pointer to the record at the specified record index.
set_file_pointer(fd, new_pos)	Position the file pointer in a binary file.
verify_ssh_mount(server)	Verify that a server is mounted.

Data_Reduction.DSN.old_VSR Module

Functions for scripts, logs and data files on the EAC, RAC, RAVI and VSR.

A lot of stuff here is either obsolete or no longer accurate

This package has functions to generate scripts, download and parse DSN logs, and manage data files. The module also initializes path names relative to a root directory whose default is:

root_dir = "/usr/local/projects/PESD/"

The observations are kept in subdirectories with names like YYYY-MM-DD in:

obs_dir  = root_dir + "observations/"

Notes

This module and the others in this package assume that sshfs has been used to mount the relevant Flight Ops radio astronomy hosts as sub-directories of localhost:/usr/local/projects/PESD/rahost. The logs and some post-processed data files will be or have been copied into localhost:/usr/local/projects/PESD/observations/YYYY-MM-DD. This directory will be created when ‘make_scripts’ is run.

Functions

check_vsr()	Check the usage of the Venus VSR BLS disk area.
clear_vsr(year, DOY)	Delete the data files on venus-vsr1 for a given date.
get_binary_record(fd, file_header_size, …)	Get the binary record at the specified index.
get_obs_date(filename)	Parse a VSR filename for the date of observation.
header_report(year, doy, start_sec, freq, …)	Format a report of the observing parameters
path_to_remote(workstation, local_path)	Form a path to an SSH mounted destination.
print_header(header)	Prints selected contents from a VSR data record header.
rx_band(freq)	Associate a waveguide band with a frequency
set_binary_record_pointer(fd, …)	Position the file pointer to the record at the specified record index.
set_file_pointer(fd, new_pos)	Position the file pointer in a binary file.
verify_ssh_mount(server)	Verify that a server is mounted.

Data_Reduction.DSN.OLSR Module

Module OLSR

For open loop science recorder files using functions.

Uses module functions to access files. Object-oriented file access is in module RSData. Supports data files from the VSR, WVSR, PVSR, OLR, etc. Recognizes RDEF, VDR, SFDU formats

Option keys are:

'startDate'       datetime.datetime
'startTime'       alternate for 'startDate' as UNIX time
'lookForDate'     set True if using 'startDate'
'endDate'         datetime.datetime
'duration'        compute 'endDate' from 'startDate'
'fixTime'         ignore the fractional second of the start time
'format'          a string with one of the recognized formats
'toPrintHeaders'  output header
'toPrintData'     output data
'leaveFileOpen'   keep reading from the same file

In __main__, date formats may be given as YYYY-MM-DD-HH:MM:SS.fff or UNIX time seconds.

Notes

The file reading functions read all the data between the specified times and their defaults. Each record is a dict which looks like this:

{'values': array([ 11. -1.j,  -5. -3.j,  -7.+13.j, ...,
                    9.+15.j, -15.-21.j, -25. +1.j]),
 'times': array([0.00000e+00, 1.00000e-06, 2.00000e-06, ...,
                 9.99997e-01, 9.99998e-01, 9.99999e-01]),
 'phase': array([-885534.63141, -885534.63141, -885534.63141, ...,
                 -885534.63141, -885534.63141, -885534.63141]),
 'phCoeff': [[-885534.63141,
              1.1125369292536007e-308,
              1.1125369292536007e-308,
              1.1125369292536007e-308]],
 'LO': 31960000000.0,
 'DDCLO': -11500000.0,
 'sampleSize': 8,
 'DATAFIL': '/usr/local/projects/SolarPatrol/Observations/dss84/2020/163/                                         NET4n004tSsMG12rOPc05-20163121517.prd',
 't0': datetime.datetime(2020, 6, 11, 12, 15, 20, 999979),
 'file': {'f':[],
 'curRecord': []}}

Example

OLSR method RDEF needs options set before being called:

In [1]: import Data_Reduction.DSN.OLSR as OLSR
In [2]: filename = 'NET4n005tSsMG12rOPc05-20163122527.prd'

In [4]: options = {'format': OLSR.checkFormat(filename)}
In [6]: options.update({'toPrintHeaders': False})
In [7]: hdr = OLSR.readHeaders(filename, options)
In [14]: options.update({'duration':10, 'toPrintData':False,
                         'lookForDate': True, 'fixTime':True})
In [15]: data = OLSR.RDEF(filename,options)

In [18]: data['values'][:20]
Out[18]:
array([-21. +5.j,  21.-21.j,  11. -3.j, -31. -5.j, -13. +1.j,   5. -1.j,
        -3. -7.j,   5.-17.j,  15. +7.j, -17. +9.j,  21. +3.j,  29. +3.j,
        19. +7.j,  13.-11.j, -11.-11.j,  -1.+43.j, -25.+13.j, -19. -3.j,
       -17.+17.j,   1.+25.j])

Todo

1. Fix the timedelta and datetime overflow errors in VDR() and SFDU().

Functions

RDEF(DATAFIL, options)	Read RDEF format datafile
SFDU(DATAFIL, options)	Read RSR format file
VDR(DATAFIL, options)	Read VDR format datafile
checkFormat(DATAFIL)	check the file format
data2float(data, format)	data sample to float
data2int(data, format)	data sample to integer
readHeaders(f, options)	Decode the file header

Data_Reduction.DSN.RSData Module

RSData - Radio Science Data

Data handler for such recorders as VSR, WVSR, OLR, etc. It recognizes the ‘RDEF’, ‘VDR’, and ‘SFDU’ formats.

This uses:

fileObject.seek(offset[, whence])

Parameters:

offset − This is the position of the read/write pointer within the file. whence − This is optional and:

* defaults to 0 which means absolute file positioning;
* 1 means seek relative to the current position and
* 2 means seek relative to the file's end.

Classes

File([name, sessionpath, filename])

class to read and convert data in radio science recorder data files

Class Inheritance Diagram

Data_Reduction.DSN.STATS Module

Support for STATS data files.

STATS binary file structure

A stats binary output files begins with a stats_hdt_t structure:

typedef struct
{
  (unsigned short header_size   /* bytes, may or may not be there */
  unsigned short spcid;         /* station id - 10, 40, 60, 21 */
  unsigned short vsrid;         /* vsr1a, vsr1b ... from enum */
  unsigned short chanid;        /* subchannel id 0,1,2,3 */
  unsigned short bps;           /* number of bits per sample - 1, 2, 4, 8,
                                   or 16 */
  unsigned long  srate;         /* number of samples per second in kilo-
                                   samples per second */
  unsigned short error;         /* hw err flag, dma error or num_samples
                                   error,
                                   0 ==> no errors */
  unsigned short year;          /* time tag - year */
  unsigned short doy;           /* time tag - day of year */
  unsigned long  sec;           /* time tag - second of day */
  double         freq;          /* in Hz */
  unsigned long  orate;         /* number of statistics samples per
                                   second */
  unsigned short nsubchan;      /* number of output sub chans */
}
stats_hdr_t;

This unpacks with “=4H Q HHH Q d Q H”

A data record looks like this:

fwrite(&doy,   sizeof(int),    1, ofp);
fwrite(&sec,   sizeof(int),    1, ofp);
fwrite(&i,     sizeof(int),    1, ofp);
fwrite(&mean,  sizeof(double), 1, ofp);
fwrite(&var,   sizeof(double), 1, ofp);
fwrite(&skew,  sizeof(double), 1, ofp);
fwrite(&kurt,  sizeof(double), 1, ofp);
fwrite(&mean,  sizeof(double), 1, ofp);
fwrite(&var,   sizeof(double), 1, ofp);
fwrite(&skew,  sizeof(double), 1, ofp);
fwrite(&kurt,  sizeof(double), 1, ofp);

which unpacks with “=LLL dddd dddd”

STATS ASCII file structure

A STATS file begins with a header. The data lines consist of:

- column 0: second, starting with 1
- column 1: sample number within the second (typically 0-999)
- column (subchannel + 2): mean
- column (subchannel + 3): r.m.s.
- column (subchannel + 4): kurtosis
- column (subchannel + 5): skewness

where subchannel = 0, 1.

Functions

find_STATS_bin_block_times(ses_date, year, …)	Gets the recording start and stop times from a binary STATS file.
find_STATS_log_block_times(ses_date, year, …)	Gets the recording times from the STATS log files.
get_STATS_ASCII_data_block(fd, nchan, nsamps)	This reads data for one block from the data file.
get_STATS_block(fd, year, orate, blk_index)	Gets the signal statistics data for a one second block.
get_binary_stats_header(fd)	Get the header from a binary stats file.
get_binary_stats_record(fd, header_size, index)	Extracts a binary record at the specified record index.
get_block_time(fd, TS0, orate, blk_index)	This gets the time of a particular STATS binary file record.
get_data_block(fd, nchan, nsamps)	Alias for get_STATS_ASCII_data_block
parse_STATS_ASCII_header(header)	Parses the header of a STATS
parse_STATS_header(header)	Extract the header from a binary STATS data file.
parse_record(year, orate, data)	This parses a data record that is in list format.
print_record(data)	Pretty-prints a STATS record that is in list format.
process_STATS_ASCII_data_line(line, nchan)	This processes one line of a STATS data file
process_STATS_ASCII_header(fd)	Process the header in a STATS file
write_binary_stats_header(header)	Write a header in binary format.

Data_Reduction.DSN.SAO Package

module for reducing SAO spectrometer data

Functions

parse_filename(fname)

Parse an SAO .hf5 filename for date and source

Classes

Data(x, y)	A subset of data extracted from an average difference spectrum
SAOexaminer([project, dss, date])	Class for examining SAOdataset objects
SAOhdf5(filename[, clobber_0, clobber_center])	Container for data in SAOspec HDF5 data files

Class Inheritance Diagram

Data_Reduction.DSN.Tid_ASCII_data Module

Tid_ASCII_data

This module handles Tidbinbilla spectroscopy data files written in ASCII format by the HP 9825 controller. The data are written in scientific notation. The precision is such that the frequencies are given to the nearest 10 kHz. As a result, computed velocities may be as much as 0.3 km/s off.

NEEDS MAJOR UPDATES FROM MODULE Observatory TO MonitorControl

Functions

check_backend(path, fname, backends)	Determine the type of back end and the number channels.
get_scan_data(data, index, block_size)	Get a scan from an ASCII file
get_scan_indices(data)	Get the line numbers where scans start
get_scans(path, groups, group_index)	Build up a dictionary of Tid scans for an observing session
group_files(files)	Group files by DOY
num_scans_in_group(path, groups)	Number of scans in each group
report_count(filename, scan_indices)	Print the number of scans in a file
sort_Tid_ASCII_files(files, dss)	Sort ASCII data files from Tidbinbilla, written with HP9825 controller.

Data_Reduction.DSN.Tid_data Module

Handles old Tidbinbilla data files

NEEDS MAJOR UPDATES FROM MODULE Observatory TO MonitorControl

Functions

configure_station(dss)	Describe the station and report on the elements and their relationships.
fill_bintable_header(tid_scan, num_backends, …)	Initializes the scantable header
fill_bintable_row(tid_scan, num_backends, …)	Writes a Tid scan to a scantable row
number_of_channels(numbers)	Deduce the number of channels from Tid ASCII file data
update_backend(numbers, backends)	Sets the attributes ‘polarizations’ and ‘num_chan’

Classes

Tid_scan(dss, numbers)

Make a Tidbinbilla spectral scan object

Class Inheritance Diagram

Data_Reduction.DSN.tipping Module

Analyze tipping curve data taken with observatoryCtrl

A tipping array consists of two 1-D arrays, the first containing the elevation in degrees and the second the system temperature for each of the four channels.

Older files (like 2014) have only one reading and it may be dBm instead of Tsys

Functions

airmass(elev)	:param elev : elevation above the horizon in degrees :type elev : float
extract_simple_tip(e)
fit_airmasses(e, t)
fit_tipping_data(e, t[, Tamb])	Fits tipping curve data from a specified file
get_tipping_data(filename)	Gets tipping data from a file made with observatoryCtrl
plot_tipcurves(e, t[, plot_title, fig_title])	Plots tipping curve data from a specified file
sort_by_time(directory, name_template)	Returns a time-ordered dict with times of files

Data_Reduction.FITS Package

A package for managing FITS files.

The core module creates and operates on SDFITS-format files.

SDFITS files are composed mainly of FITS binary tables. The basic method for creating a simple 1-table file is this:

1. Create a primary HDU::

     hdu = pyfits.PrimaryHDU()
   There are optional named arguments.
    - 'data=' allows a data array to be included in the primary HDU.
    - 'header=' allows additional header data to be included, beyond
    - the bare minimum.

2. Create the columns::

    scan_array = numpy.array([0]*tablesize)
    scan_col   = pyfits.Column(name='SCAN',format='1I',array=scan_array)
    date_array = strings.array(['']*tablesize)
    date_col   = pyfits.Column(name='DATE-OBS',format='A16',array=date_array)

3. Create a column definitions object::
    cols  = pyfits.ColDefs([scan_col,date_col])

4. Create a binary table::

    tbhdu = pyfits.new_table(cols)
   This also initializes a table header with required information.
   There are optional named arguments:
     - 'header=' allows additional header parameters to be included;
       -this is in the form of a CardList() instance
       -default None
     - 'nrows=' specifies the number of rows in the table; default 0
     - 'fill=' specifies the initial value to put in the rows; default 0
     - 'tbtype=' specifies the table type; default 'BinTableHDU'

5. Create an HDU list with the primary HDU and the table::

     hdulist = pyfits.HDUList([hdu, tbhdu])
   Additional tables may be appended with::
     hdulist.append(tb2hdu)

Functions

save_scan_data(row, scan, dateobs, source, …)

Save a scan to the FITS table

Data_Reduction.GAVRT Package

Classes for GAVRT mysql database

Example

DBPlotter (from Data_Reduction.DSN.GAVRT.Mysql.plotter) is used to reduce data stored in the LCER GAVRT MySQL database. The following example gets the coordinate data for a map made during a given session:

In [1]: from Data_Reduction.DSN.GAVRT.plotter import DBPlotter
In [2]: pl = DBPlotter()
In [3]: sp = pl.get_session_plotter(2017,233)
In [4]: map69data = sp.maps[69].get_data_from_tlogs()
In [5]: xdec,dec = sp.maps[69].get_offsets()

Databases

The databases and their schemas are described in http://gsc.lewiscenter.org/data_info/dss28_eac.php.

The server has these databases:

'dss28_eac'
'dss28_spec'
'gavrt_sources'.

Database ‘dss28_eac’

has these tables:

In [17]: dbplotter.get_public_tables()
Out[17]:
(('angles',),     ('chan_cfg',),     ('conv_cfg',), ('fiber_cfg',),
 ('five_point',), ('pointing_cfg',), ('raster',),   ('raster_cfg',),
 ('rf_cfg',),     ('rss_cfg',),      ('seti_cfg',), ('seti_frame',),
 ('tlog',),       ('weather',),      ('xpwr',),     ('xpwr_cfg',),
 ('xscan',),      ('zplot',),        ('zplot_cfg',))

Database ‘gavrt_sources’

has these tables:

'catalog',
'class',
'source'

Table columns

‘angles’ columns:

angles_id,
year, doy, utc, epoch, az, el, status

‘catalog’ columns:

catalog_id, name

‘chan_cfg’ columns:

chan_cfg_id,
year, doy, utc, epoch, chan, center_freq, tdiode

‘class’ columns:

class_id, name, description

‘conv_cfg’ columns:

conv_cfg_id,
year, doy, utc, epoch, converter, mode_a, ifbw_a, bbbw_a, atten_a,
                                  mode_b, ifbw_b, bbbw_b, atten_b, lock_status

‘five_point’ columns:

five_point_id,
xpwr_cfg_id, year, doy, utc, epoch, source_id, chan, tsrc, az, el, ha, dec,
xdec_off, dec_off

‘pointing_cfg’ columns:

pointing_cfg_id,
year, doy, utc, epoch, man, plx, semod, refrctn, delut, model

‘raster’ columns:

raster_id,
raster_cfg_id, year, doy, utc, epoch, xdecoff, decoff, ha, dec, tsrc

‘raster_cfg’ columns:

raster_cfg_id,
rss_cfg_id, year, doy, utc, epoch, source_id, chan, freq, rate, step

‘rf_cfg’ columns:

rf_cfg_id,
year, doy, utc, epoch, feed, diodex, diodey, pol, transfer

‘rss_cfg’ columns:

rss_cfg_id,
year, doy, utc, chan, sky_freq, feed, pol, nd, if_mode, if_bw, bb_bw, fiber_chan

‘source’ columns:

source_id, catalog_id, class_id,
name, RA, Dec, size_dec, size_xdec, reference, aka

‘tlog’ columns:

tlog_id,
rss_cfg_id, year, doy, utc, epoch, chan, top, integ, az, el, diode, level, cryo

‘weather’ columns:

weather_id,
datetime, pressure, temp, humidity, wind_speed, wind_dir

‘xpwr’ columns:

xpwr_id,
xpwr_cfg_id, year, doy, utc, epoch, tsys, az, el, ha, dec, offset

‘xpwr_cfg’ columns:

xpwr_cfg_id,
rss_cfg_id, source_id, cal_src_id, year, doy, utc, epoch, axis, chan, cal_flux

‘xscan’ columns:

xscan_id,
xpwr_cfg_id, year, doy, utc, epoch, tsrc, stdev, bl_stdev, az, az_offset, el,
el_offset, ha, dec, offset, bw, corr

Functions

DSS28_beamtaper(freq)	ad hoc fit to beamwidth vs frequency plot
DSS28_beamwidth(freq)	beamwidth in deg.

Classes

BoresightScan(parent, xpwr_cfg_id)	class for a single scan during a boresight
DSS28db([host, user, pw, name, port])	subclass for the DSS-28 EAC database
Map(parent, raster_cfg_id[, name])	Class for all the data and methods associated with a raster scan map
Observation([parent, name])	Class for any group of data for a single purpose.
Session(parent, year, doy)	Class for an observing session on a given year and DOY

Class Inheritance Diagram

Data_Reduction.GAVRT.mysql Module

Classes for GAVRT data using MySQL tables

This module defines the classes BaseDB and MysqlException. It also provides subclasses for reducing data stored in the DSS-28 database.

Functions

check_database(host, user, passwd, database)	Does the database exist on the host?
check_table(db_conn, table)	Checks whether a table exists.
create_GAVRT_mysql_auth([host, user, pw])	Create a fairly private authentication file for GAVRT mysql db
create_table(database, name, keys)	Creates a table with the specified column names and types.
get_databases(host, user, pw)	Command line function to recall the database names on a host
get_table_columns(db_conn, table)	Returns information about the columns in a table.
make_table(database, table_data)	Adds or populates a table in the specified database from a dictionary.
report_insert_error(error, msg)	Returns text string with cause of the insert error
show_databases(host, user, passwd)	Prints report on all the databases
update_record(db, table, fields, condition)	Updates a record in ‘table’ of data base ‘db’

Classes

BaseDB([host, user, pw, name, port])	This is a database superclass.
MysqlException(message, *args)	Handles exceptions in local module Mysql

Class Inheritance Diagram

Data_Reduction.GAVRT.plotter Module

plots data from the DSS-28 EAC database

Classes

DBPlotter()	Database class with plotting capability.
MapPlotter(parent, raster_cfg_id[, name])
SessionPlotter(parent, year, doy)	Class to add plotting capability to a Session object

Class Inheritance Diagram

Data_Reduction.OldGAVRT Package

Functions for reducing Solar Patrol data

A lot of modules under this need fixing up because of change from package Observatory to MonitorControl and algorithmic to object-oriented programming.

Functions

DSS28_beamwidth(freq)	Estimate DSS-28 beamwidth until we have something better.
extract_data(datatype, wb, start, stop, …)	Extract map from the data files and create/fill a metadata worksheet
get_session_t_files(project_path, date_info)
zenith_gain(freq)	Antenna response vs frequency for DSS-28

Classes

Observation([name, dss, date, project])	DSS-28 observations from before the tlog database table became available.
Recording(session)	This can later be subclassed, when I’ve worked out the common attributes of all recordings.

Class Inheritance Diagram

Data_Reduction.OldGAVRT.plotting Module

plot system temperatures

Functions

plot_tsys(fig, date_nums, tsys[, label, picker])

Plot system temperatures

Data_Reduction.OldGAVRT.solar Module

Functions

create_metadata_sheet(metasheet, sheetname)	Create a sheet with metadata columns
get_file_freqs_and_pols(ws, meta_column, files)	Create a dictionary files indexed by frequency and polarization
get_meta_data(meta_ws, meta_column, files)	Get metadata from session worksheet
load_meta_sheet(wb, obs_ws)	Load the observations spreadsheet or create it.
load_spreadsheet(filename[, sheet])	Load the observations spreadsheet
open_metadata_spreadsheet(filename[, …])	Load the observations metadata spreadsheet or create it.

Supporting Modules

Data_Reduction.boresights.boresight_fitter Module

This is supposed to be a general purpose boresight fitter but it has too many DSS-28 dependencies.

Classes

ScanFitter(scan)

Create an object to fit a scan in one direction to a baseline and a Gaussian

Class Inheritance Diagram

Data_Reduction.hyperfine_fit Module

hyperfine_fit - Fit Gaussian lines to hyperfine components

Functions needed to fit data:

ammonia_data()
model()
error_func()

Here is the basic procedure, assuming that the data are in arrays x(in MHz) and y:

plot(x,y)                          # to see what to expect
set_model(line)                    # see ammonia_data
parameter_guess = [.3,-0.6,width]  # Put in appropriate numbers
result = fit_data(x,y,hpf_model,parameter_guess)
best_fit = hpf_model(x,*result)    # model y-values
plot(x,best_fit)                   # overlaid on the data

Problem

Getting the ammonia hyperfine structure from the JPL catalog (v4) and calculating A_ul gives the wrong relative intensities. 10**logint gives the right ratios.

Functions

ammonia_data(ju, ku, jl, kl)
error_func(guess, x, data, data_model)	:param guess : parameter list.
fit_data(x, data, model, parameter_guess)	fit noisy data to the model
hpf_model(x, amp, x_0, w)	Builds multicomponent Gaussian profile
noise_data(SNR, width, span)	Create noisy data samples
set_model(line)	Make the model parameters global
test()	Test with simulated and real data.

Data_Reduction.maps Module

Obsolete module replaced by plotting and the MapPlotter class.

Functions

center_data(date_nums, ras, decs, body, site)	Generates a map in relative coordinates relative to a source
plot_azel(azs, els, title_str)	Plot elevation vs azimuth
plot_ra_dec(ras, decs, title_str)	Plot declination vs right ascension
plot_xdec_dec(xdecs, decs, title_str)	plot declination vs cross-declination
show_body_orientation(body, meantime, …)

Data_Reduction.SLATool Module

Spectral Line Analysis Tool

Environment for analyzing spectral line from multiple observatories. Manages various data reduction programs

Classes

SessionAnalyzer([project, dss, year, DOY])	Tool for reducing multiple data reduction sessions
Spectrum()	Generic container for a spectrum and its metadata

Class Inheritance Diagram

Data_Reduction.SLAPlotter Module

Session plotting methods

Plotting component of an environment for analyzing spectral line from multiple observatories. Manages various data reduction programs.

Classes

SessionPlotter([project, dss, year, DOY])

Class Inheritance Diagram

Data_Reduction.TAMS_tipping Module

Finds Tidbinbilla tipping curve files

Notes

Instead of ‘TidTipFinder’ having an initialization argument ‘remote’, one could just test with ‘socket.gethostname’ and do a remote mount if the response is not ‘crux’.

Classes

TidTipFinder([dss, remote, date])

Attributes.

Class Inheritance Diagram

Data_Reduction.tipping Module

Analyze tipping curve data taken with observatoryCtrl

This was superceded by class TidTipAnalyzer in Data_Reduction.dss43-tipping

A tipping array consists of two 1-D arrays, the first containing the elevation in degrees and the second the system temperature for each of the four channels.

Older files (like 2014) have only one reading and it may be dBm instead of Tsys

Functions

airmass(elev)	:param elev : elevation above the horizon in degrees :type elev : float
fit_tipcurve_data(elev, Tsys[, Tatm, method])	fits system temperatures to airmass

Classes

AtmosAnalyzer(datacube)

Analyzer for environmental data contained in a weather data cube

Class Inheritance Diagram

Indices and tables

• Index

• Module Index

• Search Page