source: trunk/doc/userguide.tex@ 1219

\documentclass[11pt]{article}
\usepackage{a4}
\usepackage{calc}
\usepackage[dvips]{graphicx}
\usepackage{makeidx}

% Adjust the page size
\addtolength{\oddsidemargin}{-0.4in}
\addtolength{\evensidemargin}{+0.4in}
\addtolength{\textwidth}{+0.8in}

\setlength{\parindent}{0mm}
\setlength{\parskip}{1ex}

\title{ATNF Spectral Analysis Package\\User Guide v2.1\\DRAFT }
\author{Chris Phillips}

\newcommand{\cmd}[1]{{\tt #1}}

\newcommand{\asaprc}[3]{
  \begin{minipage}[t]{45mm}#1\end{minipage}
  \begin{minipage}[t]{30mm}\raggedright #2\end{minipage}\hspace{3mm}
  \begin{minipage}[t]{\textwidth-75mm}#3\end{minipage}
}

\newcommand{\commanddef}[3]{
  \begin{minipage}[t]{27mm}\tt #1\end{minipage}\hspace{3mm}
  \begin{minipage}[t]{\textwidth-30mm}#2 \\ \tt #3\end{minipage}
}

\newcommand{\bigcommanddef}[3]{
  \begin{minipage}[t]{45mm}\tt #1\end{minipage}\hspace{3mm}
  \begin{minipage}[t]{\textwidth-47mm}#2 \\ \tt #3\end{minipage}
}

\makeindex

\begin{document}

\maketitle

\section{Introduction}

ASAP is a single dish spectral line processing package currently being
developed by the ATNF. It is intended to process data from all ATNF
antennas, and can probably be used for other antennas if they can
produce ``Single Dish FITS'' format. It is based on the AIPS++
package.

This userguide is being updated for the ASAP 2.1. Please report any
mistakes you find.

\section{Installation and Running}

Currently there are installations running on Linux machines at

\begin{itemize}
\item Epping - use hosts {\tt draco} or {\tt hydra}
\item Narrabri - use host {\tt kaputar}
\item Parkes - use host {\tt bourbon}
\item Mopra - use host {\tt minos}
\end{itemize}

Or use your own Linux desktop.

{\em Note. ASAP2.1 only runs on ATNF Linux machines which have been
updated to Debian Sarge and are using the ``DEBIANsarge''
/usr/local. If your favourite machine has not been upgraded, send a
request your your friendly IT support.}

\index{Running}To start asap log onto one of these Linux hosts and enter

\begin{verbatim}
  > cd /my/data/directory
  > asap
\end{verbatim}

This starts the ASAP. To quit, you need to type \verb+^+-d
(control-d) or type \cmd{\%Exit}.

\section{Interface}

\index{Interface}ASAP is written in C++ and python. The user interface
uses the ``ipython'' interactive shell, which is a simple interactive
interface to python. The user does not need to understand python to
use this, but certain aspects python affect what the user can do.  The
current interface is object oriented.

\subsection {Integer Indices are 0-relative}

Please note, all integer indices in ASAP and iPython are {\bf 0-relative}.

\subsection{Objects}
\index{objects}
The ASAP interface is based around a number of ``objects'' which the
user deals with. Objects range from the data which have been read from
disk, to tools used for fitting functions to the data. The following
main objects are used :

\begin{itemize}
  \item[\cmd{scantable}] The data container (actual spectra and header
    information)
  \item[\cmd{selector}] Allows the user to select a subsection of the
    data, such as a specified or range of beam numbers, IFs, etc.
  \item[\cmd{plotter}] A tool used to plot the spectral line data
  \item[\cmd{fitter}] A tool used to fit functions to the spectral data
  \item[\cmd{reader}] A tool which can be used to read data from disks
    into a scantable object (advanced use).
\end{itemize}

There can be many objects of the same type. Each object is referred to
by a variable name made by the user. The name of this variable is not
important and can be set to whatever the user prefers (i.e. ``s'' and
``ParkesHOH-20052002'' are equivalent).  However, having a simple and
consistent naming convention will help you a lot.

\subsection{Member Functions (functions)}

\index{Functions!member}Following the object oriented approach,
objects have associated ``member functions'' which can either be used
to modify the data in some way or change global properties of the
object. In this document member functions will be referred to simply
as functions. From the command line, the user can execute these
functions using the syntax:
\begin{verbatim}
  ASAP> out = object.function(arguments)
\end{verbatim}

Where \cmd{out} is the name of the returned variable (could be a new
scantable object, or a vector of data, or a status return),
\cmd{object} is the object variable name (set by the user),
\cmd{function} is the name of the member function and \cmd{arguments}
is a list of arguments to the function. The arguments can be provided
either though position or \cmd{name=}.  A mix of the two can be used.
E.g.

\begin{verbatim}
  ASAP> av = scans.average_time(msk,weight='tsys')
  ASAP> av = scans.average_time(mask=msk,weight='tsys')
  ASAP> av = scans.average_time(msk,tsys)
  ASAP> scans.poly_baseline(mask=msk, order=0, insitu=True)
  ASAP> scans.poly_baseline(msk,0,True)
  ASAP> scans.poly_baseline(mask, insitu=True)
\end{verbatim}

\subsection{Global Functions}

\index{Functions!global}It does not make sense to implement some functions as member
functions, typically functions which operate on more than one
scantable (e.g. time averaging of many scans). These functions will
always be referred to as global functions.

\subsection{Interactive environment}

\index{ipython!environment}ipython has a number of useful interactive
features and a few things to be aware of for the new user.

\subsubsection{String completion}

\index{ipython!string completion}Tab completion is enabled for all
function names. If you type the first few letters of a function name,
then type {\tt <TAB>} the function name will be auto completed if it
is un-ambiguous, or a list of possibilities will be
given. Auto-completion works for the user object names as well as
function names. It does not work for filenames, nor for function
arguments.

Example
\begin{verbatim}
  ASAP> scans = scantable('MyData.rpf')
  ASAP> scans.se<TAB>
  ASAP> scans.set_in<TAB>
scans.set_cursor      scans.set_freqframe   scans.set_selection
scans.set_doppler     scans.set_instrument  scans.set_unit
scans.set_fluxunit    scans.set_restfreqs

  ASAP> scans.set_instrument()
\end{verbatim}

\subsubsection{Leading Spaces}

\index{ipython!leading space}Python uses leading space to mark blocks
of code. This means that it you start a command line with a space, the
command generally will fail with an syntax error.

\subsubsection{Variable Names}

\index{ipython!variable names}During normal data processing, the user
will have to create named variables to hold spectra etc. These must
conform to the normal python syntax, specifically they cannot contain
``special'' characters such as \@ \$ etc and cannot start with a
number (but can contain numbers).  Variable (and function) names are
case sensitive.

\subsubsection{Unix Interaction}

\index{ipython!unix interaction}Basic unix shell commands (\cmd{pwd},
\cmd{ls}, \cmd{cd} etc) can be issued from within ASAP. This allows
the user to do things like look at files in the current directory. The
shell command ``\cmd{cd}'' works within ASAP, allowing the user to
change between data directories. Unix programs cannot be run this way,
but the shell escape ``$!$'' can be used to run arbitrary
programs. E.g.

\begin{verbatim}
  ASAP> pwd
  ASAP> ls
  ASAP> cd /my/data/directory
  ASAP> ! mozilla&
\end{verbatim}

\subsection{Help}

\index{Help}ASAP has built in help for all functions. To get a list of
functions type:

\begin{verbatim}
  ASAP> commands()
\end{verbatim}

To get help on specific functions, the built in help needs to be given
the object and function name. E.g.

\begin{verbatim}
  ASAP> help scantable.get_scan # or help(scantable.get_scan)
  ASAP> help scantable.stats
  ASAP> help plotter.plot
  ASAP> help fitter.plot

  ASAP> scans = scantable('mydata.asap')
  ASAP> help scans.get_scan # Same as above
\end{verbatim}

Global functions just need their name

\begin{verbatim}
  ASAP> help average_time
\end{verbatim}

Note that if you just type \cmd{help} the internal ipython help is
invoked, which is probably {\em not} what you want. Type \verb+^+-d
(control-d) to escape from this.

\subsection{Customisation - .asaprc}

\index{.asaprc}ASAP use an \cmd{.asaprc} file to control the user's
preference of default values for various functions arguments. This
includes the defaults for arguments such as \cmd{insitu}, scantable
\cmd{freqframe} and the plotters \cmd{set\_mode} values. The help on
individual functions says which arguments can be set default values
from the \cmd{.asaprc} file. To get a sample contents for the
\cmd{.asaprc} file use the command \cmd{list\_rcparameters}.

Common values include:
\begin{verbatim}
  # apply operations on the input scantable or return new one
  insitu                     : False

  # default output format when saving scantable
  scantable.save             : ASAP

  # default frequency frame to set when function
  # scantable.set_freqframe is called
  scantable.freqframe        : LSRK

  # auto averaging on read
  scantable.autoaverage      : True
\end{verbatim}

For a complete list of \cmd{.asaprc} values, see the Appendix.

\section{Scantables}
\index{Scantables}
\subsection {Description}

\subsubsection {Basic Structure}

\index{Scantable!structure}ASAP data handling works on objects called
scantables.  A scantable holds your data, and also provides functions
to operate upon it.

The building block of a scantable is an integration, which is a single
row of a scantable. Each row contains just one spectrum for each beam,
IF and polarisation. For example Parkes OH-multibeam data would
normally contain 13 beams, 1 IF and 2 polarisations, Parkes
methanol-multibeam data would contain 7 beams, 2 IFs and 2
polarisations while the Mopra 8-GHz MOPDS filterbank will produce one
beam, many IFs, and 2-4 polarisations.

All of the combinations of Beams/IFs an Polarisations are
contained in seperate rows. These rows are grouped in cycles (same time stamp).

A collection of cycles for one source is termed a scan (and each scan
has a unique numeric identifier, the SCANNO). A scantable is then a
collection of one or more scans. If you have scan-averaged your data
in time, i.e. you have averegaed all cycles within a scan, then each
scan would hold just one (averaged) integration.

Many of the functions which work on scantables can either return a new
scantable with modified data or change the scantable insitu. Which
method is used depends on the users preference. The default can be
changed via the {\tt .asaprc} resource file.

For example a Mopra scan with a  4s intergration time, two IFs and
dual polarisations has two (2s) cycles.
\begin{verbatim}
    SCANNO  CYCLENO BEAMNO IFNO POLNO
    0       0       0      0    0
    0       0       0      0    1
    0       0       0      1    0
    0       0       0      1    1
    0       1       0      0    0
    0       1       0      0    1
    0       1       0      1    0
    0       1       0      1    1
\end{verbatim}


\subsubsection {Contents}

\index{Scantable!contents}A scantable has header information and data
(a scantable is actually an AIPS++ Table and it is generally stored in
memory when you are manipulating it with ASAP.  You can save it to
disk and then browse it with the AIPS++ Table browser if you know how
to do that !).

The data are stored in columns (the length of a column is the number of
rows/spectra of course).

Two important columns are those that describe the frequency setup.  We mention
them explicitly here because you need to be able to understand the presentation
of the frequency information and possibly how to manipulate it.

These columns are called FREQ\_ID and MOLECULE\_ID.  They contain indices, for
each IF, pointing into tables with all of the frequency and rest-frequency
information for that integration. 

There are of course many other columns which contain the actual spectra,
the flags, the Tsys, the source names and so on.

There is also a function \cmd{summary} to list a summary of the scantable.
You will find this very useful.

Example:

\begin{verbatim}
  ASAP> scans = scantable('MyData.rpf')
  ASAP> scans.summary()                # Brief listing

  # Equivalent to brief summary function call
  ASAP> print scan
\end{verbatim}

The summary function gives you a scan-based summary, presenting the
scantable as a cascading view of Beams and IFs. Note that the output
of summary is redirected into your current pager specified by the
\$PAGER environment variable. If you find the screen is reset to the
original state when summary is finished (ie the output from summary
disappears), you may need to set the \$LESS environment variable to
include the \cmd{-X} option.

\subsection{Data Selection}
\label{sec:selection}

ASAP contains flexible data selection. Data can be selected based on
IF, beam, polarisation, scan number as well as values such as
Tsys. Advanced users can also make use of the AIPS++ TAQL language to
create selections based on almost any of the values recorded.

Selection is based on a \cmd{selector} object. This object is created
and various selection functions applied to it (\cmd{set\_ifs},
\cmd{set\_beams} etc). The selection object then must be applied to a
scantable using the \cmd{set\_selection} function. A single selection
object can be created and setup then applied to multiple scantables.

Once a selection has been applied, all following functions will only
``see'' the selected spectra (including functions such as
\cmd{summary}). The selection can then be reset and all spectra are
visible. Note that if functions such as \cmd{copy} are run on a
scantable with active selection, only the selected spectra are copied.

The common selection functions are:

\begin{itemize}

\item[\cmd{set\_beams}] Select beams by index number
\item[\cmd{set\_ifs}] Select ifs by index number
\item[\cmd{set\_name}] Select by source name. Can contain ``*'' as a
wildcard, e.g. ``Orion*\_R''.
\item[\cmd{set\_ifs}] Select IFs by index number
\item[\cmd{set\_polarisation}] Select by polarisation index or
name. If polarisation names are given, the data will be on-the-fly
converted (for example from linears to Stokes).
\item[\cmd{set\_query}] Set query directly. For power users only!
\item[\cmd{set\_tsys}] Select data based on Tsys. Also example of user
definable query.
\item[\cmd{reset}] Reset the selection to include all spectra.


Note that all indices are zero based.

Examples:

\begin{verbatim}
  ASAP> selection = selector()         # Create selection object
  ASAP> selection.set_ifs(0)           # Just select the first IF
  ASAP> scans.set_selection(selection) # Apply the selection
  ASAP> print scans                    # Will just show the first IF

  ASAP> selection.set_ifs([0,1])       # Select the first two IFs
  ASAP> selection.set_beams([1,3,5])   # Also select three of the beams
  ASAP> scans.set_selection(selection) # Apply the selection

  ASAP> selection.set_name('G308*')     # Select by source name

  ASAP> selection.reset()              # Turn off selection
  ASAP> scans.set_selection(selection) # Apply the reset selection

\end{verbatim}

\end{itemize}

\subsection{State}

\index{Scantable!state}Each scantable contains "state"; these are
properties applying to all of the data in the scantable.

Examples are the selection of beam, IF and polarisation,  spectral unit
(e.g. km/s), frequency reference frame (e.g. BARY) and velocity Doppler
type (e.g. RADIO).

\subsubsection{Units, Doppler and Frequency Reference Frame}

The information describing the frequency setup for each integration
is stored fundamentally in frequency in the reference frame
of observation (E.g. TOPO).

When required, this is converted to the desired reference frame
(e.g. LSRK), Doppler (e.g. OPTICAL) and unit (e.g. km/s) on-the-fly.
This is important, for example, when you are displaying the data or
fitting to it. The reference frame is set on file read to the value
set in the user \cmd{.asaprc} file.

For units, the user has the choice of frequency, velocity or channel.
The \cmd{set\_unit} function is used to set the current unit for a
scantable. All functions will (where relevant) work with the selected
unit until this changes. This is mainly important for fitting (the fits
can be computed in any of these units), plotting and mask creation.

The velocity definition can be changed with the \cmd{set\_doppler}
function, and the frequency reference frame can be changed with the
\cmd{set\_freqframe} function.

Example usage:

\begin{verbatim}
  ASAP> scans = scantable('2004-11-23_1841-P484.rpf') # Read in the data
  ASAP> scans.set_freqframe('LSRK')  # Use the LSR velocity frame
  ASAP> scans.set_unit('km/s')        # Use velocity for plots etc from now on
  ASAP> scans.set_doppler('OPTICAL')  # Use the optical velocity convention
  ASAP> scans.set_unit('MHz')         # Use frequency in MHz from now on
\end{verbatim}


\subsubsection{Rest Frequency}

\index{Scantable!rest frequency}ASAP reads the line rest frequency
from the RPFITS file when reading the data. The values stored in the
RPFITS file are not always correct and so there is a function
\cmd{set\_restfreq} to set the rest frequencies for the currently
selected data.

For each integration, there is a rest-frequency per IF (the rest
frequencies are just stored as a list with an index into them).
There are a few ways to set the rest frequencies with this function.

If you specify just one rest frequency, then it is set for all IF.

\begin{verbatim}
  # Set all IFs
  ASAP> scans.set_restfreqs(freqs=1.667359e9)
\end{verbatim}

If set a rest frequency for each IF, specify a list of frequencies (of
length the number of IFs).  Regardless of the source, the rest
frequency will be set for each IF to the corresponding value in the
provided list.

\begin{verbatim}
  # Set rest frequency for all IFs
  ASAP> scans.set_restfreqs(freqs=[1.6654018e9,1.667359e9,])

\end{verbatim}

{\em Currently the following is not implemented

In both of the above modes, you can also specify the rest frequencies via
names in a known list rather than by their values.

Examples:

\begin{verbatim}
  ASAP> scans.set_restfreqs(freqs=['OH1665','OH1667'])
\end{verbatim}
}

\subsubsection{Masks}

\index{Masks}\index{Scantable!masks}

Many tasks (fitting, baseline subtraction, statistics etc) should only
be run on range of channels. Depending on the current ``unit'' setting
this range is set directly as channels, velocity or frequency
ranges. Internally these are converted into a simple boolean mask for
each channel of the abscissa. This means that if the unit setting is
later changed, previously created mask are still valid. (This is not
true for functions which change the shape or shift the frequency
axis).  You create masks with the function \cmd{create\_mask} and this
specified the channels to be included in the selection. When setting
the mask in velocity, the conversion from velocity to channels is
based on the current selection, specified row and selected frequency
reference frame.


Note that for multi IF data with different number of channels per IF a
single mask cannot be applied to different IFs. To use a mask on such
data the selector should be applied to select all IFs with the same
number of channels.

Example :
\begin{verbatim}

  # Select channel range for baselining
  ASAP> scans.set_unit('channels')
  ASAP> msk = scans.create_mask([100,400],[600,800])

  # Select velocity range for fitting
  ASAP> scans.set_unit('km/s')
  ASAP> msk = scans.create_mask([-30,-10])
\end{verbatim}

Sometimes it is more convenient to specify the channels to be
excluded, rather included.  You can do this with the ``invert''
argument.

Example :
\begin{verbatim}
  ASAP> scans.set_unit('channels')
  ASAP> msk = scans.create_mask([0,100],[900-1023], invert=True)
\end{verbatim}

By default \cmd{create\_mask} uses the frequency setup of the first row
to convert velocities into a channel mask. If the rows in the data
cover different velocity ranges, the scantable row to use should be
specified:

\begin{verbatim}
  ASAP> scans.set_unit('km/s')
  ASAP> msk = q.create_mask([-30,-10], row=5)
\end{verbatim}

Because the mask is stored in a simple python variable, the users is
able to combine masks using simple arithmetic. To create a mask
excluding the edge channels, a strong maser feature and a birdie in
the middle of the band:

\begin{verbatim}
  ASAP> scans.set_unit('channels')
  ASAP> msk1 = q.create_mask([0,100],[511,511],[900,1023],invert=True)
  ASAP> scans.set_unit('km/s')
  ASAP> msk2 = q.create_mask([-20,-10],invert=True)

  ASAP> mask = msk1 and msk2
\end{verbatim}


\subsection{Management}

\index{Scantable!management}During processing it is possible to create
a large number of scan tables. These all consume memory, so it is best
to periodically remove unneeded scan tables. Use \cmd{list\_scans} to
print a list of all scantables and \cmd{del} to remove unneeded ones.

Example:

\begin{verbatim}
  ASAP> list_scans()
  The user created scantables are:
  ['s', 'scans', 'av', 's2', 'ss']

  ASAP> del s2
  ASAP> del ss
\end{verbatim}

\section{Data Input}

\index{Reading data}Data can be loaded in one of two ways; using the
reader object or via the scantable constructor. The scantable method
is simpler but the reader allows the user more control on what is read.

\subsection{Scantable constructor}

\index{Scantable constructor}\index{Scantable!constructor}This loads
all of the data from filename into the scantable object scans and
averages all the data within a scan (i.e.  the resulting scantable
will have one row per scan).  The recognised input file formats are
RPFITS, SDFITS (singledish fits), ASAP's scantable format and aips++
MeasurementSet2 format.

Example usage:

\begin{verbatim}
  ASAP> scan = scantable('2004-11-23_1841-P484.rpf')

  # Don't scan average the data
  ASAP> scan = scantable('2004-11-23_1841-P484.rpf', average=False)
\end{verbatim}


\subsection{Reader object}

\index{Reader object}\index{Scantable!reader object}For more control
when reading data into ASAP, the reader object should be used.  This
has the option of only reading in a range of integrations, only a
specified beam or IF and does not perform any scan averaging of the
data, allowing analysis of the individual integrations.  Note that due
to limitation of the RPFITS library, only one reader object can be
open at one time reading RPFITS files.  To read multiple RPFITS files,
the old reader must be destroyed before the new file is opened.
However, multiple readers can be created and attached to SDFITS files.


Example usage:

\begin{verbatim}
  ASAP> r = reader('2003-03-16_082048_t0002.rpf')
  ASAP> r.summary()
  ASAP> scan = r.read()
  ASAP> del r
\end{verbatim}

\section{Basic Processing}

In the following section, a simple data reduction to form a quotient
spectrum of a single source is followed.  It has been assume that the
\cmd{.asaprc} file has {\em not} been used to change the \cmd{insitu}
default value from \cmd{True}.

\subsection{Auto quotient}
\index{Auto quotient}Quotients can be computed ``automatically''. This
requires the data to have matching source/reference pairs or one
reference for multiple sources. Auto quotient assumes reference scans
have a trailing ``\_R'' in the source name for data from Parkes and
Mopra, and a trailing ``e'' or ``w'' for data fro, Tidbinbilla.

\begin{verbatim}
  ASAP> q = s.auto_quotient()
\end{verbatim}

By default the quotient spectra is calculated
to preserve continuum emission. If you wish to remove the continuum
contribution, use the \cmd{preserve} argument:

\begin{verbatim}
 ASAP> q = s.auto_quotient(preserve=True)
\end{verbatim}

If this is not sufficient the following alternative method can be used.

\subsection{Separate reference and source observations}

\index{Quotient spectra}Most data from ATNF observatories
distinguishes on and off source data using the file name. This makes
it easy to create two scantables with the source and reference
data. As long as there was exactly one reference observation for each
on source observation for following method will work.

For Mopra and Parkes data:
\begin{verbatim}
  ASAP> r = scans.get_scan('*_R')
  ASAP> s = scans.get_scan('*_S')
\end{verbatim}

For Tidbinbilla data
\begin{verbatim}
  ASAP> r = scans.get_scan('*_[ew]')
  ASAP> s = scans.get_scan('*_[^ew]')
\end{verbatim}

\subsection{Make the quotient spectra}

Use the quotient function

\begin{verbatim}
  ASAP> q = s.quotient(r)
\end{verbatim}

This uses the rows in scantable \cmd{r} as reference spectra for the
rows in scantable \cmd{s}. Scantable \cmd{r} must have either 1 row
(which is applied to all rows in \cmd{s}) or both scantables must have
the same number of rows.

\subsection{Time average separate scans}

\index{Time average}If you have observed the source with multiple
source/reference cycles you will want to scan-average the quotient
spectra together.

\begin{verbatim}
 ASAP> av = q.average_time()
\end{verbatim}

If for some you want to average multiple sets of scantables together
you can:

\begin{verbatim}
 ASAP> av = average_time(q1, q2, q3)
\end{verbatim}

The default is to use integration time weighting. The alternative is
to use none, variance, Tsys weighting, Tsys \& integration time or
median averaging.

\begin{verbatim}
 ASAP> av = average_time(q, weight='tintsys')
\end{verbatim}

To use variance based weighting, you need to supply a mask saying which
channel range you want it to calculate the variance from.

\begin{verbatim}
 ASAP> msk = scans.create_mask([200,400],[600,800])
 ASAP> av = average_time(scans, mask=msk, weight='var')
\end{verbatim}

If you have not observed your data with Doppler tracking (or run
\cmd{freq\_align} explicitally) you should align the data in frequency
before averaging.

\begin{verbatim}
 ASAP> av = scans.average_time(align=True)
\end{verbatim}

Note that, if needed, you should run \cmd{gain\_el} and \cmd{opacity}
before you average the data in time (\S \ref{sec:gainel} \&
\ref{sec:freqalign}).

\subsection{Baseline fitting}

\index{Baseline fitting}To make a baseline fit, you must first create
a mask of channels to use in the baseline fit.

\begin{verbatim}
 ASAP> msk = scans.create_mask([100,400],[600,900])
 ASAP> scans.poly_baseline(msk, order=1)
\end{verbatim}

This will fit a first order polynomial to the selected channels and subtract
this polynomial from the full spectra.

\subsubsection{Auto-baselining}

\index{Auto-baseline}The function \cmd{auto\_poly\_baseline} can be used to automatically
baseline your data without having to specify channel ranges for the
line free data. It automatically figures out the line-free emission
and fits a polynomial baseline to that data. The user can use masks to
fix the range of channels or velocity range for the fit as well as
mark the band edge as invalid.

Simple example

\begin{verbatim}
  ASAP> scans.auto_poly_baseline(order=2,threshold=5)
\end{verbatim}

\cmd{order} is the polynomial order for the fit. \cmd{threshold} is
the SNR threshold to use to deliminate line emission from
signal. Generally the value of threshold is not too critical, however
making this too large will compromise the fit (as it will include
strong line features) and making it too small will mean it cannot find
enough line free channels.


Other examples:

\begin{verbatim}
  # Don't try and fit the edge of the bandpass which is noisier
  ASAP> scans.auto_poly_baseline(edge=(500,450),order=3,threshold=3)

  # Only fit a given region around the line
  ASAP> scans.set_unit('km/s')
  ASAP> msk = scans.create_mask([-60,-20])
  ASAP> scans.auto_poly_baseline(mask=msk,order=3,threshold=3)

\end{verbatim}

\subsection{Average the polarisations}

\index{average\_pol}If you are just interested in the highest SNR for total intensity you
will want to average the parallel polarisations together.

\begin{verbatim}
 ASAP> scans.average_pol()
\end{verbatim}

\subsection{Calibration}

\index{Calibration}For most uses, calibration happens transparently as the input data
contains the Tsys measurements taken during observations. The nominal
``Tsys'' values may be in Kelvin or Jansky. The user may wish to
supply a Tsys correction or apply gain-elevation and opacity
corrections.

\subsubsection{Brightness Units}

\index{Brightness Units}RPFITS files do not contain any information as
to whether the telescope calibration was in units of Kelvin or
Janskys.  On reading the data a default value is set depending on the
telescope and frequency of observation.  If this default is incorrect
(you can see it in the listing from the \cmd{summary} function) the
user can either override this value on reading the data or later.
E.g:

\begin{verbatim}
  ASAP> scans = scantable('2004-11-23_1841-P484.rpf', unit='Jy')
  # Or in two steps
  ASAP> scans = scantable('2004-11-23_1841-P484.rpf')
  ASAP> scans.set_fluxunit('Jy')
\end{verbatim}

\subsubsection{Feed Polarisation}

\index{Brightness Units}The RPFITS files also do not contain any
information as to the feed polarisation. ASAP will set a default based
on the antenna, but this will often be wrong the data has been read,
the default can be changed using the \cmd{set\_feedtype} function with
an argument of \cmd{'linear'} or \cmd{'circular'}.

E.g:

\begin{verbatim}
  ASAP> scans = scantable('2004-11-23_1841-P484.rpf')
  ASAP> scans.set_feedtype('circular')
\end{verbatim}

\subsubsection{Tsys scaling}

\index{Tsys scaling}Sometime the nominal Tsys measurement at the
telescope is wrong due to an incorrect noise diode calibration. This
can easily be corrected for with the scale function. By default,
\cmd{scale} only scans the spectra and not the corresponding Tsys.

\begin{verbatim}
  ASAP> scans.scale(1.05, tsys=True)
\end{verbatim}

\subsubsection{Unit Conversion}

\index{Unit conversion}To convert measurements in Kelvin to Jy (and
vice versa) the global function \cmd{convert\_flux} is needed. This
converts and scales the data from K to Jy or vice-versa depending on
what the current brightness unit is set to. The function knows the
basic parameters for some frequencies and telescopes, but the user may
need to supply the aperture efficiency, telescope diameter or the Jy/K
factor.

\begin{verbatim}
  ASAP> scans.convert_flux()               # If efficency known
  ASAP> scans.convert_flux(eta=0.48)       # If telescope diameter known
  ASAP> scans.convert_flux(eta=0.48,d=35)  # Unknown telescope
  ASAP> scans.convert_flux(jypk=15)        # Alternative
\end{verbatim}

\subsubsection{Gain-Elevation and Opacity Corrections}
\label{sec:gainel}

\index{Gain-elevation}As higher frequencies (particularly $>$20~GHz)
it is important to make corrections for atmospheric opacity and
gain-elevation effects.

Note that currently the elevation is not written correctly into
Tidbinbilla rpfits files. This means that gain-elevation and opacity
corrections will not work unless these get recalculated.

\begin{verbatim}
  ASAP> scans.recalc_azel()                # recalculate az/el based on pointing
\end{verbatim}

Gain-elevation curves for some telescopes and frequencies are known to
ASAP (currently only for Tidbinbilla at 20~GHz).  In these cases
making gain-corrections is simple.  If the gain curve for your data is
not known, the user can supply either a gain polynomial or text file
tabulating gain factors at a range of elevations (see \cmd{help
scantable.gain\_el}).

Examples:

\begin{verbatim}
  ASAP> scans.gain_el()   # If gain table known
  ASAP> scans.gain_el(poly=[3.58788e-1,2.87243e-2,-3.219093e-4])
\end{verbatim}

\index{Opacity}Opacity corrections can be made with the global
function \cmd{opacity}. This should work on all telescopes as long as
a measurement of the opacity factor was made during the observation.

\begin{verbatim}
  ASAP> scans.opacity(0.083)
\end{verbatim}

Note that at 3~mm Mopra uses a paddle wheel for Tsys calibration,
which takes opacity effects into account (to first order). ASAP
opacity corrections should not be used for Mopra 3-mm data.

\subsection{Frequency Frame Alignment}
\label{sec:freqalign}

\index{Frequency alignment}\index{Velicity alignment}When time
averaging a series of scans together, it is possible that the velocity
scales are not exactly aligned.  This may be for many reasons such as
not Doppler tracking the observations, errors in the Doppler tracking
etc.  This mostly affects very long integrations or integrations
averaged together from different days.  Before averaging such data
together, they should be frequency aligned using \cmd{freq\_align}.

E.g.:

\begin{verbatim}
  ASAP> scans.freq_align()
  ASAP> av = average_time(scans)
\end{verbatim}

{\em A Global freq\_align command will be made eventually}

To average together data taken on different days, which are in
different scantables, each scantable must aligned to a common
reference time then the scantables averaged. The simplest way of
doing this is to allow ASAP to choose the reference time for the first
scantable then using this time for the subsequent scantables.

\begin{verbatim}
  ASAP> scans1.freq_align() # Copy the refeference Epoch from the output
  ASAP> scans2.freq_align(reftime='2004/11/23/18:43:35')
  ASAP> scans3.freq_align(reftime='2004/11/23/18:43:35')
  ASAP> av = average_time(scans1, scans2, scans3)
\end{verbatim}

\section{Scantable manipulation}

\index{Scantable!manipulation}While it is very useful to have many
independent sources within one scantable, it is often inconvenient for
data processing. The \cmd{get\_scan} function can be used to create a
new scantable with a selection of scans from a scantable. The
selection can either be on the source name, with simple wildcard
matching or set of scan ids. Internally this uses the selector object,
so for more complicated selection the selector should be used directly
instead.

For example:

\begin{verbatim}
  ASAP> ss = scans.get_scan(10) # Get the 11th scan (zero based)
  ASAP> ss = scans.get_scan(range(10)) # Get the first 10 scans
  ASAP> ss = scans.get_scan(range(10,20)) # Get the next 10 scans
  ASAP> ss = scans.get_scan([2,4,6,8,10]) # Get a selection of scans

  ASAP> ss = scans.get_scan('345p407') # Get a specific source
  ASAP> ss = scans.get_scan('345*')    # Get a few sources

  ASAP> r = scans.get_scan('*_R') # Get all reference sources (Parkes/Mopra)
  ASAP> s = scans.get_scan('*_S') # Get all program sources (Parkes/Mopra)
  ASAP> r = scans.get_scan('*[ew]')  # Get all reference sources (Tid)
  ASAP> s = scans.get_scan('*[^ew]') # Get all program sources (Tid)

\end{verbatim}

To copy a scantable the following does not work:

\begin{verbatim}
  ASAP> ss = scans
\end{verbatim}

as this just creates a reference to the original scantable. Any
changes made to \cmd{ss} are also seen in \cmd{scans}. To duplicate a
scantable, use the copy function.

\begin{verbatim}
  ASAP> ss = scans.copy()
\end{verbatim}

\section{Data Output}

\index{Scantable!save}\index{Saving data}ASAP can save scantables in a
variety of formats, suitable for reading into other packages. The
formats are:

\begin{itemize}
\item[ASAP] This is the internal format used for ASAP. It is the only
  format that allows the user to restore the data, fits etc. without
  loosing any information.  As mentioned before, the ASAP scantable is
  an AIPS++ Table (a memory-based table).  This function just converts
  it to a disk-based Table.  You can the access that Table with the
  AIPS++ Table browser or any other AIPS++ tool.

\item[SDFITS] The Single Dish FITS format. This format was designed to
  for interchange between packages, but few packages actually can read
  it.

%\item[FITS] This uses simple ``image'' fits to save the data, each row
%  being written to a separate fits file. This format is suitable for
%  importing the data into CLASS.

\item[ASCII] A simple text based format suitable for the user to
processing using Perl or, Python, gnuplot etc.

\item[MS2] Saves the data in an aips++ MeasurementSet V2 format.
You can also access this with the Table browser and other AIPS++
tools.

\end{itemize}

The default output format can be set in the users {\tt .asaprc} file.
Typical usages are:

\begin{verbatim}
  ASAP> scans.save('myscans') # Save in default format
  ASAP> scans.save('myscans', 'FITS') # Save as FITS for exporting into CLASS
  ASAP> scans.save('myscans', overwrite=True) # Overwrite an existing file
\end{verbatim}

\section{Plotter}

\index{Plotter}Scantable spectra can be plotted at any time. An asapplotter object is
used for plotting, meaning multiple plot windows can be active at the
same time. On start up a default asapplotter object is created called
``plotter''. This would normally be used for standard plotting.

The plotter, optionally, will run in a multipanel mode and contain
multiple plots per panel. The user must tell the plotter how they want
the data distributed. This is done using the set\_mode function. The
default can be set in the users {\tt .asaprc} file. The units (and frame
etc) of the abscissa will be whatever has previously been set by
\cmd{set\_unit}, \cmd{set\_freqframe} etc.

Typical plotter usage would be:

\begin{verbatim}
  ASAP> scans.set_unit('km/s')
  ASAP> plotter.set_mode(stacking='p',panelling='t')
  ASAP> plotter.plot(scans)
\end{verbatim}

This will plot multiple polarisation within each plot panel and each
scan row in a separate panel.

Other possibilities include:

\begin{verbatim}
  # Plot multiple IFs per panel
  ASAP> plotter.set_mode(stacking='i',panelling='t')

  # Plot multiple beams per panel
  ASAP> plotter.set_mode(stacking='b',panelling='t')

  # Plot one IF per panel, time stacked
  ASAP> plotter.set_mode('t', 'i')

  # Plot each scan in a seperate panel
  ASAP> plotter.set_mode('t', 's')

\end{verbatim}

\subsection{Plot Selection}
\label{sec:plotter_cursor}

\index{Plotter!selection}The plotter can plot up to 25 panels and
stacked spectra per panel. If you have data larger than this (or for
your own sanity) you need to select a subset of this data. This is
particularly true for multibeam or multi IF data. The selector object
should be used for this purpose. Selection can either be applied to
the scantable or directly to the plotter, the end result is the same.
You don't have to reset the scantable slection though, if you set
the selection on the plotter.

Examples:

\begin{verbatim}
  ASAP> selection = selector()
  # Select second IF
  ASAP> selection.set_ifs(1)
  ASAP> plotter.set_selection(selection)

  # Select first 4 beams
  ASAP> selection.set_beams([0,1,2,3])
  ASAP> plotter.set_selection(selection)

  # Select a few scans
  ASAP> selection.set_scans([2,4,6,10])
  ASAP> plotter.set_selection(selection)

  # Multiple selection
  ASAP> selection.set_ifs(1)
  ASAP> selection.set_scans([2,4,6,10])
  ASAP> plotter.set_selection(selection)

\end{verbatim}

\subsection{Plot Control}

\index{Plotter!control}The plotter window has a row of buttons on the
lower left. These can be used to control the plotter (mostly for
zooming the individual plots). From left to right:

\begin{itemize}

\item[Home] This will unzoom the plots to the original zoom factor

\item[Plot history] (left and right arrow). The plotter keeps a
history of zoom settings. The left arrow sets the plot zoom to the
previous value. The right arrow returns back again. This allows you,
for example, to zoom in on one feature then return the plot to how it
was previously.

\item[Pan] (The Cross) This sets the cursor to pan, or scroll mode
  allowing you to shift the plot within the window. Useful when
  zoomed in on a feature.

\item[Zoom] (the letter with the magnifying glass) lets you draw a
  rectangle around a region of interest then zooms in on that
  region. Use the plot history to unzoom again.

\item[Adjust] (rectangle with 4 arrows) adjust subplot paramaters
  (space at edge of plots)

\item[Save] (floppy disk). Save the plot as a postscript or .png file

You can also type ``g'' in the plot window to toggle on and off grid
lines. Typing 'l' turns on and off logarithmic Y-axis.

\end{itemize}

\subsection{Other control}

The plotter has a number of functions to describe the layout of the
plot. These include \cmd{set\_legend}, \cmd{set\_layout} and \cmd{set\_title}.

To set the exact velocity or channel range to be plotted use the
\cmd{set\_range} function. To reset to the default value, call
\cmd{set\_range} with no arguments. E.g.

\begin{verbatim}
  ASAP> scans.set_unit('km/s')
  ASAP> plotter.plot(scans)
  ASAP> plotter.set_range(-150,-50)
  ASAP> plotter.set_range() # To reset
\end{verbatim}

Both the range of the ``x'' and ``y'' axis can be set at once, if desired:

\begin{verbatim}
  ASAP> plotter.set_range(-10,30,-1,6.6)
\end{verbatim}

To save a hardcopy of the current plot, use the save function, e.g.

\begin{verbatim}
  ASAP> plotter.save('myplot.ps')
  ASAP> plotter.save('myplot.png', dpi=80)
\end{verbatim}

\subsection{Plotter Customisation}

The plotter allows the user to change most properties such as text
size and colour. The \cmd{commands} funcation and {\cmd help\
asapplotter} list all the possible commands that can be used with the
plotter.

\commanddef{set\_colors}{Change the default colours used for line
plotting. Colours can be given either by name, using the html standard
(e.g. red, blue or hotpink), or hexadecimal code (e.g. for black
\#000000). If less colours are specified than lines plotted , the
plotter cycles through the colours. Example:} {ASAP>
plotter.set\_colors('red blue green')\\ ASAP>
plotter.set\_colors(`\#0000 blue \#FF00FF')\\ }

\commanddef{set\_linestyles}{Change the line styles used for
plots. Allowable values are 'line', 'dashed', 'dotted', 'dashdot', 
'dashdotdot' and 'dashdashdot. Example: }{
  ASAP> plotter.set\_linestyles('line dash cotted datshot.)\\
  ASAP> plotter.set\_font(size=10)\\
}

\commanddef{set\_font}{Change the font style and size. Example}{
  ASAP> plotter.set\_font(weight='bold')\\
  ASAP> plotter.set\_font(size=10)\\
  ASAP> plotter.set\_font(style='italic')\\
}

\commanddef{set\_layout}{Change the multi-panel layout, ie now many
  rows and columns}{
  ASAP> plotter.set\_layout(3,2)
}

\commanddef{set\_legend}{Set the position, size and optional value of the legend}{
  ASAP> plotter.set\_legend(fontsize=16)\\
  ASAP> plotter.set\_legend(mode=0)  \# ASAP chooses where to put the legend\\
  ASAP> plotter.set\_legend(mode=4)  \# Put legend on lower right\\
  ASAP> plotter.set\_legend(mode=-1) \# No legend\\
  ASAP> plotter.set\_legend(mp=['RR','LL']) \# Specify legend labels\\
  ASAP> plotter.set\_legend(mp=[r'\$\^\{12\}CO\$',r'\$\^\{13\}CO\$']) \# Latex labels
}

\commanddef{set\_title}{Set the plot title. If multiple panels are
  plotted, multiple titles have to be specified}{
  ASAP> plotter.set\_title(`G323.12$-$1.79`)\\
  ASAP> plotter.set\_title([`SiO`, 'Methanol'], fontsize=18)\\
}

\subsection{Plotter Annotations}

The plotter allows various annontations (lines, arrows, text and
``spans'') to be added to the plot. These annotations are
``temporary'', when the plotter is next refreshed
(e.g. \cmd{plotter.plot} or \cmd{plotter.set\_range}) the annotations
will be removed. 

\bigcommanddef{arrow(x,y,x+dx,y+dy)}{Draw an arrow from a specifed
\cmd{(x,y)} position to \cmd{(x+dx, y+dy)}. The values are in world
coordinates. \em {HOW TO SET ARROW HEAD??}}{
  ASAP> plotter.arrow(-40,7,35,0)
}

\bigcommanddef{axhline(y, xmin, xmax)}{Draw a horizontal line at the
specifed \cmd{y} position (in world coordinates) between xmin and xmax
(in relative coordinates, ie 0.0 is the left hand edge of the plot
while 1.0 is the right side of the plot.}{
 ASAP> plotter.axhline(6.0,0.2,0.8)
}

\bigcommanddef{avhline(x, ymin, ymax)}{Draw a vertical line at the
specifed \cmd{x} position (in world coordinates) between \cmd{ymin}
and \cmd{ymax} (in relative coordinates, ie 0.0 is the left hand edge
of the plot while 1.0 is the right side of the plot).}{
 ASAP> plotter.axvline(-50.0,0.1,1.0)
}

\bigcommanddef{axhspan(ymin, ymax, \\ \hspace*{20mm}xmin,
 xmax)}{Overlay a transparent colour rectangle. \cmd{ymin} and
 \cmd{ymax} are given in world coordnates while \cmd{xmin} and
 \cmd{xmax} are given in relative coordinates}{ 
ASAP> plotter.axhspan(2,4,0.25,0.75) 
}

\bigcommanddef{axvspan(xmin, xmax, \\ \hspace*{20mm} ymin,
 ymax)}{Overlay a transparent colour rectangle. \cmd{ymin} and
 \cmd{ymax} are given in relative coordnates while \cmd{xmin} and
 \cmd{xmax} are given in world coordinates}{ 
ASAP> plotter.axvspan(-50,60,0.2,0.5) 
}

\bigcommanddef{text(x, y, str)}{Place the string \cmd{str} at the
 given \cmd{(x,y)} position in world coordinates.}{
ASAP>plotter.text(-10,7,"CO") 
}



These functions all take a set of \cmd{kwargs} commands. These can be
used to set colour, linewidth fontsize etc. These are standard
matplotlib settings. Common ones include:

\begin{tabular}{ll}
 \tt color  \\
 \tt linewidth \\
 \tt fontsize \\
 \tt fontname & Sans, Helvetica, Courier, Times etc\\
 \tt rotation & Text rotation (horizontal, vertical) \\
 \tt alpha & The alpha transparency on 0-1 scale\\
\end{tabular}

Examples: 
\begin{verbatim}
  ASAP> plotter.axhline(6.0,0.2,0.8, color='red', linewidth=3)
  ASAP> plotter.text(-10,7,"CO", fontsize=20)
\end{verbatim}

\section{Fitting}

\index{Fitting}Currently multicomponent Gaussian function is
available. This is done by creating a fitting object, setting up the
fit and actually fitting the data. Fitting can either be done on a
single scantable selection or on an entire scantable using the
\cmd{auto\_fit} function. If single value fitting is used, and the
current selection includes multiple spectra (beams, IFs, scans etc)
then the first spectrum in the scantable will be used for fitting.

\begin{verbatim}
 ASAP> f = fitter()
 ASAP> f.set_function(gauss=2) # Fit two Gaussians
 ASAP> f.set_scan(scans)
 ASAP> selection = selector()
 ASAP> selection.set_polarisations(1) # Fit the second polarisation
 ASAP> scans.set_selection(selection)
 ASAP> scans.set_unit('km/s')  # Make fit in velocity units
 ASAP> f.fit(1)                # Run the fit on the second row in the table
 ASAP> f.plot()                # Show fit in a plot window
 ASAP> f.get_parameters()      # Return the fit paramaters
\end{verbatim}

This auto-guesses the initial values of the fit and works well for data
without extra confusing features. Note that the fit is performed in
whatever unit the abscissa is set to.

If you want to confine the fitting to a smaller range (e.g. to avoid
band edge effects or RFI you must set a mask.

\begin{verbatim}
  ASAP> f = fitter()
  ASAP> f.set_function(gauss=2)
  ASAP> scans.set_unit('km/s')  # Set the mask in channel units
  ASAP> msk = s.create_mask([1800,2200])
  ASAP> scans.set_unit('km/s')  # Make fit in velocity units
  ASAP> f.set_scan(s,msk)
  ASAP> f.fit()
  ASAP> f.plot()
  ASAP> f.get_parameters()
\end{verbatim}

If you wish, the initial parameter guesses can be specified and
specific parameters can be fixed:

\begin{verbatim}
  ASAP> f = fitter()
  ASAP> f.set_function(gauss=2)
  ASAP> f.set_scan(s,msk)
  ASAP> f.fit() # Fit using auto-estimates
  # Set Peak, centre and fwhm for the second gaussian.
  # Force the centre to be fixed
  ASAP> f.set_gauss_parameters(0.4,450,150,0,1,0,component=1)
  ASAP> f.fit() # Re-run the fit
\end{verbatim}

The fitter \cmd{plot} function has a number of options to either view
the fit residuals or the individual components (by default it plots
the sum of the model components).

Examples:

\begin{verbatim}
  # Plot the residual
  ASAP> f.plot(residual=True)

  # Plot the first 2 componentsa
  ASAP> f.plot(components=[0,1])

  # Plot the first and third component plus the model sum
  ASAP> f.plot(components=[-1,0,2])  # -1 means the compoment sum
\end{verbatim}

\subsection{Fit saving}

\index{Fitter!Fit saving}One you are happy with your fit, it is
possible to store it as part of the scantable.

\begin{verbatim}
  ASAP> f.storefit()
\end{verbatim}

This will be saved to disk with the data, if the ``ASAP'' file format
is selected. Multiple fits to the same data can be stored in the
scantable.

The scantable function \cmd{get\_fit} can be used to retrieve the
stored fits. Currently the fit parameters are just printed to the
screen.

\begin{verbatim}
  ASAP> scans.get_fit(4) # Print fits for row 4
\end{verbatim}

\section{Polarisation}

\index{Polarisation}Currently ASAP only supports polarmetric analysis
on linearly polarised feeds and the cross polarisation products
measured. Other cases will be added on an as needed basis.

Conversions of linears to Stokes or Circular polarisations are done
``on-the-fly''. Leakage cannot be corrected for nor are there routines
to calibrate position angle offsets.

\subsection{Simple Calibration}

\index{Polarisation!calibration}It is possible that there is a phase
offset between polarisation which will effect the phase of the cross
polarisation correlation, and so give rise to spurious
polarisation. \cmd{rotate\_xyphase} can be used to correct for this
error. At this point, the user must know how to determine the size of
the phase offset themselves.

\begin{verbatim}
  ASAP> scans.rotate_xyphase(10.5)            # Degrees
\end{verbatim}

Note that if this function is run twice, the sum of the two values is
applied because it is done in-situ.

A correction for the receiver parallactic angle may need to be made,
generally because of how it is mounted. Use \cmd{rotate\_linpolphase}
to correct the position angle. Running this function twice results in
the sum of the corrections being applied because it is applied
in-situ.

\begin{verbatim}
  ASAP> scans.rotate_linpolphase(-45) # Degrees; correct for receiver mounting
\end{verbatim}

If the sign of the complex correlation is wrong (this can happen
depending on the correlator configuration), use \cmd{invert\_phase} to
change take the complex conjugate of the complex correlation
term. This is always performed in-situ.

\begin{verbatim}
  ASAP> scans.invert_phase()
\end{verbatim}

Depending on how the correlator is configured, ``BA'' may be
correlated insead of ``AB''. Use \cmd{swap\_linears} to correct for
this problem:

\begin{verbatim}
  ASAP> scans.swap_linears()
\end{verbatim}

\subsection{Conversion}
\label{sec:polconv}

Data can be permanently converted between linear and circular
polarisations and stokes.

\begin{verbatim}
  ASAP> stokescans = linearscans.convert_pol("stokes")
\end{verbatim}


\subsection{Plotting}
\label{sec:polplot}

\index{Polarisation!plotting}To plot Stokes values, a selector object
must be created and the set\_polarisation function used to select the
desired polarisation products.

The values which can be plotted include a selection of [I,Q,U,V], [I,
Plinear, Pangle, V], [RR, LL] or [XX, YY, Real(XY),
Imaginary(XY)]. (Plinear and Pangle are the percentage and position
angle of linear polarisation).

Example:

\begin{verbatim}
  ASAP> selection = selector()

  ASAP> selection.set_polarisations(``I Q U V'')
  ASAP  plotter.set_selection(selection);              # Select I, Q, U \& V

  ASAP> selection.set_polarisations(``I Q'')
  ASAP  plotter.set_selection(selection);              # Select just I \& Q

  ASAP> selection.set_polarisations(``RR LL'')
  ASAP  plotter.set_selection(selection);              # Select just RR \& LL

  ASAP> selection.set_polarisations(``XX YY'')
  ASAP  plotter.set_selection(selection);              # Select linears

  ASAP> selection.set_polarisations(``I Plinear'')
  ASAP  plotter.set_selection(selection);              # Fractional linear

  ASAP> selection.set_polarisations(``Pangle'')
  ASAP  plotter.set_selection(selection);              # Position angle

\end{verbatim}

Scan, beam and IF selection are also available in the selector object as
describe in section~\ref{sec:selection}.

\subsection{Saving}

\index{Polarisation!saving}When saving data using the \cmd{save}
function, the \cmd{stokes} argument can be used to save the data as
Stoke values when saving in FITS format.

Example:

\begin{verbatim}
  ASAP> scans.save('myscan.sdfits', 'SDFITS', stokes=True)
\end{verbatim}

\section{Specialised Processing}

\subsection{Multibeam MX mode}

MX mode is a specific observing approach with a multibeam where a
single source is observed cycling through each beam. The scans when
the beam is off source is used as a reference for the onsource
scan. The function \cmd{mx\_quotient} is used to make a quotient
spectrum from an MX cycle. This works averaging the ``off-source''
scans for each beam (either a median average or mean) and using this
as a reference scan in a normal quotient (for each beam). The final
spectrum for each beam is returned on a new scantable containing
single scan (it the scan numbers are re-labelled to be the same). Note
that the current version of \cmd{mx\_quotient} only handles a single
MX cycle, ie if each beam has observed the source multiple times you
will need to use the selector object multiple times to select a single
MX cycle, run \cmd{mx\_quotient} for each cycle then merge the
resulting scan tables back together.

Example:

\begin{verbatim}
  ASAP> scans = scantable('mydata.rpf')
  ASAP> q = scans.mx_quotient()
  ASAP> plotter.plot(q)
\end{verbatim}

The function \cmd{average\_beam} averages multiple beam data
together. This is need if MX mode has been used to make a long
integration on a single source. E.g.

\begin{verbatim}
  ASAP> av = q.average_beam()
\end{verbatim}

\subsection{Frequency Switching}

{\em FILL ME IN}

\section{Scantable Mathematics}

\index{Scantable!maths}It is possible to to simple mathematics
directly on scantables from the command line using the \cmd{+, -, *,
/} operators as well as their cousins \cmd{+=, -= *=, /=}. This works
between a scantable and a float. (Note that it does
not work for integers).

{\em Currently mathematics between two scantables is not available }

%  ASAP> sum = scan1+scan2
\begin{verbatim}
  ASAP> scan2 = scan1+2.0
  ASAP> scan *= 1.05
\end{verbatim}

\section{Scripting}

\index{Scripting}Because asap is based on python, it easy for the user
write their own scripts and functions to process data. This is highly
recommended as most processing of user data could then be done in a
couple of steps using a few simple user defined functions. A Python
primer is beyond the scope of this userguide. See the asap home pages
for a scripting tutorial or the main python website for comprehensive
documentation.

\hspace{1cm} http://www.atnf.csiro.au/computing/software/asap/tutorials

\hspace{1cm} http://www.python.org/doc/Introduction.html

\subsection{Running scripts}

The asap global function \cmd{execfile} reads the named text file and
executes the contained python code. This file can either contain
function definitions which will be used in subsequent processing or
just a set of commands to process a specific dataset.

\subsection{asapuserfuncs.py}

The file $\sim$/.asap/asapuserfuncs.py is automatically read in when
asap is started. The user can use this to define a set of user
functions which are automatically available each time asap is
used. The \cmd{execfile} function can be called from within this file.

\section{Worked examples}

In the following section a few examples of end-to-end processing of
some data in asap are given.

\subsection{Mopra}
\index{Mopra}

The following example is of some dual polarisation, position switched
data from Mopra. The source has been observed mulitple times split
into a number of seperate rpfits files. To make the processing easier,
the first step is to \cmd{cat} the seeprate rpfits files together and
load as a whole (future versions of asap will make this unnecessary).


\begin{verbatim}
# get a list of the individual rpfits files in the current directory
myfiles = list_files()

# Load the data into a scantable
data = scantable(myfiles)
print data

# Form the quotient spectra
q = data.auto_quotient()
print q

# Look at the spectra
plotter.plot(q)

# Set unit and reference frame
q.set_unit('km/s')
q.set_freqframe('LSRK')

# Average all scans in time, aligning in velocity
av = q.average_time(align=True)
plotter.plot(av)

# Remove the baseline
msk = av.create_mask([100,130],[160,200])
av.poly_baseline(msk,2)

# Average the two polarisations together
iav = av.average_pol()
print iav
plotter.plot(iav)

# Set a sensible velocity range on the plot
plotter.set_range(85,200)

# Smooth the data a little
av.smooth('gauss',4)
plotter.plot()

# Fit a guassian to the emission
f = fitter()
f.set_function(gauss=1)
f.set_scan(av)
f.fit()

# View the fit
f.plot()

# Get the fit parameters
f.get_parameters()

\end{verbatim}


\subsection{Parkes Polarimetry}

\index{Parkes}\index{Polarisation}The following example is processing
of some Parkes polarmetric observations of OH masers at
1.6~GHz. Because digital filters where used in the backend, the
baselines are stable enough not to require a quotient spectra. The
4~MHz bandwidth is wide enough to observe both the 1665 and 1667~MHz
OH maser transitions. Each source was observed once for about 10
minutes. Tsys information was not written to the rpfits file (a
nominal 25K values was used), so the amplitudes need to be adjusted
based on a separate log file. A simple user function is used to
simplify this, contained in a file called mypol.py:

\begin{verbatim}
def xyscale(data,xtsys=1.0,ytsys=1.0,nomtsys=25.0) :

 selection = selector()
 selection.set_polarisations(0)
 data.set_selection(selection)
 data.scale(xtsys/nomtsys)

 selection.set_polarisations(1)
 data.set_selection(selection)
 data.scale(ytsys/nomtsys)

 selection.set_polarisations(0)
 data.set_selection(selection)
 data.scale((xtsys+ytsys)/(2*nomtsys))

 selection.set_polarisations(0)
 data.set_selection(selection)
 data.scale((xtsys+ytsys)/(2*nomtsys))
\end{verbatim}

The typical asap session would be

\begin{verbatim}

# Remind ourself the name of the rpfits files
ls

# Load data from an rpfits file
d1665 = scantable('2005-10-27_0154-P484.rpf')

# Check what we have just loaded
d1665.summary()

# View the data in velocity
d1665.set_unit('km/s')
d1665.set_freqframe('LSRK')

# Correct for the known phase offset in the crosspol data
d1665.rotate_xyphase(-4)

# Create a copy of the data and set the rest frequency to the 1667 MHz
# transition
d1667 = d1665.copy()
d1667.set_restfreqs([1667.3590], 'MHz')
d1667.summary()

# Copy out the scan we wish to process
g351_5 = d1665.get_scan('351p160')
g351_7 = d1667.get_scan('351p160')

# Baseline both
msk = g351_5.create_mask([-30,-25],[-5,0])
g351_5.poly_baseline(msk,order=1)
msk = g351_7.create_mask([-30,-25],[-5,0])
g351_7.poly_baseline(msk,order=1)


# Plot the data. The plotter can only plot a single scantable
# So we must merge the two tables first

plotscans = merge(g351_5, g351_7)

plotter.plot(plotscans) # Only shows one panel

# Tell the plotter to stack polarisation and panel scans
plotter.set_mode('p','s')

# Correct for the Tsys using our predefined function
execfile('mypol.py') # Read in the function xyscale
xyscale(g351_5,23.2,22.7) # Execute it on the data
xyscale(g351_7,23.2,22.7)

# Only plot the velocity range of interest
plotter.set_range(-30,10)

# Update the plot with the baselined data
plotter.plot()

# Look at the various polarisation products
selection = selector()
selection.set_polarisations(``RR LL'')
plotter.set_selection(selection)
selection.set_polarisations(``I Plinear'')
plotter.set_selection(selection)
selection.set_polarisations(``I Q U V'')
plotter.set_selection(selection)

# Save the plot as postscript
plotter.save('g351_stokes.ps')

# Save the process spectra
plotscans.save('g351.asap')

\end{verbatim}

\subsection{Tidbinbilla}

\index{Tidbinbilla}The following example is processing of some
Tidbinbilla observations of NH$_3$ at 12~mm. Tidbinbilla has (at the
time of observations) a single polarisation, but can process two IFs
simultaneously. In the example, the first half of the observation was
observing the (1,1) and (2,2) transitions simultaneously). The second
half observed only the (4,4) transition due to bandwidth
limitations. The data is position switched, observing first an
reference to the west, then the source twice and finally reference to
the east.

\begin{verbatim}

# Load the rpfits file and inspect
d = scantable('2003-03-16_082048_t0002.rpf')
print d

# Make the quotient spectra
q = d.auto_quotient()
print q

del d

# Plot/select in velocity
q.set_freqframe('LSRK')
q.set_unit('km/s')

# Correct for gain/el effects

q.recalc_azel()  # Tid does not write the elevation
q.gain_el()
q.opacity(0.05)

# Seperate data from the (1,1)&(2,2) and (4,4) transitions
g1 = q.get_scan(range(6))     # scans 0..5
g2 = q.get_scan(range(6,12))  # scans 6..11

# Align data in velocity
g1.freq_align()
g2.freq_align()

# Average individual scans
a1 = g1.average_time()
a2 = g2.average_time()

# Rpfits file only contains a single rest frequency. Set both
a1.set_restfreqs([23694.4700e6,23722.6336e6])

plotter.plot(a1)
plotter.set_mode('i','t')

a1.auto_poly_baseline()

plotter.plot()

a1.smooth('gauss',5)
plotter.plot()


\end{verbatim}

\newpage

\section{Appendix}

\subsection{Function Summary}

\index{Functions!summary}%
\begin{verbatim}

    [The scan container]
        scantable           - a container for integrations/scans
                              (can open asap/rpfits/sdfits and ms files)
            copy            - returns a copy of a scan
            get_scan        - gets a specific scan out of a scantable
                              (by name or number)
            drop_scan       - drops a specific scan out of a scantable
                              (by number)
            set_selection   - set a new subselection of the data
            get_selection   - get the current selection object
            summary         - print info about the scantable contents
            stats           - get specified statistic of the spectra in
                              the scantable
            stddev          - get the standard deviation of the spectra
                              in the scantable
            get_tsys        - get the TSys
            get_time        - get the timestamps of the integrations
            get_sourcename  - get the source names of the scans
            get_azimuth     - get the azimuth of the scans
            get_elevation   - get the elevation of the scans
            get_parangle    - get the parallactic angle of the scans
            get_unit        - get the current unit
            set_unit        - set the abcissa unit to be used from this
                              point on
            get_abcissa     - get the abcissa values and name for a given
                              row (time)
            set_freqframe   - set the frame info for the Spectral Axis
                              (e.g. 'LSRK')
            set_doppler     - set the doppler to be used from this point on
            set_dirframe    - set the frame for the direction on the sky
            set_instrument  - set the instrument name
            set_feedtype    - set the feed type
            get_fluxunit    - get the brightness flux unit
            set_fluxunit    - set the brightness flux unit
            create_mask     - return an mask in the current unit
                              for the given region. The specified regions
                              are NOT masked
            get_restfreqs   - get the current list of rest frequencies
            set_restfreqs   - set a list of rest frequencies
            flag            - flag selected channels in the data
            lag_flag        - flag specified frequency in the data
            save            - save the scantable to disk as either 'ASAP',
                              'SDFITS' or 'ASCII'
            nbeam,nif,nchan,npol - the number of beams/IFs/Pols/Chans
            nscan           - the number of scans in the scantable
            nrow            - te number of spectra in the scantable
            history         - print the history of the scantable
            get_fit         - get a fit which has been stored witnh the data
            average_time    - return the (weighted) time average of a scan
                              or a list of scans
            average_pol     - average the polarisations together.
            average_beam    - average the beams together.
            convert_pol     - convert to a different polarisation type
            auto_quotient   - return the on/off quotient with
                              automatic detection of the on/off scans (closest
                              in time off is selected)
            mx_quotient     - Form a quotient using MX data (off beams)
            scale, *, /     - return a scan scaled by a given factor
            add, +, -       - return a scan with given value added
            bin             - return a scan with binned channels
            resample        - return a scan with resampled channels
            smooth          - return the spectrally smoothed scan
            poly_baseline   - fit a polynomial baseline to all Beams/IFs/Pols
            auto_poly_baseline - automatically fit a polynomial baseline
            recalc_azel     - recalculate azimuth and elevation based on
                              the pointing
            gain_el         - apply gain-elevation correction
            opacity         - apply opacity correction
            convert_flux    - convert to and from Jy and Kelvin brightness
                              units
            freq_align      - align spectra in frequency frame
            invert_phase    - Invert the phase of the cross-correlation
            swap_linears    - Swap XX and YY
            rotate_xyphase  - rotate XY phase of cross correlation
            rotate_linpolphase - rotate the phase of the complex
                                 polarization O=Q+iU correlation
            freq_switch     - perform frequency switching on the data
            stats           - Determine the specified statistic, e.g. 'min'
                              'max', 'rms' etc.
            stddev          - Determine the standard deviation of the current
                              beam/if/pol
     [Selection]
         selector              - a selection object to set a subset of a scantable
            set_scans          - set (a list of) scans by index
            set_cycles         - set (a list of) cycles by index
            set_beams          - set (a list of) beamss by index
            set_ifs            - set (a list of) ifs by index
            set_polarisations  - set (a list of) polarisations by name
                                 or by index
            set_names          - set a selection by name (wildcards allowed)
            set_tsys           - set a selection by tsys thresholds
            reset              - unset all selections
            +                  - merge to selections

     [Math] Mainly functions which operate on more than one scantable

            average_time    - return the (weighted) time average
                              of a list of scans
            quotient        - return the on/off quotient
            simple_math     - simple mathematical operations on two scantables,
                              'add', 'sub', 'mul', 'div'
            quotient        - build quotient of the given on and off scans
                              (matched pairs and 1 off/n on are valid)
            merge           - merge a list of scantables

     [Line Catalog]
        linecatalog              - a linecatalog wrapper, taking an ASCII or
                                   internal format table
            summary              - print a summary of the current selection
            set_name             - select a subset by name pattern, e.g. '*OH*'
            set_strength_limits  - select a subset by line strength limits
            set_frequency_limits - select a subset by frequency limits
            reset                - unset all selections
            save                 - save the current subset to a table (internal
                                   format)
            get_row              - get the name and frequency from a specific
                                   row in the table
     [Fitting]
        fitter
            auto_fit        - return a scan where the function is
                              applied to all Beams/IFs/Pols.
            commit          - return a new scan where the fits have been
                              commited.
            fit             - execute the actual fitting process
            store_fit       - store the fit parameters in the data (scantable)
            get_chi2        - get the Chi^2
            set_scan        - set the scantable to be fit
            set_function    - set the fitting function
            set_parameters  - set the parameters for the function(s), and
                              set if they should be held fixed during fitting
            set_gauss_parameters - same as above but specialised for individual
                                   gaussian components
            get_parameters  - get the fitted parameters
            plot            - plot the resulting fit and/or components and
                              residual
    [Plotter]
        asapplotter         - a plotter for asap, default plotter is
                              called 'plotter'
            plot            - plot a scantable
            plot_lines      - plot a linecatalog overlay
            save            - save the plot to a file ('png' ,'ps' or 'eps')
            set_mode        - set the state of the plotter, i.e.
                              what is to be plotted 'colour stacked'
                              and what 'panelled'
            set_selection   - only plot a selected part of the data
            set_range       - set a 'zoom' window [xmin,xmax,ymin,ymax]
            set_legend      - specify user labels for the legend indeces
            set_title       - specify user labels for the panel indeces
            set_abcissa     - specify a user label for the abcissa
            set_ordinate    - specify a user label for the ordinate
            set_layout      - specify the multi-panel layout (rows,cols)
            set_colors      - specify a set of colours to use
            set_linestyles  - specify a set of linestyles to use if only
                              using one color
            set_font        - set general font properties, e.g. 'family'
            set_histogram   - plot in historam style
            set_mask        - set a plotting mask for a specific polarization
            text            - draw text annotations either in data or relative
                              coordinates
            arrow           - draw arrow annotations either in data or relative
                              coordinates
            set_abcissa     - specify a user label for the abcissa
            set_ordinate    - specify a user label for the ordinate
            set_layout      - specify the multi-panel layout (rows,cols)
            set_colors      - specify a set of colours to use
            set_linestyles  - specify a set of linestyles to use if only
                              using one color
            set_font        - set general font properties, e.g. 'family'
            set_histogram   - plot in historam style
            set_mask        - set a plotting mask for a specific polarization
            text            - draw text annotations either in data or relative
                              coordinates
            arrow           - draw arrow annotations either in data or relative
                              coordinates
            axhline,axvline - draw horizontal/vertical lines
            axhspan,axvspan - draw horizontal/vertical regions

        xyplotter           - matplotlib/pylab plotting functions

    [Reading files]
        reader              - access rpfits/sdfits files
            open            - attach reader to a file
            close           - detach reader from file
            read            - read in integrations
            summary         - list info about all integrations

    [General]
        commands            - this command
        print               - print details about a variable
        list_scans          - list all scantables created bt the user
        list_files          - list all files readable by asap (default rpf)
        del                 - delete the given variable from memory
        range               - create a list of values, e.g.
                              range(3) = [0,1,2], range(2,5) = [2,3,4]
        help                - print help for one of the listed functions
        execfile            - execute an asap script, e.g. execfile('myscript')
        list_rcparameters   - print out a list of possible values to be
                              put into .asaprc
        rc                  - set rc parameters from within asap
        mask_and,mask_or,
        mask_not            - boolean operations on masks created with
                              scantable.create_mask
\end{verbatim}

\subsection{ASCII output format}

\subsection{.asaprc settings}
\index{.asaprc}
\asaprc{verbose}{{\bf True}/False}{Print verbose output, good to disable in scripts}

\asaprc{insitu}{{\bf True}/False}{Apply operations on the input
scantable or return new one}

\asaprc{useplotter}{{\bf True}/False}{Preload a default plotter}

\asaprc{plotter.gui}{{\bf True}/False}{Do we want a GUI or plot to a
file}

\asaprc{plotter.stacking}{{\bf Pol} Beam IF Scan Time}{Default mode for
colour stacking}

\asaprc{plotter.panelling}{Pol Beam IF {\bf Scan} Time}{Default mode
for panelling}

\asaprc{plotter.ganged}{{\bf True}/False}{Push panels together, to
share axislabels}

\asaprc{plotter.decimate}{True/{\bf False}}{Decimate the number of
points plotted by a factor of nchan/1024}

\asaprc{plotter.histogram}{True/{\bf False}}{Plot spectrum using
histogram rather than lines.}

{\em MALTE TO FIX}

\asaprc{plotter.colours}{}{Set default colours for plotting}

\asaprc{plotter.colours}{}{Set default line styles}

\asaprc{plotter.papersze}{{\bf A4}}{}

% scantable
\asaprc{scantable.save}{{\bf ASAP} SDFITS FITS ASCII MS2}{Default output
format when saving}

\asaprc{scantable.autoaverage}{{\bf True}/False}{Auto averaging on
read}

\asaprc{scantable.freqframe}{{\bf LSRK} TOPO BARY etc}{default
frequency frame to set when function scantable.set\_freqframe is
called or the data is imported}

\asaprc{scantable.verbosesummary}{True/{\bf False}}{Control the level
of information printed by summary}

\asaprc{scantable.storage}{{\bf memory}/disk}{Storage of scantables in
memory of via based disk tables}

\subsection{Installation}

\index{Installation}ASAP depends on a number of third-party libraries which you must
have installed before attempting to build ASAP. These are:

\begin{itemize}
\item AIPS++
\item Boost
\item Matplotlib
\item python/ipython
\end{itemize}

Debian Linux is currently supported and we intend also
to support other popular Linux flavours, Solaris and Mac.

Of the dependencies, AIPS++ is the most complex to install.

\printindex

\end{document}