source: branches/Release12/doc/cookbook.tex@ 3037

Last change on this file since 3037 was 790, checked in by phi196, 19 years ago

Added index, Mopra example, updated function sumary

  • Property svn:eol-style set to native
  • Property svn:keywords set to Author Date Id Revision
File size: 60.0 KB
RevLine 
[534]1\documentclass[11pt]{article}
2\usepackage{a4}
[763]3\usepackage{calc}
[534]4\usepackage[dvips]{graphicx}
[790]5\usepackage{makeidx}
[534]6
7% Adjust the page size
8\addtolength{\oddsidemargin}{-0.4in}
9\addtolength{\evensidemargin}{+0.4in}
10\addtolength{\textwidth}{+0.8in}
11
12\setlength{\parindent}{0mm}
13\setlength{\parskip}{1ex}
14
[763]15\title{ATNF Spectral Analysis Package\\User Guide }
[534]16\author{Chris Phillips}
17
18\newcommand{\cmd}[1]{{\tt #1}}
19
[763]20\newcommand{\asaprc}[3]{
21 \begin{minipage}[t]{45mm}#1\end{minipage}
22 \begin{minipage}[t]{30mm}\raggedright #2\end{minipage}\hspace{3mm}
23 \begin{minipage}[t]{\textwidth-75mm}#3\end{minipage}
24}
25
[790]26\makeindex
27
28
[534]29\begin{document}
30
31\maketitle
32
33\section{Introduction}
34
[544]35ASAP is a single dish spectral line processing package currently being
36developed by the ATNF. It is intended to process data from all ATNF
[763]37antennas, and can probably be used for other antennas if they can
[544]38produce ``Single Dish FITS'' format. It is based on the AIPS++
39package.
40
[534]41%\section{Documentation Standards}
42
[738]43%In most of the examples in this document, it has been assumed that the
[534]44
[544]45\section{Installation and Running}
[534]46
[738]47Currently there are installations running on Linux machines at
[534]48
49\begin{itemize}
50\item Epping - use hosts {\tt draco} or {\tt hydra}
51\item Narrabri - use host {\tt kaputar}
[537]52\item Parkes - use host {\tt bourbon}
[534]53\item Mopra - use host {\tt minos}
54\end{itemize}
55
[790]56\index{Running}To start asap log onto one of these Linux hosts and enter
[534]57
58\begin{verbatim}
59 > cd /my/data/directory
60 > asap
61\end{verbatim}
62
[544]63This starts the ASAP. To quit, you need to type \verb+^+-d
64(control-d).
[534]65
66\section{Interface}
67
[790]68\index{Interface}ASAP is written in C++ and python. The user interface uses the
[534]69``ipython'' interactive shell, which is a simple interactive interface
70to python. The user does not need to understand python to use this,
71but certain aspects python affect what the user can do. The current
72interface is object oriented. In the future, we will build a
73functional (non object oriented) shell on top of this to ease
74interactive use.
75
76\subsection {Integer Indices are 0-relative}
77
78Please note, all integer indices in ASAP and iPython are {\bf 0-relative}.
79
80\subsection{Objects}
[790]81\index{objects}
[534]82The ASAP interface is based around a number of ``objects'' which the
83user deals with. Objects range from the data which have been read from
84disk, to tools used for fitting functions to the data. The following
85main objects are used :
86
87\begin{itemize}
[544]88 \item[\cmd{scantable}] The data container (actual spectra and header
89 information)
90 \item[\cmd{fitter}] A tool used to fit functions to the spectral data
91 \item[\cmd{plotter}] A tool used to plot the spectral line data
92 \item[\cmd{reader}] A tool which can be used to read data from disks
[534]93 into a scantable object.
94\end{itemize}
95
[738]96There can be many objects of the same type. Each object is referred to
[534]97by a variable name made by the user. The name of this variable is not
[763]98important and can be set to whatever the user prefers (i.e. ``s'' and
[534]99``ParkesHOH-20052002'' are equivalent). However, having a simple and
100consistent naming convention will help you a lot.
101
[544]102\subsection{Member Functions (functions)}
[534]103
[790]104\index{Functions!member}Following the object oriented approach,
105objects have associated ``member functions'' which can either be used
106to modify the data in some way or change global properties of the
107object. In this document member functions will be referred to simply
108as functions. From the command line, the user can execute these
109functions using the syntax:
[534]110\begin{verbatim}
111 ASAP> out = object.function(arguments)
112\end{verbatim}
113
114Where \cmd{out} is the name of the returned variable (could be a new
[544]115scantable object, or a vector of data, or a status return),
116\cmd{object} is the object variable name (set by the user),
117\cmd{function} is the name of the member function and \cmd{arguments}
118is a list of arguments to the function. The arguments can be provided
119either though position or \cmd{name=}. A mix of the two can be used.
120E.g.
[534]121
122\begin{verbatim}
123 ASAP> av = scans(msk,weight='tsys')
124 ASAP> av = scans(mask=msk,weight='tsys')
[763]125 ASAP> av = scans(msk,tsys)
[544]126 ASAP> scans.polybaseline(mask=msk, order=0, insitu=True)
[534]127 ASAP> scans.polybaseline(msk,0,True)
128 ASAP> scans.polybaseline(mask, insitu=True)
129\end{verbatim}
130
131\subsection{Global Functions}
132
[790]133\index{Functions!global}It does not make sense to implement some functions as member
[544]134functions, typically functions which operate on more than one
135scantable (e.g. time averaging of many scans). These functions will
136always be referred to as global functions.
[534]137
[538]138\subsection{Interactive environment}
[534]139
[790]140\index{ipython!environment}ipython has a number of useful interactive
141features and a few things to be aware of for the new user.
[534]142
143\subsubsection{String completion}
144
[790]145\index{ipython!string completion}Tab completion is enabled for all
146function names. If you type the first few letters of a function name,
147then type {\tt <TAB>} the function name will be auto completed if it
148is un-ambiguous, or a list of possibilities will be
149given. Auto-completion works for the user object names as well as
150function names. It does not work for filenames, nor for function
151arguments.
[534]152
153Example
154\begin{verbatim}
155 ASAP> scans = scantable('MyData.rpf')
156 ASAP> scans.se<TAB>
157scans.set_cursor scans.set_freqframe scans.set_unit scans.setpol
[738]158scans.set_doppler scans.set_instrument scans.setbeam
159scans.set_fluxunit scans.set_restfreqs scans.setif
[534]160 ASAP> scans.set_in<TAB>
[774]161 ASAP> scans.set_instrument()
[534]162\end{verbatim}
163
[544]164\subsubsection{Leading Spaces}
165
[790]166\index{ipython!leading space}Python uses leading space to mark blocks
167of code. This means that it you start a command line with a space, the
168command generally will fail with an syntax error.
[544]169
[763]170\subsubsection{Variable Names}
171
[790]172\index{ipython!variable names}During normal data processing, the user
173will have to create named variables to hold spectra etc. These must
174conform to the normal python syntax, specifically they cannot contain
175``special'' characters such as \@ \$ etc and cannot start with a
176number (but can contain numbers). Variable (and function) names are
177case sensitive.
[763]178
[534]179\subsubsection{Unix Interaction}
180
[790]181\index{ipython!unix interaction}Basic unix shell commands (\cmd{pwd},
182\cmd{ls}, \cmd{cd} etc) can be issued from within ASAP. This allows
183the user to do things like look at files in the current directory. The
184shell command ``\cmd{cd}'' works within ASAP, allowing the user to
185change between data directories. Unix programs cannot be run this way,
186but the shell escape ``$!$'' can be used to run arbitrary
187programs. E.g.
[534]188
189\begin{verbatim}
190 ASAP> pwd
191 ASAP> ls
192 ASAP> ! mozilla&
193\end{verbatim}
194
195\subsection{Help}
196
[790]197\index{Help}ASAP has built in help for all functions. To get a list of
198functions type:
[534]199
[538]200\begin{verbatim}
[774]201 ASAP> commands()
[538]202\end{verbatim}
203
204To get help on specific functions, the built in help needs to be given
205the object and function name. E.g.
206
207\begin{verbatim}
208 ASAP> help scantable.get_scan
209 ASAP> help scantable.stats
210 ASAP> help plotter.plot
211 ASAP> help fitter.plot
212
213 ASAP> scans = scantable('mydata.asap')
214 ASAP> help scans.get_scan # Same as above
[544]215\end{verbatim}
[538]216
[544]217Global functions just need their name
[538]218
[544]219\begin{verbatim}
[738]220 ASAP> help average_time
[538]221\end{verbatim}
222
223Note that if you just type \cmd{help} the internal ipython help is
224invoked, which is probably {\em not} what you want. Type \verb+^+-d
225(control-d) to escape from this.
226
[738]227\subsection{Customisation - .asaprc}
[534]228
[790]229\index{.asaprc}ASAP use an \cmd{.asaprc} file to control the user's
230preference of default values for various functions arguments. This
231includes the defaults for arguments such as \cmd{insitu}, scantable
232\cmd{freqframe} and the plotters \cmd{set\_mode} values. The help on
233individual functions says which arguments can be set default values
234from the \cmd{.asaprc} file. To get a sample contents for the
235\cmd{.asaprc} file use then command \cmd{list\_rcparameters}.
[534]236
[538]237Common values include:
238\begin{verbatim}
239 # apply operations on the input scantable or return new one
240 insitu : False
241
[738]242 # default output format when saving scantable
[538]243 scantable.save : 'ASAP'
244
245 # default frequency frame to set when function
246 # scantable.set_freqframe is called
247 scantable.freqframe : 'LSRK'
248
249 # auto averaging on read
250 scantable.autoaverage : True
251\end{verbatim}
252
[763]253For a complete list of \cmd{.asaprc} values, see the appendix.
254
[534]255\section{Scantables}
[790]256\index{Scantables}
[534]257\subsection {Description}
258
259\subsubsection {Basic Structure}
260
[790]261\index{Scantable!structure}ASAP data handling works on objects called
262scantables. A scantable holds your data, and also provides functions
263to operate upon it.
[534]264
265The building block of a scantable is an integration, which is a single
266row of a scantable. Each row contains spectra for each beam, IF and
267polarisation. For example Parkes multibeam data would contain many
268beams, one IF and 2-4 polarisations, while the new Mopra 8-GHz
269filterbank will eventually produce one beam, many IFs, and 2-4
270polarisations.
271
272A collection of sequential integrations (rows) for one source is termed
273a scan (and each scan has a unique numeric identifier, the ScanID). A
274scantable is then a collection of one or more scans. If you have
275scan-averaged your data in time, then each scan would hold just one
276(averaged) integration.
277
278Many of the functions which work on scantables can either return a
279new scantable with modified data or change the scantable insitu. Which
280method is used depends on the users preference. The default can be
281changed via the {\tt .asaprc} resource file.
282
283\subsubsection {Contents}
284
[790]285\index{Scantable!contents}A scantable has header information and data
286(a scantable is actually an AIPS++ Table and it is stored in Memory
287when you are manipulating it with ASAP. You can store it to disk and
288then browse it with the AIPS++ Table browser if you know how to do
289that !).
[534]290
291The data are stored in columns (the length of a column is the number of
[738]292rows/integrations of course).
[534]293
294Two important columns are those that describe the frequency setup. We mention
[538]295them explicitly here because you need to be able to understand the presentation
[534]296of the frequency information and possibly how to manipulate it.
297
298These columns are called FreqID and RestFreqID. They contain indices, for
299each IF, pointing into tables with all of the frequency information for that
300integration. More on these below when we discuss the \cmd{summary} function
301in the next subsection.
302
303There are of course many other columns which contain the actual spectra,
304the flags, the Tsys, the source names and so on, but those are a little
305more transparently handled.
306
307\subsection{Management}
308
[790]309\index{Scantable!management}During processing it is possible to create
310a large number of scan tables. These all consume memory, so it is best
311to periodically remove unneeded scan tables. Use \cmd{list\_scans} to
312print a list of all scantables and \cmd{del} to remove unneeded ones.
[534]313
314Example:
315
316\begin{verbatim}
[774]317 ASAP> list_scans()
[534]318 The user created scantables are:
319 ['s', 'scans', 'av', 's2', 'ss']
320
[738]321 ASAP> del s2
[534]322 ASAP> del ss
323\end{verbatim}
324
325There is also a function \cmd{summary} to list a summary of the scantable.
326You will find this very useful.
327
328Example:
329
330\begin{verbatim}
331 ASAP> scans = scantable('MyData.rpf')
332 ASAP> scans.summary() # Brief listing
333 ASAP> scans.summary(verbose=True) # Include frequency information
[544]334
335 # Equivalent to brief summary function call
[738]336 ASAP> print scan
[534]337\end{verbatim}
338
339Most of what the \cmd{summary} function prints out is obvious. However,
[738]340it also prints out the FreqIDs and RestFreqIDs to which we alluded above.
[534]341These are the last column of the listing.
342
343The summary function gives you a scan-based summary. So it lists all of
344the FreqIDs and RestFreqIDs that it encountered for each scan. If you'd
345like to see what each FreqID actually means, then set the verbose
[738]346argument to True and the frequency table will be listed at the end.
[534]347FreqID of 3 say, refers to the fourth row of the frequency table (ASAP
348is 0-relative). The list of rest frequencies, to which the RestFreqIDs
349refer, is always listed.
350
[544]351%You can copy one scantable to another with the \cmd{copy} function.
[534]352
[544]353%Example:
[534]354
[544]355%\begin{verbatim}
356% ASAP> scans = scantable('MyData.rpf')
357% ASAP> scan2 = scans.copy()
358%\end{verbatim}
[534]359
360\subsection{State}
361
[790]362\index{Scantable!state}Each scantable contains "state"; these are
363properties applying to all of the data in the scantable.
[534]364
365Examples are the selection of beam, IF and polarisation, spectral unit
[767]366(e.g. km/s), frequency reference frame (e.g. BARY) and velocity Doppler
[534]367type (e.g. RADIO).
368
369\subsubsection{Units, Doppler and Frequency Reference Frame}
370
371The information describing the frequency setup for each integration
372is stored fundamentally in frequency in the reference frame
[738]373of observation (E.g. TOPO).
[534]374
[544]375When required, this is converted to the desired reference frame
376(e.g. LSRK), Doppler (e.g. OPTICAL) and unit (e.g. km/s) on-the-fly.
377This is important, for example, when you are displaying the data or
378fitting to it.
[534]379
380For units, the user has the choice of frequency, velocity or channel.
381The \cmd{set\_unit} function is used to set the current unit for a
382scantable. All functions will (where relevant) work with the selected
383unit until this changes. This is mainly important for fitting (the fits
[738]384can be computed in any of these units), plotting and mask creation.
[534]385
[544]386The velocity definition can be changed with the \cmd{set\_doppler}
387function, and the frequency reference frame can be changed with the
[534]388\cmd{set\_freqframe} function.
389
390Example usage:
391
392\begin{verbatim}
393 ASAP> scans = scantable('2004-11-23_1841-P484.rpf') # Read in the data
394 ASAP> scans.set_freqframe('LSRK') # Use the LSR velocity frame
395 ASAP> scans.set_unit('km/s') # Use velocity for plots etc from now on
396 ASAP> scans.set_doppler('OPTICAL') # Use the optical velocity convention
397 ASAP> scans.set_unit('MHz') # Use frequency in MHz from now on
398\end{verbatim}
399
400
401\subsubsection{Rest Frequency}
402
[790]403\index{Scantable!rest frequency}ASAP reads the line rest frequency
404from the RPFITS file when reading the data. The values stored in the
405RPFITS file are not always correct and so there is a function
406\cmd{set\_restfreq} to set the rest frequencies.
[534]407
408For each integration, there is a rest-frequency per IF (the rest
409frequencies are just stored as a list with an index into them).
410There are a few ways to set the rest frequencies with this function.
411
412If you specify just one rest frequency, then it is selected for the
413specified source and IF and added to the list of rest frequencies.
414
415\begin{verbatim}
[538]416 # Select for specified source/IF
[738]417 ASAP> scans.set_restfreqs(freqs=1.667359e9, source='NGC253', theif=0)
[538]418
419 # Select for all sources and IFs
[738]420 ASAP> scans.set_restfreqs(freqs=1.667359e9)
[534]421\end{verbatim}
422
423
424If you specify a list of frequencies, then it must be of length the
425number of IFs. Regardless of the source, the rest frequency will be set
426for each IF to the corresponding value in the provided list. The
427internally stored list of rest frequencies will be replaced by this
428list.
429
430
431\begin{verbatim}
[763]432 # Set rest frequency for all IFs
433 ASAP> scans.set_restfreqs(freqs=[1.6654018e9,1.667359e9,])
[538]434
[534]435\end{verbatim}
436
437In both of the above modes, you can also specify the rest frequencies via
438names in a known list rather than by their values.
439
440Examples:
441
442\begin{verbatim}
443 ASAP> scans.lines() # Print list of known lines
444 ASAP> scans.set_restfreqs(lines=['OH1665','OH1667'])
445\end{verbatim}
446
447
448\subsection{Data Selection}
449
[790]450\index{Scantable!data selection}Data selection is currently fairly limited. This
451will be improved in the future.
[534]452
453
454\subsubsection{Cursor}
455
[790]456\index{Scantable!cursor}Generally the user will want to run functions
457on all rows in a scantable. This allows very fast reduction of
458data. There are situations when functions should only operate on
459specific elements of the spectra. This is handled by the scantable
460cursor, which allows the user to select a single beam, IF and
461polarisation combination.
[534]462
463Example :
464
465\begin{verbatim}
466 ASAP> scans.set_cursor(0,2,1) # beam, IF, pol
467 ASAP> scans.smooth(allaxes=F) # in situ by default or .aipsrc
468\end{verbatim}
469
470\subsubsection{Row number}
471
[790]472\index{Scantable!row selection}Most functions work on all rows of a
473scan table. Exceptions are the fitter and plotter. If you wish to only
474operate on a selected set of scantable rows, use the \cmd{get\_scan}
475function to copy the rows into a new scantable.
[534]476
477\subsubsection{Allaxes}
478
[790]479\index{allaxes}\index{Scantable!allaxes}Many functions have an
480\cmd{allaxes} option which controls whether the function will operate
481on all elements within a scantable row, or just those selected with
482the current cursor. The default is taken from the users {\tt .asaprc}
483file.
[534]484
485\subsubsection{Masks}
486
[790]487\index{Masks}\index{Scantable!masks}Many tasks (fitting, baseline
488subtraction, statistics etc) should only be run on range of
489channels. Depending on the current ``unit'' setting this range is set
490directly as channels, velocity or frequency ranges. Internally these
491are converted into a simple boolean mask for each channel of the
492abscissa. This means that if the unit setting is later changed,
493previously created mask are still valid. (This is not true for
494functions which change the shape or shift the frequency axis). You
495create masks with the function \cmd{create\_mask} and this specified
[534]496the channels to be included in the selection.
497
498When setting the mask in velocity, the conversion from velocity
499to channels is based on the current cursor setting, selected row and
[544]500selected frequency reference frame.
[534]501
502Example :
503\begin{verbatim}
504
505 # Select channel range for baselining
506 ASAP> scans.set_unit('channels')
[544]507 ASAP> msk = scans.create_mask([100,400],[600,800])
[738]508
[534]509 # Select velocity range for fitting
510 ASAP> scans.set_unit('km/s')
[544]511 ASAP> msk = scans.create_mask([-30,-10])
[534]512\end{verbatim}
513
[544]514Sometimes it is more convenient to specify the channels to be
515excluded, rather included. You can do this with the ``invert''
516argument.
[534]517
518Example :
519\begin{verbatim}
520 ASAP> scans.set_unit('channels')
[738]521 ASAP> msk = scans.create_mask([0,100],[900-1023], invert=True)
[534]522\end{verbatim}
523
[544]524By default \cmd{create\_mask} uses the frequency setup of the first row
525to convert velocities into a channel mask. If the rows in the data
526cover different velocity ranges, the scantable row to use should be
527specified:
528
529\begin{verbatim}
530 ASAP> scans.set_unit('km/s')
531 ASAP> msk = q.create_mask([-30,-10], row=5)
532\end{verbatim}
533
[534]534Because the mask is stored in a simple python variable, the users is
535able to combine masks using simple arithmetic. To create a mask
536excluding the edge channels, a strong maser feature and a birdie in
537the middle of the band:
538
539\begin{verbatim}
540 ASAP> scans.set_unit('channels')
541 ASAP> msk1 = q.create_mask([0,100],[511,511],[900,1023],invert=True)
542 ASAP> scans.set_unit('km/s')
543 ASAP> msk2 = q.create_mask([-20,-10],invert=True)
544
545 ASAP> mask = msk1 and msk2
546\end{verbatim}
547
548
549\section{Data Input}
550
[790]551\index{Reading data}Data can be loaded in one of two ways; using the reader object or via
[534]552the scantable constructor. The scantable method is simpler but the
553reader allow the user more control on what is read.
554
555\subsection{Scantable constructor}
556
[790]557\index{Scantable constructor}\index{Scantable!constructor}This loads
558all of the data from filename into the scantable object scans and
559averages all the data within a scan (i.e. the resulting scantable
[534]560will have one row per scan). The recognised input file formats are
561RPFITS, SDFITS (singledish fits), ASAP's scantable format and aips++
[738]562MeasurementSet2 format.
[534]563
564Example usage:
565
566\begin{verbatim}
567 ASAP> scan = scantable('2004-11-23_1841-P484.rpf')
[544]568
569 # Don't scan average the data
570 ASAP> scan = scantable('2004-11-23_1841-P484.rpf', average=False)
[534]571\end{verbatim}
572
573
574\subsection{Reader object}
575
[790]576\index{Reader object}\index{Scantable!reader object}For more control
577when reading data into ASAP, the reader object should be used. This
578has the option of only reading in a range of integrations and does not
579perform any scan averaging of the data, allowing analysis of the
580individual integrations. Note that due to limitation of the RPFITS
581library, only one reader object can be open at one time reading RPFITS
582files. To read multiple RPFITS files, the old reader must be
583destroyed before the new file is opened. However, multiple readers
584can be created and attached to SDFITS files.
[534]585
586
587Example usage:
588
589\begin{verbatim}
590 ASAP> r = reader('2003-03-16_082048_t0002.rpf')
[774]591 ASAP> r.summary()
[534]592 ASAP> scan = r.read()
593 ASAP> s = r.read(range(100)) # To read in the first 100 integrations
594 ASAP> del r
595\end{verbatim}
596
597\section{Basic Processing}
598
599In the following section, a simple data reduction to form a quotient
[544]600spectrum of a single source is followed. It has been assume that the
601\cmd{.asaprc} file has {\em not} been used to change the \cmd{insitu}
602default value from \cmd{True}.
[534]603
604%\subsection{Editing}
605
606%How and when?
[738]607\subsection{Auto quotient}
[790]608\index{Auto quotient}Quotients can be computed ``automatically''. This
609requires the data to have matching source/reference pairs or one
610reference for multiple sources. Auto quotient assumes reference scans
611have a trailing ``\_R'' in the source name for data from Parkes and
612Mopra, and a trailing ``e'' or ``w'' for data fro, Tidbinbilla.
[534]613
[738]614\begin{verbatim}
615 ASAP> q = s.auto_quotient()
616\end{verbatim}
617
618If this is not sufficient the following alternative method can be used.
619
[534]620\subsection{Separate reference and source observations}
621
[790]622\index{Quotient spectra}Most data from ATNF observatories
623distinguishes on and off source data using the file name. This makes
624it easy to create two scantables with the source and reference
625data. As long as there was exactly one reference observation for each
626on source observation for following method will work.
[534]627
628For Mopra and Parkes data:
629\begin{verbatim}
630 ASAP> r = scans.get_scan('*_R')
631 ASAP> s = scans.get_scan('*_S')
632\end{verbatim}
633
634For Tidbinbilla data
635\begin{verbatim}
636 ASAP> r = scans.get_scan('*_[ew]')
637 ASAP> s = scans.get_scan('*_[^ew]')
638\end{verbatim}
639
640\subsection{Make the quotient spectra}
641
642Use the quotient function
643
644\begin{verbatim}
645 ASAP> q = s.quotient(r)
646\end{verbatim}
647
648This uses the rows in scantable \cmd{r} as reference spectra for the
649rows in scantable \cmd{s}. Scantable \cmd{r} must have either 1 row
650(which is applied to all rows in \cmd{s}) or both scantables must have
651the same number of rows. By default the quotient spectra is calculated
[544]652to preserve continuum emission. If you wish to remove the continuum
[534]653contribution, use the \cmd{preserve} argument:
654
655\begin{verbatim}
656 ASAP> q = s.quotient(r, preserve=True)
657\end{verbatim}
658
659\subsection{Time average separate scans}
660
[790]661\index{Time average}If you have observed the source with multiple
662source/reference cycles you will want to scan-average the quotient
663spectra together.
[534]664
665\begin{verbatim}
666 ASAP> av = average_time(q)
667\end{verbatim}
668
[544]669If for some you want to average multiple sets of scantables together
670you can:
[534]671
672\begin{verbatim}
673 ASAP> av = average_time(q1, q2, q3)
674\end{verbatim}
675
[544]676The default is to use integration time weighting. The alternative is
[767]677to use none, variance, Tsys weighting or Tsys \& integration time.
[534]678
[544]679\begin{verbatim}
680 ASAP> av = average_time(q, weight='tintsys')
681\end{verbatim}
682
[534]683To use variance based weighting, you need to supply a mask saying which
684channel range you want it to calculate the variance from.
685
686\begin{verbatim}
[544]687 ASAP> msk = scans.create_mask([200,400],[600,800])
688 ASAP> av = average_time(scans, mask=msk, weight='var')
[534]689\end{verbatim}
690
[790]691Note that, if needed, you should run \cmd{freq\_align}, \cmd{gain\_el}
692and \cmd{opacity} before you average the data in time (\S
693\ref{sec:gainel} \& \ref{sec:freqalign}).
694
[534]695\subsection{Baseline fitting}
696
[790]697\index{Baseline fitting}To make a baseline fit, you must first create
698a mask of channels to use in the baseline fit.
[534]699
700\begin{verbatim}
701 ASAP> msk = scans.create_mask([100,400],[600,900])
[738]702 ASAP> scans.poly_baseline(msk, 1)
[534]703\end{verbatim}
704
705This will fit a first order polynomial to the selected channels and subtract
706this polynomial from the full spectra.
707
708\subsubsection{Auto-baselining}
709
[790]710\index{Auto-baseline}The function \cmd{auto\_poly\_baseline} can be used to automatically
[767]711baseline your data without having to specify channel ranges for the
712line free data. It automatically figures out the line-free emission
713and fits a polynomial baseline to that data. The user can use masks to
714fix the range of channels or velocity range for the fit as well as
715mark the band edge as invalid.
[534]716
717Simple example
718
719\begin{verbatim}
720 ASAP> scans.auto_poly_baseline(order=2,threshold=5)
721\end{verbatim}
722
723\cmd{order} is the polynomial order for the fit. \cmd{threshold} is
724the SNR threshold to use to deliminate line emission from
[548]725signal. Generally the value of threshold is not too critical, however
726making this too large will compromise the fit (as it will include
727strong line features) and making it too small will mean it cannot find
728enough line free channels.
[534]729
[548]730
[534]731Other examples:
732
733\begin{verbatim}
734 # Don't try and fit the edge of the bandpass which is noisier
735 ASAP> scans.auto_poly_baseline(edge=(500,450),order=3,threshold=3)
736
737 # Only fit a given region around the line
738 ASAP> scans.set_unit('km/s')
739 ASAP> msk = scans.create_mask((-60,-20))
740 ASAP> scans.auto_poly_baseline(mask=msk,order=3,threshold=3)
741
742\end{verbatim}
743
744\subsection{Average the polarisations}
745
[790]746\index{average\_pol}If you are just interested in the highest SNR for total intensity you
[534]747will want to average the parallel polarisations together.
748
749\begin{verbatim}
750 ASAP> scans.average_pol()
751\end{verbatim}
752
753\subsection{Calibration}
754
[790]755\index{Calibration}For most uses, calibration happens transparently as the input data
[534]756contains the Tsys measurements taken during observations. The nominal
757``Tsys'' values may be in Kelvin or Jansky. The user may wish to
758supply a Tsys correction or apply gain-elevation and opacity
759corrections.
760
761\subsubsection{Brightness Units}
762
[790]763\index{Brightness Units}RPFITS files do not contain any information as
764to whether the telescope calibration was in units of Kelvin or
765Janskys. On reading the data a default value is set depending on the
766telescope and frequency of observation. If this default is incorrect
767(you can see it in the listing from the \cmd{summary} function) the
768user can either override this value on reading the data or later.
769E.g:
[534]770
771\begin{verbatim}
772 ASAP> scans = scantable(('2004-11-23_1841-P484.rpf', unit='Jy')
773 # Or in two steps
774 ASAP> scans = scantable(('2004-11-23_1841-P484.rpf')
775 ASAP> scans.set_fluxunit('Jy)
776\end{verbatim}
777
778\subsubsection{Tsys scaling}
779
[790]780\index{Tsys scaling}Sometime the nominal Tsys measurement at the
781telescope is wrong due to an incorrect noise diode calibration. This
782can easily be corrected for with the scale function. By default,
783\cmd{scale} only scans the spectra and not the corresponding Tsys.
[534]784
785\begin{verbatim}
[738]786 ASAP> scans.scale(1.05, tsys=True)
[534]787\end{verbatim}
788
789\subsubsection{Unit Conversion}
790
[790]791\index{Unit conversion}To convert measurements in Kelvin to Jy (and
792vice versa) the global function \cmd{convert\_flux} is needed. This
793converts and scales the data from K to Jy or vice-versa depending on
794what the current brightness unit is set to. The function knows the
795basic parameters for some frequencies and telescopes, but the user may
796need to supply the aperture efficiency, telescope diameter or the Jy/K
797factor.
[534]798
799\begin{verbatim}
[774]800 ASAP> scans.convert_flux() # If efficency known
[534]801 ASAP> scans.convert_flux(eta=0.48) # If telescope diameter known
802 ASAP> scans.convert_flux(eta=0.48,d=35) # Unknown telescope
803 ASAP> scans.convert_flux(jypk=15) # Alternative
804\end{verbatim}
805
806\subsubsection{Gain-Elevation and Opacity Corrections}
[790]807\label{sec:gainel}
[534]808
[790]809\index{Gain-elevation}As higher frequencies (particularly $>$20~GHz)
810it is important to make corrections for atmospheric opacity and
811gain-elevation effects.
[534]812
[774]813Note that currently the elevation is not written correctly into
[763]814Tidbinbilla rpfits files. This means that gain-elevation and opacity
[774]815corrections will not work unless these get recalculated.
[763]816
[774]817\begin{verbatim}
818 ASAP> scans.recalc_azel() # recalculate az/el based on pointing
819\end{verbatim}
820
[544]821Gain-elevation curves for some telescopes and frequencies are known to
[790]822ASAP (currently only for Tidbinbilla at 20~GHz). In these cases
823making gain-corrections is simple. If the gain curve for your data is
824not known, the user can supply either a gain polynomial or text file
[534]825tabulating gain factors at a range of elevations (see \cmd{help
[544]826scantable.gain\_el}).
[534]827
828Examples:
829
830\begin{verbatim}
831 ASAP> scans.gain_el() # If gain table known
832 ASAP> scans.gain_el(poly=[3.58788e-1,2.87243e-2,-3.219093e-4])
833\end{verbatim}
834
[790]835\index{Opacity}Opacity corrections can be made with the global
836function \cmd{opacity}. This should work on all telescopes as long as
837a measurement of the opacity factor was made during the observation.
[534]838
839\begin{verbatim}
840 ASAP> scans.opacity(0.083)
841\end{verbatim}
842
843Note that at 3~mm Mopra uses a paddle wheel for Tsys calibration,
844which takes opacity effects into account (to first order). ASAP
[544]845opacity corrections should not be used for Mopra 3-mm data.
[534]846
847\subsection{Frequency Frame Alignment}
[790]848\label{sec:freqalign}
[534]849
[790]850\index{Frequency alignment}\index{Velicity alignment}When time
851averaging a series of scans together, it is possible that the velocity
852scales are not exactly aligned. This may be for many reasons such as
853not Doppler tracking the observations, errors in the Doppler tracking
854etc. This mostly affects very long integrations or integrations
855averaged together from different days. Before averaging such data
856together, they should be frequency aligned using \cmd{freq\_align}.
[534]857
858E.g.:
859
860\begin{verbatim}
861 ASAP> scans.freq_align()
862 ASAP> av = average_time(scans)
863\end{verbatim}
864
865\cmd{freq\_align} has two modes of operations controlled by the
866\cmd{perif} argument. By default it will align each source and freqid
867separately. This is needed for scan tables containing multiple
[763]868sources. However if scan-based Doppler tracking has been made at the
869observatory, each row will have a different freqid. In these cases run
870with \cmd{perif=True} and all rows of a source will be aligned to the
871same frame. In general \cmd{perif=True} will be needed for most
[534]872observations as Doppler tracking of some form is made at Parkes, Tid
873and Mopra.
874
875\begin{verbatim}
876 ASAP> scans.freq_align(perif=True)
877\end{verbatim}
878
879To average together data taken on different days, which are in
880different scantables, each scantable must aligned to a common
881reference time then the scantables averaged. The simplest way of
882doing this is to allow ASAP to choose the reference time for the first
[738]883scantable then using this time for the subsequent scantables.
[534]884
885\begin{verbatim}
886 ASAP> scans1.freq_align() # Copy the refeference Epoch from the output
887 ASAP> scans2.freq_align(reftime='2004/11/23/18:43:35')
888 ASAP> scans3.freq_align(reftime='2004/11/23/18:43:35')
889 ASAP> av = average_time(scans1, scans2, scans3)
890\end{verbatim}
891
892\section{Scantable manipulation}
893
[790]894\index{Scantable!manipulation}While it is very useful to have many
895independent sources within one scantable, it is often inconvenient for
896data processing. The \cmd{get\_scan} function can be used to create a
897new scantable with a selection of scans from a scantable. The
898selection can either be on the source name, with simple wildcard
899matching or set of scan ids.
[534]900
901For example:
902
903\begin{verbatim}
904 ASAP> ss = scans.get_scan(10) # Get the 11th scan (zero based)
905 ASAP> ss = scans.get_scan(range(10)) # Get the first 10 scans
[538]906 ASAP> ss = scans.get_scan(range(10,20)) # Get the next 10 scans
[534]907 ASAP> ss = scans.get_scan([2,4,6,8,10]) # Get a selection of scans
908
909 ASAP> ss = scans.get_scan('345p407') # Get a specific source
910 ASAP> ss = scans.get_scan('345*') # Get a few sources
911
912 ASAP> r = scans.get_scan('*_R') # Get all reference sources (Parkes/Mopra)
913 ASAP> s = scans.get_scan('*_S') # Get all program sources (Parkes/Mopra)
914 ASAP> r = scans.get_scan('*_[ew]') # Get all reference sources (Tid)
915 ASAP> s = scans.get_scan('*_[^ew]') # Get all program sources (Tid)
916
917\end{verbatim}
918
919To copy a scantable the following does not work:
920
921\begin{verbatim}
922 ASAP> ss = scans
923\end{verbatim}
924
[544]925as this just creates a reference to the original scantable. Any
926changes made to \cmd{ss} are also seen in \cmd{scans}. To duplicate a
[534]927scantable, use the copy function.
928
929\begin{verbatim}
[774]930 ASAP> ss = scans.copy()
[534]931\end{verbatim}
932
933\section{Data Output}
934
[790]935\index{Scantable!save}\index{Saving data}ASAP can save scantables in a
936variety of formats, suitable for reading into other packages. The
937formats are:
[534]938
939\begin{itemize}
940\item[ASAP] This is the internal format used for ASAP. It is the only
[544]941 format that allows the user to restore the data, fits etc. without
942 loosing any information. As mentioned before, the ASAP scantable is
943 an AIPS++ Table (a memory-based table). This function just converts
944 it to a disk-based Table. You can the access that Table with the
945 AIPS++ Table browser or any other AIPS++ tool.
[534]946
[544]947\item[SDFITS] The Single Dish FITS format. This format was designed to
948 for interchange between packages, but few packages actually can read
949 it.
[534]950
951\item[FITS] This uses simple ``image'' fits to save the data, each row
[544]952 being written to a separate fits file. This format is suitable for
953 importing the data into CLASS.
[534]954
955\item[ASCII] A simple text based format suitable for the user to
956processing using Perl or, Python, gnuplot etc.
957
958\item[MS2] Saves the data in an aips++ MeasurementSet V2 format.
959You can also access this with the Table browser and other AIPS++
960tools.
961
962\end{itemize}
963
[738]964The default output format can be set in the users {\tt .asaprc} file.
[534]965Typical usages are:
966
967\begin{verbatim}
968 ASAP> scans.save('myscans') # Save in default format
969 ASAP> scans.save('myscans', 'FITS') # Save as FITS for exporting into CLASS
970
971 ASAP> scans.save('myscans', stokes=True) # Convert raw polarisations into Stokes
972 ASAP> scans.save('myscans', overwrite=True) # Overwrite an existing file
973\end{verbatim}
974
975
976\section{Plotter}
977
[790]978\index{Plotter}Scantable spectra can be plotted at any time. An asapplotter object is
[534]979used for plotting, meaning multiple plot windows can be active at the
980same time. On start up a default asapplotter object is created called
981``plotter''. This would normally be used for standard plotting.
982
[538]983The plotter, optionally, will run in a multipanel mode and contain
[534]984multiple plots per panel. The user must tell the plotter how they want
985the data distributed. This is done using the set\_mode function. The
986default can be set in the users {\tt .asaprc} file. The units (and frame
[538]987etc) of the abscissa will be whatever has previously been set by
988\cmd{set\_unit}, \cmd{set\_freqframe} etc.
[534]989
990Typical plotter usage would be:
991
992\begin{verbatim}
993 ASAP> scans.set_unit('km/s')
994 ASAP> plotter.set_mode(stacking='p',panelling='t')
995 ASAP> plotter.plot(scans)
996\end{verbatim}
997
998This will plot multiple polarisation within each plot panel and each
[544]999scan row in a separate panel.
[534]1000
[538]1001Other possibilities include:
[534]1002
1003\begin{verbatim}
1004 # Plot multiple IFs per panel
1005 ASAP> plotter.set_mode(stacking='i',panelling='t')
[538]1006
1007 # Plot multiple beams per panel
1008 ASAP> plotter.set_mode(stacking='b',panelling='t')
1009
1010 # Plot one IF per panel, time stacked
1011 ASAP> plotter.set_mode('t', 'i')
1012
1013 # Plot each scan in a seperate panel
1014 ASAP> plotter.set_mode('t', 's')
1015
[534]1016\end{verbatim}
1017
[538]1018\subsection{Plot Selection}
1019\label{sec:plotter_cursor}
1020
[790]1021\index{Plotter!selection}The plotter can plot up to 25 panels and
1022stacked spectra per panel. If you have data larger than this (or for
1023your own sanity) you need to select a subset of this data. This is
1024particularly true for multibeam or multi IF data. The plotter
1025\cmd{set\_cursor} function is used to select a subset of the data. The
1026arguments \cmd{row}, \cmd{beam} and \cmd{IF} all accept a vector of
1027indices corresponding to row, beam or IF selection. Only the selected
1028data will be plotted. To select on polarisation, see
1029section~\ref{sec:polplot}.
[538]1030
1031Examples:
1032
1033\begin{verbatim}
1034 # Select second IF
1035 ASAP> plotter.set_cursor(IF=[1])
1036
1037 # Select first 4 beams
1038 ASAP> plotter.set_cursor(beam=[0,1,2,3])
1039
1040 # Select a few rows
1041 ASAP> plotter.set_cursor(row=[2,4,6,10])
1042
1043 # Multiple selection
1044 ASAP> plotter.set_cursor(IF=[1], beam=[0,2], row=range(10))
1045\end{verbatim}
1046
[544]1047Note that the plotter cursor selection is independent of the scantable
1048cursor.
[534]1049
[544]1050\subsection{Plot Control}
1051
[790]1052\index{Plotter!control}The plotter window has a row of buttons on the
1053lower left. These can be used to control the plotter (mostly for
1054zooming the individual plots). From left to right:
[534]1055
1056\begin{itemize}
1057
1058\item[Home] This will unzoom the plots to the original zoom factor
1059
1060\item[Plot history] (left and right arrow). The plotter keeps a
1061history of zoom settings. The left arrow sets the plot zoom to the
1062previous value. The right arrow returns back again. This allows you,
1063for example, to zoom in on one feature then return the plot to how it
1064was previously.
1065
1066\item[Pan] (The Cross) This sets the cursor to pan, or scroll mode
1067 allowing you to shift the plot within the window. Useful when
1068 zoomed in on a feature.
1069
1070\item[Zoom] (the letter with the magnifying glass) lets you draw a
1071 rectangle around a region of interest then zooms in on that
[738]1072 region. Use the plot history to unzoom again.
[534]1073
1074\item[Save] (floppy disk). Save the plot as a postscript or .png file
1075
[790]1076You can also type ``g'' in the plot window to toggle on and off grid
1077lines. Typing 'l' turns on and off logarithmic Y-axis.
1078
[534]1079\end{itemize}
1080
1081\subsection{Other control}
1082
1083The plotter has a number of functions to describe the layout of the
1084plot. These include \cmd{set\_legend}, \cmd{set\_layout} and \cmd{set\_title}.
1085
1086To set the exact velocity or channel range to be plotted use the
1087\cmd{set\_range} function. To reset to the default value, call
1088\cmd{set\_range} with no arguments. E.g.
1089
1090\begin{verbatim}
1091 ASAP> scans.set_unit('km/s')
1092 ASAP> plotter.plot(scans)
1093 ASAP> plotter.set_range(-150,-50)
[544]1094 ASAP> plotter.set_range() # To reset
[534]1095\end{verbatim}
1096
[544]1097Both the range of the ``x'' and ``y'' axis can be set at once, if desired:
1098
1099\begin{verbatim}
1100 ASAP> plotter.set_range(-10,30,-1,6.6)
1101\end{verbatim}
1102
[738]1103To save a hardcopy of the current plot, use the save function, e.g.
[534]1104
1105\begin{verbatim}
1106 ASAP> plotter.save('myplot.ps')
1107\end{verbatim}
1108
1109\section{Fitting}
1110
[790]1111\index{Fitting}Currently multicomponent Gaussian function is
1112available. This is done by creating a fitting object, setting up the
1113fit and actually fitting the data. Fitting can either be done on a
1114single scantable row/cursor selection or on an entire scantable using
1115the \cmd{auto\_fit} function.
[534]1116
1117\begin{verbatim}
1118 ASAP> f = fitter()
1119 ASAP> f.set_function(gauss=2) # Fit two Gaussians
1120 ASAP> f.set_scan(scans)
1121 ASAP> scans.set_cursor(0,0,1) # Fit the second polarisation
1122 ASAP> scans.set_unit('km/s') # Make fit in velocity units
1123 ASAP> f.fit(1) # Run the fit on the second row in the table
1124 ASAP> f.plot() # Show fit in a plot window
1125 ASAP> f.get_parameters() # Return the fit paramaters
1126\end{verbatim}
1127
1128This auto-guesses the initial values of the fit and works well for data
1129without extra confusing features. Note that the fit is performed in
1130whatever unit the abscissa is set to.
1131
1132If you want to confine the fitting to a smaller range (e.g. to avoid
1133band edge effects or RFI you must set a mask.
1134
1135\begin{verbatim}
1136 ASAP> f = fitter()
1137 ASAP> f.set_function(gauss=2)
1138 ASAP> scans.set_unit('km/s') # Set the mask in channel units
1139 ASAP> msk = s.create_mask([1800,2200])
1140 ASAP> scans.set_unit('km/s') # Make fit in velocity units
1141 ASAP> f.set_scan(s,msk)
1142 ASAP> f.fit()
1143 ASAP> f.plot()
1144 ASAP> f.get_parameters()
1145\end{verbatim}
1146
[544]1147If you wish, the initial parameter guesses can be specified and
1148specific parameters can be fixed:
[534]1149
1150\begin{verbatim}
1151 ASAP> f = fitter()
1152 ASAP> f.set_function(gauss=2)
1153 ASAP> f.set_scan(s,msk)
1154 ASAP> f.fit() # Fit using auto-estimates
[738]1155 # Set Peak, centre and fwhm for the second gaussian.
[534]1156 # Force the centre to be fixed
1157 ASAP> f.set_gauss_parameters(0.4,450,150,0,1,0,component=1)
1158 ASAP> f.fit() # Re-run the fit
1159\end{verbatim}
1160
1161The fitter \cmd{plot} function has a number of options to either view
1162the fit residuals or the individual components (by default it plots
1163the sum of the model components).
1164
1165Examples:
1166
1167\begin{verbatim}
1168 # Plot the residual
[738]1169 ASAP> f.plot(residual=True)
[534]1170
1171 # Plot the first 2 componentsa
[738]1172 ASAP> f.plot(components=[0,1])
[534]1173
1174 # Plot the first and third component plus the model sum
1175 ASAP> f.plot(components=[-1,0,2]) # -1 means the compoment sum
1176\end{verbatim}
1177
[544]1178\subsection{Fit saving}
1179
[790]1180\index{Fitter!Fit saving}One you are happy with your fit, it is
1181possible to store it as part of the scantable.
[544]1182
1183\begin{verbatim}
1184 ASAP> f.storefit()
1185\end{verbatim}
1186
1187This will be saved to disk with the data, if the ``ASAP'' file format
1188is selected. Multiple fits to the same data can be stored in the
[738]1189scantable.
[544]1190
1191The scantable function \cmd{get\_fit} can be used to retrieve the
1192stored fits. Currently the fit parameters are just printed to the
1193screen.
1194
1195\begin{verbatim}
1196 ASAP> scans.get_fit(4) # Print fits for row 4
1197\end{verbatim}
1198
[534]1199\section{Polarisation}
1200
[790]1201\index{Polarisation}Currently ASAP only supports polarmetric analysis
1202on linearly polarised feeds and the cross polarisation products
1203measured. Other cases will be added on an as needed basic.
[534]1204
[538]1205Conversions of linears to Stokes or Circular polarisations are done
[544]1206``on-the-fly''. Leakage cannot be corrected for nor are these routines
[738]1207able to calibrate position angle offsets.
[534]1208
[538]1209\subsection{Simple Calibration}
1210
[763]1211{\em Currently the receiver position angle is not read from the RPFITS
1212file and a position angle of zero is assumed. This severely hampers
1213correct handling of polarimetry. In the future we aim to define a
1214general framework and populate the RPFITS files with the data required
1215for transparent polarimetric calibration.}
[538]1216
[790]1217\index{Polarisation!calibration}It is possible that there is a phase
1218offset between polarisation which will effect the phase of the cross
1219polarisation correlation, and so give rise to spurious
1220polarisation. \cmd{rotate\_xyphase} can be used to correct for this
1221error. At this point, the user must know how to determine the size of
1222the phase offset themselves.
[538]1223
1224\begin{verbatim}
[546]1225 ASAP> scans.rotate_xyphase(10.5) # Degrees
[538]1226\end{verbatim}
1227
1228Note that if this function is run twice, the sum of the two values is
[546]1229applied because it is done in-situ.
[538]1230
[546]1231A correction for the receiver parallactic angle may need to be made,
1232either because of how it is mounted or if parallactifiying had to track
[538]1233at 90 degrees rather than 0. Use \cmd{rotate\_linpolphase} to correct
1234the position angle. Running this function twice results in the sum of
[546]1235the corrections being applied because it is applied in-situ.
[538]1236
1237\begin{verbatim}
[546]1238 ASAP> scans.rotate_linpolphase(-20) # Degrees; correct for receiver mounting
[538]1239
1240 # Receiver was tracking 90 degrees rather than 0
[738]1241 ASAP> scans.rotate_linpolphase(90)
[538]1242\end{verbatim}
1243
1244\subsection{Plotting}
1245\label{sec:polplot}
1246
[790]1247\index{Polarisation!plotting}To plot Stokes values, the plotter
1248\cmd{set\_cursor} function should be called first using the \cmd{pol}
1249argument. The values which can be plotted include a selection of
1250[I,Q,U,V], [I, Plinear, Pangle, V], [RR, LL] or [XX, YY, Real(XY),
1251Imaginary(XY)]. (Plinear and Pangle are the percentage and position
1252angle of linear polarisation). Conversion to circular polarisations
1253are currently not available.
[538]1254
1255Example:
1256
1257\begin{verbatim}
[763]1258 ASAP> plotter.set_cursor(pol=``I Q'')
1259 ASAP> plotter.set_cursor(pol=``RR LL'')
1260 ASAP> plotter.set_cursor(pol=``XX YY'')
1261 ASAP> plotter.set_cursor(pol=``I Plinear'')
[538]1262\end{verbatim}
1263
1264Row, beam and IF selection are also available in \cmd{set\_cursor} as
1265describe in section~\ref{sec:plotter_cursor}.
1266
1267\subsection{Saving}
1268
[790]1269\index{Polarisation!saving}When saving data using the \cmd{save}
1270function, the \cmd{stokes} argument can be used to save the data as
1271Stoke values when saving in FITS format.
[538]1272
1273Example:
1274
1275\begin{verbatim}
1276 ASAP> scans.save('myscan.sdfits', 'SDFITS', stokes=True)
1277\end{verbatim}
1278
[534]1279
[763]1280\section{Scantable Mathematics}
1281
[790]1282\index{Scantable!maths}It is possible to to simple mathematics
1283directly on scantables from the command line using the \cmd{+, -, *,
1284/} operators as well as their cousins \cmd{+=, -= *=, /=}. This works
1285between two scantables or a scantable and a float. (Note that it does
1286not work for integers).
[763]1287
[534]1288\begin{verbatim}
[763]1289 ASAP> sum = scan1+scan2
1290 ASAP> scan2 = scan1+2.0
1291 ASAP> scan *= 1.05
1292\end{verbatim}
1293
1294\section{Scripting}
1295
[790]1296\index{Scripting}Because asap is based on python, it easy for the user
1297write their own scripts and functions to process data. This is highly
1298recommended as most processing of user data could then be done in a
1299couple of steps using a few simple user defined functions. A Python
1300primer is beyond the scope of this userguide. See the asap home pages
1301for a scripting tutorial or the main python website for comprehensive
1302documentation.
[763]1303
1304\hspace{1cm} http://www.atnf.csiro.au/computing/software/asap/tutorials
1305\hspace{1cm} http://www.python.org/doc/Introduction.html
1306
1307\subsection{Running scripts}
1308
1309The asap global function \cmd{execfile} reads the named text file and
1310executes the contained python code. This file can either contain
1311function definitions which will be used in subsequent processing or
1312just a set of commands to process a specific dataset.
1313
1314\subsection{asapuserfuncs.py}
1315
1316The file $\sim$/.asap/asapuserfuncs.py is automatically read in when
1317asap is started. The user can use this to define a set of user
1318functions which are automatically available each time asap is
1319used. The \cmd{execfile} function can be called from within this file.
1320
1321\section{Worked examples}
1322
1323In the following section a few examples of end-to-end processing of
[774]1324some data in asap are given.
[763]1325
1326\subsection{Mopra}
[790]1327\index{Mopra}
[763]1328
[790]1329The following example is of some dual polarisation, position switched
1330data from Mopra. The source has been observed mulitple times split
1331into a number of seperate rpfits files. To make the processing easier,
1332the first step is to \cmd{cat} the seeprate rpfits files together and
1333load as a whole (future versions of asap will make this unnecessary).
1334
1335
1336\begin{verbatim}
1337# Concatenate the individual rpfits files together before loading
1338!cat 2005-06*.rpf > data.rpf
1339
1340# Load the data into a scantable
1341data = scantable('data.rpf')
1342print data
1343
1344# Form the quotient spectra
1345q = data.auto_quotient()
1346print q
1347
1348# Look at the spectra
1349plotter.plot(q)
1350
1351# Velocity align the data before averaging
1352q.set_unit('km/s')
1353q.set_freqframe('LSRK')
1354q.freq_align()
1355
1356# Average all scans in time
1357av = q.average_time()
1358plotter.plot(av)
1359
1360# Remove the baseline
1361msk = av.create_mask([100,130],[160,200])
1362av.poly_baseline(msk,2)
1363
1364# Average the two polarisations together
1365iav = av.average_pol()
1366print iav
1367plotter.plot(iav)
1368
1369# Set a sensible velocity range on the plot
1370plotter.set_range(85,200)
1371
1372# Smooth the data a little
1373av.smooth('gauss',4)
1374plotter.plot()
1375
1376# Fit a guassian to the emission
1377f = fitter()
1378f.set_function(gauss=1)
1379f.set_scan(av)
1380f.fit()
1381
1382# View the fit
1383f.plot()
1384
1385# Get the fit parameters
1386f.get_parameters()
1387
1388\end{verbatim}
1389
1390
[763]1391\subsection{Parkes Polarimetry}
1392
[790]1393\index{Parkes}\index{Polarisation}The following example is processing
1394of some Parkes polarmetric observations of OH masers at
13951.6~GHz. Because digital filters where used in the backend, the
1396baselines are stable enough not to require a quotient spectra. The
13974~MHz bandwidth is wide enough to observe both the 1665 and 1667~MHz
1398OH maser transitions. Each source was observed once for about 10
1399minutes. Tsys information was not written to the rpfits file (a
1400nominal 25K values was used), so the amplitudes need to be adjusted
1401based on a separate log file. A simple user function is used to
1402simplify this, contained in a file called mypol.py:
[763]1403
1404\begin{verbatim}
1405def xyscale(data,xtsys=1.0,ytsys=1.0,nomtsys=25.0) :
1406
1407 data.set_cursor(pol=0)
1408 data.scale(xtsys/nomtsys,allaxes=False)
1409
1410 data.set_cursor(pol=1)
1411 data.scale(ytsys/nomtsys,allaxes=False)
1412
1413 data.set_cursor(pol=2)
1414 data.scale((xtsys+ytsys)/(2*nomtsys),allaxes=False)
1415
1416 data.set_cursor(pol=3)
1417 data.scale((xtsys+ytsys)/(2*nomtsys),allaxes=False)
1418\end{verbatim}
1419
1420The typical asap session would be
1421
1422\begin{verbatim}
[774]1423
[763]1424# Remind ourself the name of the rpfits files
[774]1425ls
[763]1426
1427# Load data from an rpfits file
1428d1665 = scantable('2005-10-27_0154-P484.rpf')
1429
1430# Check what we have just loaded
1431d1665.summary
1432
1433# View the data in velocity
1434d1665.set_unit('km/s')
1435d1665.set_freqframe('LSRK')
1436
1437# Correct for the known phase offset in the crosspol data
1438d1665.rotate_xyphase(-4)
1439
[774]1440# Create a copy of the data and set the rest frequency to the 1667 MHz
[763]1441# transition
1442d1667 = d1665.copy()
1443d1667.set_restfreqs(lines=['OH1667'])
1444d1667.summary
1445
1446# Copy out the scan we wish to process
1447g351_5 = d1665.get_scan('351p160')
1448g351_7 = d1667.get_scan('351p160')
1449
1450# Plot the data
1451plotter.plot(g351_5,g351_7) # Only shows one panel
1452
1453# Tell the plotter to stack polarisation and panel scans
1454plotter.set_mode('p','s')
1455
1456# Correct for the Tsys using our predefined function
1457execfile('mypol.py') # Read in the function
1458xyscale(g351_5,23.2,22.7) # Execute it on the data
1459xyscale(g351_7,23.2,22.7)
1460
1461# Only plot the velocity range of interest
1462plotter.set_range(-30,10)
1463
1464# Baseline the data
1465msk = g351_5.create_mask([-20,-15],[0,5])
1466g351_5.poly_baseline(msk,1)
1467msk = g351_7.create_mask([-20,-15],[0,5])
1468g351_7.poly_baseline(msk,1)
1469
1470# Update the plot with the baselined data
1471plotter.plot()
1472
1473# Look at the various polarisation products
1474plotter.set_cursor(pol='RR LL')
1475plotter.set_cursor(pol='I Plinear')
1476plotter.set_cursor(pol='I Q U V')
1477
1478# Save the plot as postscript
1479plotter.save('g361_stokes.ps')
1480
1481# Save the process spectra
1482g351_5.save('junk5.asap')
1483g351_7.save('junk7.asap')
1484
1485\end{verbatim}
1486
1487\subsection{Tidbinbilla}
1488
[790]1489\index{Tidbinbilla}The following example is processing of some
1490Tidbinbilla observations of NH$_3$ at 12~mm. Tidbinbilla has (at the
1491time of observations) a single polarisation, but can process two IFs
1492simultaneously. In the example, the first half of the observation was
1493observing the (1,1) and (2,2) transitions simultaneously). The second
1494half observed only the (4,4) transition due to bandwidth
1495limitations. The data is position switched, observing first an
1496reference to the west, then the source twice and finally reference to
1497the east.
[767]1498
1499\begin{verbatim}
1500
1501# Load the rpfits file and inspect
1502d = scantable('2003-03-16_082048_t0002.rpf')
1503print d
1504
1505# Make the quotient spectra
1506q = d.auto_quotient()
1507print q
1508
1509# Plot/select in velocity
1510q.set_freqframe('LSRK')
1511q.set_unit('km/s')
1512
1513# Seperate data from the (1,1)&(2,2) and (4,4) transitions
1514g1 = q.get_scan(range(6)) # Rows 0..5
1515g2 = q.get_scan(range(6,12)) # Rows 6..11
1516
[774]1517# Align data in velocity
[767]1518g1.freq_align(perif=True)
1519g2.freq_align(perif=True)
1520
1521# Average individual scans
1522a1 = g1.average_time()
1523a2 = g2.average_time()
1524
1525# Rpfits file only contrains a single rest frequency. Set both
1526a1.set_restfreqs(freqs= [23694.4700e6,23722.6336e6])
1527
1528plotter.plot(a1,a2)
1529plotter.set_mode('i','s')
1530x = raw_input()
1531
1532a1.auto_poly_baseline()
1533a2.auto_poly_baseline()
1534
1535plotter.plot()
1536
1537a1.smooth('gauss',5)
1538a2.smooth('gauss',5)
1539plotter.plot()
1540
1541\end{verbatim}
1542
[763]1543\newpage
1544
1545\section{Appendix}
1546
1547\subsection{Function Summary}
1548
[790]1549\index{Functions!summary}%
[763]1550\begin{verbatim}
[738]1551 [The scan container]
[534]1552 scantable - a container for integrations/scans
1553 (can open asap/rpfits/sdfits and ms files)
1554 copy - returns a copy of a scan
1555 get_scan - gets a specific scan out of a scantable
1556 summary - print info about the scantable contents
1557 set_cursor - set a specific Beam/IF/Pol 'cursor' for
1558 further use
1559 get_cursor - print out the current cursor position
1560 stats - get specified statistic of the spectra in
1561 the scantable
1562 stddev - get the standard deviation of the spectra
1563 in the scantable
1564 get_tsys - get the TSys
1565 get_time - get the timestamps of the integrations
[790]1566 get_azimuth - get the azimuth of the scans
1567 get_elevation - get the elevation of the scans
1568 get_parangle - get the parallactic angle of the scans
[534]1569 get_unit - get the currnt unit
1570 set_unit - set the abcissa unit to be used from this
1571 point on
1572 get_abcissa - get the abcissa values and name for a given
1573 row (time)
1574 set_freqframe - set the frame info for the Spectral Axis
1575 (e.g. 'LSRK')
1576 set_doppler - set the doppler to be used from this point on
1577 set_instrument - set the instrument name
1578 get_fluxunit - get the brightness flux unit
1579 set_fluxunit - set the brightness flux unit
1580 create_mask - return an mask in the current unit
1581 for the given region. The specified regions
1582 are NOT masked
1583 get_restfreqs - get the current list of rest frequencies
1584 set_restfreqs - set a list of rest frequencies
1585 lines - print list of known spectral lines
1586 flag_spectrum - flag a whole Beam/IF/Pol
1587 save - save the scantable to disk as either 'ASAP'
1588 or 'SDFITS'
1589 nbeam,nif,nchan,npol - the number of beams/IFs/Pols/Chans
1590 history - print the history of the scantable
[544]1591 get_fit - get a fit which has been stored witnh the data
[738]1592 average_time - return the (weighted) time average of a scan
[534]1593 or a list of scans
1594 average_pol - average the polarisations together.
1595 The dimension won't be reduced and
1596 all polarisations will contain the
1597 averaged spectrum.
[738]1598 auto_quotient - return the on/off quotient with
1599 automatic detection of the on/off scans
[534]1600 quotient - return the on/off quotient
1601 scale - return a scan scaled by a given factor
[738]1602 add - return a scan with given value added
[534]1603 bin - return a scan with binned channels
1604 resample - return a scan with resampled channels
1605 smooth - return the spectrally smoothed scan
1606 poly_baseline - fit a polynomial baseline to all Beams/IFs/Pols
[738]1607 auto_poly_baseline - automatically fit a polynomial baseline
[774]1608 recalc_azel - recalculate azimuth and elevation based on
1609 the pointing
[534]1610 gain_el - apply gain-elevation correction
1611 opacity - apply opacity correction
1612 convert_flux - convert to and from Jy and Kelvin brightness
1613 units
1614 freq_align - align spectra in frequency frame
1615 rotate_xyphase - rotate XY phase of cross correlation
1616 rotate_linpolphase - rotate the phase of the complex
1617 polarization O=Q+iU correlation
1618 [Math] Mainly functions which operate on more than one scantable
1619
[738]1620 average_time - return the (weighted) time average
[534]1621 of a list of scans
1622 quotient - return the on/off quotient
[544]1623 simple_math - simple mathematical operations on two scantables,
1624 'add', 'sub', 'mul', 'div'
[534]1625 [Fitting]
1626 fitter
1627 auto_fit - return a scan where the function is
1628 applied to all Beams/IFs/Pols.
1629 commit - return a new scan where the fits have been
1630 commited.
1631 fit - execute the actual fitting process
[544]1632 store_fit - store the fit paramaters in the data (scantable)
[534]1633 get_chi2 - get the Chi^2
1634 set_scan - set the scantable to be fit
1635 set_function - set the fitting function
1636 set_parameters - set the parameters for the function(s), and
1637 set if they should be held fixed during fitting
[544]1638 set_gauss_parameters - same as above but specialised for individual
1639 gaussian components
[534]1640 get_parameters - get the fitted parameters
1641 plot - plot the resulting fit and/or components and
1642 residual
1643 [Plotter]
1644 asapplotter - a plotter for asap, default plotter is
1645 called 'plotter'
1646 plot - plot a (list of) scantable
1647 save - save the plot to a file ('png' ,'ps' or 'eps')
1648 set_mode - set the state of the plotter, i.e.
1649 what is to be plotted 'colour stacked'
1650 and what 'panelled'
[544]1651 set_cursor - only plot a selected part of the data
1652 set_range - set a 'zoom' window
[534]1653 set_legend - specify user labels for the legend indeces
1654 set_title - specify user labels for the panel indeces
1655 set_ordinate - specify a user label for the ordinate
1656 set_abcissa - specify a user label for the abcissa
1657 set_layout - specify the multi-panel layout (rows,cols)
[738]1658
[534]1659 [Reading files]
1660 reader - access rpfits/sdfits files
1661 read - read in integrations
1662 summary - list info about all integrations
1663
1664 [General]
1665 commands - this command
1666 print - print details about a variable
1667 list_scans - list all scantables created bt the user
1668 del - delete the given variable from memory
1669 range - create a list of values, e.g.
1670 range(3) = [0,1,2], range(2,5) = [2,3,4]
1671 help - print help for one of the listed functions
[538]1672 execfile - execute an asap script, e.g. execfile('myscript')
[544]1673 list_rcparameters - print out a list of possible values to be
[763]1674 put into \$HOME/.asaprc
[534]1675 mask_and,mask_or,
1676 mask_not - boolean operations on masks created with
1677 scantable.create_mask
[738]1678
[534]1679 Note:
1680 How to use this with help:
1681 # function 'summary'
1682 [xxx] is just a category
[544]1683 Every 'sub-level' in this list should be replaces by a '.' Period when
[738]1684 using help
[534]1685 Example:
1686 ASAP> help scantable # to get info on ths scantable
1687 ASAP> help scantable.summary # to get help on the scantable's
1688 ASAP> help average_time
[544]1689
[534]1690\end{verbatim}
1691
1692\subsection{Installation}
1693
[790]1694\index{Installation}ASAP depends on a number of third-party libraries which you must
[534]1695have installed before attempting to build ASAP. These are:
1696
1697\begin{itemize}
1698\item AIPS++
1699\item Boost
1700\item Matplotlib
[738]1701\item python/ipython
[534]1702\end{itemize}
1703
1704Debian Linux is currently supported and we intend also
1705to support other popular Linux flavours, Solaris and Mac.
1706
1707Of the dependencies, AIPS++ is the most complex to install.
1708
1709\subsection{ASCII output format}
1710
1711\subsection{.asaprc settings}
[790]1712\index{.asaprc}
[763]1713\asaprc{verbose}{{\bf True}/False}{Print verbose output}
1714
1715\asaprc{insitu}{{\bf True}/False}{Apply operations on the input
1716scantable or return new one}
1717
1718% plotting
1719
1720\asaprc{useplotter}{{\bf True}/False}{Preload a default plotter}
1721
1722\asaprc{plotter.gui}{{\bf True}/False}{Do we want a GUI or plot to a
1723file}
1724
1725\asaprc{plotter.stacking}{{\bf Pol} Beam IF Scan Time}{Default mode for
1726colour stacking}
1727
1728\asaprc{plotter.panelling}{Pol Beam IF {\bf Scan} Time}{Default mode
1729for panelling}
1730
1731\asaprc{plotter.ganged}{{\bf True}/False}{Push panels together, to
1732share axislabels}
1733
1734\asaprc{plotter.decimate}{True/{\bf False}}{Decimate the number of
1735points plotted by a factor of nchan/1024}
1736
1737% default colours/linestyles
1738%\asaprc{plotter.colours}{.}{.}
1739%\asaprc{plotter.linestyles{.}{.}
1740
1741% scantable
1742\asaprc{scantable.save}{{\bf ASAP} SDFITS FITS ASCII MS2}{Default output
[774]1743format when saving}
[763]1744
1745\asaprc{scantable.autoaverage}{{\bf True}/False}{Auto averaging on
1746read}
1747
1748\asaprc{scantable.freqframe}{{\bf LSRK} TOPO BARY etc}{default
1749frequency frame to set when function scantable.set\_freqframe is
1750called}
1751
1752\asaprc{scantable.allaxes}{{\bf True}/False}{Apply action to all axes
1753not just the cursor location}
1754
1755\asaprc{scantable.plotter}{{\bf True}/False}{Use internal plotter}
1756
1757\asaprc{scantable.verbosesummary}{True/{\bf False}}{Control the level
1758of information printed by summary}
1759
[790]1760\printindex
1761
[534]1762\end{document}
[767]1763
Note: See TracBrowser for help on using the repository browser.