source: branches/Release2.0/doc/userguide.tex@ 3090

Last change on this file since 3090 was 1049, checked in by phi196, 19 years ago

Ticker #9 final asap2.0 updates

  • Property svn:eol-style set to native
  • Property svn:keywords set to Author Date Id Revision
File size: 63.7 KB
Line 
1\documentclass[11pt]{article}
2\usepackage{a4}
3\usepackage{calc}
4\usepackage[dvips]{graphicx}
5\usepackage{makeidx}
6
7% Adjust the page size
8\addtolength{\oddsidemargin}{-0.4in}
9\addtolength{\evensidemargin}{+0.4in}
10\addtolength{\textwidth}{+0.8in}
11
12\setlength{\parindent}{0mm}
13\setlength{\parskip}{1ex}
14
15\title{ATNF Spectral Analysis Package\\User Guide v2.0 }
16\author{Chris Phillips}
17
18\newcommand{\cmd}[1]{{\tt #1}}
19
20\newcommand{\asaprc}[3]{
21 \begin{minipage}[t]{45mm}#1\end{minipage}
22 \begin{minipage}[t]{30mm}\raggedright #2\end{minipage}\hspace{3mm}
23 \begin{minipage}[t]{\textwidth-75mm}#3\end{minipage}
24}
25
26\makeindex
27
28
29\begin{document}
30
31\maketitle
32
33\section{Introduction}
34
35ASAP is a single dish spectral line processing package currently being
36developed by the ATNF. It is intended to process data from all ATNF
37antennas, and can probably be used for other antennas if they can
38produce ``Single Dish FITS'' format. It is based on the AIPS++
39package.
40
41This userguide is currently being updated for the ASAP 2.0 release
42which has a number of significant changes which affect the user
43interface. Please report any mistakes you find in this userguide.
44
45\section{Installation and Running}
46
47Currently there are installations running on Linux machines at
48
49\begin{itemize}
50\item Epping - use hosts {\tt draco} or {\tt hydra}
51\item Narrabri - use host {\tt kaputar}
52\item Parkes - use host {\tt bourbon}
53\item Mopra - use host {\tt minos}
54\end{itemize}
55
56\index{Running}To start asap log onto one of these Linux hosts and enter
57
58\begin{verbatim}
59 > cd /my/data/directory
60 > asap
61\end{verbatim}
62
63This starts the ASAP. To quit, you need to type \verb+^+-d
64(control-d) or type \cmd{\%Exit}.
65
66\section{Interface}
67
68\index{Interface}ASAP is written in C++ and python. The user interface uses the
69``ipython'' interactive shell, which is a simple interactive interface
70to python. The user does not need to understand python to use this,
71but certain aspects python affect what the user can do. The current
72interface is object oriented. In the future, we will build a
73functional (non object oriented) shell on top of this to ease
74interactive use.
75
76\subsection {Integer Indices are 0-relative}
77
78Please note, all integer indices in ASAP and iPython are {\bf 0-relative}.
79
80\subsection{Objects}
81\index{objects}
82The ASAP interface is based around a number of ``objects'' which the
83user deals with. Objects range from the data which have been read from
84disk, to tools used for fitting functions to the data. The following
85main objects are used :
86
87\begin{itemize}
88 \item[\cmd{scantable}] The data container (actual spectra and header
89 information)
90 \item[\cmd{selector}] Allows the user to select a subsection of the
91 data, such as a specified or ramge of beam numbers, IFs, etc.
92 \item[\cmd{plotter}] A tool used to plot the spectral line data
93 \item[\cmd{fitter}] A tool used to fit functions to the spectral data
94 \item[\cmd{reader}] A tool which can be used to read data from disks
95 into a scantable object (advanced use).
96\end{itemize}
97
98There can be many objects of the same type. Each object is referred to
99by a variable name made by the user. The name of this variable is not
100important and can be set to whatever the user prefers (i.e. ``s'' and
101``ParkesHOH-20052002'' are equivalent). However, having a simple and
102consistent naming convention will help you a lot.
103
104\subsection{Member Functions (functions)}
105
106\index{Functions!member}Following the object oriented approach,
107objects have associated ``member functions'' which can either be used
108to modify the data in some way or change global properties of the
109object. In this document member functions will be referred to simply
110as functions. From the command line, the user can execute these
111functions using the syntax:
112\begin{verbatim}
113 ASAP> out = object.function(arguments)
114\end{verbatim}
115
116Where \cmd{out} is the name of the returned variable (could be a new
117scantable object, or a vector of data, or a status return),
118\cmd{object} is the object variable name (set by the user),
119\cmd{function} is the name of the member function and \cmd{arguments}
120is a list of arguments to the function. The arguments can be provided
121either though position or \cmd{name=}. A mix of the two can be used.
122E.g.
123
124\begin{verbatim}
125 ASAP> av = scans.average_time(msk,weight='tsys')
126 ASAP> av = scans.average_time(mask=msk,weight='tsys')
127 ASAP> av = scans.average_time(msk,tsys)
128 ASAP> scans.poly_baseline(mask=msk, order=0, insitu=True)
129 ASAP> scans.poly_baseline(msk,0,True)
130 ASAP> scans.poly_baseline(mask, insitu=True)
131\end{verbatim}
132
133\subsection{Global Functions}
134
135\index{Functions!global}It does not make sense to implement some functions as member
136functions, typically functions which operate on more than one
137scantable (e.g. time averaging of many scans). These functions will
138always be referred to as global functions.
139
140\subsection{Interactive environment}
141
142\index{ipython!environment}ipython has a number of useful interactive
143features and a few things to be aware of for the new user.
144
145\subsubsection{String completion}
146
147\index{ipython!string completion}Tab completion is enabled for all
148function names. If you type the first few letters of a function name,
149then type {\tt <TAB>} the function name will be auto completed if it
150is un-ambiguous, or a list of possibilities will be
151given. Auto-completion works for the user object names as well as
152function names. It does not work for filenames, nor for function
153arguments.
154
155Example
156\begin{verbatim}
157 ASAP> scans = scantable('MyData.rpf')
158 ASAP> scans.se<TAB>
159 ASAP> scans.set_in<TAB>
160scans.set_cursor scans.set_freqframe scans.set_selection
161scans.set_doppler scans.set_instrument scans.set_unit
162scans.set_fluxunit scans.set_restfreqs
163
164 ASAP> scans.set_instrument()
165\end{verbatim}
166
167\subsubsection{Leading Spaces}
168
169\index{ipython!leading space}Python uses leading space to mark blocks
170of code. This means that it you start a command line with a space, the
171command generally will fail with an syntax error.
172
173\subsubsection{Variable Names}
174
175\index{ipython!variable names}During normal data processing, the user
176will have to create named variables to hold spectra etc. These must
177conform to the normal python syntax, specifically they cannot contain
178``special'' characters such as \@ \$ etc and cannot start with a
179number (but can contain numbers). Variable (and function) names are
180case sensitive.
181
182\subsubsection{Unix Interaction}
183
184\index{ipython!unix interaction}Basic unix shell commands (\cmd{pwd},
185\cmd{ls}, \cmd{cd} etc) can be issued from within ASAP. This allows
186the user to do things like look at files in the current directory. The
187shell command ``\cmd{cd}'' works within ASAP, allowing the user to
188change between data directories. Unix programs cannot be run this way,
189but the shell escape ``$!$'' can be used to run arbitrary
190programs. E.g.
191
192\begin{verbatim}
193 ASAP> pwd
194 ASAP> ls
195 ASAP> cd /my/data/directory
196 ASAP> ! mozilla&
197\end{verbatim}
198
199\subsection{Help}
200
201\index{Help}ASAP has built in help for all functions. To get a list of
202functions type:
203
204\begin{verbatim}
205 ASAP> commands()
206\end{verbatim}
207
208To get help on specific functions, the built in help needs to be given
209the object and function name. E.g.
210
211\begin{verbatim}
212 ASAP> help scantable.get_scan # or help(scantable.get_scan)
213 ASAP> help scantable.stats
214 ASAP> help plotter.plot
215 ASAP> help fitter.plot
216
217 ASAP> scans = scantable('mydata.asap')
218 ASAP> help scans.get_scan # Same as above
219\end{verbatim}
220
221Global functions just need their name
222
223\begin{verbatim}
224 ASAP> help average_time
225\end{verbatim}
226
227Note that if you just type \cmd{help} the internal ipython help is
228invoked, which is probably {\em not} what you want. Type \verb+^+-d
229(control-d) to escape from this.
230
231\subsection{Customisation - .asaprc}
232
233\index{.asaprc}ASAP use an \cmd{.asaprc} file to control the user's
234preference of default values for various functions arguments. This
235includes the defaults for arguments such as \cmd{insitu}, scantable
236\cmd{freqframe} and the plotters \cmd{set\_mode} values. The help on
237individual functions says which arguments can be set default values
238from the \cmd{.asaprc} file. To get a sample contents for the
239\cmd{.asaprc} file use then command \cmd{list\_rcparameters}.
240
241Common values include:
242\begin{verbatim}
243 # apply operations on the input scantable or return new one
244 insitu : False
245
246 # default output format when saving scantable
247 scantable.save : ASAP
248
249 # default frequency frame to set when function
250 # scantable.set_freqframe is called
251 scantable.freqframe : LSRK
252
253 # auto averaging on read
254 scantable.autoaverage : True
255\end{verbatim}
256
257For a complete list of \cmd{.asaprc} values, see the Appendix.
258
259\section{Scantables}
260\index{Scantables}
261\subsection {Description}
262
263\subsubsection {Basic Structure}
264
265\index{Scantable!structure}ASAP data handling works on objects called
266scantables. A scantable holds your data, and also provides functions
267to operate upon it.
268
269The building block of a scantable is an integration, which is a single
270row of a scantable. Each row contains just one spectrum for each beam, IF and
271polarisation. For example Parkes multibeam data would contain many
272beams, one IF and 2-4 polarisations, while the new Mopra 8-GHz
273filterbank will eventually produce one beam, many IFs, and 2-4
274polarisations. All of the combinations of Beams/IFs an Polarisations are
275contained in seperate rows. These rows are grouped in cycles (same time stamp).
276
277A collection of cycles for one source is termed
278a scan (and each scan has a unique numeric identifier, the SCANNO). A
279scantable is then a collection of one or more scans. If you have
280scan-averaged your data in time, i.e. you have averegaed all cycles within
281a scan, then each scan would hold just one (averaged) integration.
282
283Many of the functions which work on scantables can either return a
284new scantable with modified data or change the scantable insitu. Which
285method is used depends on the users preference. The default can be
286changed via the {\tt .asaprc} resource file.
287
288For example a Mopra scan with a 4s intergration time, two IFs and
289dual polarisations has two (2s) cycles.
290
291\begin{verbatim}
292SCANNO CYCLENO BEAMNO IFNO POLNO
2930 0 0 0 0
2940 0 0 0 1
2950 0 0 1 0
2960 0 0 1 1
2970 1 0 0 0
2980 1 0 0 1
2990 1 0 1 0
3000 1 0 1 1
301
302\end{verbatim}
303
304
305\subsubsection {Contents}
306
307\index{Scantable!contents}A scantable has header information and data
308(a scantable is actually an AIPS++ Table and it is generally stored in
309memory when you are manipulating it with ASAP. You can save it to
310disk and then browse it with the AIPS++ Table browser if you know how
311to do that !).
312
313The data are stored in columns (the length of a column is the number of
314rows/spectra of course).
315
316
317Two important columns are those that describe the frequency setup. We mention
318them explicitly here because you need to be able to understand the presentation
319of the frequency information and possibly how to manipulate it.
320
321These columns are called FREQ\_ID and MOLECULE\_ID. They contain indices, for
322each IF, pointing into tables with all of the frequency and rest-frequency
323information for that integration.
324
325There are of course many other columns which contain the actual spectra,
326the flags, the Tsys, the source names and so on.
327
328There is also a function \cmd{summary} to list a summary of the scantable.
329You will find this very useful.
330
331Example:
332
333\begin{verbatim}
334 ASAP> scans = scantable('MyData.rpf')
335 ASAP> scans.summary() # Brief listing
336
337 # Equivalent to brief summary function call
338 ASAP> print scan
339\end{verbatim}
340
341The summary function gives you a scan-based summary, presenting the
342scantable as a cascading view of Beams and IFs. Note that the output
343of summary is redirected into your current pager specified by the
344\$PAGER environment variable. If you find the screen is reset to the
345original state when summary is finished (ie the output from summary
346disappears), you may need to set the \$LESS environment variable to
347include the \cmd{-X} option.
348
349\subsection{Data Selection}
350\label{sec:selection}
351
352ASAP contains flexible data selection. Data can be selected based on
353IF, beam, polarisation, scan number as well as values such as
354Tsys. Advanced users can also make use of the AIPS++ TAQL language to
355create selections based on almost any of the values recorded.
356
357Selection is based on a \cmd{selector} object. This object is created
358and various selection functions applied to it (\cmd{set\_ifs},
359\cmd{set\_beams} etc). The selection object then must be applied to a
360scantable using the \cmd{set\_selection} function. A single selection
361object can be created and setup then applied to multiple scantables.
362
363Once a selection has been applied, all following functions will only
364``see'' the selected spectra (including functions such as
365\cmd{summary}). The selection can then be reset and all spectra are
366visible. Note that if functions such as \cmd{copy} are run on a
367scantable with active selection, only the selected spectra are copied.
368
369The common selection functions are:
370
371\begin{itemize}
372
373\item[\cmd{set\_beams}] Select beams by index number
374\item[\cmd{set\_ifs}] Select ifs by index number
375\item[\cmd{set\_name}] Select by source name. Can contain ``*'' as a
376wildcard, e.g. ``Orion*\_R''.
377\item[\cmd{set\_ifs}] Select ifs by index number
378\item[\cmd{set\_polarisation}] Select but polarisation index or
379name. If polarisation names are given, the data will be on-the-fly
380converted (for example from linears to Stokes).
381\item[\cmd{set\_query}] Set query directly. For power users only!
382\item[\cmd{set\_tsys}] Select data based on Tsys. Also example of user
383definable query.
384\item[\cmd{reset}] Reset the selection to include all spectra.
385
386
387Note that all indices are zero based.
388
389Examples:
390
391\begin{verbatim}
392 ASAP> selection = selector() # Create selection object
393 ASAP> selection.set_ifs(0) # Just select the first IF
394 ASAP> scans.set_selection(selection) # Apply the selection
395 ASAP> print scans # Will just show the first IF
396
397 ASAP> selection.set_ifs([0,1]) # Select the first two IFs
398 ASAP> selection.set_beams([1,3,5]) # Also select three of the beams
399 ASAP> scans.set_selection(selection) # Apply the selection
400
401 ASAP> selection.set_name('G308*') # Select by source name
402
403 ASAP> selection.reset() # Turn off selection
404 ASAP> scans.set_selection(selection) # Apply the reset selection
405
406\end{verbatim}
407
408\end{itemize}
409
410\subsection{State}
411
412\index{Scantable!state}Each scantable contains "state"; these are
413properties applying to all of the data in the scantable.
414
415Examples are the selection of beam, IF and polarisation, spectral unit
416(e.g. km/s), frequency reference frame (e.g. BARY) and velocity Doppler
417type (e.g. RADIO).
418
419\subsubsection{Units, Doppler and Frequency Reference Frame}
420
421The information describing the frequency setup for each integration
422is stored fundamentally in frequency in the reference frame
423of observation (E.g. TOPO).
424
425When required, this is converted to the desired reference frame
426(e.g. LSRK), Doppler (e.g. OPTICAL) and unit (e.g. km/s) on-the-fly.
427This is important, for example, when you are displaying the data or
428fitting to it. The reference frame is set on file read to the value
429set in the user \cmd{.asaprc} file.
430
431For units, the user has the choice of frequency, velocity or channel.
432The \cmd{set\_unit} function is used to set the current unit for a
433scantable. All functions will (where relevant) work with the selected
434unit until this changes. This is mainly important for fitting (the fits
435can be computed in any of these units), plotting and mask creation.
436
437The velocity definition can be changed with the \cmd{set\_doppler}
438function, and the frequency reference frame can be changed with the
439\cmd{set\_freqframe} function.
440
441Example usage:
442
443\begin{verbatim}
444 ASAP> scans = scantable('2004-11-23_1841-P484.rpf') # Read in the data
445 ASAP> scans.set_freqframe('LSRK') # Use the LSR velocity frame
446 ASAP> scans.set_unit('km/s') # Use velocity for plots etc from now on
447 ASAP> scans.set_doppler('OPTICAL') # Use the optical velocity convention
448 ASAP> scans.set_unit('MHz') # Use frequency in MHz from now on
449\end{verbatim}
450
451
452\subsubsection{Rest Frequency}
453
454\index{Scantable!rest frequency}ASAP reads the line rest frequency
455from the RPFITS file when reading the data. The values stored in the
456RPFITS file are not always correct and so there is a function
457\cmd{set\_restfreq} to set the rest frequencies for the currently
458selected data.
459
460For each integration, there is a rest-frequency per IF (the rest
461frequencies are just stored as a list with an index into them).
462There are a few ways to set the rest frequencies with this function.
463
464If you specify just one rest frequency, then it is set for all IF.
465
466\begin{verbatim}
467 # Set all IFs
468 ASAP> scans.set_restfreqs(freqs=1.667359e9)
469\end{verbatim}
470
471If set a rest frequency for each IF, specify a list of frequencies (of
472length the number of IFs). Regardless of the source, the rest
473frequency will be set for each IF to the corresponding value in the
474provided list.
475
476\begin{verbatim}
477 # Set rest frequency for all IFs
478 ASAP> scans.set_restfreqs(freqs=[1.6654018e9,1.667359e9,])
479
480\end{verbatim}
481
482{\em Currently this is not implemented}
483
484In both of the above modes, you can also specify the rest frequencies via
485names in a known list rather than by their values.
486
487Examples:
488
489\begin{verbatim}
490 ASAP> scans.set_restfreqs(freqs=['OH1665','OH1667'])
491\end{verbatim}
492
493
494\subsubsection{Masks}
495
496\index{Masks}\index{Scantable!masks}
497
498Many tasks (fitting, baseline subtraction, statistics etc) should only
499be run on range of channels. Depending on the current ``unit'' setting
500this range is set directly as channels, velocity or frequency
501ranges. Internally these are converted into a simple boolean mask for
502each channel of the abscissa. This means that if the unit setting is
503later changed, previously created mask are still valid. (This is not
504true for functions which change the shape or shift the frequency
505axis). You create masks with the function \cmd{create\_mask} and this
506specified the channels to be included in the selection. When setting
507the mask in velocity, the conversion from velocity to channels is
508based on the current cursor setting, selected row and selected
509frequency reference frame.
510
511
512Note that for multi IF data with different number of channels per IF a
513single mask cannot be applied to different IFs. To use a mask on such
514data the selector should be applied to select all IFs with the same
515number of channels.
516
517Example :
518\begin{verbatim}
519
520 # Select channel range for baselining
521 ASAP> scans.set_unit('channels')
522 ASAP> msk = scans.create_mask([100,400],[600,800])
523
524 # Select velocity range for fitting
525 ASAP> scans.set_unit('km/s')
526 ASAP> msk = scans.create_mask([-30,-10])
527\end{verbatim}
528
529Sometimes it is more convenient to specify the channels to be
530excluded, rather included. You can do this with the ``invert''
531argument.
532
533Example :
534\begin{verbatim}
535 ASAP> scans.set_unit('channels')
536 ASAP> msk = scans.create_mask([0,100],[900-1023], invert=True)
537\end{verbatim}
538
539By default \cmd{create\_mask} uses the frequency setup of the first row
540to convert velocities into a channel mask. If the rows in the data
541cover different velocity ranges, the scantable row to use should be
542specified:
543
544\begin{verbatim}
545 ASAP> scans.set_unit('km/s')
546 ASAP> msk = q.create_mask([-30,-10], row=5)
547\end{verbatim}
548
549Because the mask is stored in a simple python variable, the users is
550able to combine masks using simple arithmetic. To create a mask
551excluding the edge channels, a strong maser feature and a birdie in
552the middle of the band:
553
554\begin{verbatim}
555 ASAP> scans.set_unit('channels')
556 ASAP> msk1 = q.create_mask([0,100],[511,511],[900,1023],invert=True)
557 ASAP> scans.set_unit('km/s')
558 ASAP> msk2 = q.create_mask([-20,-10],invert=True)
559
560 ASAP> mask = msk1 and msk2
561\end{verbatim}
562
563
564\subsection{Management}
565
566\index{Scantable!management}During processing it is possible to create
567a large number of scan tables. These all consume memory, so it is best
568to periodically remove unneeded scan tables. Use \cmd{list\_scans} to
569print a list of all scantables and \cmd{del} to remove unneeded ones.
570
571Example:
572
573\begin{verbatim}
574 ASAP> list_scans()
575 The user created scantables are:
576 ['s', 'scans', 'av', 's2', 'ss']
577
578 ASAP> del s2
579 ASAP> del ss
580\end{verbatim}
581
582\section{Data Input}
583
584\index{Reading data}Data can be loaded in one of two ways; using the
585reader object or via the scantable constructor. The scantable method
586is simpler but the reader allows the user more control on what is read.
587
588\subsection{Scantable constructor}
589
590\index{Scantable constructor}\index{Scantable!constructor}This loads
591all of the data from filename into the scantable object scans and
592averages all the data within a scan (i.e. the resulting scantable
593will have one row per scan). The recognised input file formats are
594RPFITS, SDFITS (singledish fits), ASAP's scantable format and aips++
595MeasurementSet2 format.
596
597Example usage:
598
599\begin{verbatim}
600 ASAP> scan = scantable('2004-11-23_1841-P484.rpf')
601
602 # Don't scan average the data
603 ASAP> scan = scantable('2004-11-23_1841-P484.rpf', average=False)
604\end{verbatim}
605
606
607\subsection{Reader object}
608
609\index{Reader object}\index{Scantable!reader object}For more control
610when reading data into ASAP, the reader object should be used. This
611has the option of only reading in a range of integrations, only a
612specified beam or IF and does not perform any scan averaging of the
613data, allowing analysis of the individual integrations. Note that due
614to limitation of the RPFITS library, only one reader object can be
615open at one time reading RPFITS files. To read multiple RPFITS files,
616the old reader must be destroyed before the new file is opened.
617However, multiple readers can be created and attached to SDFITS files.
618
619
620Example usage:
621
622\begin{verbatim}
623 ASAP> r = reader('2003-03-16_082048_t0002.rpf')
624 ASAP> r.summary()
625 ASAP> scan = r.read()
626 ASAP> del r
627\end{verbatim}
628
629\section{Basic Processing}
630
631In the following section, a simple data reduction to form a quotient
632spectrum of a single source is followed. It has been assume that the
633\cmd{.asaprc} file has {\em not} been used to change the \cmd{insitu}
634default value from \cmd{True}.
635
636\subsection{Auto quotient}
637\index{Auto quotient}Quotients can be computed ``automatically''. This
638requires the data to have matching source/reference pairs or one
639reference for multiple sources. Auto quotient assumes reference scans
640have a trailing ``\_R'' in the source name for data from Parkes and
641Mopra, and a trailing ``e'' or ``w'' for data fro, Tidbinbilla.
642
643\begin{verbatim}
644 ASAP> q = s.auto_quotient()
645\end{verbatim}
646
647By default the quotient spectra is calculated
648to preserve continuum emission. If you wish to remove the continuum
649contribution, use the \cmd{preserve} argument:
650
651\begin{verbatim}
652 ASAP> q = s.auto_quotient(preserve=True)
653\end{verbatim}
654
655%If this is not sufficient the following alternative method can be used.
656%
657%\subsection{Separate reference and source observations}
658%
659%\index{Quotient spectra}Most data from ATNF observatories
660%distinguishes on and off source data using the file name. This makes
661%it easy to create two scantables with the source and reference
662%data. As long as there was exactly one reference observation for each
663%on source observation for following method will work.
664%
665%For Mopra and Parkes data:
666%\begin{verbatim}
667% ASAP> r = scans.get_scan('*_R')
668% ASAP> s = scans.get_scan('*_S')
669%\end{verbatim}
670%
671%For Tidbinbilla data
672%\begin{verbatim}
673% ASAP> r = scans.get_scan('*_[ew]')
674% ASAP> s = scans.get_scan('*_[^ew]')
675%\end{verbatim}
676%
677%\subsection{Make the quotient spectra}
678%
679%Use the quotient function
680%
681%\begin{verbatim}
682% ASAP> q = s.quotient(r)
683%\end{verbatim}
684%
685%This uses the rows in scantable \cmd{r} as reference spectra for the
686%rows in scantable \cmd{s}. Scantable \cmd{r} must have either 1 row
687%(which is applied to all rows in \cmd{s}) or both scantables must have
688%the same number of rows.
689
690\subsection{Time average separate scans}
691
692\index{Time average}If you have observed the source with multiple
693source/reference cycles you will want to scan-average the quotient
694spectra together.
695
696\begin{verbatim}
697 ASAP> av = q.average_time()
698\end{verbatim}
699
700If for some you want to average multiple sets of scantables together
701you can:
702
703\begin{verbatim}
704 ASAP> av = average_time(q1, q2, q3)
705\end{verbatim}
706
707The default is to use integration time weighting. The alternative is
708to use none, variance, Tsys weighting or Tsys \& integration time.
709
710\begin{verbatim}
711 ASAP> av = average_time(q, weight='tintsys')
712\end{verbatim}
713
714To use variance based weighting, you need to supply a mask saying which
715channel range you want it to calculate the variance from.
716
717\begin{verbatim}
718 ASAP> msk = scans.create_mask([200,400],[600,800])
719 ASAP> av = average_time(scans, mask=msk, weight='var')
720\end{verbatim}
721
722If you have not observed your data with Doppler tracking (or run
723\cmd{freq\_align} explicitally) you should align the data in frequency
724before averaging.
725
726\begin{verbatim}
727 ASAP> av = scans.average_time(align=True)
728\end{verbatim}
729
730Note that, if needed, you should run \cmd{gain\_el} and \cmd{opacity}
731before you average the data in time (\S \ref{sec:gainel} \&
732\ref{sec:freqalign}).
733
734\subsection{Baseline fitting}
735
736\index{Baseline fitting}To make a baseline fit, you must first create
737a mask of channels to use in the baseline fit.
738
739\begin{verbatim}
740 ASAP> msk = scans.create_mask([100,400],[600,900])
741 ASAP> scans.poly_baseline(msk, order=1)
742\end{verbatim}
743
744This will fit a first order polynomial to the selected channels and subtract
745this polynomial from the full spectra.
746
747\subsubsection{Auto-baselining}
748
749\index{Auto-baseline}The function \cmd{auto\_poly\_baseline} can be used to automatically
750baseline your data without having to specify channel ranges for the
751line free data. It automatically figures out the line-free emission
752and fits a polynomial baseline to that data. The user can use masks to
753fix the range of channels or velocity range for the fit as well as
754mark the band edge as invalid.
755
756Simple example
757
758\begin{verbatim}
759 ASAP> scans.auto_poly_baseline(order=2,threshold=5)
760\end{verbatim}
761
762\cmd{order} is the polynomial order for the fit. \cmd{threshold} is
763the SNR threshold to use to deliminate line emission from
764signal. Generally the value of threshold is not too critical, however
765making this too large will compromise the fit (as it will include
766strong line features) and making it too small will mean it cannot find
767enough line free channels.
768
769
770Other examples:
771
772\begin{verbatim}
773 # Don't try and fit the edge of the bandpass which is noisier
774 ASAP> scans.auto_poly_baseline(edge=(500,450),order=3,threshold=3)
775
776 # Only fit a given region around the line
777 ASAP> scans.set_unit('km/s')
778 ASAP> msk = scans.create_mask([-60,-20])
779 ASAP> scans.auto_poly_baseline(mask=msk,order=3,threshold=3)
780
781\end{verbatim}
782
783\subsection{Average the polarisations}
784
785\index{average\_pol}If you are just interested in the highest SNR for total intensity you
786will want to average the parallel polarisations together.
787
788\begin{verbatim}
789 ASAP> scans.average_pol()
790\end{verbatim}
791
792\subsection{Calibration}
793
794\index{Calibration}For most uses, calibration happens transparently as the input data
795contains the Tsys measurements taken during observations. The nominal
796``Tsys'' values may be in Kelvin or Jansky. The user may wish to
797supply a Tsys correction or apply gain-elevation and opacity
798corrections.
799
800\subsubsection{Brightness Units}
801
802\index{Brightness Units}RPFITS files do not contain any information as
803to whether the telescope calibration was in units of Kelvin or
804Janskys. On reading the data a default value is set depending on the
805telescope and frequency of observation. If this default is incorrect
806(you can see it in the listing from the \cmd{summary} function) the
807user can either override this value on reading the data or later.
808E.g:
809
810\begin{verbatim}
811 ASAP> scans = scantable('2004-11-23_1841-P484.rpf', unit='Jy')
812 # Or in two steps
813 ASAP> scans = scantable('2004-11-23_1841-P484.rpf')
814 ASAP> scans.set_fluxunit('Jy)
815\end{verbatim}
816
817\subsubsection{Tsys scaling}
818
819\index{Tsys scaling}Sometime the nominal Tsys measurement at the
820telescope is wrong due to an incorrect noise diode calibration. This
821can easily be corrected for with the scale function. By default,
822\cmd{scale} only scans the spectra and not the corresponding Tsys.
823
824\begin{verbatim}
825 ASAP> scans.scale(1.05, tsys=True)
826\end{verbatim}
827
828\subsubsection{Unit Conversion}
829
830\index{Unit conversion}To convert measurements in Kelvin to Jy (and
831vice versa) the global function \cmd{convert\_flux} is needed. This
832converts and scales the data from K to Jy or vice-versa depending on
833what the current brightness unit is set to. The function knows the
834basic parameters for some frequencies and telescopes, but the user may
835need to supply the aperture efficiency, telescope diameter or the Jy/K
836factor.
837
838\begin{verbatim}
839 ASAP> scans.convert_flux() # If efficency known
840 ASAP> scans.convert_flux(eta=0.48) # If telescope diameter known
841 ASAP> scans.convert_flux(eta=0.48,d=35) # Unknown telescope
842 ASAP> scans.convert_flux(jypk=15) # Alternative
843\end{verbatim}
844
845\subsubsection{Gain-Elevation and Opacity Corrections}
846\label{sec:gainel}
847
848\index{Gain-elevation}As higher frequencies (particularly $>$20~GHz)
849it is important to make corrections for atmospheric opacity and
850gain-elevation effects.
851
852Note that currently the elevation is not written correctly into
853Tidbinbilla rpfits files. This means that gain-elevation and opacity
854corrections will not work unless these get recalculated.
855
856\begin{verbatim}
857 ASAP> scans.recalc_azel() # recalculate az/el based on pointing
858\end{verbatim}
859
860Gain-elevation curves for some telescopes and frequencies are known to
861ASAP (currently only for Tidbinbilla at 20~GHz). In these cases
862making gain-corrections is simple. If the gain curve for your data is
863not known, the user can supply either a gain polynomial or text file
864tabulating gain factors at a range of elevations (see \cmd{help
865scantable.gain\_el}).
866
867Examples:
868
869\begin{verbatim}
870 ASAP> scans.gain_el() # If gain table known
871 ASAP> scans.gain_el(poly=[3.58788e-1,2.87243e-2,-3.219093e-4])
872\end{verbatim}
873
874\index{Opacity}Opacity corrections can be made with the global
875function \cmd{opacity}. This should work on all telescopes as long as
876a measurement of the opacity factor was made during the observation.
877
878\begin{verbatim}
879 ASAP> scans.opacity(0.083)
880\end{verbatim}
881
882Note that at 3~mm Mopra uses a paddle wheel for Tsys calibration,
883which takes opacity effects into account (to first order). ASAP
884opacity corrections should not be used for Mopra 3-mm data.
885
886\subsection{Frequency Frame Alignment}
887\label{sec:freqalign}
888
889\index{Frequency alignment}\index{Velicity alignment}When time
890averaging a series of scans together, it is possible that the velocity
891scales are not exactly aligned. This may be for many reasons such as
892not Doppler tracking the observations, errors in the Doppler tracking
893etc. This mostly affects very long integrations or integrations
894averaged together from different days. Before averaging such data
895together, they should be frequency aligned using \cmd{freq\_align}.
896
897E.g.:
898
899\begin{verbatim}
900 ASAP> scans.freq_align()
901 ASAP> av = average_time(scans)
902\end{verbatim}
903
904{\em A Global freq\_align command will be made eventually}
905
906To average together data taken on different days, which are in
907different scantables, each scantable must aligned to a common
908reference time then the scantables averaged. The simplest way of
909doing this is to allow ASAP to choose the reference time for the first
910scantable then using this time for the subsequent scantables.
911
912\begin{verbatim}
913 ASAP> scans1.freq_align() # Copy the refeference Epoch from the output
914 ASAP> scans2.freq_align(reftime='2004/11/23/18:43:35')
915 ASAP> scans3.freq_align(reftime='2004/11/23/18:43:35')
916 ASAP> av = average_time(scans1, scans2, scans3)
917\end{verbatim}
918
919\section{Scantable manipulation}
920
921\index{Scantable!manipulation}While it is very useful to have many
922independent sources within one scantable, it is often inconvenient for
923data processing. The \cmd{get\_scan} function can be used to create a
924new scantable with a selection of scans from a scantable. The
925selection can either be on the source name, with simple wildcard
926matching or set of scan ids. Internally this uses the selector object,
927so for more complicated selection the selector should be used directly
928instead.
929
930For example:
931
932\begin{verbatim}
933 ASAP> ss = scans.get_scan(10) # Get the 11th scan (zero based)
934 ASAP> ss = scans.get_scan(range(10)) # Get the first 10 scans
935 ASAP> ss = scans.get_scan(range(10,20)) # Get the next 10 scans
936 ASAP> ss = scans.get_scan([2,4,6,8,10]) # Get a selection of scans
937
938 ASAP> ss = scans.get_scan('345p407') # Get a specific source
939 ASAP> ss = scans.get_scan('345*') # Get a few sources
940
941 ASAP> r = scans.get_scan('*_R') # Get all reference sources (Parkes/Mopra)
942 ASAP> s = scans.get_scan('*_S') # Get all program sources (Parkes/Mopra)
943 ASAP> r = scans.get_scan('*[ew]') # Get all reference sources (Tid)
944 ASAP> s = scans.get_scan('*[^ew]') # Get all program sources (Tid)
945
946\end{verbatim}
947
948To copy a scantable the following does not work:
949
950\begin{verbatim}
951 ASAP> ss = scans
952\end{verbatim}
953
954as this just creates a reference to the original scantable. Any
955changes made to \cmd{ss} are also seen in \cmd{scans}. To duplicate a
956scantable, use the copy function.
957
958\begin{verbatim}
959 ASAP> ss = scans.copy()
960\end{verbatim}
961
962\section{Data Output}
963
964\index{Scantable!save}\index{Saving data}ASAP can save scantables in a
965variety of formats, suitable for reading into other packages. The
966formats are:
967
968\begin{itemize}
969\item[ASAP] This is the internal format used for ASAP. It is the only
970 format that allows the user to restore the data, fits etc. without
971 loosing any information. As mentioned before, the ASAP scantable is
972 an AIPS++ Table (a memory-based table). This function just converts
973 it to a disk-based Table. You can the access that Table with the
974 AIPS++ Table browser or any other AIPS++ tool.
975
976\item[SDFITS] The Single Dish FITS format. This format was designed to
977 for interchange between packages, but few packages actually can read
978 it.
979
980\item[FITS] This uses simple ``image'' fits to save the data, each row
981 being written to a separate fits file. This format is suitable for
982 importing the data into CLASS.
983
984\item[ASCII] A simple text based format suitable for the user to
985processing using Perl or, Python, gnuplot etc.
986
987\item[MS2] Saves the data in an aips++ MeasurementSet V2 format.
988You can also access this with the Table browser and other AIPS++
989tools.
990
991\end{itemize}
992
993The default output format can be set in the users {\tt .asaprc} file.
994Typical usages are:
995
996\begin{verbatim}
997 ASAP> scans.save('myscans') # Save in default format
998 ASAP> scans.save('myscans', 'FITS') # Save as FITS for exporting into CLASS
999 ASAP> scans.save('myscans', overwrite=True) # Overwrite an existing file
1000\end{verbatim}
1001
1002
1003\section{Plotter}
1004
1005\index{Plotter}Scantable spectra can be plotted at any time. An asapplotter object is
1006used for plotting, meaning multiple plot windows can be active at the
1007same time. On start up a default asapplotter object is created called
1008``plotter''. This would normally be used for standard plotting.
1009
1010The plotter, optionally, will run in a multipanel mode and contain
1011multiple plots per panel. The user must tell the plotter how they want
1012the data distributed. This is done using the set\_mode function. The
1013default can be set in the users {\tt .asaprc} file. The units (and frame
1014etc) of the abscissa will be whatever has previously been set by
1015\cmd{set\_unit}, \cmd{set\_freqframe} etc.
1016
1017Typical plotter usage would be:
1018
1019\begin{verbatim}
1020 ASAP> scans.set_unit('km/s')
1021 ASAP> plotter.set_mode(stacking='p',panelling='t')
1022 ASAP> plotter.plot(scans)
1023\end{verbatim}
1024
1025This will plot multiple polarisation within each plot panel and each
1026scan row in a separate panel.
1027
1028Other possibilities include:
1029
1030\begin{verbatim}
1031 # Plot multiple IFs per panel
1032 ASAP> plotter.set_mode(stacking='i',panelling='t')
1033
1034 # Plot multiple beams per panel
1035 ASAP> plotter.set_mode(stacking='b',panelling='t')
1036
1037 # Plot one IF per panel, time stacked
1038 ASAP> plotter.set_mode('t', 'i')
1039
1040 # Plot each scan in a seperate panel
1041 ASAP> plotter.set_mode('t', 's')
1042
1043\end{verbatim}
1044
1045\subsection{Plot Selection}
1046\label{sec:plotter_cursor}
1047
1048\index{Plotter!selection}The plotter can plot up to 25 panels and
1049stacked spectra per panel. If you have data larger than this (or for
1050your own sanity) you need to select a subset of this data. This is
1051particularly true for multibeam or multi IF data. The selector object
1052should be used for this purpose. Selection can either be applied to
1053the scantable or directly to the plotter, the end result is the same.
1054You don't have to reset the scantable slection though, if you set
1055the selection on the plotter.
1056
1057Examples:
1058
1059\begin{verbatim}
1060 ASAP> selection = selector()
1061 # Select second IF
1062 ASAP> selection.set_ifs(1)
1063 ASAP> plotter.set_selection(selection)
1064
1065 # Select first 4 beams
1066 ASAP> selection.set_beams([0,1,2,3])
1067 ASAP> plotter.set_selection(selection)
1068
1069 # Select a few scans
1070 ASAP> selection.set_scans([2,4,6,10])
1071 ASAP> plotter.set_selection(selection)
1072
1073 # Multiple selection
1074 ASAP> selection.set_ifs(1)
1075 ASAP> selection.set_scans([2,4,6,10])
1076 ASAP> plotter.set_selection(selection)
1077
1078\end{verbatim}
1079
1080\subsection{Plot Control}
1081
1082\index{Plotter!control}The plotter window has a row of buttons on the
1083lower left. These can be used to control the plotter (mostly for
1084zooming the individual plots). From left to right:
1085
1086\begin{itemize}
1087
1088\item[Home] This will unzoom the plots to the original zoom factor
1089
1090\item[Plot history] (left and right arrow). The plotter keeps a
1091history of zoom settings. The left arrow sets the plot zoom to the
1092previous value. The right arrow returns back again. This allows you,
1093for example, to zoom in on one feature then return the plot to how it
1094was previously.
1095
1096\item[Pan] (The Cross) This sets the cursor to pan, or scroll mode
1097 allowing you to shift the plot within the window. Useful when
1098 zoomed in on a feature.
1099
1100\item[Zoom] (the letter with the magnifying glass) lets you draw a
1101 rectangle around a region of interest then zooms in on that
1102 region. Use the plot history to unzoom again.
1103
1104\item[Adjust] (rectangle with 4 arrows) adjust subplot paramaters
1105 (space at edge of plots)
1106
1107\item[Save] (floppy disk). Save the plot as a postscript or .png file
1108
1109You can also type ``g'' in the plot window to toggle on and off grid
1110lines. Typing 'l' turns on and off logarithmic Y-axis.
1111
1112\end{itemize}
1113
1114\subsection{Other control}
1115
1116The plotter has a number of functions to describe the layout of the
1117plot. These include \cmd{set\_legend}, \cmd{set\_layout} and \cmd{set\_title}.
1118
1119To set the exact velocity or channel range to be plotted use the
1120\cmd{set\_range} function. To reset to the default value, call
1121\cmd{set\_range} with no arguments. E.g.
1122
1123\begin{verbatim}
1124 ASAP> scans.set_unit('km/s')
1125 ASAP> plotter.plot(scans)
1126 ASAP> plotter.set_range(-150,-50)
1127 ASAP> plotter.set_range() # To reset
1128\end{verbatim}
1129
1130Both the range of the ``x'' and ``y'' axis can be set at once, if desired:
1131
1132\begin{verbatim}
1133 ASAP> plotter.set_range(-10,30,-1,6.6)
1134\end{verbatim}
1135
1136To save a hardcopy of the current plot, use the save function, e.g.
1137
1138\begin{verbatim}
1139 ASAP> plotter.save('myplot.ps')
1140 ASAP> plotter.save('myplot.png', dpi=80)
1141\end{verbatim}
1142
1143\section{Fitting}
1144
1145\index{Fitting}Currently multicomponent Gaussian function is
1146available. This is done by creating a fitting object, setting up the
1147fit and actually fitting the data. Fitting can either be done on a
1148single scantable selection or on an entire scantable using the
1149\cmd{auto\_fit} function. If single value fitting is used, and the
1150current selection includes multiple spectra (beams, IFs, scans etc)
1151then the first spectrum in the scantable will be used for fitting.
1152
1153\begin{verbatim}
1154 ASAP> f = fitter()
1155 ASAP> f.set_function(gauss=2) # Fit two Gaussians
1156 ASAP> f.set_scan(scans)
1157 ASAP> selection = selector()
1158 ASAP> selection.set_polarisations(1) # Fit the second polarisation
1159 ASAP> scans.set_selection(selection)
1160 ASAP> scans.set_unit('km/s') # Make fit in velocity units
1161 ASAP> f.fit(1) # Run the fit on the second row in the table
1162 ASAP> f.plot() # Show fit in a plot window
1163 ASAP> f.get_parameters() # Return the fit paramaters
1164\end{verbatim}
1165
1166This auto-guesses the initial values of the fit and works well for data
1167without extra confusing features. Note that the fit is performed in
1168whatever unit the abscissa is set to.
1169
1170If you want to confine the fitting to a smaller range (e.g. to avoid
1171band edge effects or RFI you must set a mask.
1172
1173\begin{verbatim}
1174 ASAP> f = fitter()
1175 ASAP> f.set_function(gauss=2)
1176 ASAP> scans.set_unit('km/s') # Set the mask in channel units
1177 ASAP> msk = s.create_mask([1800,2200])
1178 ASAP> scans.set_unit('km/s') # Make fit in velocity units
1179 ASAP> f.set_scan(s,msk)
1180 ASAP> f.fit()
1181 ASAP> f.plot()
1182 ASAP> f.get_parameters()
1183\end{verbatim}
1184
1185If you wish, the initial parameter guesses can be specified and
1186specific parameters can be fixed:
1187
1188\begin{verbatim}
1189 ASAP> f = fitter()
1190 ASAP> f.set_function(gauss=2)
1191 ASAP> f.set_scan(s,msk)
1192 ASAP> f.fit() # Fit using auto-estimates
1193 # Set Peak, centre and fwhm for the second gaussian.
1194 # Force the centre to be fixed
1195 ASAP> f.set_gauss_parameters(0.4,450,150,0,1,0,component=1)
1196 ASAP> f.fit() # Re-run the fit
1197\end{verbatim}
1198
1199The fitter \cmd{plot} function has a number of options to either view
1200the fit residuals or the individual components (by default it plots
1201the sum of the model components).
1202
1203Examples:
1204
1205\begin{verbatim}
1206 # Plot the residual
1207 ASAP> f.plot(residual=True)
1208
1209 # Plot the first 2 componentsa
1210 ASAP> f.plot(components=[0,1])
1211
1212 # Plot the first and third component plus the model sum
1213 ASAP> f.plot(components=[-1,0,2]) # -1 means the compoment sum
1214\end{verbatim}
1215
1216\subsection{Fit saving}
1217
1218\index{Fitter!Fit saving}One you are happy with your fit, it is
1219possible to store it as part of the scantable.
1220
1221\begin{verbatim}
1222 ASAP> f.storefit()
1223\end{verbatim}
1224
1225This will be saved to disk with the data, if the ``ASAP'' file format
1226is selected. Multiple fits to the same data can be stored in the
1227scantable.
1228
1229The scantable function \cmd{get\_fit} can be used to retrieve the
1230stored fits. Currently the fit parameters are just printed to the
1231screen.
1232
1233\begin{verbatim}
1234 ASAP> scans.get_fit(4) # Print fits for row 4
1235\end{verbatim}
1236
1237\section{Polarisation}
1238
1239\index{Polarisation}Currently ASAP only supports polarmetric analysis
1240on linearly polarised feeds and the cross polarisation products
1241measured. Other cases will be added on an as needed basis.
1242
1243Conversions of linears to Stokes or Circular polarisations are done
1244``on-the-fly''. Leakage cannot be corrected for nor are there routines
1245to calibrate position angle offsets.
1246
1247\subsection{Simple Calibration}
1248
1249\index{Polarisation!calibration}It is possible that there is a phase
1250offset between polarisation which will effect the phase of the cross
1251polarisation correlation, and so give rise to spurious
1252polarisation. \cmd{rotate\_xyphase} can be used to correct for this
1253error. At this point, the user must know how to determine the size of
1254the phase offset themselves.
1255
1256\begin{verbatim}
1257 ASAP> scans.rotate_xyphase(10.5) # Degrees
1258\end{verbatim}
1259
1260Note that if this function is run twice, the sum of the two values is
1261applied because it is done in-situ.
1262
1263A correction for the receiver parallactic angle may need to be made,
1264generally because of how it is mounted. Use \cmd{rotate\_linpolphase}
1265to correct the position angle. Running this function twice results in
1266the sum of the corrections being applied because it is applied
1267in-situ.
1268
1269\begin{verbatim}
1270 ASAP> scans.rotate_linpolphase(-45) # Degrees; correct for receiver mounting
1271\end{verbatim}
1272
1273If the sign of the complex correlation is wrong (this can happen
1274depending on the correlator configuration), use \cmd{invert\_phase} to
1275change take the complex conjugate of the complex correlation
1276term. This is always performed in-situ.
1277
1278\begin{verbatim}
1279 ASAP> scans.invert_phase()
1280\end{verbatim}
1281
1282Depending on how the correlator is configured, ``BA'' may be
1283correlated insead of ``AB''. Use \cmd{swap\_linears} to correct for
1284this problem:
1285
1286\begin{verbatim}
1287 ASAP> scans.swap_linears()
1288\end{verbatim}
1289
1290\subsection{Conversion}
1291\label{sec:polconv}
1292Data can be converted between linear and circular polarisations and stokes.
1293
1294\begin{verbatim}
1295 ASAP> stokescans = linearscans.convert_pol("stokes")
1296\end{verbatim}
1297
1298
1299\subsection{Plotting}
1300\label{sec:polplot}
1301
1302\index{Polarisation!plotting}To plot Stokes values, a selector object
1303must be created and the set\_polarisation function used to select the
1304desired polarisation products.
1305
1306The values which can be plotted include a selection of [I,Q,U,V], [I,
1307Plinear, Pangle, V], [RR, LL] or [XX, YY, Real(XY),
1308Imaginary(XY)]. (Plinear and Pangle are the percentage and position
1309angle of linear polarisation).
1310
1311Example:
1312
1313\begin{verbatim}
1314 ASAP> selection = selector()
1315
1316 ASAP> selection.set_polarisations(``I Q U V'')
1317 ASAP plotter.set_selection(selection); # Select I, Q, U \& V
1318
1319 ASAP> selection.set_polarisations(``I Q'')
1320 ASAP plotter.set_selection(selection); # Select just I \& Q
1321
1322 ASAP> selection.set_polarisations(``RR LL'')
1323 ASAP plotter.set_selection(selection); # Select just RR \& LL
1324
1325 ASAP> selection.set_polarisations(``XX YY'')
1326 ASAP plotter.set_selection(selection); # Select linears
1327
1328 ASAP> selection.set_polarisations(``I Plinear'')
1329 ASAP plotter.set_selection(selection); # Fractional linear
1330
1331 ASAP> selection.set_polarisations(``Pangle'')
1332 ASAP plotter.set_selection(selection); # Position angle
1333
1334\end{verbatim}
1335
1336Scan, beam and IF selection are also available in the selector object as
1337describe in section~\ref{sec:selection}.
1338
1339\subsection{Saving}
1340
1341\index{Polarisation!saving}When saving data using the \cmd{save}
1342function, the \cmd{stokes} argument can be used to save the data as
1343Stoke values when saving in FITS format.
1344
1345Example:
1346
1347\begin{verbatim}
1348 ASAP> scans.save('myscan.sdfits', 'SDFITS', stokes=True)
1349\end{verbatim}
1350
1351\section{Scantable Mathematics}
1352
1353\index{Scantable!maths}It is possible to to simple mathematics
1354directly on scantables from the command line using the \cmd{+, -, *,
1355/} operators as well as their cousins \cmd{+=, -= *=, /=}. This works
1356between a scantable and a float. (Note that it does
1357not work for integers).
1358
1359{\em Currently mathematics between two scantables is not available }
1360
1361% ASAP> sum = scan1+scan2
1362\begin{verbatim}
1363 ASAP> scan2 = scan1+2.0
1364 ASAP> scan *= 1.05
1365\end{verbatim}
1366
1367\section{Scripting}
1368
1369\index{Scripting}Because asap is based on python, it easy for the user
1370write their own scripts and functions to process data. This is highly
1371recommended as most processing of user data could then be done in a
1372couple of steps using a few simple user defined functions. A Python
1373primer is beyond the scope of this userguide. See the asap home pages
1374for a scripting tutorial or the main python website for comprehensive
1375documentation.
1376
1377\hspace{1cm} http://www.atnf.csiro.au/computing/software/asap/tutorials
1378
1379\hspace{1cm} http://www.python.org/doc/Introduction.html
1380
1381\subsection{Running scripts}
1382
1383The asap global function \cmd{execfile} reads the named text file and
1384executes the contained python code. This file can either contain
1385function definitions which will be used in subsequent processing or
1386just a set of commands to process a specific dataset.
1387
1388\subsection{asapuserfuncs.py}
1389
1390The file $\sim$/.asap/asapuserfuncs.py is automatically read in when
1391asap is started. The user can use this to define a set of user
1392functions which are automatically available each time asap is
1393used. The \cmd{execfile} function can be called from within this file.
1394
1395\section{Worked examples}
1396
1397In the following section a few examples of end-to-end processing of
1398some data in asap are given.
1399
1400\subsection{Mopra}
1401\index{Mopra}
1402
1403The following example is of some dual polarisation, position switched
1404data from Mopra. The source has been observed mulitple times split
1405into a number of seperate rpfits files. To make the processing easier,
1406the first step is to \cmd{cat} the seeprate rpfits files together and
1407load as a whole (future versions of asap will make this unnecessary).
1408
1409
1410\begin{verbatim}
1411# get a list of the individual rpfits files in the current directory
1412myfiles = list_files()
1413
1414# Load the data into a scantable
1415data = scantable(myfiles)
1416print data
1417
1418# Form the quotient spectra
1419q = data.auto_quotient()
1420print q
1421
1422# Look at the spectra
1423plotter.plot(q)
1424
1425# Set unit and reference frame
1426q.set_unit('km/s')
1427q.set_freqframe('LSRK')
1428
1429# Average all scans in time, aligning in velocity
1430av = q.average_time(align=True)
1431plotter.plot(av)
1432
1433# Remove the baseline
1434msk = av.create_mask([100,130],[160,200])
1435av.poly_baseline(msk,2)
1436
1437# Average the two polarisations together
1438iav = av.average_pol()
1439print iav
1440plotter.plot(iav)
1441
1442# Set a sensible velocity range on the plot
1443plotter.set_range(85,200)
1444
1445# Smooth the data a little
1446av.smooth('gauss',4)
1447plotter.plot()
1448
1449# Fit a guassian to the emission
1450f = fitter()
1451f.set_function(gauss=1)
1452f.set_scan(av)
1453f.fit()
1454
1455# View the fit
1456f.plot()
1457
1458# Get the fit parameters
1459f.get_parameters()
1460
1461\end{verbatim}
1462
1463
1464\subsection{Parkes Polarimetry}
1465
1466\index{Parkes}\index{Polarisation}The following example is processing
1467of some Parkes polarmetric observations of OH masers at
14681.6~GHz. Because digital filters where used in the backend, the
1469baselines are stable enough not to require a quotient spectra. The
14704~MHz bandwidth is wide enough to observe both the 1665 and 1667~MHz
1471OH maser transitions. Each source was observed once for about 10
1472minutes. Tsys information was not written to the rpfits file (a
1473nominal 25K values was used), so the amplitudes need to be adjusted
1474based on a separate log file. A simple user function is used to
1475simplify this, contained in a file called mypol.py:
1476
1477\begin{verbatim}
1478def xyscale(data,xtsys=1.0,ytsys=1.0,nomtsys=25.0) :
1479
1480 selection = selector()
1481 selection.set_polarisations(0)
1482 data.set_selection(selection)
1483 data.scale(xtsys/nomtsys)
1484
1485 selection.set_polarisations(1)
1486 data.set_selection(selection)
1487 data.scale(ytsys/nomtsys)
1488
1489 selection.set_polarisations(0)
1490 data.set_selection(selection)
1491 data.scale((xtsys+ytsys)/(2*nomtsys))
1492
1493 selection.set_polarisations(0)
1494 data.set_selection(selection)
1495 data.scale((xtsys+ytsys)/(2*nomtsys))
1496\end{verbatim}
1497
1498The typical asap session would be
1499
1500\begin{verbatim}
1501
1502# Remind ourself the name of the rpfits files
1503ls
1504
1505# Load data from an rpfits file
1506d1665 = scantable('2005-10-27_0154-P484.rpf')
1507
1508# Check what we have just loaded
1509d1665.summary()
1510
1511# View the data in velocity
1512d1665.set_unit('km/s')
1513d1665.set_freqframe('LSRK')
1514
1515# Correct for the known phase offset in the crosspol data
1516d1665.rotate_xyphase(-4)
1517
1518# Create a copy of the data and set the rest frequency to the 1667 MHz
1519# transition
1520d1667 = d1665.copy()
1521d1667.set_restfreqs([1667.3590], 'MHz')
1522d1667.summary()
1523
1524# Copy out the scan we wish to process
1525g351_5 = d1665.get_scan('351p160')
1526g351_7 = d1667.get_scan('351p160')
1527
1528# Baseline both
1529msk = g351_5.create_mask([-30,-25],[-5,0])
1530g351_5.poly_baseline(msk,order=1)
1531msk = g351_7.create_mask([-30,-25],[-5,0])
1532g351_7.poly_baseline(msk,order=1)
1533
1534
1535# Plot the data. The plotter can only plot a single scantable
1536# So we must merge the two tables first
1537
1538plotscans = merge(g351_5, g351_7)
1539
1540plotter.plot(plotscans) # Only shows one panel
1541
1542# Tell the plotter to stack polarisation and panel scans
1543plotter.set_mode('p','s')
1544
1545# Correct for the Tsys using our predefined function
1546execfile('mypol.py') # Read in the function xyscale
1547xyscale(g351_5,23.2,22.7) # Execute it on the data
1548xyscale(g351_7,23.2,22.7)
1549
1550# Only plot the velocity range of interest
1551plotter.set_range(-30,10)
1552
1553# Update the plot with the baselined data
1554plotter.plot()
1555
1556# Look at the various polarisation products
1557selection = selector()
1558selection.set_polarisations(``RR LL'')
1559plotter.set_selection(selection)
1560selection.set_polarisations(``I Plinear'')
1561plotter.set_selection(selection)
1562selection.set_polarisations(``I Q U V'')
1563plotter.set_selection(selection)
1564
1565# Save the plot as postscript
1566plotter.save('g351_stokes.ps')
1567
1568# Save the process spectra
1569plotscans.save('g351.asap')
1570
1571\end{verbatim}
1572
1573\subsection{Tidbinbilla}
1574
1575\index{Tidbinbilla}The following example is processing of some
1576Tidbinbilla observations of NH$_3$ at 12~mm. Tidbinbilla has (at the
1577time of observations) a single polarisation, but can process two IFs
1578simultaneously. In the example, the first half of the observation was
1579observing the (1,1) and (2,2) transitions simultaneously). The second
1580half observed only the (4,4) transition due to bandwidth
1581limitations. The data is position switched, observing first an
1582reference to the west, then the source twice and finally reference to
1583the east.
1584
1585\begin{verbatim}
1586
1587# Load the rpfits file and inspect
1588d = scantable('2003-03-16_082048_t0002.rpf')
1589print d
1590
1591# Make the quotient spectra
1592q = d.auto_quotient()
1593print q
1594
1595del d
1596
1597# Plot/select in velocity
1598q.set_freqframe('LSRK')
1599q.set_unit('km/s')
1600
1601# Correct for gain/el effects
1602
1603q.recalc_azel() # Tid does not write the elevation
1604q.gain_el()
1605q.opacity(0.05)
1606
1607# Seperate data from the (1,1)&(2,2) and (4,4) transitions
1608g1 = q.get_scan(range(6)) # scans 0..5
1609g2 = q.get_scan(range(6,12)) # scans 6..11
1610
1611# Align data in velocity
1612g1.freq_align()
1613g2.freq_align()
1614
1615# Average individual scans
1616a1 = g1.average_time()
1617a2 = g2.average_time()
1618
1619# Rpfits file only contains a single rest frequency. Set both
1620a1.set_restfreqs([23694.4700e6,23722.6336e6])
1621
1622plotter.plot(a1)
1623plotter.set_mode('i','t')
1624
1625a1.auto_poly_baseline()
1626
1627plotter.plot()
1628
1629a1.smooth('gauss',5)
1630plotter.plot()
1631
1632
1633\end{verbatim}
1634
1635\newpage
1636
1637\section{Appendix}
1638
1639\subsection{Function Summary}
1640
1641\index{Functions!summary}%
1642\begin{verbatim}
1643ASAP> commands()
1644 [The scan container]
1645 scantable - a container for integrations/scans
1646 (can open asap/rpfits/sdfits and ms files)
1647 copy - returns a copy of a scan
1648 get_scan - gets a specific scan out of a scantable
1649 (by name or number)
1650 set_selection - set a new subselection of the data
1651 get_selection - get the current selection object
1652 summary - print info about the scantable contents
1653 stats - get specified statistic of the spectra in
1654 the scantable
1655 stddev - get the standard deviation of the spectra
1656 in the scantable
1657 get_tsys - get the TSys
1658 get_time - get the timestamps of the integrations
1659 get_sourcename - get the source names of the scans
1660 get_azimuth - get the azimuth of the scans
1661 get_elevation - get the elevation of the scans
1662 get_parangle - get the parallactic angle of the scans
1663 get_unit - get the current unit
1664 set_unit - set the abcissa unit to be used from this
1665 point on
1666 get_abcissa - get the abcissa values and name for a given
1667 row (time)
1668 set_freqframe - set the frame info for the Spectral Axis
1669 (e.g. 'LSRK')
1670 set_doppler - set the doppler to be used from this point on
1671 set_dirframe - set the frame for the direction on the sky
1672 set_instrument - set the instrument name
1673 get_fluxunit - get the brightness flux unit
1674 set_fluxunit - set the brightness flux unit
1675 create_mask - return an mask in the current unit
1676 for the given region. The specified regions
1677 are NOT masked
1678 get_restfreqs - get the current list of rest frequencies
1679 set_restfreqs - set a list of rest frequencies
1680 flag_spectrum - flag data
1681 save - save the scantable to disk as either 'ASAP'
1682 or 'SDFITS'
1683 nbeam,nif,nchan,npol - the number of beams/IFs/Pols/Chans
1684 nscan - the number of scans in the scantable
1685 nrow - te number of spectra in the scantable
1686 history - print the history of the scantable
1687 get_fit - get a fit which has been stored witnh the data
1688 average_time - return the (weighted) time average of a scan
1689 or a list of scans
1690 average_pol - average the polarisations together.
1691 The dimension won't be reduced and
1692 all polarisations will contain the
1693 averaged spectrum.
1694 convert_pol - convert to a different polarisation type
1695 auto_quotient - return the on/off quotient with
1696 automatic detection of the on/off scans
1697 (matched pairs and 1 off - n on)
1698 scale - return a scan scaled by a given factor
1699 add - return a scan with given value added
1700 bin - return a scan with binned channels
1701 resample - return a scan with resampled channels
1702 smooth - return the spectrally smoothed scan
1703 poly_baseline - fit a polynomial baseline to all Beams/IFs/Pols
1704 auto_poly_baseline - automatically fit a polynomial baseline
1705 recalc_azel - recalculate azimuth and elevation based on
1706 the pointing
1707 gain_el - apply gain-elevation correction
1708 opacity - apply opacity correction
1709 convert_flux - convert to and from Jy and Kelvin brightness
1710 units
1711 freq_align - align spectra in frequency frame
1712 rotate_xyphase - rotate XY phase of cross correlation
1713 rotate_linpolphase - rotate the phase of the complex
1714 polarization O=Q+iU correlation
1715 freq_switch - perform frequency switching on the data
1716 stats - Determine the specified statistic, e.g. 'min'
1717 'max', 'rms' etc.
1718 stddev - Determine the standard deviation of the current
1719 beam/if/pol
1720
1721 [Math] Mainly functions which operate on more than one scantable
1722
1723 average_time - return the (weighted) time average
1724 of a list of scans
1725 quotient - return the on/off quotient
1726 simple_math - simple mathematical operations on two scantables,
1727 'add', 'sub', 'mul', 'div'
1728 [Fitting]
1729 fitter
1730 auto_fit - return a scan where the function is
1731 applied to all Beams/IFs/Pols.
1732 commit - return a new scan where the fits have been
1733 commited.
1734 fit - execute the actual fitting process
1735 store_fit - store the fit parameters in the data (scantable)
1736 get_chi2 - get the Chi^2
1737 set_scan - set the scantable to be fit
1738 set_function - set the fitting function
1739 set_parameters - set the parameters for the function(s), and
1740 set if they should be held fixed during fitting
1741 set_gauss_parameters - same as above but specialised for individual
1742 gaussian components
1743 get_parameters - get the fitted parameters
1744 plot - plot the resulting fit and/or components and
1745 residual
1746 [Plotter]
1747 asapplotter - a plotter for asap, default plotter is
1748 called 'plotter'
1749 plot - plot a scantable
1750 save - save the plot to a file ('png' ,'ps' or 'eps')
1751 set_mode - set the state of the plotter, i.e.
1752 what is to be plotted 'colour stacked'
1753 and what 'panelled'
1754 set_selection - only plot a selected part of the data
1755 set_range - set a 'zoom' window [xmin,xmax,ymin,ymax]
1756 set_legend - specify user labels for the legend indeces
1757 set_title - specify user labels for the panel indeces
1758 set_abcissa - specify a user label for the abcissa
1759 set_ordinate - specify a user label for the ordinate
1760 set_layout - specify the multi-panel layout (rows,cols)
1761 set_colors - specify a set of colours to use
1762 set_linestyles - specify a set of linestyles to use if only
1763 using one color
1764 set_mask - set a plotting mask for a specific polarization
1765
1766 [Reading files]
1767 reader - access rpfits/sdfits files
1768 open - attach reader to a file
1769 close - detach reader from file
1770 read - read in integrations
1771 summary - list info about all integrations
1772
1773 [General]
1774 commands - this command
1775 print - print details about a variable
1776 list_scans - list all scantables created bt the user
1777 list_files - list all files readable by asap (default rpf)
1778 del - delete the given variable from memory
1779 range - create a list of values, e.g.
1780 range(3) = [0,1,2], range(2,5) = [2,3,4]
1781 help - print help for one of the listed functions
1782 execfile - execute an asap script, e.g. execfile('myscript')
1783 list_rcparameters - print out a list of possible values to be
1784 put into $HOME/.asaprc
1785 mask_and,mask_or,
1786 mask_not - boolean operations on masks created with
1787 scantable.create_mask
1788
1789 Note:
1790 How to use this with help:
1791 # function 'summary'
1792 [xxx] is just a category
1793 Every 'sub-level' in this list should be replaces by a '.' Period when
1794 using help
1795 Example:
1796 ASAP> help scantable # to get info on ths scantable
1797 ASAP> help scantable.summary # to get help on the scantable's
1798 ASAP> help average_time
1799
1800
1801\end{verbatim}
1802
1803\subsection{ASCII output format}
1804
1805\subsection{.asaprc settings}
1806\index{.asaprc}
1807\asaprc{verbose}{{\bf True}/False}{Print verbose output, good to disable in scripts}
1808
1809\asaprc{insitu}{{\bf True}/False}{Apply operations on the input
1810scantable or return new one}
1811
1812% plotting
1813
1814\asaprc{useplotter}{{\bf True}/False}{Preload a default plotter}
1815
1816\asaprc{plotter.gui}{{\bf True}/False}{Do we want a GUI or plot to a
1817file}
1818
1819\asaprc{plotter.stacking}{{\bf Pol} Beam IF Scan Time}{Default mode for
1820colour stacking}
1821
1822\asaprc{plotter.panelling}{Pol Beam IF {\bf Scan} Time}{Default mode
1823for panelling}
1824
1825\asaprc{plotter.ganged}{{\bf True}/False}{Push panels together, to
1826share axislabels}
1827
1828\asaprc{plotter.decimate}{True/{\bf False}}{Decimate the number of
1829points plotted by a factor of nchan/1024}
1830
1831% default colours/linestyles
1832%\asaprc{plotter.colours}{.}{.}
1833%\asaprc{plotter.linestyles{.}{.}
1834
1835% scantable
1836\asaprc{scantable.save}{{\bf ASAP} SDFITS FITS ASCII MS2}{Default output
1837format when saving}
1838
1839\asaprc{scantable.autoaverage}{{\bf True}/False}{Auto averaging on
1840read}
1841
1842\asaprc{scantable.freqframe}{{\bf LSRK} TOPO BARY etc}{default
1843frequency frame to set when function scantable.set\_freqframe is
1844called or the data is imported}
1845
1846\subsection{Installation}
1847
1848\index{Installation}ASAP depends on a number of third-party libraries which you must
1849have installed before attempting to build ASAP. These are:
1850
1851\begin{itemize}
1852\item AIPS++
1853\item Boost
1854\item Matplotlib
1855\item python/ipython
1856\end{itemize}
1857
1858Debian Linux is currently supported and we intend also
1859to support other popular Linux flavours, Solaris and Mac.
1860
1861Of the dependencies, AIPS++ is the most complex to install.
1862
1863\printindex
1864
1865\end{document}
1866
Note: See TracBrowser for help on using the repository browser.