source: trunk/admin/requirements1.tex@ 1790

Last change on this file since 1790 was 769, checked in by phi196, 19 years ago

ASAP release 1 requirements

  • Property svn:eol-style set to native
  • Property svn:keywords set to Author Date Id Revision
File size: 43.5 KB
Line 
1\documentclass[11pt]{article}
2\usepackage{a4}
3\usepackage{calc}
4\usepackage{smartref}
5\usepackage[dvips]{graphicx}
6
7% Adjust the page size
8\addtolength{\oddsidemargin}{-0.4in}
9\addtolength{\evensidemargin}{+0.4in}
10\addtolength{\textwidth}{+0.8in}
11
12\setlength{\parindent}{0mm}
13\setlength{\parskip}{1ex}
14
15
16\title{ATNF Single Dish Spectral Line Software Requirements }
17\author{Chris Phillips \& Malte Marquarding}
18
19\newcounter{requirement}
20\addtoreflist{requirement}
21\newcommand{\reqref}[1]{R\ref{#1}-\requirementref{#1}}
22
23\newcommand{\makenote}[1]{{\bf \tt \em#1}}
24
25\newcommand{\anitem}[2]{\smallskip \parbox[t]{2cm}{#1}\parbox[t]{\textwidth-2cm}{#2}}
26
27\newcommand{\showreqcounter}{{\stepcounter{requirement}
28 \bf R\arabic{section}.\arabic{subsection}-\arabic{requirement}}}
29
30\newcommand{\requirement}[2]{
31 \hspace*{2mm}\begin{minipage}{\textwidth-2mm}
32 \setlength{\parindent}{-2mm}
33 \showreqcounter\ #1 \\
34 \hspace*{1cm} {\em Priority #2}
35 \end{minipage}
36}
37
38\newcommand{\extendedrequirement}[2]{
39 \hspace*{2mm}\begin{minipage}{\textwidth-2mm}
40 \setlength{\parindent}{-2mm}
41 \showreqcounter\ #1
42 \hspace*{1cm} {\em Priority #2}
43 \end{minipage}
44}
45
46\newcommand{\reqeqn}[1]{\\\hspace*{1cm} $#1$}
47
48\let\oldsection\section
49\renewcommand{\section}[1]{\setcounter{requirement}{0}\oldsection{#1}}
50
51\let\oldsubsection\subsection
52\renewcommand{\subsection}[1]{\setcounter{requirement}{0}\oldsubsection{#1}}
53
54\begin{document}
55
56\maketitle
57
58\section{Introduction}
59
60The spectral line single-dish software {\tt spc} at the ATNF has
61become increasingly difficult to maintain. There also have been many
62requests for features which are not possible to implement without
63major change to this package. The interface is based on outdated
64operating systems and programming languages. The decision was made to
65replace {\tt spc} with a new package which supports existing and
66future ATNF single-dish instrumentation. A survey of users was taken
67into account creating this requirements document.
68
69\section{Scope}
70
71The software should be able to process all spectral line single-dish
72observations from ATNF telescopes (Parkes, Mopra \& Tidbinbilla). This
73includes reading the data produced by the telescope, calibration and
74reduction of the data and basic analysis of the data such as fitting
75line profiles.
76
77It has been assumed that the following processing is out of the scope
78of the new software.
79\begin{itemize}
80\item Raster or ``on-the-fly'' mapping (This is handled by
81``livedata'' and gridzilla).
82\item Very complex or specific data processing. (A route into
83 Class\footnote{Part of the GLIDAS software package, produced by
84 Institut de Radio Astronomie Millime\'trique http://www.iram.fr}
85 should be available for advanced processing).
86%%TODO%% give example
87\item Continuum data.
88\item Pulsar timing observations.
89\end{itemize}
90
91\section{Priorities}
92
93Requirements have been given a value of 0 to 3. The ``0'' priority
94functions will be implemented in a demonstrator version of the
95software (with no GUI interface). The other requirements will be
96implemented mainly depending on priority, with ``1'' the
97highest. Priority 3 and some priority 2 requirements will probably not
98get implemented in the first released version of the software.
99
100\section{User Interface}
101
102The user interface (UI) is the most important part of a single dish
103processing package, but probably the most difficult to get right. The
104UI for this software will consist of three parts.
105\begin{itemize}
106 \item A graphical user interface (GUI).
107 \item An interactive command line interface (CLI).
108 \item A scriptable interface for batch processing.
109\end{itemize}
110
111The CLI and scriptable interface may essentially be the same.
112
113The software does not {\em need} to be able to run solely from a
114``vt100'' style terminal. It can be assumed that the user is running
115the software from within a windowed (i.e. X11) environment. This will
116mean it will not necessarily be possible to run the software remotely
117over a slow network connection (e.g. internationally or from home).
118Where possible, operations on the data should be possible from all
119three aspects of the user interface.
120
121The user interface needs to be implemented so that the user can easily
122and transparently work on spectra either one at a time or by
123processing multiple spectra in parallel. This means there must be an
124easy way to select specific or multiple spectra to display or process.
125
126\subsection{Graphical User Interface}
127
128The GUI is intended to be the main interactive interface with the
129software.
130
131\smallskip
132\requirement{It should be simple, intuitive and
133uncluttered. Specifically, use of many windows simultaneously should
134be discouraged, as should hiding functionality behind layers of dialog
135boxes.}{1}
136
137\requirement{The plotting window should be a major component of the GUI
138control, not a separate isolated window.}{1}
139
140\requirement{The interface should use minimal ``always visible''
141 controls, with use of pull down menus and maybe a toolbar for
142 frequency used functions. }{1}
143
144\requirement{Keyboard shortcuts should be available.}{2}
145
146\requirement{Most user preferences (i.e. keywords in the CLI) should
147be presented in a popup, tabbed, dialog box.}{2}
148
149\requirement{When performing line profile fitting, a spreadsheet type
150window should be viewable which shows the current parameter values
151(amplitude, velocity etc) for each line fitted and allow the user to
152change these parameters or set the current value as fixed. This gui
153should stay synchronised with any CLI changes to these values.}{2}
154
155\subsection{Command Line Interface}
156
157\requirement{While the GUI should be the main interface for new users
158and for basic manipulation, some tasks can be more efficiently
159performed using a CLI. A virtual CLI could be integrated as part of
160the GUI.}{1}
161
162\requirement{The CLI should have a keyword/argument form and never
163prompt the user for specific values (the user should be able to change
164values which are retained until they wants to change them again).}{1}
165
166\requirement{The CLI should be case insensitive and accept minimum
167matching and short forms of keywords.}{2}
168
169\requirement{The user must be able to quickly and easily see from the
170command line the available routines and keywords which affect it, so
171they can see which parameters may need changing.}{1}
172
173\subsection{Scripting}
174
175\requirement{It must be possible to run the software in a scripting
176mode. This would be to process large amounts of data in a routine
177manner and also to automatically reproduce specific plots etc (So the
178scripting must have full control of the plotter). Preferably the
179scripting ``language'' and the CLI would be the same. }{1}
180
181\requirement{It would be worthwhile having a method to auto-generate
182scripts (for reduction or plotting) from current spectra history, or
183some similar method.}{3}
184
185\section{Plotter}
186
187The plotter should be fully interactive and be an integral part of the
188GUI and software interface.
189
190\requirement{It must be able to produce plots of publishable
191quality. This includes being able to specify line thickness, character
192size, colours, position of axis ticks, axis titles etc and producing
193hard copies in postscript and .png format.}{0}
194
195\requirement{It must be possible to flexibly select the data to plot
196 (e.g. Tsys vs time etc as well as plots such as amplitude vs channel
197 number or velocity). Preferably any of the header values for a
198 selection of scans could be plotted on a scatter plot (e.g. Tsys vs
199 elevation)}{2}
200
201\requirement{It must be possible to overlay multiple spectra on a
202 single plot using different colours and/or different line
203 styles. (Including multiple stokes data and multiple IFs).}{1}
204
205\requirement{It must be possible to plot either the individual
206 integrations (in either a stacked fashion, or using a new subplot
207 per integration) or some type of average of all the integrations in
208 a scan.}{2}
209
210\requirement{Multi-panelling of spectra in an n$\times$m size grid. If
211 more spectra than can fit on the plot matrix are to be plotted, then
212 it must be possible to step back and forth between the viewable
213 spectra (i.e. ``multi-page'' plots). It must be possible to quickly
214 and easily change the number of plots per page, and define the ``n''
215 and ``m'' values.}{1}
216
217\requirement{When using multi-panelling, it should be possible to
218 easily change the display to flip between a single spectra display
219 and multi-panels display (i.e. zoom into a specific sub-window).}{3}
220
221\requirement{It must be possible to interactively zoom the plot
222 (channel range selection and amplitude of the spectra etc.) This
223 includes both GUI control of the zooming as well as command line
224 control of either the zoom factor or directly specifying the zoom
225 bounds. }{1}
226
227\requirement{On a single plot, it should be possible to plot the full
228 spectrum and a zoomed copy of the data (using a different lie style)
229 to see weak features. The user must be able to specify the zoom
230 factor.}{3}
231
232\requirement{Optionally when stacking multiple spectral plots in one
233 subwindow, a (user definable) offset in the ``y'' direction should
234 be added to each subsequent spectra.}{2}
235
236\requirement{The plotter should automatically update to reflect user
237processing, either from the CLI or GUI. The user should have to option
238to turn this feature off if they so wish.}{2}
239
240\requirement{It should be possible to plot individual integrations
241(possibly from multiple scans) in a ``waterfall'' plot. This is an
242image based display, where spectral channel is along the x-axis of the
243plot, time (or integration number) along the y-axis and greyscale or
244colour represent the amplitude of spectra. Interactive zooming and
245panning of this image should be supported. }{3}
246
247\requirement{When plotting ``waterfall'' plots, it should be possible
248to interactively select regions or points and mark them as invalid
249(i.e. to remove RFI affected data). The plotter should also show the
250time/velocity of the pixel beneath the cursor.}{3}
251
252\requirement{It should be possible to export the ``waterfall'' plot
253images as a FITs file, for user specific analysis.}{3}
254
255\requirement{Line markers overlays, read from a catalogue should be
256 optionally available. This would include the full Lovas catalogue,
257 the JPL line catalogue, radio recombination lines and a simple user
258 definable catalogue. The lines would be Doppler corrected to the
259 line velocity. The user must be able to plot just a sub-section of
260 the lines in any specific catalogue (to avoid clutter).}{2}
261
262\requirement{Optionally plot fitted functions (e.g line profiles or
263 baseline fit). If multiple components (e.g. Gaussians) have been
264 fit, it should be possible to show the individual functions or the
265 sum of the components}{1}
266
267\requirement{It should be possible to plot the residual data with or
268 without subtraction of fit functions. This includes plotting the
269 spectra with or without baseline removal and the residual after
270 subtracting Gaussian fits. The default should be to plot the data
271 with baseline subtracted but profile fits not subtracted.}{2}
272
273\requirement{Basic header data (source name, molecule, observation
274 time etc) should be optionally shown, either on the plot or next to
275 it. This may either consist of a set of values, or only one or two
276 values the user specifically wants to see (source name and molecule,
277 for example). Preferably the user would be able to define where on
278 the plot the header info would appear.}{2}
279
280\requirement{Optionally, relevant data such as the current mouse
281 position should be displayed (maybe with a mode to display an
282 extended cross, horizontal or vertical line at the current cursor
283 position).}{2}
284
285\requirement{The user should be able to define simple
286 annotations. This would include text overlay and probably simple
287 graphics (lines, arrows etc).}{3}
288
289The user must be able to use the plotter window to interactively set
290initial values and ranges used for fitting functions etc. The use of
291keyboard ``shortcuts'' or other similar ``power user'' features should
292be available to save the time of experienced users.
293
294The plotter should be used to set the following values:
295
296\requirement{Range of spectral points needed for specific tasks (See
297 requirement \reqref{ref:chansel})}{1}
298
299\requirement{Initial Gaussian parameters (velocity, width, amplitude)
300for profile fitting.}{1}
301
302\requirement{Change the parameter values of existing line profile
303fits, or channel ranges used for baseline fits.}{3}
304
305\section{Functionality}
306
307\subsection{Import/export}
308
309The software needs a set of import/export functions to deal with a
310variety of data formats and to be able to exchange data with other
311popular packages. These functions should be flexible enough to allow
312the user to perform analysis functions in an different package and
313re-import the data (or vice versa). The import function must be
314modular enough to easily add new file formats when the need arises.
315To properly import data, extra information may have to be read from
316secondary calibration files (such as GTP, Gated Total Power, for 3~mm
317wavelength data taken with Mopra). The import functions should be
318flexible enough to gracefully handle data files with missing headers
319etc. They should also be able to make telescope and date specific
320corrections to the data (for ATNF observatories).
321
322The software must be able to read (import) the following file formats.
323
324\requirement{The rpfits file format produced by all current ATNF
325 correlators.}{0}
326
327\requirement{SDFITS (currently written by {\tt SPC}).}{1}
328
329\requirement{Simple ``image'' FITS (used by CLASS}{2}
330
331\requirement{Historic ATNF single dish formats (Spectra, SPC,
332SLAP). Possibly a set of routines to translate these formats to SDFITs
333would suffice.}{3}
334
335\requirement{PSRFIT for pulsar spectroscopy.}
336
337\requirement{For online analysis, the software should be able to read
338an rpfits file which is is still currently open for writing by the
339telescope backend processor.}{1}
340
341\requirement{Data which has been observed in either a fixed frequency
342or Doppler tracked fashion needs to be handled transparently.}{1}
343
344The software should be able to export the data in the following formats.
345
346\requirement{Single Dish FITS.}{1}
347
348\requirement{Simple ``image'' FITS (used by CLASS. It must be possible
349to to export multiple spectra simultaneously, using default file name
350and postfix.}{2}
351
352\requirement{In a format which can be imported by other popular
353 packages such as Class. }{2}
354
355\requirement{Simple ascii format, suitable for use with programs such
356 as Perl, Python, SuperMongo etc.}{2}
357
358\requirement{The exported data should retain as much header data as
359possible. It should also be possible to request specific data be
360written in the desired form (B1950 coordinates, optical velocity
361definition etc).}{2}
362
363\requirement{The import function should apply relevant corrections
364(especially those which are time dependent) to specific
365telescopes. See $\S$\ref{sec:issues} for a list of currently known
366issues.}{1}
367
368\subsection{Sky subtraction}
369\label{sec:skysubtraction}
370To remove the effects of the passband filter shape and atmospheric
371fluctuations across the band, sky subtraction must be performed on the
372data. The software must be able to do sky subtraction using both
373position switching (quotient spectra) and frequency switching
374techniques.
375
376\requirement{\label{ref:skysub} Position switched sky subtraction
377should be implemented using the algorithm \medskip\reqeqn{T_{ref}
378\times \frac{S}{R} - T_{sig}} -- removes continuum\bigskip
379\reqeqn{T_{ref} \times \frac{S}{R} - T_{ref}} -- preserves
380continuum\medskip}{0}
381
382\requirement{The user should be able to specify an arbitrarily complex
383reference/source order (which repeats), which can then be used to make
384perform multiple sky subtractions in parallel.}{3}
385
386\requirement{Frequency switched sky subtraction should be
387 supported. (Ref. Liszt, 1997, A\&AS, 124, 183) }{2}
388
389%\requirement{For wideband multibit sampled data it may be desirable or
390%even required to assume Tsys has a frequency dependency. Appropriate
391%sky subtraction algorithms will need to be investigated.}{3}
392
393\requirement{For pulsar binned data, the (user specified) off pulse
394bins can be used as the reference spectra. Due to potentially rapid
395amplitude fluctuations, sky subtractions may need to be done on a per
396integration basis.}{3}
397
398Multibeam systems can observe in a nodding fashion (called MX mode at
399Parkes), where the telescope position is nodded between scans so that
400the source is observed in turn by two beams and a reference spectra
401for one beam is obtained while the other is observing the target source.
402
403\requirement{For multibeam systems, it must be possible to perform sky
404subtraction with the source and reference in an alternate pair of
405beams}{2}
406
407\subsection{Baseline removal}
408
409Baseline removal is needed to correct for imperfections in sky
410subtraction. Depending on the stability of the system, the residual
411spectral baseline errors can be small or quite large. Baseline removal
412is usually done by fitting a function to the (user specified) line
413free channels.
414
415\requirement{The software must be able to do baseline removal by
416 fitting a n'th order polynomials to the line free channels using a
417 least squares method.}{0}
418
419%\requirement{The baseline fitting should be reversible, i.e. the fit
420%parameters must be retained.}{2}
421
422\requirement{Removal of standing wave ripples should be done by
423fitting a Sine function to the line free channels.}{2}
424
425\requirement{``Robust'' fitting functions should be available, which are more
426 tolerant to RFI.}{1}
427
428\requirement{Automatic techniques for baselining should be
429 investigated.}{3}
430
431\subsection{Line Profile Fitting}
432
433The user will want to fit multicomponent line profiles to the data in
434a simple manner and be able to manipulate the exact fitting
435parameters.
436
437\requirement{The software must be able to do multi-component Gaussian
438 fitting of the spectra. The initial amplitude, width and velocity of
439 each component should be able to be set by the user and specific
440 values to be fit should be easily set.}{0}
441
442\requirement{The reduce Chi square (or similar statistic) of the fit
443should given to the user, so that they can easily see if adding extra
444components give a statistically significant improvement to the fit.}{1}
445
446%\requirement{The fit parameters should be stored with the data so that
447%the user can work on multiple data sets simultaneously and experiment
448%with different fitting values. These values should be saved to disk
449%along with the data.}{1}
450
451\requirement{For multiple polarisation data, the individual stokes
452parameters or polarisation products should be fit independently.}{1}
453
454\requirement{There should be an easy way of exporting the fit
455parameter from multiple spectra, e.g. as an ASCII table.}{2}
456
457\requirement{It should be also possible to do constrained fitting of
458multiple hyperfine components, e.g. the NH$_3$ hyperfine
459components. (The constraints may be either the frequency separation of
460the individual components or the amplitude ratio etc.)}{3}
461
462\requirement{It must be possible to alter the line profile fit
463parameter values by hand at any stage.}{2}
464
465\requirement{It must be possible to ``fix'' particular values of the
466 line profile parameters, so that only subset of lines or (say) the
467 width of a line is fit.}{1}
468
469\requirement{The software should allow hooks for line profile shapes
470other than Gaussian to be added in the future, possible user
471specified.}{2}
472
473%\makenote{Should it be possible to attach multiple sets of fits to the
474%data (similar to CL tables in classic AIPS), so the user can
475%experiment with different ways of fitting the data?}
476
477%\makenote{Should calculations of rotational temperatures etc be
478%handled when fitting hyperfine components, or should the user be doing
479%this themselves?}
480
481\subsection{Calibration}
482
483The software should handle all basic system temperature (Tsys) and
484gain calibration as well as opacity corrections where relevant. The
485Tsys value should be contained in the rpfits files. The actual
486application of the T$_{\mbox{sys}}$ factor will be applied as part of
487the sky subtraction ($\S$\ref{sec:skysubtraction}). The units of Tsys
488recorded in the data may be either in Jy or Kelvin, which will affect
489how the data is calibrated. The rpfits file does {\em not} distinguish
490if the flux units are Kelvin or Janskys.
491
492\requirement{Gain elevation corrections should be implemented using a
493elevation dependent polynomial. The polynomial coefficients will be
494telescope and frequency dependent. They will also have a (long term)
495time dependence.}{1}
496
497\requirement{The user may wish to supply their own gain polynomial.}{2}
498
499\requirement{When required by the user, the spectral units must be
500converted from Kelvin to Jansky. At higher (3mm) frequencies this
501conversion is often not applied. The conversion factor is\medskip
502\reqeqn{\mbox{Flux (Jy)} = \frac{T \times 2 k_b \times
50310^{26}}{\eta A}},\medskip\\where $k_b$ is Boltzmann's constant, A is
504the illuminated area of the telescope and $\eta$ is the efficiency of
505the telescope (frequency, telescope and time dependent)}{1}
506%%TODO%% Use planks formula here?
507
508\requirement{In some cases the recorded Tsys values will be
509wrong. There needs to be a mechanism to scale the Tsys value and the
510spectrum if the Tsys value has already been applied (i.e. a simple and
511consistent rescaling factor).}{2}
512
513\requirement{The data may need to be corrected for opacity effects,
514particularly at frequencies of 20~GHz and higher. The opacity factor
515to apply is given by\medskip\reqeqn{C_o = e^{\tau/cos(z)}}\medskip\\
516where $\tau$ is the opacity and z is the zenith angle (90-El). These
517corrections will generally be derived from periodic ``skydip''
518measurements. These values will not be contained in the rpfits files,
519so there should be a simple way of the software obtaining them and
520interpolating in time (the user should not {\em have} to type them in,
521but may want to). Reading in an ascii file which contains the skydip
522data along with a time-stamp would be one possibility.}{1}
523
524\requirement{For wideband, multibit observations, the software should
525have the option to handle Tsys which varies across the band. The exact
526implementation will have to be decided once experience is gained with
527the new Mopra digital filterbank. This will affect the sky subtraction
528algorithms (requirement \reqref{ref:skysub}).}{2}
529
530%\makenote{Is the dependence of gain on frequency weak enough for one
531%set of coefficients for each receiver, or is a full frequency dependent
532%set of values needed?}
533
534%\makenote{Should it be possible to read ``correct'' Tsys values from
535%an external ascii file?}
536
537\subsection{Editing \& RFI robustness}
538
539In a data set with many observations, individual spectra may be
540corrupted or the data may be affected by RFI and ``birdies''. The user
541needs to be able to easily flag individual spectra or channels. This
542may affect other routines such as sky-subtraction, as this will
543disrupt the reference/source sequence.
544
545\requirement{The user must be able to set an entire spectra or part
546thereof (individual polarisation, IF etc) as being invalid. The
547effected channels should either be blanked or interpolated depending
548on the user wishes. When blanked data is plotted, the plotting routine
549should also either interpolate the data on the fly or show a blank in
550the spectrum, depending on the users preferences.}{1}
551
552\requirement{The user must be able to indicate an individual spectral
553point or range of spectral points are invalid. This should be applied
554to an individual spectra, or set of spectra.}{1}
555
556\requirement{The user should be able to plot the average spectral flux
557across the band, or part of the band, as a function of time and
558interactively select sections of data which should be marked as
559invalid (similar to IBLED in classic aips).}{3}
560
561\requirement{Where relevant, fitting routines etc should have the
562option of selecting RFI tolerant (``robust'') algorithms. This will
563require investigating alternate fitting routines other than the
564least-squares approach.}{3}
565
566\requirement{A routine to automatically find birdies or RFI corrupted
567data and indicate the data as invalid would be useful.}{3}
568
569\requirement{Other routines must be able to cope with portions of data
570which are marked as invalid.}{1}
571
572\subsection{Spectra mathematics and manipulation}
573
574A flexible suite of mathematical operations on the spectra should be
575possible. This should include options such as adding, subtracting,
576averaging and scaling the data. For common operations such as
577averaging and smoothing, it must be simple for the user to invoke the
578function (i.e. not to have to start up a complex spectral
579calculator). Where it makes sense, it should be possible to manipulate
580multiple spectra simultaneously.
581
582The spectral manipulations which should be available are:
583
584\requirement{Add or subtract multiple spectra.}{1}
585
586\requirement{Averaging multiple spectra, with optional weighting based
587on Tsys, integration or rms. If the velocity of the spectra to be
588averaged is different, the data should be automatically aligned in
589velocity.}{0}
590
591\requirement{Various robust averaging possibilities (e.g. median
592averaging, clipped means etc) should be possible.}{2}
593
594\requirement{Re-sampling or re-binning of the data to a lower (or
595higher) spectral resolution (i.e. change the number of spectral
596points). The re-sampling factor may not necessarily be an integer.}{2}
597
598\requirement{It must be possible to shift the data in
599 ``frequency/velocity''. This should include channel, frequency and
600 velocity shifts of an arbitrary amount.}{1}
601
602\requirement{Spectral smoothing of the data. Hanning, Tukey, boxcar
603and Gaussian smoothing of variable widths should be possible.}{1}
604
605\requirement{Scaling of the spectra.}{1}
606
607\requirement{Calculate basic statistical values (maximum, minimum,
608rms, mean) on a range of spectral points. The range may not be
609contiguous. The calculated rms value should be retained with the
610spectra so it can be optionally used for weighted averaging of
611spectra.}{1}
612
613\requirement{It must be possible to calculate the flux integral over a
614range of channels. The units should be Jy.km/s (or Kelvin.km/s). The
615channel range for the calculation should be specific via the GUI or
616CLI.}{2}
617
618\requirement{It must be possible to calculate the numerical ``width''
619of a line (full width at half maximum type measurement). This should
620be calculated by specifying a channel range and finding the maximum
621value in this range and then finding the interpolated crossing point
622of the data as a user defined fraction of the maximum (default
62350\%). The profile width and velocity mid-point should then be
624computed. If the profile shape is complex (e.g. double arch) with
625multiple crossing points of the fraction value, the minimum and
626maximum width values should be calculated. There should be the option
627of using a user specified ``maximum value''.}{2}
628
629\requirement{The user must be able to easily change the rest-frequency
630to which the velocity is referenced.}{1}
631
632\requirement{FFT filtering for high- and lowpass filtering and
633tapering.}{3}
634
635\requirement{It should be possible to FFT the data to and from power
636 spectra to the autocorrelation function.}{2}
637
638\requirement{The user may wish to compute the cross correlation
639function of two spectra. The result should be a standard ``spectra'',
640which can be displayed and analysed using other functions (max, rms
641etc).}{3}
642
643\requirement{Complex experiment specific processing can often be done
644using a series of the simple of basic functions. A spectral calculator
645options should be added to the CLI to perform a series of
646manipulations on a set of spectra.}{3}
647
648The user may want to perform specific analysis on the data using the
649functionality above, but wish to do the manipulation between two
650polarisations or IFs. Allowing the functions to also, optionally,
651specify specific polarisations or IF would be an implementation and
652interface nightmare. The simplest solution is the allow the data to be
653``split'' into separate spectra.
654
655\requirement{It must be possible to take multi IF, multibeam or
656polarisation data and split out the individual spectral portions to
657form self contained spectra.}{2}
658
659\subsection{Polarimetry}
660
661The software must fully support polarmetric analysis. This includes
662calibration and basic conversions. Observations may be made with
663linear or circular feed and the backend may or may not compute the
664cross polarisation products. As such the software must cope with a
665variety of conversions. The software should be able to calculate
666stokes parameters with or without solving for leakage terms.
667
668%\makenote{It is debatable whether stokes I is the sum or average or
669%two dual polarisation measurements.}
670
671\requirement{All functions on the data (calibration, sky subtraction
672spectral mathematics) must support arbitrary, multiple, polarisation
673(linear, circular \& stokes and single, dual \& cross
674polarisations.}{1}
675
676\requirement{It must be possible to calculate stokes I from single or
677dual polarisation observations.}{1}
678
679\requirement{Average a mixture of dual polarisation and single
680polarisation data and form average stokes I (e.g. for a long
681observation of a source, in which one polarisation is missing for some
682time.}{3}
683
684\requirement{Full stokes parameters should be obtained from dual pol
685(linear or circular) observations where the cross polarisation
686products have been calculated.}{1}
687
688%\requirement{If the observations used linear polarisations and the
689%cross polarisations were not computed, the source needs to be
690%observed with the feeds set at least 3 different parallactic angles
691%(note that if dual linear feeds are available, 2 orthogonal
692%parallactic angles are obtained at once). The Stokes parameters can be
693%solved using a least squares fit to the equation:
694%\reqeqn{Iu/2 + Ip * cos^2 (PA + p)},\\
695%where PA is the linear feed position angle, p is the polarisation
696%angle, Iu and Ip and the unpolarised and linearly polarised
697%intensity. {\em Stolen from SPC. I need to write this in more useful
698%language. Is this technique likely to be used anymore?.}}{3}
699
700\requirement{If dual circular polarisation measurements are taken,
701without computing the cross products, the software should still be
702able to compute stokes I and V.}{2}
703
704\requirement{The software should be able to calculate leakage terms
705from a calibrator source and correct the data either before or after
706conversion to Stokes. (ref. Johnston, 2002, PASA, 19, 277)}{3}
707
708\requirement{The software should be able to determine absolute
709position angle from a calibrator source and correct the data either
710before or after conversion to Stokes.}{3}
711
712\requirement{Zeeman splitting factors should be derived from
713(previous) profile fitting and the left and right circular
714polarisations. The velocity shift varies linearly with the magnetic
715field, but the scaling factor depends on the molecule and
716transition. Scaling factor for common transitions should be known by
717the software and the user able to enter factors for less common
718transitions. Correctly identifying Zeeman pairs is crucial in getting
719the correct result. The software should attempt to make an initial
720guess of pairs (based on component velocity and width) but make the
721user confirm and override the pairing if required.}{3}
722
723\subsection{Data Selection}
724While the software is running the user will usually have loaded
725multiple (possibly many) spectra each of which may have multiple IFs,
726data from multiple beams and multiple polarisations. The user will
727want to be able to quickly flip from considering one spectra to
728another and, where relevant, want to perform parallel processing on
729multiple spectra at once (e.g. baselining a sequence of on/off
730observations of the same source which will later be averaged
731together).
732
733\requirement{The software needs an easy-to-use mechanism to select
734either individual or multiple spectra for viewing, parallel processing
735etc.}{1}
736
737\requirement{An easy-to-use mechanism to select individual IFs, beams
738or polarisations is needed.}{1}
739
740\requirement{\label{ref:chansel} The range of spectral points to use
741for baseline removal, statistical calculations, RFI editing, analysis
742etc must be easily set by the user from both the CLI and GUI. From the
743CLI there must be the option of setting the range using a variety of
744units (channel number, velocity, frequency). The selection range will
745probably not be a contiguous set of channels, but many sets of
746disjoint channel ranges. For some tasks (such as baseline subtraction
747and statistical values), the channel range should be retained and be
748available as a plot overlay.}{1}
749
750\requirement{When performing baseline subtraction on many spectra
751simultaneously, the software should have a mechanism for identifying
752``on'' and ``off'' spectra and automatically selecting the signal and
753quotient spectra. The algorithm needs to cope with on/off/on/off
754sequences as well as off/on/on/off. If an individual quotient spectra
755has been marked as invalid, an alternative should be found.}{3}
756
757\requirement{The software should be able to select sets of sources
758based on simple regular expression type filtering (wild cards) on a
759range of header values. Examples include G309$*$ or G309$*$w to select
760on source name, or NH3$*$ to select on molecule name.}{3}
761
762\subsection{Plugins}
763
764\requirement{It would be desirable to support ``plugins'', user
765definable functions for specific processing. The plugin code must have
766full access (read/write) to the spectra data and headers. It should
767also be possible to create new spectra which the software treats the
768same as the original data. Preferably some (limited) access to the
769plotter would also be possible and a number of the standard functions
770accessible as ``library'' routines.}{3}
771
772\subsection{Pipelining}
773
774\requirement{Some sort of pipelining mode is required. This would
775involve doing a quotient spectra, applying appropriate calibration and
776possibly fitting a Gaussian to any lines present.}{3}
777
778\subsection{Methanol Multibeam Survey}
779
780The software may need to support reduction of data from the methanol
781multibeam project. If so the pipelining will need to be flexible and
782powerful enough to support this.
783
784\subsection{Miscellaneous functionality}
785
786\requirement{The software should be able to take a simple ``grid'' of
787observations (normally a set of observations in a cross pattern on the
788sky) and, for a subset of channels, fit the position of the
789emission. The fit positions should be either plotted on the screen or
790exported in a simple ascii form.}{3}
791
792\requirement{The kinematic distance of a source should be calculated
793using basic Galactic rotation models. Multiple Galactic rotation
794models must be supported and a mechanism for easily adding more.}{3}
795
796\requirement{For 1420 MHz observations of HI, the rms (Tsys) values
797vary significantly across the band. The software should be able to
798compute the rms as a function of frequency across the spectra from the
799off-pulse data and then be able to plot n-sigma error bars on the
800spectra.}{3}
801
802\requirement{It should be possible to take a selection of calibrated
803spectra which are then passed to the ``Gridzilla'' program to produce
804an image cube. Analysis of this cube would be done using external
805programs (e.g. Miriad, aips++)}{3}
806
807\section{Help}
808
809\requirement{There should be built-in web-based documentation, which
810can be easily kept up-to-date}{1}
811
812\requirement{A short and simple end-to-end cookbook for basic data
813analysis should be available.}{1}
814
815\section{Data and meta-data}
816
817\requirement{The software must be capable of handling multi-IF
818 (potentially dozens of IFs) and multi-beam data with arbitrary
819 polarisation (e.g. single pol, dual pol, full stokes etc).}{1}
820
821\requirement{The software should handle pulsar binned data for pulsar
822absorption experiments.}{3}
823
824\subsection{History}
825
826\requirement{A user viewable history of data processing steps should
827be kept as part of the data. Where possible this should be retained
828when data is imported from other packages.}{1}
829
830\requirement{It should be possible to use the history information to
831create template pipeline scripts for batch processing.}{2}
832
833\subsection{Multiple IFs}
834
835\requirement{If multiple IFs are present (currently Tidbinbilla can
836produce two IFs and the new wideband spectrometer for Mopra may have
837dozens of IFs), the software should handle the data
838transparently. Potentially each IF may have a significantly different
839sky frequency and be observing a different molecule or transition with
840a different rest frequency. From the users point of view,
841simultaneously obtained IFs should be kept within the same
842``container'' (not split into a myriad of separate ``container'').}{1}
843
844%\makenote{Does the case of multiple lines (so multiple
845%rest frequencies) within a single IF need special attention?}
846
847\subsection{Multibeam}
848
849\requirement{Basic handling of multibeam data should be possible (ie
850in general each beam will be treated as a separate observation, but
851all within the same container. The user should be able to view or
852process either individual beams or all beams in parallel.}{2}
853
854\requirement{The use of a single beam observing a source and the
855rest of the beams as reference beams for sky-subtraction should be
856investigated.}{2}
857
858\subsection{Robust fitting}
859
860\requirement{If robust fitting using median filtering is used, then
861the individual integrations from the observations should {\em not} be
862averaged when the data is imported, but retained within a single
863container. Inspection of this data should be optionally of the averaged
864or individual data.}{2}
865
866\subsection{Fit parameters}
867
868\requirement{The fitting parameters for functions which have been fit
869to the data (e.g. for baseline removal or Gaussian fits) should be
870retained as an integral part of the data and stored permantely on
871disk.}{1}
872
873\requirement{It must be possible to export fitting values in an
874appropriate form. (i.e. ascii text format)}{1}
875
876\requirement{It should be possible to ``undo'' functions which have
877been subtracted from the data (e.g. baseline polynomials).}{3}
878
879\subsection{Coordinate frames and units}
880
881\requirement{Coordinate frames and unit selection and handling needs
882to be flexible and relatively transparent to the user (i.e. if the
883users preference is for LSRK velocities, they do not need to worry
884about the reference frame in which the data was observed).}{1}
885
886\requirement{At a minimum the following reference frames and
887conventions should be handled: \setlength{\parindent}{0pt}
888
889\smallskip
890\anitem{Position}{(RA,Dec) in J2000 \& B1950 (as well as other
891 arbitrary epochs), Galactic, (Az,El).}
892
893\anitem{Frequency}{Velocity (Topocentric, Geocentric, Barycentric,
894 Heliocentric, kinematical LSR, dynamical LSR, Rest), Frequency
895 (MHz, GHz), channel number.}
896
897\anitem{Velocity}{ Optical, Radio, Relativistic.}
898
899\anitem{Flux}{ Jansky, Kelvin (mJy etc).}}{1}
900
901\requirement{All data should be internally labelled with the
902 appropriate coordinate frame and units. If this information is
903 ambiguous for some reason, it should be set when the data is imported
904 and the user should not have to worry about it again.}{1}
905
906\requirement{It should be clear to the user what coordinate frame
907(velocity, position etc) the data is being presented as.}{1}
908
909\subsection{Meta-data}
910
911A comprehensive set of header data should be read from the input data
912files. In general all meta-data available in the rpfits file should be
913retained. The user may wish to enter some specific values by hand.
914
915\requirement{All header data should be viewable and editable by the
916user. This includes changes such as scaling the given Tsys values.}{2}
917
918\requirement{Missing header data should be handled gracefully,
919i.e. the software should fill the values with ``blanks'' and be able
920to continue to process the data if possible.}{1}
921
922\requirement{The user must be able to add missing header data, which
923is not present in the RPFITs file. It must be possible to add the same
924header data to multiple scans simultaneously.}{2}
925
926\extendedrequirement{
927The following header data would be required per scan:
928\begin{itemize}
929 \item Source name
930 \item Scan type (signal or reference)
931 \item Integration time
932 \item Scan length (actual time of observation, $\ge$ integration time)
933 \item Telescope
934 \item UT time and date of observation
935 \item Telescope elevation of observation
936 \item Parallactic angle
937 \item Beam size
938 \item Scan ID
939 \item Observer
940 \item Project
941 \item Polarisation
942 \item Receiver
943 \item Telescope coordinates
944 \item Weather info (temperature, pressure, humidity)
945 \item LO chain setup
946 \item User axis display preference (LSR velocity, frequency etc).
947\end{itemize}
948}{1}
949
950\extendedrequirement{
951\label{req:if}
952The following header data is required for each IF, beam etc:
953\begin{itemize}
954\item Source coordinates and coordinate frame
955\item Frequency/velocity axis definition and type
956\item System Temperature
957\item Antenna gain (if Tsys measured in Kelvin)
958\item Beam number
959\item Molecule rest frequency$^\dagger$
960\item Molecular name$^\dagger$
961\item Molecular transition$^\dagger$
962\item Molecular formula$^\dagger$
963\end{itemize}
964}{1}
965
966\requirement{The molecular formula could be stored with embedded
967superscripted and subscripted symbols for ``pretty'' printing on the
968plotted, but printed in plain text on the CLI or in ascii output}{3}
969
970Some molecular line rest-frequencies are close enough that two or more
971molecules or transitions may be observed in a single IF. Typical
972examples include the 1665/1667~MHz OH maser pair, NH$_3$ transitions,
973and many observations in the 3~mm band.
974
975\vspace{\parskip}
976\requirement{The software should optionally support multiple lines per
977IF, by storing a set of rest frequencies per IF, rather than a single
978value. The header values in requirement \reqref{req:if} marked with a
979$\dagger$ would all have to be stored as an array of values rather
980than a scalar. A simple mechanism must be posible to change the
981currently ``active'' rest-frequency.}{3}
982
983\section{Installation}
984
985\requirement{It must be possible for astronomers to install the
986software at their own institute with either a moderate amount of OS
987experience or some help from the local system administrators. This
988includes installation on a central ``NFS'' server as well as local
989desk-tops.}{1}
990
991\requirement{The software must run on Solaris and all major flavours
992of Linux (Fedora/Redhat, Debian, etc).
993 }{1}
994
995\requirement{It must be possible for users to
996install the software on their (unix) laptops and run with no network
997connection. }{1}
998
999\requirement{It should be relatively easy to upgrade to the lastest
1000version of the software.}{2}
1001
1002\requirement{The software should run on MacOS/X}{3}
1003
1004\requirement{It would be desirable for the software to run on
1005Windows.}{3}
1006
1007\section{Known Issues}
1008\label{sec:issues}
1009The following issue are known problems with the data from ATNF
1010telescopes, which probably should be automatically corrected for if at
1011all possible. The best place to do this is while loading the data.
1012
1013\subsection{General}
1014
1015\begin{itemize}
1016 \item All polarisations in the RPFITS files are labelled as
1017 XX/YY. These need to be relabelled as LL/RR when appropriate.
1018\end{itemize}
1019
1020\subsection{Mopra}
1021
1022\begin{itemize}
1023\item Data obtained in 2002 \& 2003 (and probably before) have an
1024 error in the frequency headers (this may be corrected by an external
1025 program). \makenote{Nedd Ladd}
1026
1027\item The (RA,Dec) positions in the data file are in date coordinates
1028 not J2000. This causes problems for packages like Class when
1029 averaging the data. \makenote{Maria Hunt}
1030
1031\item It is possible Tsys calibration is inconsistent currently.
1032 \makenote{Cormac Purcell??}
1033
1034\end{itemize}
1035
1036\subsection{Parkes}
1037
1038\begin{itemize}
1039\item For pulsar data the automatic gain control is disabled. This
1040means the nominal Tsys measurement does not change and Tsys per
1041integration is encoded in a non-standard way. \makenote{Simon
1042Johnston}
1043\end{itemize}
1044
1045\subsection{Tidbinbilla}
1046
1047\begin{itemize}
1048\item All 20-GHz data is calibrated in flux units of Kelvin.
1049\end{itemize}
1050
1051\section{Acknowledgements}
1052
1053We would like to thank the following people for input directly or from
1054previous documents: Maria Hunt, Paul Jones, Jim Caswell \& Dave
1055McConnell, Frank Briggs, Juergen Ott, Daniel Pisano, Jim Lovell, Ray
1056Norris, WIm Brouw, Maxim Voronkov, Vincent McIntyre, Jessica Chapman,
1057Simon Johnston, Nedd Ladd, Lister Staveley-Smith, Cormac Purcell \&
1058Rachel Deacon.
1059
1060\end{document}
Note: See TracBrowser for help on using the repository browser.