1 | % ----------------------------------------------------------------------- |
---|
2 | % intro.tex: Introduction, and guide to what Duchamp is doing. |
---|
3 | % ----------------------------------------------------------------------- |
---|
4 | % Copyright (C) 2006, Matthew Whiting, ATNF |
---|
5 | % |
---|
6 | % This program is free software; you can redistribute it and/or modify it |
---|
7 | % under the terms of the GNU General Public License as published by the |
---|
8 | % Free Software Foundation; either version 2 of the License, or (at your |
---|
9 | % option) any later version. |
---|
10 | % |
---|
11 | % Duchamp is distributed in the hope that it will be useful, but WITHOUT |
---|
12 | % ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or |
---|
13 | % FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License |
---|
14 | % for more details. |
---|
15 | % |
---|
16 | % You should have received a copy of the GNU General Public License |
---|
17 | % along with Duchamp; if not, write to the Free Software Foundation, |
---|
18 | % Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307, USA |
---|
19 | % |
---|
20 | % Correspondence concerning Duchamp may be directed to: |
---|
21 | % Internet email: Matthew.Whiting [at] atnf.csiro.au |
---|
22 | % Postal address: Dr. Matthew Whiting |
---|
23 | % Australia Telescope National Facility, CSIRO |
---|
24 | % PO Box 76 |
---|
25 | % Epping NSW 1710 |
---|
26 | % AUSTRALIA |
---|
27 | % ----------------------------------------------------------------------- |
---|
28 | \secA{Introduction and getting going quickly} |
---|
29 | |
---|
30 | \secB{About Duchamp} |
---|
31 | |
---|
32 | This document provides a user's guide to \duchamp, an object-finder |
---|
33 | for use on spectral-line data cubes. The basic execution of \duchamp |
---|
34 | is to read in a FITS data cube, find sources in the cube, and produce |
---|
35 | a text file of positions, velocities and fluxes of the detections, as |
---|
36 | well as a postscript file of the spectra of each detection. |
---|
37 | |
---|
38 | \duchamp has been designed to search for objects in particular sorts |
---|
39 | of data: those with relatively small, isolated objects in a large |
---|
40 | amount of background or noise. Examples of such data are extragalactic |
---|
41 | \hi surveys, or maser surveys. \duchamp searches for groups of |
---|
42 | connected voxels (or pixels) that are all above some flux |
---|
43 | threshold. No assumption is made as to the shape of detections, and |
---|
44 | the only size constraints applied are those specified by the user. |
---|
45 | |
---|
46 | \duchamp has been written as a three-dimensional finder, but it is |
---|
47 | possible to run it on a two-dimensional image (\ie one with no |
---|
48 | frequency or velocity information), or indeed a one-dimensional array, |
---|
49 | and many of the features of the program will work fine. The focus, |
---|
50 | however, is on object detection in three dimensions, one of which is a |
---|
51 | spectral dimension. Note, in particular, that it does not do any |
---|
52 | fitting of source profiles, a feature common (and desirable) for many |
---|
53 | two-dimensional source finders. This is beyond the current scope of |
---|
54 | \duchamp, whose aim is reliable detection of spectral-line objects. |
---|
55 | |
---|
56 | \secB{What to do} |
---|
57 | |
---|
58 | So, you have a FITS cube, and you want to find the sources in it. What |
---|
59 | do you do? First, you need to get \duchamp: there are instructions in |
---|
60 | Appendix~\ref{app-install} for obtaining and installing it. Once you |
---|
61 | have it running, the first step is to make an input file that contains |
---|
62 | the list of parameters. Brief and detailed examples are shown in |
---|
63 | Appendix~\ref{app-input}. This file provides the input file name, the |
---|
64 | various output files, and defines various parameters that control the |
---|
65 | execution. |
---|
66 | |
---|
67 | The standard way to run \duchamp is by the command |
---|
68 | \begin{quote} |
---|
69 | {\footnotesize |
---|
70 | \texttt{> Duchamp -p [parameter file]} |
---|
71 | } |
---|
72 | \end{quote} |
---|
73 | replacing \texttt{[parameter file]} with the name of the file listing |
---|
74 | the parameters. |
---|
75 | |
---|
76 | An even easier way is to use the default values for all parameters |
---|
77 | (these are given in Appendix~\ref{app-param} and in the file |
---|
78 | \texttt{InputComplete} included in the distribution directory) and use |
---|
79 | the syntax |
---|
80 | \begin{quote} |
---|
81 | {\footnotesize |
---|
82 | \texttt{> Duchamp -f [FITS file]} |
---|
83 | } |
---|
84 | \end{quote} |
---|
85 | where \texttt{[FITS file]} is the file you wish to search. |
---|
86 | |
---|
87 | The default action includes displaying a map of detected objects in a |
---|
88 | PGPLOT X-window. This can be disabled by setting the parameter |
---|
89 | \texttt{flagXOutput = false} or using the \texttt{-x} command-line |
---|
90 | option, as in |
---|
91 | \begin{quote} |
---|
92 | {\footnotesize |
---|
93 | \texttt{> Duchamp -x -p [parameter file]} |
---|
94 | } |
---|
95 | \end{quote} |
---|
96 | and similarly for the \texttt{-f} case. |
---|
97 | |
---|
98 | Once a FITS file and parameters have been set, the program will then |
---|
99 | work away and give you the list of detections and their spectra. The |
---|
100 | program execution is summarised below, and detailed in |
---|
101 | \S\ref{sec-flow}. Information on inputs is in \S\ref{sec-param} and |
---|
102 | Appendix~\ref{app-param}, and descriptions of the output is in |
---|
103 | \S\ref{sec-output}. |
---|
104 | |
---|
105 | \secB{Guide to terminology and conventions} |
---|
106 | |
---|
107 | First, a brief note on the use of terminology in this guide. \duchamp |
---|
108 | is designed to work on FITS ``cubes''. These are FITS\footnote{FITS is |
---|
109 | the Flexible Image Transport System -- see \citet{hanisch01} or |
---|
110 | websites such as |
---|
111 | \href{http://fits.cv.nrao.edu/FITS.html}{http://fits.cv.nrao.edu/FITS.html} |
---|
112 | for details.} image arrays with (at least) three dimensions. They |
---|
113 | are assumed to have the following form: the first two dimensions |
---|
114 | (referred to as $x$ and $y$) are spatial directions (that is, relating |
---|
115 | to the position on the sky -- often, but not necessarily, |
---|
116 | corresponding to Equatorial or Galactic coordinates), while the third |
---|
117 | dimension, $z$, is the spectral direction, which can correspond to |
---|
118 | frequency, wavelength, or velocity. The three dimensional analogue of |
---|
119 | pixels are ``voxels'', or volume cells -- a voxel is defined by a |
---|
120 | unique $(x,y,z)$ location and has a single value of flux, intensity |
---|
121 | or brightness (or something equivalent) associated with it. |
---|
122 | |
---|
123 | Sometimes, some pixels in a FITS file are labelled as BLANK -- that |
---|
124 | is, they are given a nominal value, defined by FITS header keywords |
---|
125 | \textsc{blank, bscale, \& bzero}, that marks them as not having a flux |
---|
126 | value. These are often used to pad a cube out so that it has a |
---|
127 | rectangular spatial shape. \duchamp has the ability to avoid these: |
---|
128 | see \S\ref{sec-blank}. |
---|
129 | |
---|
130 | Note that it is possible for the FITS file to have more than three |
---|
131 | dimensions (for instance, there could be a fourth dimension |
---|
132 | representing a Stokes parameter). Only the two spatial dimensions and |
---|
133 | the spectral dimension are read into the array of pixel values that is |
---|
134 | searched for objects. All other dimensions are ignored\footnote{This |
---|
135 | actually means that the first pixel only of that axis is used, and the |
---|
136 | array is read by the \texttt{fits\_read\_subsetnull} command from the |
---|
137 | \textsc{cfitsio} library.}. Herein, we discuss the data in terms of |
---|
138 | the three basic dimensions, but you should be aware it is possible for |
---|
139 | the FITS file to have more than three. Note that the order of the |
---|
140 | dimensions in the FITS file does not matter. |
---|
141 | |
---|
142 | With this setup, each spatial pixel (a given $(x,y)$ coordinate) can |
---|
143 | be said to be a single spectrum, while a slice through the cube |
---|
144 | perpendicular to the spectral direction at a given $z$-value is a |
---|
145 | single channel, with the 2-D image in that channel called a channel |
---|
146 | map. |
---|
147 | |
---|
148 | Detection involves locating a contiguous group of voxels with fluxes |
---|
149 | above a certain threshold. \duchamp makes no assumptions as to the |
---|
150 | size or shape of the detected features, other than having |
---|
151 | user-selected minimum size criteria. Features that are detected are |
---|
152 | assumed to be positive. The user can choose to search for negative |
---|
153 | features by setting an input parameter -- this inverts the cube prior |
---|
154 | to the search (see \S\ref{sec-detection} for details). |
---|
155 | |
---|
156 | \secB{A summary of the execution steps} |
---|
157 | |
---|
158 | The basic flow of the program is summarised here -- all steps are |
---|
159 | discussed in more detail in the following sections. |
---|
160 | \begin{enumerate} |
---|
161 | \item The necessary parameters are recorded. |
---|
162 | |
---|
163 | How this is done depends on the way the program is run from the |
---|
164 | command line. If the \texttt{-p} option is used, the parameter file |
---|
165 | given on the command line is read in, and the parameters therein are |
---|
166 | read. All other parameters are given their default values (listed in |
---|
167 | Appendix~\ref{app-param}). |
---|
168 | |
---|
169 | If the \texttt{-f} option is used, all parameters are assigned their |
---|
170 | default values. |
---|
171 | |
---|
172 | \item The FITS image is located and read in to memory. |
---|
173 | |
---|
174 | The file given is assumed to be a valid FITS file. As discussed |
---|
175 | above, it can have any number of dimensions, but \duchamp only |
---|
176 | reads in the two spatial and the one spectral dimensions. A subset |
---|
177 | of the FITS array can be given (see \S\ref{sec-input} for details). |
---|
178 | |
---|
179 | \item If requested, a FITS file containing a previously reconstructed |
---|
180 | or smoothed array is read in. |
---|
181 | |
---|
182 | When a cube is either smoothed or reconstructed with the \atrous |
---|
183 | wavelet method, the result can be saved to a FITS file, so that |
---|
184 | subsequent runs of \duchamp can read it in to save having to re-do |
---|
185 | the calculations (as they can be relatively time-intensive). |
---|
186 | |
---|
187 | \item \label{step-blank} If requested, BLANK pixels are trimmed from |
---|
188 | the edges, and the baseline of each spectrum is removed. |
---|
189 | |
---|
190 | BLANK pixels, while they are ignored by all calculations in |
---|
191 | \duchamp, do increase the size in memory of the array above that |
---|
192 | absolutely needed. This step trims them from the spatial edges, |
---|
193 | recording the amount trimmed so that they can be added back in |
---|
194 | later. |
---|
195 | |
---|
196 | A spectral baseline (or bandpass) can also be removed at this point |
---|
197 | as well. This may be necessary if there is a ripple or other |
---|
198 | large-scale feature present that will hinder detection of faint |
---|
199 | sources. |
---|
200 | |
---|
201 | \item If the reconstruction method is requested, and the reconstructed |
---|
202 | array has not been read in at Step 3 above, the cube is |
---|
203 | reconstructed using the \atrous wavelet method. |
---|
204 | |
---|
205 | This step uses the \atrous method to determine the amount of |
---|
206 | structure present at various scales. A simple thresholding technique |
---|
207 | then removes random noise from the cube, leaving the significant |
---|
208 | signal. This process can greatly reduce the noise level in the cube, |
---|
209 | enhancing the detectability of sources. |
---|
210 | |
---|
211 | \item Alternatively (and if requested), the cube is smoothed, either |
---|
212 | spectrally or spatially. |
---|
213 | |
---|
214 | This step presents two options. The first considers each spectrum |
---|
215 | individually, and convolves it with a Hanning filter (with width |
---|
216 | chosen by the user). The second considers each channel map |
---|
217 | separately, and smoothes it with a Gaussian kernel of size and shape |
---|
218 | chosen by the user. This step can help to reduce the amount of noise |
---|
219 | visible in the cube and enhance fainter sources. |
---|
220 | |
---|
221 | \item A threshold for the cube is then calculated, based on the pixel |
---|
222 | statistics (unless a threshold is manually specified by the user). |
---|
223 | |
---|
224 | The threshold can either be chosen as a simple $n\sigma$ threshold |
---|
225 | (\ie a certain number of standard deviations above the mean), or |
---|
226 | calculated via the ``False Discovery Rate'' method. Alternatively, |
---|
227 | the threshold can be specified as a simple flux value, without care |
---|
228 | as to the statistical significance (\eg ``I want every source |
---|
229 | brighter than 10mJy''). |
---|
230 | |
---|
231 | By default, the full cube is used for the statistics calculation, |
---|
232 | although the user can nominate a subsection of the cube to be used |
---|
233 | instead. |
---|
234 | |
---|
235 | \item Searching for objects then takes place, using the requested |
---|
236 | thresholding method. |
---|
237 | |
---|
238 | The cube is searched one channel-map at a time. Detections are |
---|
239 | compared to already detected objects and either combined with a |
---|
240 | neighbouring one or added to the end of the list. |
---|
241 | |
---|
242 | \item The list of objects is condensed by merging neighbouring objects |
---|
243 | and removing those deemed unacceptable. |
---|
244 | |
---|
245 | While some merging has been done in the previous step, this process |
---|
246 | is a much more rigorous comparison of each object with every other |
---|
247 | one. If a pair of objects lie within requested limits, they are |
---|
248 | combined. |
---|
249 | |
---|
250 | After the merging is done, the list is culled (although see comment |
---|
251 | for the next step). There are certain criteria the user can specify |
---|
252 | that objects must meet: minimum numbers of spatial pixels and |
---|
253 | spectral channels, and minimum separations between neighbouring |
---|
254 | objects. Those that do not meet these criteria are deleted |
---|
255 | from the list. |
---|
256 | |
---|
257 | \item If requested, the objects are ``grown'' down to a lower |
---|
258 | threshold, and then the merging step is done a second time. |
---|
259 | |
---|
260 | In this case, each object has pixels in its neighbourhood examined, |
---|
261 | and if they are above a secondary threshold, they are added to the |
---|
262 | object. The merging process is done a second time in case two |
---|
263 | objects have grown over the top of one another. Note that the |
---|
264 | rejection part of the previous step is not done until the end of the |
---|
265 | second merging process. |
---|
266 | |
---|
267 | \item The baselines and trimmed pixels are replaced prior to output. |
---|
268 | |
---|
269 | This is just the inverse of step~\#\ref{step-blank}. |
---|
270 | |
---|
271 | \item The details of the detections are written to screen and to the |
---|
272 | requested output file. |
---|
273 | |
---|
274 | Crucial properties of each detection are provided, showing its |
---|
275 | location, extent, and flux. These are presented in both pixel |
---|
276 | coordinates and world coordinates (\eg sky position and |
---|
277 | velocity). Any warning flags are also printed, showing detections to |
---|
278 | be wary of. Alternative output options are available, such as a |
---|
279 | VOTable or a Karma annotation file. |
---|
280 | |
---|
281 | \item Maps showing the spatial location of the detections are written. |
---|
282 | |
---|
283 | These are 2-dimensional maps, showing where each detection lies on |
---|
284 | the spatial coverage of the cube. This is provided as an aid to the |
---|
285 | user so that a quick idea of the distribution of object positions |
---|
286 | can be gained \eg are all the detections on the edge? |
---|
287 | |
---|
288 | Two maps are provided: one is a 0th moment map, showing the 0th |
---|
289 | moment (\ie a map of the integrated flux) of each detection in its |
---|
290 | appropriate position, while the second is a ``detection map'', |
---|
291 | showing the number of times each spatial pixel was detected in the |
---|
292 | searching routines (including those pixels rejected at step 9 and so |
---|
293 | not in any of the final detections). |
---|
294 | |
---|
295 | These maps are written to postscript files, and the 0th moment map |
---|
296 | can also be displayed in a PGPLOT X-window. |
---|
297 | |
---|
298 | \item The integrated or peak spectra of each detection are written to a |
---|
299 | postscript file. |
---|
300 | |
---|
301 | The spectral equivalent of the maps -- what is the spectral profile |
---|
302 | of each detection? Also provided here are basic information for each |
---|
303 | object (a summary of the information in the results file), as well |
---|
304 | as a 0th moment map of the detection. |
---|
305 | |
---|
306 | \item If requested, a text file containing all spectra is written. |
---|
307 | |
---|
308 | This file will contain the peak or integrated spectra for each |
---|
309 | source, as a function of the appropriate spectral coordinate. The |
---|
310 | file is a multi-column ascii text file, suitable for import into |
---|
311 | other software packages. |
---|
312 | |
---|
313 | \item If requested, the reconstructed or smoothed array can be written |
---|
314 | to a new FITS file. |
---|
315 | |
---|
316 | If either of these procedures were done, the resulting array can be |
---|
317 | saved as a FITS file for later use. The FITS header will be the same |
---|
318 | as the input file, with a few additional keywords to identify the |
---|
319 | file. |
---|
320 | |
---|
321 | \end{enumerate} |
---|
322 | |
---|