[303] | 1 | % ----------------------------------------------------------------------- |
---|
| 2 | % intro.tex: Introduction, and guide to what Duchamp is doing. |
---|
| 3 | % ----------------------------------------------------------------------- |
---|
| 4 | % Copyright (C) 2006, Matthew Whiting, ATNF |
---|
| 5 | % |
---|
| 6 | % This program is free software; you can redistribute it and/or modify it |
---|
| 7 | % under the terms of the GNU General Public License as published by the |
---|
| 8 | % Free Software Foundation; either version 2 of the License, or (at your |
---|
| 9 | % option) any later version. |
---|
| 10 | % |
---|
| 11 | % Duchamp is distributed in the hope that it will be useful, but WITHOUT |
---|
| 12 | % ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or |
---|
| 13 | % FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License |
---|
| 14 | % for more details. |
---|
| 15 | % |
---|
| 16 | % You should have received a copy of the GNU General Public License |
---|
| 17 | % along with Duchamp; if not, write to the Free Software Foundation, |
---|
| 18 | % Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307, USA |
---|
| 19 | % |
---|
| 20 | % Correspondence concerning Duchamp may be directed to: |
---|
| 21 | % Internet email: Matthew.Whiting [at] atnf.csiro.au |
---|
| 22 | % Postal address: Dr. Matthew Whiting |
---|
| 23 | % Australia Telescope National Facility, CSIRO |
---|
| 24 | % PO Box 76 |
---|
| 25 | % Epping NSW 1710 |
---|
| 26 | % AUSTRALIA |
---|
| 27 | % ----------------------------------------------------------------------- |
---|
[158] | 28 | \secA{Introduction and getting going quickly} |
---|
| 29 | |
---|
[309] | 30 | \secB{About Duchamp} |
---|
| 31 | |
---|
[158] | 32 | This document provides a user's guide to \duchamp, an object-finder |
---|
[258] | 33 | for use on spectral-line data cubes. The basic execution of \duchamp |
---|
[158] | 34 | is to read in a FITS data cube, find sources in the cube, and produce |
---|
| 35 | a text file of positions, velocities and fluxes of the detections, as |
---|
| 36 | well as a postscript file of the spectra of each detection. |
---|
| 37 | |
---|
[309] | 38 | \duchamp has been designed to search for objects in particular sorts |
---|
| 39 | of data: those with relatively small, isolated objects in a large |
---|
| 40 | amount of background or noise. Examples of such data are extragalactic |
---|
| 41 | \hi surveys, or maser surveys. \duchamp searches for groups of |
---|
| 42 | connected voxels (or pixels) that are all above some flux |
---|
| 43 | threshold. No assumption is made as to the shape of detections, and |
---|
| 44 | the only size constraints applied are those specified by the user. |
---|
[298] | 45 | |
---|
[447] | 46 | \duchamp has been written as a three-dimensional finder, but it is |
---|
| 47 | possible to run it on a two-dimensional image (\ie one with no |
---|
| 48 | frequency or velocity information), or indeed a one-dimensional array, |
---|
| 49 | and many of the features of the program will work fine. The focus, |
---|
| 50 | however, is on object detection in three dimensions, one of which is a |
---|
| 51 | spectral dimension. Note, in particular, that it does not do any |
---|
| 52 | fitting of source profiles, a feature common (and desirable) for many |
---|
| 53 | two-dimensional source finders. This is beyond the current scope of |
---|
| 54 | \duchamp, whose aim is reliable detection of spectral-line objects. |
---|
| 55 | |
---|
[232] | 56 | \secB{What to do} |
---|
| 57 | |
---|
[158] | 58 | So, you have a FITS cube, and you want to find the sources in it. What |
---|
| 59 | do you do? First, you need to get \duchamp: there are instructions in |
---|
| 60 | Appendix~\ref{app-install} for obtaining and installing it. Once you |
---|
| 61 | have it running, the first step is to make an input file that contains |
---|
| 62 | the list of parameters. Brief and detailed examples are shown in |
---|
| 63 | Appendix~\ref{app-input}. This file provides the input file name, the |
---|
| 64 | various output files, and defines various parameters that control the |
---|
| 65 | execution. |
---|
| 66 | |
---|
[258] | 67 | The standard way to run \duchamp is by the command |
---|
[158] | 68 | \begin{quote} |
---|
[294] | 69 | {\footnotesize |
---|
| 70 | \texttt{> Duchamp -p [parameter file]} |
---|
| 71 | } |
---|
[158] | 72 | \end{quote} |
---|
| 73 | replacing \texttt{[parameter file]} with the name of the file listing |
---|
| 74 | the parameters. |
---|
| 75 | |
---|
| 76 | An even easier way is to use the default values for all parameters |
---|
| 77 | (these are given in Appendix~\ref{app-param} and in the file |
---|
[231] | 78 | \texttt{InputComplete} included in the distribution directory) and use |
---|
| 79 | the syntax |
---|
[158] | 80 | \begin{quote} |
---|
[294] | 81 | {\footnotesize |
---|
| 82 | \texttt{> Duchamp -f [FITS file]} |
---|
| 83 | } |
---|
[158] | 84 | \end{quote} |
---|
| 85 | where \texttt{[FITS file]} is the file you wish to search. |
---|
| 86 | |
---|
[294] | 87 | The default action includes displaying a map of detected objects in a |
---|
| 88 | PGPLOT X-window. This can be disabled by setting the parameter |
---|
[298] | 89 | \texttt{flagXOutput = false} or using the \texttt{-x} command-line |
---|
[294] | 90 | option, as in |
---|
| 91 | \begin{quote} |
---|
| 92 | {\footnotesize |
---|
| 93 | \texttt{> Duchamp -x -p [parameter file]} |
---|
| 94 | } |
---|
| 95 | \end{quote} |
---|
| 96 | and similarly for the \texttt{-f} case. |
---|
[158] | 97 | |
---|
[294] | 98 | Once a FITS file and parameters have been set, the program will then |
---|
| 99 | work away and give you the list of detections and their spectra. The |
---|
| 100 | program execution is summarised below, and detailed in |
---|
| 101 | \S\ref{sec-flow}. Information on inputs is in \S\ref{sec-param} and |
---|
| 102 | Appendix~\ref{app-param}, and descriptions of the output is in |
---|
| 103 | \S\ref{sec-output}. |
---|
| 104 | |
---|
[158] | 105 | \secB{Guide to terminology and conventions} |
---|
| 106 | |
---|
[258] | 107 | First, a brief note on the use of terminology in this guide. \duchamp |
---|
[158] | 108 | is designed to work on FITS ``cubes''. These are FITS\footnote{FITS is |
---|
| 109 | the Flexible Image Transport System -- see \citet{hanisch01} or |
---|
| 110 | websites such as |
---|
| 111 | \href{http://fits.cv.nrao.edu/FITS.html}{http://fits.cv.nrao.edu/FITS.html} |
---|
[258] | 112 | for details.} image arrays with (at least) three dimensions. They |
---|
| 113 | are assumed to have the following form: the first two dimensions |
---|
| 114 | (referred to as $x$ and $y$) are spatial directions (that is, relating |
---|
| 115 | to the position on the sky -- often, but not necessarily, |
---|
| 116 | corresponding to Equatorial or Galactic coordinates), while the third |
---|
| 117 | dimension, $z$, is the spectral direction, which can correspond to |
---|
| 118 | frequency, wavelength, or velocity. The three dimensional analogue of |
---|
| 119 | pixels are ``voxels'', or volume cells -- a voxel is defined by a |
---|
[265] | 120 | unique $(x,y,z)$ location and has a single value of flux, intensity |
---|
[258] | 121 | or brightness (or something equivalent) associated with it. |
---|
[158] | 122 | |
---|
[285] | 123 | Sometimes, some pixels in a FITS file are labelled as BLANK -- that |
---|
| 124 | is, they are given a nominal value, defined by FITS header keywords |
---|
| 125 | \textsc{blank, bscale, \& bzero}, that marks them as not having a flux |
---|
| 126 | value. These are often used to pad a cube out so that it has a |
---|
| 127 | rectangular spatial shape. \duchamp has the ability to avoid these: |
---|
| 128 | see \S\ref{sec-blank}. |
---|
| 129 | |
---|
[158] | 130 | Note that it is possible for the FITS file to have more than three |
---|
[232] | 131 | dimensions (for instance, there could be a fourth dimension |
---|
| 132 | representing a Stokes parameter). Only the two spatial dimensions and |
---|
| 133 | the spectral dimension are read into the array of pixel values that is |
---|
| 134 | searched for objects. All other dimensions are ignored\footnote{This |
---|
| 135 | actually means that the first pixel only of that axis is used, and the |
---|
| 136 | array is read by the \texttt{fits\_read\_subsetnull} command from the |
---|
[160] | 137 | \textsc{cfitsio} library.}. Herein, we discuss the data in terms of |
---|
| 138 | the three basic dimensions, but you should be aware it is possible for |
---|
| 139 | the FITS file to have more than three. Note that the order of the |
---|
| 140 | dimensions in the FITS file does not matter. |
---|
[158] | 141 | |
---|
[232] | 142 | With this setup, each spatial pixel (a given $(x,y)$ coordinate) can |
---|
| 143 | be said to be a single spectrum, while a slice through the cube |
---|
| 144 | perpendicular to the spectral direction at a given $z$-value is a |
---|
[265] | 145 | single channel, with the 2-D image in that channel called a channel |
---|
| 146 | map. |
---|
[158] | 147 | |
---|
| 148 | Detection involves locating a contiguous group of voxels with fluxes |
---|
[258] | 149 | above a certain threshold. \duchamp makes no assumptions as to the |
---|
[158] | 150 | size or shape of the detected features, other than having |
---|
[160] | 151 | user-selected minimum size criteria. Features that are detected are |
---|
| 152 | assumed to be positive. The user can choose to search for negative |
---|
| 153 | features by setting an input parameter -- this inverts the cube prior |
---|
| 154 | to the search (see \S\ref{sec-detection} for details). |
---|
[158] | 155 | |
---|
[232] | 156 | \secB{A summary of the execution steps} |
---|
| 157 | |
---|
| 158 | The basic flow of the program is summarised here -- all steps are |
---|
| 159 | discussed in more detail in the following sections. |
---|
| 160 | \begin{enumerate} |
---|
| 161 | \item The necessary parameters are recorded. |
---|
| 162 | |
---|
| 163 | How this is done depends on the way the program is run from the |
---|
| 164 | command line. If the \texttt{-p} option is used, the parameter file |
---|
| 165 | given on the command line is read in, and the parameters therein are |
---|
| 166 | read. All other parameters are given their default values (listed in |
---|
| 167 | Appendix~\ref{app-param}). |
---|
| 168 | |
---|
| 169 | If the \texttt{-f} option is used, all parameters are assigned their |
---|
| 170 | default values. |
---|
| 171 | |
---|
| 172 | \item The FITS image is located and read in to memory. |
---|
| 173 | |
---|
| 174 | The file given is assumed to be a valid FITS file. As discussed |
---|
[258] | 175 | above, it can have any number of dimensions, but \duchamp only |
---|
[232] | 176 | reads in the two spatial and the one spectral dimensions. A subset |
---|
| 177 | of the FITS array can be given (see \S\ref{sec-input} for details). |
---|
| 178 | |
---|
| 179 | \item If requested, a FITS file containing a previously reconstructed |
---|
| 180 | or smoothed array is read in. |
---|
| 181 | |
---|
[285] | 182 | When a cube is either smoothed or reconstructed with the \atrous |
---|
| 183 | wavelet method, the result can be saved to a FITS file, so that |
---|
| 184 | subsequent runs of \duchamp can read it in to save having to re-do |
---|
| 185 | the calculations (as they can be relatively time-intensive). |
---|
[232] | 186 | |
---|
| 187 | \item \label{step-blank} If requested, BLANK pixels are trimmed from |
---|
| 188 | the edges, and the baseline of each spectrum is removed. |
---|
| 189 | |
---|
[285] | 190 | BLANK pixels, while they are ignored by all calculations in |
---|
| 191 | \duchamp, do increase the size in memory of the array above that |
---|
| 192 | absolutely needed. This step trims them from the spatial edges, |
---|
| 193 | recording the amount trimmed so that they can be added back in |
---|
| 194 | later. |
---|
[232] | 195 | |
---|
[285] | 196 | A spectral baseline (or bandpass) can also be removed at this point |
---|
| 197 | as well. This may be necessary if there is a ripple or other |
---|
[265] | 198 | large-scale feature present that will hinder detection of faint |
---|
| 199 | sources. |
---|
[232] | 200 | |
---|
| 201 | \item If the reconstruction method is requested, and the reconstructed |
---|
| 202 | array has not been read in at Step 3 above, the cube is |
---|
[258] | 203 | reconstructed using the \atrous wavelet method. |
---|
[232] | 204 | |
---|
[258] | 205 | This step uses the \atrous method to determine the amount of |
---|
[232] | 206 | structure present at various scales. A simple thresholding technique |
---|
| 207 | then removes random noise from the cube, leaving the significant |
---|
| 208 | signal. This process can greatly reduce the noise level in the cube, |
---|
| 209 | enhancing the detectability of sources. |
---|
| 210 | |
---|
[275] | 211 | \item Alternatively (and if requested), the cube is smoothed, either |
---|
| 212 | spectrally or spatially. |
---|
[232] | 213 | |
---|
[275] | 214 | This step presents two options. The first considers each spectrum |
---|
| 215 | individually, and convolves it with a Hanning filter (with width |
---|
| 216 | chosen by the user). The second considers each channel map |
---|
[285] | 217 | separately, and smoothes it with a Gaussian kernel of size and shape |
---|
| 218 | chosen by the user. This step can help to reduce the amount of noise |
---|
| 219 | visible in the cube and enhance fainter sources. |
---|
[232] | 220 | |
---|
| 221 | \item A threshold for the cube is then calculated, based on the pixel |
---|
| 222 | statistics (unless a threshold is manually specified by the user). |
---|
| 223 | |
---|
| 224 | The threshold can either be chosen as a simple $n\sigma$ threshold |
---|
[285] | 225 | (\ie a certain number of standard deviations above the mean), or |
---|
| 226 | calculated via the ``False Discovery Rate'' method. Alternatively, |
---|
| 227 | the threshold can be specified as a simple flux value, without care |
---|
| 228 | as to the statistical significance (\eg ``I want every source |
---|
| 229 | brighter than 10mJy''). |
---|
[232] | 230 | |
---|
[265] | 231 | By default, the full cube is used for the statistics calculation, |
---|
| 232 | although the user can nominate a subsection of the cube to be used |
---|
| 233 | instead. |
---|
| 234 | |
---|
[232] | 235 | \item Searching for objects then takes place, using the requested |
---|
| 236 | thresholding method. |
---|
| 237 | |
---|
[265] | 238 | The cube is searched one channel-map at a time. Detections are |
---|
[264] | 239 | compared to already detected objects and either combined with a |
---|
| 240 | neighbouring one or added to the end of the list. |
---|
[232] | 241 | |
---|
| 242 | \item The list of objects is condensed by merging neighbouring objects |
---|
| 243 | and removing those deemed unacceptable. |
---|
| 244 | |
---|
[264] | 245 | While some merging has been done in the previous step, this process |
---|
| 246 | is a much more rigorous comparison of each object with every other |
---|
| 247 | one. If a pair of objects lie within requested limits, they are |
---|
| 248 | combined. |
---|
[232] | 249 | |
---|
[264] | 250 | After the merging is done, the list is culled (although see comment |
---|
| 251 | for the next step). There are certain criteria the user can specify |
---|
| 252 | that objects must meet: minimum numbers of spatial pixels and |
---|
| 253 | spectral channels, and minimum separations between neighbouring |
---|
| 254 | objects. Those that do not meet these criteria are deleted |
---|
| 255 | from the list. |
---|
| 256 | |
---|
| 257 | \item If requested, the objects are ``grown'' down to a lower |
---|
| 258 | threshold, and then the merging step is done a second time. |
---|
| 259 | |
---|
| 260 | In this case, each object has pixels in its neighbourhood examined, |
---|
| 261 | and if they are above a secondary threshold, they are added to the |
---|
| 262 | object. The merging process is done a second time in case two |
---|
| 263 | objects have grown over the top of one another. Note that the |
---|
| 264 | rejection part of the previous step is not done until the end of the |
---|
| 265 | second merging process. |
---|
| 266 | |
---|
[232] | 267 | \item The baselines and trimmed pixels are replaced prior to output. |
---|
| 268 | |
---|
| 269 | This is just the inverse of step~\#\ref{step-blank}. |
---|
| 270 | |
---|
| 271 | \item The details of the detections are written to screen and to the |
---|
| 272 | requested output file. |
---|
| 273 | |
---|
| 274 | Crucial properties of each detection are provided, showing its |
---|
| 275 | location, extent, and flux. These are presented in both pixel |
---|
| 276 | coordinates and world coordinates (\eg sky position and |
---|
| 277 | velocity). Any warning flags are also printed, showing detections to |
---|
[265] | 278 | be wary of. Alternative output options are available, such as a |
---|
| 279 | VOTable or a Karma annotation file. |
---|
[232] | 280 | |
---|
| 281 | \item Maps showing the spatial location of the detections are written. |
---|
| 282 | |
---|
| 283 | These are 2-dimensional maps, showing where each detection lies on |
---|
| 284 | the spatial coverage of the cube. This is provided as an aid to the |
---|
| 285 | user so that a quick idea of the distribution of object positions |
---|
| 286 | can be gained \eg are all the detections on the edge? |
---|
| 287 | |
---|
| 288 | Two maps are provided: one is a 0th moment map, showing the 0th |
---|
[265] | 289 | moment (\ie a map of the integrated flux) of each detection in its |
---|
| 290 | appropriate position, while the second is a ``detection map'', |
---|
| 291 | showing the number of times each spatial pixel was detected in the |
---|
| 292 | searching routines (including those pixels rejected at step 9 and so |
---|
| 293 | not in any of the final detections). |
---|
[232] | 294 | |
---|
| 295 | These maps are written to postscript files, and the 0th moment map |
---|
| 296 | can also be displayed in a PGPLOT X-window. |
---|
| 297 | |
---|
| 298 | \item The integrated or peak spectra of each detection are written to a |
---|
| 299 | postscript file. |
---|
| 300 | |
---|
| 301 | The spectral equivalent of the maps -- what is the spectral profile |
---|
| 302 | of each detection? Also provided here are basic information for each |
---|
| 303 | object (a summary of the information in the results file), as well |
---|
| 304 | as a 0th moment map of the detection. |
---|
| 305 | |
---|
| 306 | \item If requested, the reconstructed or smoothed array can be written |
---|
| 307 | to a new FITS file. |
---|
| 308 | |
---|
| 309 | If either of these procedures were done, the resulting array can be |
---|
| 310 | saved as a FITS file for later use. The FITS header will be the same |
---|
| 311 | as the input file, with a few additional keywords to identify the |
---|
| 312 | file. |
---|
| 313 | |
---|
| 314 | \end{enumerate} |
---|
| 315 | |
---|