source: trunk/CHANGES

Last change on this file was 1439, checked in by MatthewWhiting, 7 years ago

Adding changes for the imminent 1.6.2 release (#527)

File size: 67.2 KB
RevLine 
[1439]1Duchamp version 1.6.2 (2017/09/25)
2----------------------------------
3
4A patch release after a long hiatus! This addresses a bunch of bugs
5and other issues that have been identified:
6
7 * The plotting of spectra was failing when spectralMethod=peak was
8   used. This has been fixed, with the consequent fixing of the
9   verification scripts.
10 * Numerous patches that were identified during the course of
11   ASKAPsoft development have been incorporated back into the Duchamp
12   code-base. These largely either provided additional flexibility for
13   particular functions & classes, or fix bugs. The latter include:
14    - Applying offsets for Object3D spatial maps
15    - Correctly setting offsets in 2D images
16    - Fixes to some compilation errors when using more recent (gcc 6+)
17      compilers
18 * The sizes of ellipses plotted in Karma annotation files where 60x
19   too big (being given in arcmin rather than degrees).
20 * A few fixes to the user guide have also been made, correcting some
21   minor errors in equations.
22   
23
[1389]24Duchamp version 1.6.1 (2014/05/01)
25----------------------------------
26
27Principal reason for the release is to fix a memory leak. This was
28occuring when making blank pixel masks, particularly in the
29parameterisation step. It was apparent with large files and large
30numbers of sources. Thanks to Quang Nguyen for raising the bug and
31helping with the identification of the problem.
32
[1397]33For those interested in the code base itself, all boolean masks have
34been moved to vector<bool> types, with the consequent change of
35interface for some functions (including the statistics functions,
36Gaussian smoothing, and some spectral utility functions).
37
[1389]38A few other potential memory leaks were identified and fixed.
39
40To aid the debugging, the progress bar now also reports the percentage
41complete as well as the set of hashes.
42
43
[1375]44Duchamp version 1.6 (2014/04/22)
[1362]45--------------------------------
46
47Improvements and bug fixes:
48
49* When searching for negative features while using baseline
50  subtraction, the parameterisation of the resulting detections was
51  not being done correctly. Duchamp now correctly identifies the peak
52  absorption location and measures the peak as being the peak
53  absorption from the fitted continuum level. A future release will
54  add more absorption-line-oriented parameters.
55
56* The way the borders of detections are displayed in annotation files
57  has been improved. Duchamp now writes them in a polygon or
58  join-the-dots fashion. This reduces the size of the annotation
59  files, and is now compatible with the CASA viewer.
60
61* The VOTables now refer to version 1.3 of the VOTable schema. Note
62  that they still use the old COOSYS method of specifying the
63  coordinates. Moving to the preferred STC method will happen in a
64  future release.
65
66* There were some problems with the storage and recovery of parameters
67  in the binary catalogue - these have been fixed.
68
69* The BITPIX keyword in the mask FITS files that Duchamp writes are
70  now SHORT_IMG, although they can be bigger if warranted by the
71  number of objects.
72
[1373]73* Duchamp can now read .fits.gz files, and have the output FITS files
74  appropriately named in the automatic way (the .gz suffix was not
75  being recognised & removed properly).
76
[1362]77* Some UCDs in the VOTable were missing semicolons between elements.
78
[1372]79* When calculating the cube statistics, Duchamp makes a mask showing
80  the locations of blank pixels, flagged channels, and the statistics
81  subsection are, so that only the required pixels are used in the
82  calculations. If none of these apply, however (that is, if there are
83  no blank pixels or flagged channels, and the user is not requesting
84  a particular stats subsection via the statsec parameter), then the
85  mask is not created. This saves additional memory allocation, which
86  can be important for large cubes. Thanks to Andrew Walsh for
87  identifying this problem.
88
[1375]89* Duchamp exits better (ie. without throwing a segmentation fault)
90  when the requested FITS file does not exist.
91
[1378]92* The reference cube used for the verification script has been changed
93  - the sources therein are now smaller spatially (more appropriately
94  so for the pixel size), which gives a better idea as to the
95  performance.
96
[1362]97Documentation improvmements
98
99* The InputComplete file is more up-to-date, and has information on
100  the type and default values of each possible input parameter (this
101  information is also reflected in the User Guide). This is in
[1375]102  preparation for a parameter-editing GUI that will be released in a
[1362]103  forthcoming version.
104
105* The User's Guide now has a section on common installation issues,
106  based on feedback and experience with new users.
107
108Build changes:
109
110* Duchamp is now linked with the C++ compiler. Accordingly, the
111  configure script searches for libgfortran when the gfortran compiler
[1375]112  is specified as the F77 compiler (ie. it has been used to build
113  pgplot).
[1362]114
115* To allow Duchamp to build on Mac OS X 10.9 (Mavericks), configure
116  searches for the libc++ library as well as stdc++. Duchamp has been
117  successfully built on 10.9, with fresh installs of pgplot (5.2),
118  wcslib (4.20) and cfitsio (3.36), but, as always, your mileage may
119  vary. Please contact me via the Trac tickets if you encounter
120  problems.
121
[1373]122* A number of warnings that came to light with recent XCode upgrades
[1375]123  on the Mac have been fixed, as well as those from an upgrade to
124  gcc-4.4.5 on debian.
[1362]125
126
127Code base changes:
128
129* There are new wcs-pix conversion functions in fitsHeader that allow
130  the specification of a single set of pixel and world coordinates
131  (rather than arrays).
132
133* There is cleaner separation of the writing functions in the
134  VOTableCatalogueWriter class.
135
136* The management of the wavelet filters has been improved, with new
[1376]137  classes to handle the different filter types, and a factory class to
138  generate them. The interface via the Param class is largely the
139  same, however.
[1362]140
[1376]141* The annotation file changes noted above involve a new PixelInfo
142  class called Line, with associated files. The method for drawing the
143  borders has been moved into its own file (drawBorders.cc) to allow
144  dependence on the pgplot library.
145
[1375]146* The function Cube::reInvert() has been removed - use Cube::invert()
147  instead. Note that the interface for the latter now has two optional
148  bool parameters.
149
[1362]150* There are improved front-end functions for the writing of arrays to
151  the various FITS files.
152
[1375]153
154
[1286]155Duchamp version 1.5 (2013/08/29)
[1249]156--------------------------------
157
[1280]158New features & algorithms:
[1249]159
160* There is now a more flexible way to specify channels that should be
161  ignored by the source-finding. Use flaggedChannels to provide a list
162  of single channels or channel ranges that should be ignored. This
163  replaces the flagMW/minMW/maxMW parameters, extending their
164  functionality and allowing more than one set of channels to be
165  specified.
166
[1278]167* There is now an alternative way to find (and remove) spectral
168  baselines, using a median baselining algorithm. This calculates the
169  baseline value for a given pixel to be the median value within a box
170  of a specified size centred on that pixel. There are two input
171  parameters now available, baselineType (atrous or median), and
172  baselineBoxWidth (the width of the median box). Consult the User's
173  Guide for further information.
174
[1280]175* There is now more flexibility in running the spatial smoothing. The
176  cutoff value in determining the size of the smoothing kernel can now
177  be set via the input parameters, as can the way the edge pixels get
178  treated. Conult the User's Guide for details.
179
[1287]180* You can now also specify the maximum number of voxels, spatial
181  pixels and/or channels, in a way analogous to the minimum
182  values. Detections with more than the maximum are rejected. Negative
183  values (such as the defaults) mean this comparison is not done.
184
[1274]185Output changes:
186
187* The results files now include the Nchan and Nspatpix columns,
188  showing the number of distinct channels and the number of spatial
189  pixels in each detection. For clarity, the Npix column has been
190  renamed Nvoxel.
191
192* The VOTable now includes the pixel location X, Y, Z columns, and the
193  pixel ranges - this makes it much closer to the ASCII results file
194  (it is just missing the string versions of RA/DEC, and F_tot when
195  F_int is calculated).
196
[1266]197Bug fixes:
[1249]198
[1266]199* Negative velocity (and other) values were resulting in the column
200  widths not adjusting correctly, leading to columns running into each
201  other. This has been fixed, so that the spacing in the results files
202  should be correct.
203
[1270]204* Fixed a problem applicable to cubes with blank/masked pixels in the
205  same spectra as detections. The velocity width calculations could
206  sometimes fail due to the mask not being applied.
[1266]207
[1249]208Code changes:
209
210* The plotting code has been refactored, improving its class structure
211  and inheritance. No functional change from this should be evident,
212  but it improves the maintainability of the code.
213
214
[1229]215Duchamp version 1.4 (2013/06/20)
216--------------------------------
217
218Installation:
219
220* The "make lib" command will now create a shared library
221  libduchamp.1.4.so as well as the static library libduchamp.1.4.a
222  (formerly just called libduchamp.a).
223
224Bug Fixes:
225
226* The orientation of ellipses in the DS9 region files are now correct
227  (they were wrong by 90 degrees).
228
229* Corrected the output FITS filenames so that they work when the input
230  filename does *not* end in ".fits".
231
232Code changes:
233
234* The code that manages the output columns for the ASCII and VOTable
235  output has been overhauled and streamlined. Little functional change
236  should be evident, but it should be much more robust. And easier to
237  use in derived software.
238
[1213]239Duchamp version 1.3.2 (2013/05/02)
240----------------------------------
241
242Bug fixes:
243
244* The units for the integrated flux (for F_int) were not being
245  calculated for cases of a FITS file without a spectral
246  axis. Additionally, a vestigial default value meant this column had
247  units of "counts". This has been fixed so that a FITS file with
248  Jy/beam units will have F_int values in Jy.
249
250* The ellipse fitting to find the MAJ/MIN/PA was not working correctly
251  in the case of negative searches. This has been fixed.
252
253* The scaling of MAJ/MIN values was not being done correctly in all
254  cases, so that the values were out by a factor of 60 or 3600. This
255  was particularly the case when the ellipse fitting failed (due to
256  small numbers of pixels, for instance).
257
258* The principle axes calculations have been simplified slightly, and
259  now are done only on pixels above half the peak in the moment-0 map
260  (to estimate the FWHM).
261
[1202]262Duchamp version 1.3.1 (2013/04/18)
263----------------------------------
264
265Bug fixes:
266
267* The kernel used in the 2D smoothing was not normalised properly,
268  which meant the fluxes in the smoothed array were scaled
269  incorrectly. The kernel is now normalised, so that the fluxes in the
270  smoothed array decrease (and to an increasing degree with larger
271  kernels, as expected).
272
273* The way the position angle of the smoothing kernel was interpreted
274  was wrong. It now conforms to the typical astronomical position
275  angle definition of degrees east of north.
276
277* The Z_PEAK column was being left out of the results file for
278  Duchamp-1.3. It has been put back in.
279
280* The calculation of S/Nmax has changed so that when smoothing or
281  reconstruction is done, the peak flux used is measured in the
282  reconstructed/smoothed array (where the noise is measured).
283
284* The flux errors are now not reported in the case of smoothing, since
285  we don't measure the noise in the original input image array.
286
287Other fixes:
288
289* The vertical range of the spectral plots now properly takes into
290  account all spectra (not just the input array), and quirks with a MW
291  range lying outside the plotted range have been fixed.
292
293* The flux scaling for the moment-0 map plot now ignores negative
294  values, meaning the scale should be sensible.
295
296* The spelling of the Object2D::getPrincipalAxes() has been corrected.
297
298* Minor code optimisations have been made to the smoothing code.
299
300* The subsection class is now initialised with a blank string.
301
[1181]302Duchamp version 1.3 (2013/03/08)
303--------------------------------
[1150]304
305New features:
306
[1159]307* New source parameters:
[1150]308
[1159]309  # Each source now has a measurement of the major & minor axis sizes,
310    along with the position angle of the major axis. The sizes are the
311    full-width at half maximum, measured from the moment-0 map. These
312    now appear in the catalogue and in the spectral plots, with the
313    ellipse drawn on the moment map cutout in the spectral plots. The
314    old measurements of w_RA & w_DEC are still made and included in
315    the full catalogue, but are no longer reported in the smaller
316    catalogue printed to screen (nor in the XML/VOTable output).
[1150]317
[1159]318  # Each source also now has a measurement of the error on the
319    integrated and total fluxes. This is calculated from summing in
[1178]320    quadrature the statistical noise on each detected voxel. This is
321    from a request by Sara Shakouri.
[1150]322
[1159]323* Output files - new types and changes to the way they are written:
[1150]324
[1159]325  # Ability to create CASA region format files showing the location of
326    the detected sources. This is analogous to the Karma annotation
327    files or DS9 region files, for use with the casa viewer, but can
328    also be imported into casapy.
[1150]329
[1159]330  # Ability to save baseline values to a FITS file (in the same way as
331    for the reconstructed or smoothed array).
[1150]332
[1159]333  # Ability to save the moment-0 mask to a FITS file (that is, a mask
334    indicating which spatial pixels form part of the moment-0 map).
[1150]335
[1159]336  # There is an improved way to store the detected pixels for later
337    re-use. While previously the detected pixels were written as scans
338    to the logfile, and then read in when the usePrevious flag was
339    set, we now write the scans and other metadata to a binary
340    catalogue that can be read in again. Writing as a binary file
341    leads to much better compression of the data, particularly when
342    combined with the run-length encoding. It is anticipated future
343    tools will be provided to allow additional interaction with these
344    binary catalogues.
345
346  # There are slight changes to the format of the ASCII catalogues,
347    where lines not containing sources are prefixed with a comment
348    character '#'. This will facilitate their use in other software.
349
[1161]350* There is a new command-line parameter, -t THRESH, which specifies
351  the flux threshold for searching. It is designed to be used with the
352  -f option, as a quick way of searching a FITS image to a specified
353  flux level.
[1159]354
[1150]355Bug fixes:
356
357* A bug in the spatial smoothing has been fixed. The mask was being
358  interpreted incorrectly, so that no valid pixels were getting
359  smoothed.
360
361* The WCS for the moment-0 FITS cube has been fixed (specifically, the
362  axis names).
363
364* When usePrevious was used, the behaviour in previous versions was to
365  always write individual spectra. This is now controlled by a new
366  parameter flagPlotIndividualSpectra.
367
368* Minor improvements to the output when verbose=false.
369
370* Improvements to the Makefile, so that header files from a
371  previously-installed version of Duchamp do not get used instead of
372  those from the current version.
373
374* The vertical range for the spectral plots now take into account the
375  baseline and reconstructed/smoothed arrays as well (outside of any
376  MW range, of course).
377
378Other
379
380* The code-base for reading from and writing to FITS files has been
381  overhauled and streamlined. It can now be found in the FitsIO
[1180]382  directory. The interface for the writing of FITS files via the Cube
383  class has not changed.
[1150]384
[1161]385* The User Guide has been slightly re-ordered to provide more explicit
386  information about the source parameters.
[1150]387
[1161]388
[1107]389Duchamp version 1.2.2 (2012/10/29)
390----------------------------------
391
392Bug fixes:
393
394* The reading of reconstructed & smoothed arrays in the case of large
395  (>2GB) cubes has been improved. While the most recent versions of
396  cfitsio were able to handle these, earlier versions would result in
397  a segementation fault using the previous interface. An alternative
398  interface has been implemented that avoids these faults, even for
399  the older cfitsio libraries.
400
401* A bug affecting the writing of FITS mask files (apparently for the
402  case of flagging by the object number) has been fixed.
403
404* The moment map occasionally failed if sources lay spatially within
405  the blank pixel region of the first channel. This check has been
406  removed and the moment map should be produced successfully in all
407  situations.
408
409Other:
410
411* The UCDs used in the VOTable catalogue output have been
412  updated. Several of these were shared between different columns, and
413  so they have been changed so that each UCD is unique to one
414  column. The column ID strings have been updated for clarity.
415
416* The determination of default output image filenames has been
417  improved -- it previously assumed the input filename ended in
418  '.fits', but this is no longer the case.
419
[1087]420Duchamp version 1.2.1 (2012/10/09)
421----------------------------------
422
423New feature:
424
425* Duchamp can now produce region files to annotate images displayed in
426  SAOImage DS9. There are new parameters flagDS9 and ds9File that work
427  in the same way as the Karma annotation file parameters. The region
428  files can also be displayed in (at least some versions of)
429  casaviewer.
430
431* There is a new script to fix occasional problems with the
432  automatically-detected Makefile, and the verification script has
433  been updated and improved.
434
435Bug fixes:
436
437* The main bug fix for this release corrects a problem with the
438  velocity width measurements made when searching for negative
439  detections. It was possible to get very small (and incorrect) widths
440  in some cases - this was due to the integrated spectrum not being
441  inverted correctly.
442
443* In the case of smoothing, the stats of the full cube that were being
444  reported in the output files were actually those of the un-smoothed
445  (input cube). This only affected the output files, not the detection
446  process itself.
447
448* The units were not being reported in the VOTable. This has been
449  fixed.
450
451* The UCDs for the spectral parameters in the VOTable were not being
452  adapted to the user requests, so didn't always match the types of
453  the parameters. All should be consistent now.
454
455Other changes:
456
457* The codebase relating to the creation of and writing to output files
458  (results, logs, annotation, VOTables) has been overhauled, with a
[1088]459  cleaner interface. Most of this code is in the new subdirectory
460  src/Outputs, to where the columns code has been moved as well.
[1087]461
[1088]462  This was instigated by ASKAP/Selavy development, but also allowed
463  the expansion of output formats to include DS9 region files. No
464  change to the user should be visible, but anyone using the software
465  will notice differences. Please get in touch if changes are
466  difficult to follow.
467
[1035]468Duchamp version 1.2 (2012/08/08)
[962]469--------------------------------
[928]470
[951]471Key changes:
472
473* The way the wavelet reconstruction is done in the presence of BLANK
474  voxels is slightly different, specifically when there were BLANK
475  voxels on the edge. While BLANK voxels are always ignored by the
476  reconstruction (and left in place), the previous approach dealt with
477  BLANKS on the edge by moving the effective edge of the image inwards
478  until the first non-blank voxel. This meant that the size of the
479  dimensions used to determine the number of scales, and the location
480  where boundary reflections took place where different to the input
481  image size.
482
483  However, this meant that BLANK voxels on the edge were treated
484  differently to those not on the edge, so this inconsistency was
485  removed. The full image dimensions are always used to calculate the
486  number of scales, and reflections (which is how the reconstruction
487  deals with the boundaries) will occur at the actual image edges.
488
489  If your data has this property (BLANK pixels at the image
490  boundaries), then any reconstruction will be affected (most likely
491  only in a small way) and potentially any subsequent searching. You
[1035]492  are advised to check this (for instance by saving the results of the
493  resonstruction from 1.1.13 and 1.2 and comparing).
[951]494
[1035]495* The way the spectral world coordinate system is dealt with has been
496  improved. Previously Duchamp tried to convert to a velocity axis,
497  and got a bit grumpy if that didn't work well. Now it will express
498  the spectral units in the axis type indicated by the FITS
499  header. Duchamp will also allow you to change this type by providing
500  an appropriate WCS type as an input parameter. You may also provide
501  a rest frequency in the parameter file and this will be used to
502  calculate velocities. And, as before, you can provide specific units
503  for the spectral quantities to be converted into.
504
[951]505* Since v1.1.13 was released, the journal paper describing Duchamp has
506  been published. It can be found at MNRAS 421, 3242 (2012), or
507  http://onlinelibrary.wiley.com/doi/10.1111/j.1365-2966.2012.20548.x/full. It
508  includes descriptions of the various algorithms used for Duchamp, as
509  well as some indicative comparisons of the relative completeness &
510  reliability. Please cite this paper when describing Duchamp.
511
512Other new features:
513
[1035]514* The default value of snrCut has changed from 3 to 5. This is
515  espeically important if you use the -f option, but also if you don't
516  provide snrCut in your parameter file.
517
[951]518* The XML VOTable output now includes a more complete list of
519  parameters, matching the output given in text form in the results &
520  log output files. Only those parameters affecting the results are
521  included - things like the output file names and so on are left out.
522
523* The minVoxels parameter is now adjusted to be at least (minPix +
524  minChannels - 1). If it is changed, a warning message is triggered.
525
526* The warning/error reporting has been overhauled, with an improved
527  interface for developers. Also, some ERROR instances that previously
528  resulted in the program terminating now allow it to continue
529  (provided that is possible given the error).
530
[1026]531* It is now possible to give the convergence criterion for the wavelet
532  reconstruction via the parameter reconConvergence. When the relative
533  change in standard deviation of the residuals from the
534  reconstruction falls below this value, the a trous algorithm stops
535  iterating.
536
[991]537* The way one-dimensional arrays were dealt with was unsatisfactory,
538  both with the way positions were registered and the format of the
539  graphical output. Improvements have been made such that a 1D data
540  structure is assumed to be a spectrum, and so positions in the
541  spectrum are reported as z-pixels (or the equivalent world
542  coordinate). Instead of a 2D moment map, the X-windows display will
543  show the spectrum with all detected pixels plus the final list of
544  detections highlighted, along with the detection and growth
[1035]545  thresholds. This plot will also be written to the "detectionMap"
546  image.
[991]547
548* All spectral plots will now show the detection (and growth, if
549  applicable) thresholds when plotting the peak flux spectrum.
550
551* Accessing FITS files when a subsection string is provided has been
552  streamlined. The cfitsio package, which handles the FITS access,
553  creates a temporary file containing this image section, and this
554  meant the previous approach was slow, particularly for large
555  files/sections. Now the file is only opened once and the overhead of
556  creating the temporary file is only encountered once.
557
558* Several aspects of the processing & output did not handle a
559  non-existent WCS well. This has been improved, so that all stages of
560  execution will proceed without errors.
561
[928]562Bug fixes:
563
[941]564* The absence of a BLANK keyword when trimming will now not trigger a
565  failure, only a warning message.
[928]566
[941]567* The flux unit conversion (eg. from Jy/beam to mJy/beam) was not
568  being applied correctly when an existing reconstructed or smoothed
569  array was being read in. This is now applied properly.
570
571* When saving reconstructed or smoothed arrays, the data type is now
572  set to FLOAT, so that the correct values are written (and
573  subsequently read in). The previous version was potentially saving
574  to a FITS file with integer data type (this depended on the type of
[951]575  the input array). In this case, the full precision of data values
576  would not be preserved, so reading back in and doing an identical
577  search could yield different sources.
[941]578
579* For certain cases, the reconstruction attempted to use a very large
580  number of scales. This has been fixed.
581
[1026]582* The maximum scale for the reconstruction was not always being
583  applied. The scale ranges are now enforced for all types of
584  reconstructions.
585
[941]586* When saving a mask image while using a subsection of a cube, the
587  offsets for the subsection were not being accounted for
588  correctly. This is now fixed.
589
[991]590* The detection map plots were not recording pixels added from the
591  growing stage. This has been fixed.
592
[941]593* The RA & Dec now have appropriate precision, rather than an
594  arbitrary 2 decimal places on the seconds value. The RA will be one
595  decimal place more than the Dec, and they will reflect the degree of
596  precision in the image pixels.
597
598* A number of data type conversions (typically ints to floats) led to
599  errors in some calculations, such as WCS conversions. Care has been
600  taken to make sure all indexing is done appropriately with no
601  overflow, and that all conversions (where necessary) are done
602  appropriately.
603
604* It was possible to get world positions looking like 11:59:60.0! Not
605  any more.
606
[1020]607* Reading previous detections from the log file (via the use of the
608  usePrevious parameter) did result in error messages due to slightly
609  incompatible formats. These have been fixed, and all graphical
610  output should work properly.
[928]611
[1020]612* When using a spectral search, having a blank pixel in the first
613  channel could lead to that entire spectrum being ignored, leading to
614  inconsistent results between spectral and spatial searches. This
615  issue has been fixed, with both types of searches returning the same
616  catalogue.
617
618
[928]619Code base development notes:
620
[951]621* Most instances of integer overflow due to insufficient precision
622  have been tracked down - many of these were indexing variables of
623  type int, which have been changed to size_t.
[928]624
[1020]625* Importantly, for code interfacing with the duchamp::Cube and related
626  classes, axisDim is now an array of size_t elements, rather than
627  long.
628
[951]629* Object3D now carries its spatial map with it, rather than
630  calculating it on the fly when requested. This saves a lot of
631  unnecessary computational effort.
[928]632
[951]633* The warning/error interface has improved, making use of C++
634  exception handling. This should be compatible with external logging
635  infrastructure (such as log4cxx).
636
[1035]637* The interface to the FitsHeader & WCS structures for external code
638  has been improved, so that the WCS can be set up via function calls.
[951]639
640
641
[852]642Duchamp version 1.1.13 (2011/04/10)
643-------------------------------------
644
645Bug fixes:
646
647* The median statistics calculations had a bug in the test for whether
648  the array under consideration was of even size. The correction for
649  this can mean small differences in the resulting values. This can
650  flow through to calculated detection thresholds - please check this
651  with your data if you use this mode!
652
653* The reconstruction code had a bug where the incorrect number of
654  scales was being used. This occurred when the blank pixel flag was
655  set, and when the spectral dimension was smaller than both the
656  spatial dimensions.
657
658Other:
659
660* The default value of the blank pixel flag has been set to false,
661  rather than true. For Duchamp, this should not matter, as it is set
662  when the FITS file is read in. For external code using, for
663  instance, the reconstruction algorithms, this default may be
664  important. Having it true was contributing to the second bug listed
665  above. It also makes more sense for it to be false unless explicitly
666  set in some way.
667
668* The reconstruction code has been streamlined a little, reducing its
669  memory impact and using more appropriate variable types
670  (eg. unsigned ints instead of ints and so forth).
671
672* For those who care about the actual code base, the statistics
673  functions have been expanded, and separated from getStats.cc into
674  "normal" and "robust" versions.
675
676
[836]677Duchamp version 1.1.12 (2011/03/18)
678-------------------------------------
679
680Bug fixes:
681
682* The velocity width calculations had a bug that occasionally led to
683  very large values. The widths are calculated using the integrated
684  spectrum, using a threshold relative to its peak. In v.1.1.10 and
685  1.1.11, however, the peak was searched for over the entire spectral
686  range, rather than just the range for the current object. This led
687  to spurious values. This has been fixed, so that the peak is only
688  searched for within the detected ranage of the object.
689
690* Both the pixel and statistics subsections are now more carefully
691  examined, with errors reported if there are no valid pixels in the
692  pixel subsection, or if the statistics subsection does not intersect
693  with the pixel subsection.
694
695Other:
696
697* The configure script has been slightly improved so that the pgsbox
698  library is only added if PGPlot has been enabled - this is
699  particularly necessary with the slightly changed build procedure for
700  recent wcslib versions.
701
702* In a related note, one file - momentMap.cc - had some #include
703  statements that failed when PGPlot (and hence pgsbox in WCSLIB) was
704  not enabled. These were actually unnecesssary and have been removed.
705
706* There is a new input parameter - flagPlotSpectra - that allows the
707  user to turn off the production of the spectral plots. It defaults
708  to true (meaning the plots are produced).
709
710
[819]711Duchamp version 1.1.11 (2011/03/01)
[799]712-------------------------------------
713
714Improvements:
715
716* The largest changes in this release involved improving the
717  efficiency of the code, resulting in further speed-ups. The code
718  affected was the low-level object-handling code, with the merging
719  stages in particular showing good speed-ups.
720
721* The default value for the beam size, when the beam parameters are
722  not in the FITS header, has been dropped. Now the default behaviour
723  is to simply not make any beam corrections if the beam is not
724  specified by either the FITS file or the input parameters. Thanks to
725  Tobias Westemeier for the suggestion.
726
727* The reporting of the beam size has changed slightly to provide a bit
728  more information.
729
730Bug Fixes:
731
[819]732* The statistics section (statSec) could be larger than the image
733  subsection (if provided), yet the pixels required would never be
734  read in from the FITS file. The behaviour has been changed so that
735  the statSec, if requested, is trimmed to be wholly within the image
736  section.
737
738* It was possible to request, for instance, spectral smoothing on a 2D
739  image. This resulted in a blank smoothed array, which led to
740  problems in the calculations of statistics (and hence the
741  threshold), as well as subsequent calculations. This has been fixed,
742  so that in cases like this the smoothing is turned off (and a
743  warning message is displayed).
744
[799]745* There was a problem with using a MW range and subsectioning when
746  growing objects, as a result of changes in 1.1.10. This has been
747  fixed.
748
749* Inversion was failing when no array was allocated (not a problem for
750  standard Duchamp, but affected its use as a library by other code).
751
[812]752* Writing text spectra would only occur when PGPlot output was enabled
753  in the build (it was combined with the plotting of the spectral
[810]754  graphs). This has been changed so that this output can be requested
[812]755  without requiring PGPlot.
[810]756
[804]757* The equation for the beam size in Sec.4.2 of the User Guide was
758  incorrect and has been fixed.
759
[819]760* Minor change to the configure script to improve the detection of the
761  PGPlot library.
762
763Code development notes:
764
765* The main work has been additions and improvements to the interfaces
766  for the Param and the Section classes, as well as new classes to
767  manage the beam information. The efficiency changes mentioned above
768  also affected the internals of some of the PixelInfo classes, and
769  provided additional functions for judging the closeness of
770  neighbouring objects.
771
[760]772Duchamp version 1.1.10 (2010/11/15)
[574]773-------------------------------------
[542]774
[760]775Improvements:
776
777* The major change for this release is a greatly improved growing
778  algorithm. The previous incarnation was very inefficient (testing
779  all pixels in an object, most of which only have neighbours that are
780  already in the object), and could take an unnecessarily large amount
781  of time in the case of extended sources. The improved algorithm
782  makes use of voxel flags to quickly identify those voxels not yet in
783  an object and only examines them. For a certain test case
784  (observations of extended Galactic HI), the time required reduced
[761]785  from about a day to about a minute! (Note that compact sources, such
786  as HI galaxies, will not show this degree of speedup...) Many thanks
787  to Tara Murphy for pointing out this bug and her help in
788  diagnosing/fixing the problem.
[760]789
790* There have been a number of further improvements to the code to
791  speed it up, based on some profiling. The most important was the
792  function to add pixels to a 2D object, but there were several other
793  minor changes as well.
794
795* The minimum size of the detections can now also be specified in
796  terms of the total number of voxels (in addition or instead of
797  specifying the minimum spatial pixels or channels).
798
799* The way memory is allocated when reading the image has been
800  streamlined. The amount of memory being allocated is also now
801  reported - this is in an effort to allow users to identify problems
802  with running out of memory.
803
804* The version number is now reported in the results and log files.
805
806* Only the filename of the input image (ie. not the leading
807  directories) is now shown on the detection and moment-0 maps. This
808  is to avoid problems with large directory paths being too wide for
809  the plot.
810
811* The verification script was improved for readability (although no
812  functional change was made).
813
814Bug fixes:
815
816* Growing was not being done when a single source was found. Now
817  fixed.
818
819* For particular WCS setups, the RA can be interpreted as being
820  negative, which corrupted the Source ID (ie. the IAU-format
821  string). This has been fixed.
822
823* When the flux units were changed, problems had been seen when
824  writing a reconstructed cube to a FITS file. Any unit change is now
825  undone prior to writing and there should now be no problem.
826
827* Filenames with spaces in them were not handled well. New functions
828  were included and moved to a more obvious location. These filenames
829  now work as expected.
830
831* Spectral axes of type FREQ that had a rest frequency were not being
832  put into velocity units. Now fixed.
833
834* Improvements to the output to screen, making sure everything is
835  covered by the verbose flag.
836
837* Better management of the allocation of arrays.
838
839* The minSize parameter in Image was never initialised, leading to
840  unpredictable behaviour. Now fixed.
841
842Code development notes:
843
844* New functionality includes: a function to return a vector list of
845  voxels in an Object3D or Detection; a new generic Search function
846  (basically a front-end to each of the individual functions); more
[766]847  complete copy constructors for the DataArray classes; new methods
848  for finding the velocity widths.
[760]849
850Duchamp version 1.1.9 (2010/03/26)
851-------------------------------------
852
[654]853Improvements:
[616]854
[680]855* There is now the ability to save the moment-0 map as a FITS
856  file. This is controlled by the parameters flagOutputMomentMap and
857  fileOutputMomentMap, which work in the same way for the mask file.
858
[697]859* There is a new parameter flagRejectBeforeMerge that, when true,
[680]860  rejects sources according to the minPix/minChannels requirements
861  before the merging and growing is done. This defaults to false (so
862  default behaviour is as before). Thanks to Tobias Westmeier for
863  suggesting this and the above improvement.
864
[697]865* There is also a new parameter flagTwoStageMerging. When true (the
866  default), newly-detected objects are merged with the first close
867  object in the list. When false, they are simply added to the end of
868  the list. In both cases, a second round of merging is done to pick
869  up all close matches.
870
[680]871* The way the beam is specified in the absence of the BMAJ/BMIN/BPA
872  parameters has been slightly changed. The use of "beamSize" was
873  somewhat ambiguous (it meant the number of pixels covered by the
874  beam, not the FWHM), so new parameters have been introduced. The
875  beam can be specified by beamArea (which replaces beamSize), or
876  beamFWHM (the FWHM of the beam in pixels). If beamFWHM is provided,
877  it will be used, else the beamArea will be.
878
[687]879* There is a new parameter searchType that allows the user to choose
880  how the searching is done. The behaviour up till now has been to
881  search the 2D channel maps one at a time to build up the 3D
882  detections. This is the default, and can be obtained by setting
883  searchType=spatial. Alternatively, you can set searchType=spectral
884  to do the searching in the 1D spectra.
885
[654]886* Profiling of the code has led to a few changes that have sped up
887  some calculations, particularly those involving the median. Rather
888  than using std::sort, the code now simply uses std::nth_element,
889  which shows a substantial speedup factor.
890
891* When using the objectList parameter, the maps will now only show the
892  selected objects, rather than all detected objects.
893
894* There is a new verification test that looks at the "growing"
895  functionality.
896
897Bug fixes:
898
[670]899* When saving a mask FITS file, there was an occasional memory leak
900  that caused Duchamp to crash (only when the cube was suitably
901  large). This has been fixed.
902
[680]903* The median calculation had a bug that led to an incorrect value
904  being reported. The difference will not be great (the element of an
905  array being reported as the median was out by one), but could be
906  noticeable depending on the dataset.
907
908* The MW range parameters did not respond to the use of a
909  subsection. They should refer to channel ranges in the original
910  cube, and changes have been made to the code to reflect this.
911
[654]912* There was an occasional error in writing out FITS keywords, due to
913  the way comment strings were intialised. This has been improved and
914  made more robust.
915
916* Testing with gcc-4.3.2 showed a number of compilation warnings &
917  errors, which have been fixed.
918
919* The Gaussian smoothing kernel was too small - it is now made big
920  enough to encompass all possible values. It was also being
921  incorrectly scaled. This had no functional effect on the detection
922  of objects, but for other uses was incorrect. This has been
923  rectified.
924
925* Some outputs that weren't caught by the verbose flag now are.
926
927
928Code development notes:
929
930  The following notes only apply if you want to use the code as a
931  library, rather than the stand-alone package. These have generally
932  come out of the ASKAP software development. If you just use the
933  stand-alone package, they won't be a concern.
934
[616]935* The API for storing detected objects has changed, making it more
[654]936  robust from a programming point of view.
[616]937
[654]938* Additional versions of some functions (and the Gaussian smoothing
939  class) have been added - this has come out of the ASKAP development
940  work.
941
942* A number of instances of "unsigned int" being used for loops have
943  been changed to iterators or "size_t". There shouldn't be any
944  functional change from this -- the code should just build more
945  cleanly.
946
[687]947* The Detection class now records whether the parameters have been
948  set, and unsets that flag when objects are merged. This allows
949  external programs to track which objects have been merged and deal
950  with them appropriately.
[654]951
[687]952
[616]953Duchamp version 1.1.8 (2009/07/20)
954----------------------------------
955
[542]956Improvements:
957
[572]958* The final source list can now be sorted on one of a range of object
959  parameters: x-value, y-value, z-value, ra, dec, vel, w50, iflux,
960  pflux (integrated and peak flux respectively), or snr. Simply use
961  the new input parameter sortingParam. Its default value is vel, so
962  the default behaviour stays the same.
963
[543]964* The FDR method involves calculating how many pixels are
965  correlated. This previously involved multiplying the beam area by 2,
966  implying that 2 neighbouring channels were correlated. This factor
967  is now a user-selectable parameter, so that if you know how many
968  spectral channels should be correlated, you can set the parameter
969  FDRnumCorChan and the FDR calculation will be done correctly.
970
[542]971* Screen output now has more information on the growing threshold when
972  the cube statistics are printed.
973
974
975Bug fixes:
976
[584]977* When reading in previous detections, there was a bug causing a
978  segmentation fault. This has been fixed and made more robust.
979
[579]980* There was an apparent bug when writing out the reconstructed cubes
981  to a FITS file that caused the new file to be very large. Changing
982  the cfitsio function used has fixed this.
983
[574]984* Using NCP projections near the equator could result in a large
985  stream of error messages, due to coordinate transformations being
986  done in blank regions outside of the acceptable range of world
987  coordinates. This has been fixed.
988
[542]989* The function that grows objects had a major bug, that stopped it
990  working when flagAdjacent=false. This has been fixed and should work
991  fine.
992
[616]993* The input parameter spectraTextFile was not being read
994  properly. This has been remedied.
995
996* The units for the integrated flux were wrong for some types of pixel
997  flux units. This has been fixed.
998
999* A minor bug caused incorrect calculation of the 20% widths.
1000
[542]1001* All calls to uint were changed to unsigned int, as some compilers
1002  had problems recognising that they were the same thing.
1003
1004* src/fitsHeader.cc was missing an include statement for stdlib.h,
1005  again causing some compilers to fail.
1006
1007
[529]1008Duchamp version 1.1.7 (2009/01/16)
[520]1009----------------------------------
1010
1011Improvements:
1012
[522]1013* The mask file that can be created with flagMask can now specify
1014  which object a given detected pixel belongs to, by taking the object
1015  ID as its value. This applies if you set flagMaskWithObjectNum=true
1016  (it defaults to false, which means each detected pixel has the value
1017  1). Non-detected pixels still have the value 0.
[520]1018
[535]1019* The names of the output FITS files are now able to be specified
1020  directly by input parameters. These will override the default names
1021  that Duchamp provides.
1022
[520]1023* The Karma annotation file now specifies a font - previously it did
1024  not and so was at the mercy of whatever was specified in other
1025  annotation files being viewed at the same time. Please let me know
1026  if the font I've chosen (lucidasans-12) is not suitable.
1027
1028* The default flag, used in the absence of 'E', 'S' or 'N', is now '-'
1029  rather than blank. This will ensure that each column in the results
1030  file has something in it.
1031
[522]1032* The configure script is somewhat more robust to non-standard
1033  systems, particularly when the X11 libraries are in unusual
1034  places. It can also deal with the slightly different way that
1035  wcslib-4.3 installs itself.
1036
[520]1037* The installation part of the Makefile was altered so that the
1038  symbolic link Duchamp that points to Duchamp-1.1.7 does not have the
1039  absolute path encoded in it. This will enable the files to be moved
1040  around at a later date.
1041
1042Bug Fixes:
1043
1044* Fixed a bug that caused problems when the flux units were less than
1045  5 characters long (e.g. 'K')
1046
1047* The new velocity width values were not getting proper headers in the
1048  VOTable output. This has been remedied.
1049
1050* Minor improvements to help non-pgplot use
1051
1052* Minor fix in WCS functionality.
1053
1054* Minor fixes to the FitsHeader class.
1055
1056* Improved FITS data reading, that doesn't require a nullarray
1057  array. This will reduce memory usage.
1058
1059* Improved subsection parsing, particularly for non-standard WCS
1060  systems.
1061
1062* Improving the memory allocation of the Cube class.
1063
1064* Fixed a bug that meant the growthThreshold values were sometimes
1065  being lost.
1066
1067* Improving the way the growing and merging is done, particularly in
1068  the case of non-adjacent merging.
1069
[529]1070* Fixed a scaling bug in the GaussSmooth function.
1071
[520]1072Other:
1073
1074* Numerous changes to the code that do not affect how Duchamp itself
1075  runs, but were necessary for code development with the Duchamp
1076  libraries (primarily as a part of the ASKAP code development).
1077
[522]1078
[482]1079Duchamp version 1.1.6 (2008/09/08)
[480]1080----------------------------------
1081
1082New features:
1083
1084* The velocity widths can now be quoted in two additional ways: the
1085  full width at 50% of the peak flux (ie. the FWHM) and the full width
1086  at 20% of the peak flux (w_50 and w_20 respectively). The w_50
1087  parameter is now quoted in the results printed to the screen
1088  (instead of w_VEL), but all three are written to the results file.
1089
1090* The facility now exists to bypass the searching step and go straight
1091  to the output stage using a list of previously detected objects. The
1092  list is extracted from a previously-created log file. The user can
1093  change the way the spectra are plotted (e.g. choose to plot
1094  integrated rather than peak spectra), and select a subset of the
1095  detected objects to be printed to individual postscript files. Use
1096  the input parameter usePrevious (set to true) to use this facility,
1097  and provide a list of objects using the objectList parameter
1098  (eg. 1,2,4,7-10,15)
1099
[482]1100* A new parameter "growthThreshold" has been created, that goes with
1101  "threshold". If you manually specify a threshold, and you wish to
1102  grow detections, you need to manually specify a growthThreshold (in
1103  flux units).
1104
[480]1105Bug fixes:
1106
[482]1107* Cubes that do not have a spectral axis (such as Stokes cubes) were
1108  causing memory leaks and crashing, due to some assumptions of the
1109  existence of the spectral axis. These bugs have been fixed, so that
1110  Duchamp will run on such cubes, although the WCS transforms may not
1111  always work appropriately. Please let me know via bug reports of
1112  difficulties you encounter!
[480]1113
[482]1114* The growing function has had a few bugs fixed and is now more
1115  efficient.
1116
1117* The FDR method was previously overriding any manually-specified
1118  threshold. This has been fixed so that if the user specifies
1119  threshold=X, then the FDR method is not called.
1120
1121* Some undefined things were happening with textual output when no
1122  objects were detected. This has been fixed.
1123
1124* Memory leaks caused by FITS WCS irregularities have been fixed
1125  (specifically, if a FITS header had NAXIS=2 but more than two WCS
1126  axes were defined).
1127
1128
1129
[438]1130Duchamp version 1.1.5 (2008/03/20)
1131----------------------------------
1132
1133Bug fixes:
1134* A major bug was discovered in the calculation of integrated
1135  fluxes. The flux was incorrectly being weighted by the spatial size
1136  of the detection, resulting in a (often considerable) overestimation
[447]1137  of the integrated flux. This has been corrected.
[438]1138
[447]1139* The beam size calculation was also incorrect, being out by a factor
1140  of 1/ln(2).
[438]1141
[447]1142* The "make install" command previously failed to work properly if the
1143  library had not been created. This has been fixed so that the
1144  installation works even if there is no library file.
1145
1146New Features:
1147* You can now provide a new flux unit string, so that the pixel fluxes
1148  will be converted to these new fluxes before any calculations are
1149  done. This is done with the newFluxUnits input parameter. The new
1150  units should be directly convertible from the old ones (given by the
1151  BUNIT header -- e.g. if BUNIT = Jy/beam, you can give newFluxUnits
1152  as mJy/beam, but not mJy).
1153
1154* You can now specify the precision used in the output files for flux,
1155  velocity and S/N ratio columns, via new input parameters precFlux,
1156  precVel and precSNR. These will apply to the VOTable output as well
1157  as the text table and information on the spectral plots.
1158
1159* The Karma annotation files previously provided just a circle at the
1160  spatial location of the detected object. You now have the option to
1161  draw a border around the detected pixels (in a similar manner to the
1162  PGPLOT moment maps). Karma does not do this perfectly, so the lines
1163  may not line up exactly with the pixel borders, so the option
1164  remains to keep using the circles. This is controlled by the
1165  parameter "annotationType", with possible options being "borders"
1166  (the default) or "circles".
1167
1168* There is a new function getMetadata() that loads the header and WCS
1169  data, but no pixel information. This is only of interest if you are
1170  doing code development with the Duchamp library -- it does not
1171  affect the running of Duchamp itself.
1172
[425]1173Duchamp version 1.1.4 (2008/02/15)
1174----------------------------------
1175
1176Most of the changes incorporated in this update result from ASKAP code
1177development (that makes use of some of the Duchamp functionality),
1178but do not actually affect the way Duchamp runs (other than, in a
1179couple of cases, making it a bit faster). Changes that are
1180relevant for regular users of Duchamp are as follows:
1181
1182* A new option has been made available (thanks to Mary Putman et al
1183  for the suggestion!) to save the spectra of all detected objects in
1184  a text file. This has the form of one column for the spectral
1185  coordinates and one column of fluxes for each object. This is
1186  enabled by the input parameters flagTextSpectra and spectraTextFile.
1187
1188* Slight changes to the output include he formatting of the way the
1189  detected objects are written to the log file. This meant the
1190  verification files were also updated. Also the cube statistics are
1191  not printed to the results file if a user threshold is specified (as
1192  they are not calculated!).
1193
1194* The determination of beam size has been made more reliable, as has
1195  the BUNIT determination.
1196
1197* Fixed a bug that meant source merging was not done for 2D images
1198  (not usually a problem, as the source-finding takes care of this,
1199  but came up in parallelisation development).
1200
[406]1201Duchamp version 1.1.3 (2008/01/08)
1202----------------------------------
[404]1203
1204A minor update that just improves the build process. The location of
1205the wcslib include files was not specified correctly, so that some
1206machines were unable to find them. The configure script has been fixed
1207so that the compilation should work in all cases.
1208
[405]1209Some minor changes have also been made to the code in the PixelMap
[412]1210directory, and to param.cc, to fix warnings or errors that may appear
1211in compilation.
[404]1212
[405]1213However, there are no changes to the actual workings of the code in
1214this minor version, so if you have a working version of Duchamp 1.1.2
1215there is probably no reason to update.
1216
[389]1217Duchamp version 1.1.2 (2007/12/07)
1218----------------------------------
[349]1219
[363]1220Compilation:
1221
[349]1222* The compilation had problems working correctly when the cfitsio and
[373]1223  wcs libraries were not in standard locations. The configure script
[349]1224  has been tweaked to make sure the libraries are found.
1225
1226* Problems with compiling in the absence of pgplot have been fixed,
1227  and it should now compile fine.
1228
[362]1229* Improved the configure script for use with Mac OS X and gfortran.
[349]1230
[363]1231
1232Bugfixing:
1233
[362]1234* A major problem with the text-based output was fixed. The widths of
1235  the columns were not being calculated correctly, and were running
[373]1236  together. This should no longer be the case.
[349]1237
[374]1238* There was a bug in the way Duchamp interpreted spectral axes,
1239  particularly AIPS types such as FELO-HEL, leading to incorrect
1240  scaling of the velocity axis. This has been fixed, so that
1241  non-standard types are now dealt with appropriately and the velocity
1242  scaling should be accurate.
[362]1243
[376]1244* Another bug meant the beam correction was always applied to the
1245  integrated spectra, regardless of whether the flux units involved
1246  the beam (e.g. Jy/beam). This has been fixed. Note that the values
1247  of the integrated flux presented in the results table were always
1248  calculated correctly.
1249
[389]1250* Other minor changes to improve the speed and efficiency, especially
1251  of the merging code.
1252
[363]1253New user parameters:
1254
[381]1255* There is now the possibility to save a FITS file that contains a
1256  mask array, indicating where the detected objects are. This is
1257  governed by the flagOutputMask parameter.
1258
[363]1259* The header information that is printed at the start of the results
1260  file -- that is, the list of parameters, the statistics of the cube,
1261  and the number of detections -- can now be written to a separate
1262  file. Set the flagSeparateHeader parameter to true or 1 and it will
1263  go to the file indicated by the headerFile parameter (the default is
1264  duchamp-Results.hdr).
1265
[362]1266* A "scaleMax" parameter has been added, so that the wavelet
1267  reconstruction can include only a specified range of scales (for
[373]1268  instance, if one wants to search just for narrow features such as
[362]1269  absorption lines).
1270
[389]1271* A "flagRobustStats" parameter has been added. This allows the user
1272  to elect to use robust statistics (median and median absolute
1273  deviation from the median) to estimate the noise parameters. If set
1274  to false, the mean and standard deviation will be used. The default
1275  value is true.
[362]1276
[369]1277New Feature:
[362]1278
[369]1279* When Duchamp is run on a 2D image (i.e. with no spectral output), it
1280  now produces a postscript file that shows information each
1281  individual source plus the image cutout of the source. This is
1282  similar to the spectral output file, but without the spectra. The
1283  filename is given by the spectraFile parameter.
1284
[381]1285* Ability to save a FITS file containing a mask array, with pixel
1286  values of 1 for pixels in a detected object, and 0 elsewhere.
[369]1287
[381]1288
[328]1289Duchamp version 1.1.1 (2007/07/20)
1290------------------------------------
1291
[329]1292Minor update to the release, with just a few changes:
[328]1293
1294* The verification script was generating diffs due to differences in
1295  the precision of some calculations when done on different machines.
1296  One of these was the integrated flux, and this has been corrected so
1297  that the same value should appear on all machines.
1298  The verification script has also changed so that just the number of
1299  sources and their positions are tested, as well as the
1300  logfiles. This should avoid the reporting of spurious differences.
1301
1302* The grey-scale used in the image cutouts (in the spectral plots) has
1303  been inverted. This makes it consistent with the scale used in the
1304  maps, and hopefully easier to see when printing.
1305
[338]1306* The configure/make system has been altered to make it easier to
1307  install, and to specify different external libraries. In particular,
1308  it is now not necessary to have PGPLOT installed on your system to
1309  run Duchamp. In the absence of PGPLOT, Duchamp will run as normal,
1310  just without any graphical output. Some source code needed to be
[344]1311  restructured to make this work. Please read the README file, or the
1312  Installation appendix in the User's Guide, for complete details.
[328]1313
[329]1314* The time of completion is now written to the log file, so that the
1315  user can have some idea of how long the program took.
[328]1316
1317
[310]1318Duchamp version 1.1 (2007/05/17)
[259]1319------------------------------------
1320
1321Changes made since the last numbered release:
1322
[295]1323EFFICIENCY OF EXECUTION:
1324
[259]1325* Major change to the way detections are stored. They are now recorded
1326  via run-length encoding in the x-y plane, and stored as collections
1327  of channel-maps. This improves the memory usage and the speed of
[298]1328  execution (particularly for routines such as the merging
1329  routines). The memory usage has also been improved by changing the
1330  way the list of detected objects is stored.
[259]1331
[295]1332SMOOTHING:
1333
[279]1334* Addition of ability to spatially smooth the cube before
1335  searching. This is done with a Gaussian kernel, the size and shape
[287]1336  of which can be specified by the user, using the parameters kernMaj
1337  & kernMin (which are the FWHM for the major & minor axes -- if
1338  kernMin is not given it is assumed to match kernMaj) and kernPA (the
1339  position angle of the major axis).  There is a new parameter
[279]1340  smoothType that distinguishes this method from the spectral (Hanning)
1341  smoothing already in use.
1342
[295]1343DETECTIONS:
1344
[264]1345* Changed the way detection is done -- now only search in 2D images,
1346  keeping all detections (ie. no minimum-pixel requirement). This
1347  should be the same as doing both the 1D and 2D searches with the
1348  minimum pixel requirement in, but simpler. The rejection is done at
[295]1349  the end after all the merging and growing has been done. This leads
1350  to a larger number of intermediate detections (ie.  the number
1351  before merging), which may increase the run-time slightly, although
1352  this is offset by the changes to the efficiency (see above).
[264]1353
1354* Increased the range over which objects can be grown to match the
1355  threshSpatial & threshVelocity parameters, as well as rejecting new
1356  pixels based on the BLANK and MW tests.
1357
[295]1358INPUTS:
[259]1359
[287]1360* There has been a subtle change in the way BLANK pixels are dealt
1361  with. To enable trimming of BLANK pixels, users should use the new
1362  parameter flagTrim. The parameters flagBlankPix and blankPixVal are
[295]1363  no longer accessible via the parameter file -- warning messages are
1364  given if they are provided in the parameter file.
[287]1365
[295]1366* Added ability to specify a statsec, so that the statistics are only
1367  calculated from a given subsection of the cube, rather than the full
1368  cube.
1369
1370* The subsections can also specify the ability to remove a border of n
1371  pixels from the edges of an axis (ie. n from the start and from the
1372  end) -- as requested in Trac ticket #5.
1373
1374* Changed the default value of the reconDim parameter from 3 to 1, and
1375  the default value of the flagATrous parameter to false.
1376
1377OUTPUTS:
1378
[279]1379* The output now includes three different estimates of the "centre" of
1380  the detection: peak location, centroid (flux-weighted), or average.
1381
[295]1382* Some slight improvements to the user feedback during execution.
1383
[282]1384* Spectral output now includes the baseline, if one was fitted. The
1385  format of the plot headers is slightly changed, to stop information
1386  falling off the edge of the page.
[279]1387
[295]1388* Added command-line option -x to directly disable the PGPLOT X-window
1389  map plotting. This overrides the setting of flagXOutput in the
1390  parameter file.
[259]1391
[295]1392* Fallback spectral units (in case the FITS header fails to record
1393  them) changed to SPC, but code has been fixed so that any problem
1394  should be caught before this becomes an issue.
1395
1396BUGFIXES:
1397
[259]1398* Fixed bug (Trac ticket #4) that meant the velocity units were not
1399  correctly dealt with. Duchamp should be more robust for different
1400  velocity axis setups now.
1401
1402* Fixed bug (ticket #9) with the VOTable output, where the tags were
1403  not closing.
1404
[305]1405* Fixed a bug that meant the integrated fluxes were not calculated
1406  properly -- no allowance was made for the number of spatial
1407  pixels. This should also work properly for 2D images now as well.
1408
[264]1409* Fixed minor bug in calculating the flux threshold from the p-value
1410  threshold when the FDR method is used. This did not affect the
1411  actual thresholding, merely the quoted flux threshold in the results
1412  file. Also, the MW channels are kept out of the p-value threshold
1413  determination if they are being flagged.
1414
[259]1415
1416
[211]1417Duchamp version 1.0.7 (2006/11/22)
1418----------------------------------
1419
1420One new feature, and some improvements and bug-fixes.
1421
[264]1422* The user can now elect to Hanning-smooth the spectral direction of
1423  the cube before the searching is done. This can be done instead of the
1424  wavelet reconstruction (although if both are selected, the
1425  reconstruction takes precendence). This functionality is switched on
1426  and off by the flagSmooth parameter, and the width of the Hanning
1427  kernel is governed by the hanningWidth parameter.
[211]1428
[264]1429* The smoothed array can also be written to a FITS file and read in at
1430  the start in the same manner as the reconstructed array.
[211]1431
[264]1432* Several improvements have been made to the memory
1433  management. Several memory leaks were fixed, and the memory usage of
1434  Duchamp is considerably less, which will be a help for running it on
1435  large cubes. The speed of some routines has also been improved, by
1436  removing unnecessarily repetitive loops.
[211]1437
[287]1438* The sorting routines have been changed, and should now be slightly quicker
[215]1439  (and a bit more robust).
1440
[287]1441* Enabled the scale bar (plotted on the 0th moment map in the spectral
1442  output file) to be able to deal with much finer spatial scales (as
1443  may be expected from VLBI data).
[211]1444
[215]1445* Other minor changes to code and presentation.
[211]1446
1447
[194]1448Duchamp version 1.0.6 (2006/11/01)
1449----------------------------------
1450
1451Some incremental improvements and changes to the code:
1452
1453* The statistics used for the detection are now calculated once for
1454  the entire cube, rather than being calculated separately for each
1455  channel map and spectrum. This results in a uniform threshold value
1456  across the entire cube. The way the statistics calculations are
1457  implemented in the code was changed, using a new statistics class.
1458
1459* This means we can report a peak signal-to-noise value for each
1460  detection. This is given in the text-based output, as well as in the
1461  summary information above the spectral output for each source.
1462
[195]1463* The user can now elect not to have the moment map displayed in a
1464  PGPlot window, through the use of the new flagXOutput
1465  parameter. This should help in cases of running Duchamp on a dumb
1466  terminal (with no X-display capabilities), or as part of a batch
1467  script where the continual popping-up of the PGPlot window might
1468  become annoying.
1469
[265]1470* There are different Flags reported for the case of an object lying on
[194]1471  the spatial edge and spectral edge of the cube (E and S respectively).
1472
1473* A few minor bug fixes:
1474  - When providing a subsection, any axes with the range given by *
1475    had the pixel values offset incorrectly. This has been fixed.
1476  - Negative searches now correctly invert the reconstructed array
1477    when it is read in from a file.
1478
1479* Other minor changes to the code, to improve memory handling and
1480  remove redundant arrays and variables.
1481
1482
[146]1483Duchamp version 1.0.5 (2006/09/06)
[142]1484----------------------------------
1485
[146]1486Since version 1.0, there have been a range of mostly minor
1487changes. These are generally bug-fixes, thanks to feedback from a
[157]1488number of users, as well as a few extra features.
[146]1489
[157]1490Here is a summary of the changes since version 1.0:
[142]1491
[157]1492USER INTERFACE:
[146]1493
[142]1494* The user is now able to enter the value of flag parameters in the
1495  parameter file as strings, as well as integers (so one can enter
1496  true or 1 to give the same result).
1497
[157]1498GRAPHICAL OUTPUT:
1499
[142]1500* Added a new function that draws the edge of the blank region on the
[146]1501  maps and image cutouts. There is a new flag parameter --
1502  drawBlankEdges -- that controls the use of this function (the
1503  default is true).
1504  Also, the edge of the field is plotted on the image cutouts as a
1505  yellow line. This distinguishes it from the purple blank pixel
1506  dividing line.
[142]1507
[146]1508* The size of the tick mark in the image cutout (indicating the
1509  spatial scale of the image) is now adaptable, so that it shows the
1510  scale that best fits the image (ranging from 1 arcsec to 15
1511  degrees). (This is a change from the previous standard length of 15
1512  arcmin.)
[142]1513
[113]1514* The zoomed spectrum had its flux scale determined by all points in
[146]1515  the zoom box,including those flagged by the Milky Way (MW)
1516  range. This has been fixed so that any MW channels will not affect
1517  the flux scaling, potentially improving the appearance of objects
1518  detected close in velocity to strong MW emission or absorption.
[112]1519
[157]1520* Improved Karma Annotation file output, so that files without good
1521  WCS are dealt with appropriately.
1522
1523TEXT-BASED OUTPUT:
1524
[113]1525* The position resolution in the IAU-format names has been increased,
1526  so that distinct objects close to one another are more readily
1527  distinguished.  Overlapping sources at different velocities may
1528  still have the same name, but this is a more general problem.
[112]1529
[146]1530* The presentation of the results in tabular format has been improved,
1531  so that the precision of faint sources is sufficient to show their
1532  flux, for instance, is not 0. This also applies to the information
1533  presented above the spectral plots.
[112]1534
[146]1535* Improved error and warning reporting, so that the formatting and
1536  presentation of information is clearer.
1537
[157]1538FITS I/O RELATED:
1539
[172]1540* The previous version was did not deal well with 4 (or greater)
1541  dimensions in the FITS file -- the memory allocation was badly done,
1542  and the axes needed to be in a particular order. This has been
1543  fixed, so that the FITS I/O routines can now handle any sized FITS
1544  file, with axes in any order (as long as there are two spatial and one
1545  spectral axes).
[157]1546
1547* When the FITS file does not have a rest frequency defined, there is
1548  no way to transform the frequency axis into velocity. In this case,
1549  the spectral axis is left as frequency, and put into units of MHz.
1550
[172]1551* If the beam size is not indicated by the FITS header (through the
1552  BMAJ and BMIN keywords), the user can manually specify the size
1553  through the parameter file (using parameter beamSize).
1554
1555* The FITS I/O routines make use of the command fits_file_exists. This
1556  was introduced into the CFITSIO package from version 2.5, so Duchamp
1557  will only work with this version or higher. Version 3+ is preferred.
1558
[157]1559OTHER BUG-FIXING:
1560
1561* Several bugs related to the blank-pixels were dealt with.
1562  The correct values were not propagating through correctly, meaning
1563  that the default values were being used. Also, there was the
1564  potential for blank pixels to be detected when the FDR method was
1565  used. Finally, the image cutout displays occasionally did not
1566  obscure the blank pixels. All these problems should be fixed now.
1567
1568* The FDR setup routine was occasionally failing with a segmentation
1569  fault, due to it accessing unallocated memory. This has been fixed.
1570
1571* Fixed bugs that affected the way 2D images (rather than 3D cubes)
1572  were dealt with. The minChannels test failed when minChannels=0.
1573
[165]1574* If there is no beam information in the FITS header, the user can now
1575  enter a beam size as a parameter.
1576
[146]1577* Other minor fixes to the code, to improve its structure and
1578  robustness.
1579
1580
[112]1581Duchamp version 1.0 (2006/06/30)
1582--------------------------------
1583
[113]1584The first full release version.
1585
1586Several major changes compared to previous version:
1587
1588* Incorporated a FitsHeader class, to keep track of header information
1589  (including WCS) from the FITS file. All units-related calculations
1590  are done with this class, such as conversion to velocity, and
1591  pixel-to-WCS conversion.  The main advantage of this is improved
1592  speed of execution, more adaptable to different units, and improved
1593  code (easier for me!).
1594
1595* Able to select the dimensionality of the reconstruction, rather than
1596  just doing it in 3-D.
1597
1598* Robust text output, so that the columns are sized correctly given
1599  the data being written, and the relevant units (derived from the
1600  FITS header) are included in the output table.
1601
1602* Improved the way the reconstructed image is saved and read back in,
1603  with a better naming scheme and a better set of new FITS header
1604  keywords. The MW headers are now not written.
1605
1606Other, more minor changes include:
1607
1608* MW channels now not set to 0, but just ignored for searching
1609  purposes (and for the scaling of the spectral output -- they are
1610  plotted though and highlighted with a hashed box).
1611
1612* The scale bar in the moment map in the spectral output file has its
1613  length indicated.
1614
1615* Improved VOTable output, with more information about the way Duchamp
1616  was run.
1617
1618* Made sure all statistics calculations in the reconstruction are
1619  based on robust (ie. median) methods.
1620
1621Also moved to an autoconf based configure/make compilation, for ease of use.
1622
1623
1624Duchamp version 0.9.2 (2006/06/27)
1625--------------------------------
1626
1627* Added the use of warning flags for detections: for either edge
1628  location or negative enclosed flux.
1629
1630* A new command line option added so that the user can specify a FITS
1631  file to be searched with the default parameters, rather than giving
1632  a full parameter file.
1633
1634* The scale bar on the moment cutout now works in general case, using
1635  an iterative process to get to the correct length.
1636
1637* RA axes are now done correctly on the full-scale maps.
1638
1639* Various coding fixes, improving the readability of the code,
1640  particularly regarding the declaration of temporary variables.
1641
1642* Changed the way the time of execution is called : now using time.h
1643  functions, rather than a system call.
1644
1645* Changed output in Detection/outputDetection.cc for the pixel
1646  information of detected objects (the info printed above the spectral
1647  plots) -- now print out number of voxels in object, and have
1648  improved the formatting a little.
1649
1650* Improved the functionality of saving the FITS file containing the
1651  reconstructed array. Added ability to save and read back in the MW
1652  parameters when saving a reconstructed cube (if a reconstructed cube
1653  has different MW range to that being considered, it would cause
1654  problems). ReadRecon now just returns a FAILURE and will reconstruct
1655  the cube anew.
1656
1657* Changed default value of flagGrowth to 0.
1658
1659
1660
1661Duchamp version 0.9 (2006/05/05)
1662--------------------------------
1663
1664First public release version.
Note: See TracBrowser for help on using the repository browser.