source: trunk/CHANGES @ 965

Last change on this file since 965 was 962, checked in by MatthewWhiting, 12 years ago

Changing version for next release to 1.2

File size: 45.4 KB
Line 
1Duchamp version 1.2 (2012/06/XX)
2--------------------------------
3
4Key changes:
5
6* The way the wavelet reconstruction is done in the presence of BLANK
7  voxels is slightly different, specifically when there were BLANK
8  voxels on the edge. While BLANK voxels are always ignored by the
9  reconstruction (and left in place), the previous approach dealt with
10  BLANKS on the edge by moving the effective edge of the image inwards
11  until the first non-blank voxel. This meant that the size of the
12  dimensions used to determine the number of scales, and the location
13  where boundary reflections took place where different to the input
14  image size.
15
16  However, this meant that BLANK voxels on the edge were treated
17  differently to those not on the edge, so this inconsistency was
18  removed. The full image dimensions are always used to calculate the
19  number of scales, and reflections (which is how the reconstruction
20  deals with the boundaries) will occur at the actual image edges.
21
22  If your data has this property (BLANK pixels at the image
23  boundaries), then any reconstruction will be affected (most likely
24  only in a small way) and potentially any subsequent searching. You
25  are advised to check this.
26
27* Since v1.1.13 was released, the journal paper describing Duchamp has
28  been published. It can be found at MNRAS 421, 3242 (2012), or
29  http://onlinelibrary.wiley.com/doi/10.1111/j.1365-2966.2012.20548.x/full. It
30  includes descriptions of the various algorithms used for Duchamp, as
31  well as some indicative comparisons of the relative completeness &
32  reliability. Please cite this paper when describing Duchamp.
33
34Other new features:
35
36* The XML VOTable output now includes a more complete list of
37  parameters, matching the output given in text form in the results &
38  log output files. Only those parameters affecting the results are
39  included - things like the output file names and so on are left out.
40
41* The minVoxels parameter is now adjusted to be at least (minPix +
42  minChannels - 1). If it is changed, a warning message is triggered.
43
44* The warning/error reporting has been overhauled, with an improved
45  interface for developers. Also, some ERROR instances that previously
46  resulted in the program terminating now allow it to continue
47  (provided that is possible given the error).
48
49Bug fixes:
50
51* The absence of a BLANK keyword when trimming will now not trigger a
52  failure, only a warning message.
53
54* The flux unit conversion (eg. from Jy/beam to mJy/beam) was not
55  being applied correctly when an existing reconstructed or smoothed
56  array was being read in. This is now applied properly.
57
58* When saving reconstructed or smoothed arrays, the data type is now
59  set to FLOAT, so that the correct values are written (and
60  subsequently read in). The previous version was potentially saving
61  to a FITS file with integer data type (this depended on the type of
62  the input array). In this case, the full precision of data values
63  would not be preserved, so reading back in and doing an identical
64  search could yield different sources.
65
66* For certain cases, the reconstruction attempted to use a very large
67  number of scales. This has been fixed.
68
69* When saving a mask image while using a subsection of a cube, the
70  offsets for the subsection were not being accounted for
71  correctly. This is now fixed.
72
73* The RA & Dec now have appropriate precision, rather than an
74  arbitrary 2 decimal places on the seconds value. The RA will be one
75  decimal place more than the Dec, and they will reflect the degree of
76  precision in the image pixels.
77
78* A number of data type conversions (typically ints to floats) led to
79  errors in some calculations, such as WCS conversions. Care has been
80  taken to make sure all indexing is done appropriately with no
81  overflow, and that all conversions (where necessary) are done
82  appropriately.
83
84* It was possible to get world positions looking like 11:59:60.0! Not
85  any more.
86
87New features:
88
89*
90
91Code base development notes:
92
93* Most instances of integer overflow due to insufficient precision
94  have been tracked down - many of these were indexing variables of
95  type int, which have been changed to size_t.
96
97* Object3D now carries its spatial map with it, rather than
98  calculating it on the fly when requested. This saves a lot of
99  unnecessary computational effort.
100
101* The warning/error interface has improved, making use of C++
102  exception handling. This should be compatible with external logging
103  infrastructure (such as log4cxx).
104
105*
106
107
108
109Duchamp version 1.1.13 (2011/04/10)
110-------------------------------------
111
112Bug fixes:
113
114* The median statistics calculations had a bug in the test for whether
115  the array under consideration was of even size. The correction for
116  this can mean small differences in the resulting values. This can
117  flow through to calculated detection thresholds - please check this
118  with your data if you use this mode!
119
120* The reconstruction code had a bug where the incorrect number of
121  scales was being used. This occurred when the blank pixel flag was
122  set, and when the spectral dimension was smaller than both the
123  spatial dimensions.
124
125Other:
126
127* The default value of the blank pixel flag has been set to false,
128  rather than true. For Duchamp, this should not matter, as it is set
129  when the FITS file is read in. For external code using, for
130  instance, the reconstruction algorithms, this default may be
131  important. Having it true was contributing to the second bug listed
132  above. It also makes more sense for it to be false unless explicitly
133  set in some way.
134
135* The reconstruction code has been streamlined a little, reducing its
136  memory impact and using more appropriate variable types
137  (eg. unsigned ints instead of ints and so forth).
138
139* For those who care about the actual code base, the statistics
140  functions have been expanded, and separated from getStats.cc into
141  "normal" and "robust" versions.
142
143
144Duchamp version 1.1.12 (2011/03/18)
145-------------------------------------
146
147Bug fixes:
148
149* The velocity width calculations had a bug that occasionally led to
150  very large values. The widths are calculated using the integrated
151  spectrum, using a threshold relative to its peak. In v.1.1.10 and
152  1.1.11, however, the peak was searched for over the entire spectral
153  range, rather than just the range for the current object. This led
154  to spurious values. This has been fixed, so that the peak is only
155  searched for within the detected ranage of the object.
156
157* Both the pixel and statistics subsections are now more carefully
158  examined, with errors reported if there are no valid pixels in the
159  pixel subsection, or if the statistics subsection does not intersect
160  with the pixel subsection.
161
162Other:
163
164* The configure script has been slightly improved so that the pgsbox
165  library is only added if PGPlot has been enabled - this is
166  particularly necessary with the slightly changed build procedure for
167  recent wcslib versions.
168
169* In a related note, one file - momentMap.cc - had some #include
170  statements that failed when PGPlot (and hence pgsbox in WCSLIB) was
171  not enabled. These were actually unnecesssary and have been removed.
172
173* There is a new input parameter - flagPlotSpectra - that allows the
174  user to turn off the production of the spectral plots. It defaults
175  to true (meaning the plots are produced).
176
177
178Duchamp version 1.1.11 (2011/03/01)
179-------------------------------------
180
181Improvements:
182
183* The largest changes in this release involved improving the
184  efficiency of the code, resulting in further speed-ups. The code
185  affected was the low-level object-handling code, with the merging
186  stages in particular showing good speed-ups.
187
188* The default value for the beam size, when the beam parameters are
189  not in the FITS header, has been dropped. Now the default behaviour
190  is to simply not make any beam corrections if the beam is not
191  specified by either the FITS file or the input parameters. Thanks to
192  Tobias Westemeier for the suggestion.
193
194* The reporting of the beam size has changed slightly to provide a bit
195  more information.
196
197Bug Fixes:
198
199* The statistics section (statSec) could be larger than the image
200  subsection (if provided), yet the pixels required would never be
201  read in from the FITS file. The behaviour has been changed so that
202  the statSec, if requested, is trimmed to be wholly within the image
203  section.
204
205* It was possible to request, for instance, spectral smoothing on a 2D
206  image. This resulted in a blank smoothed array, which led to
207  problems in the calculations of statistics (and hence the
208  threshold), as well as subsequent calculations. This has been fixed,
209  so that in cases like this the smoothing is turned off (and a
210  warning message is displayed).
211
212* There was a problem with using a MW range and subsectioning when
213  growing objects, as a result of changes in 1.1.10. This has been
214  fixed.
215
216* Inversion was failing when no array was allocated (not a problem for
217  standard Duchamp, but affected its use as a library by other code).
218
219* Writing text spectra would only occur when PGPlot output was enabled
220  in the build (it was combined with the plotting of the spectral
221  graphs). This has been changed so that this output can be requested
222  without requiring PGPlot.
223
224* The equation for the beam size in Sec.4.2 of the User Guide was
225  incorrect and has been fixed.
226
227* Minor change to the configure script to improve the detection of the
228  PGPlot library.
229
230Code development notes:
231
232* The main work has been additions and improvements to the interfaces
233  for the Param and the Section classes, as well as new classes to
234  manage the beam information. The efficiency changes mentioned above
235  also affected the internals of some of the PixelInfo classes, and
236  provided additional functions for judging the closeness of
237  neighbouring objects.
238
239Duchamp version 1.1.10 (2010/11/15)
240-------------------------------------
241
242Improvements:
243
244* The major change for this release is a greatly improved growing
245  algorithm. The previous incarnation was very inefficient (testing
246  all pixels in an object, most of which only have neighbours that are
247  already in the object), and could take an unnecessarily large amount
248  of time in the case of extended sources. The improved algorithm
249  makes use of voxel flags to quickly identify those voxels not yet in
250  an object and only examines them. For a certain test case
251  (observations of extended Galactic HI), the time required reduced
252  from about a day to about a minute! (Note that compact sources, such
253  as HI galaxies, will not show this degree of speedup...) Many thanks
254  to Tara Murphy for pointing out this bug and her help in
255  diagnosing/fixing the problem.
256
257* There have been a number of further improvements to the code to
258  speed it up, based on some profiling. The most important was the
259  function to add pixels to a 2D object, but there were several other
260  minor changes as well.
261
262* The minimum size of the detections can now also be specified in
263  terms of the total number of voxels (in addition or instead of
264  specifying the minimum spatial pixels or channels).
265
266* The way memory is allocated when reading the image has been
267  streamlined. The amount of memory being allocated is also now
268  reported - this is in an effort to allow users to identify problems
269  with running out of memory.
270
271* The version number is now reported in the results and log files.
272
273* Only the filename of the input image (ie. not the leading
274  directories) is now shown on the detection and moment-0 maps. This
275  is to avoid problems with large directory paths being too wide for
276  the plot.
277
278* The verification script was improved for readability (although no
279  functional change was made).
280
281Bug fixes:
282
283* Growing was not being done when a single source was found. Now
284  fixed.
285
286* For particular WCS setups, the RA can be interpreted as being
287  negative, which corrupted the Source ID (ie. the IAU-format
288  string). This has been fixed.
289
290* When the flux units were changed, problems had been seen when
291  writing a reconstructed cube to a FITS file. Any unit change is now
292  undone prior to writing and there should now be no problem.
293
294* Filenames with spaces in them were not handled well. New functions
295  were included and moved to a more obvious location. These filenames
296  now work as expected.
297
298* Spectral axes of type FREQ that had a rest frequency were not being
299  put into velocity units. Now fixed.
300
301* Improvements to the output to screen, making sure everything is
302  covered by the verbose flag.
303
304* Better management of the allocation of arrays.
305
306* The minSize parameter in Image was never initialised, leading to
307  unpredictable behaviour. Now fixed.
308
309Code development notes:
310
311* New functionality includes: a function to return a vector list of
312  voxels in an Object3D or Detection; a new generic Search function
313  (basically a front-end to each of the individual functions); more
314  complete copy constructors for the DataArray classes; new methods
315  for finding the velocity widths.
316
317Duchamp version 1.1.9 (2010/03/26)
318-------------------------------------
319
320Improvements:
321
322* There is now the ability to save the moment-0 map as a FITS
323  file. This is controlled by the parameters flagOutputMomentMap and
324  fileOutputMomentMap, which work in the same way for the mask file.
325
326* There is a new parameter flagRejectBeforeMerge that, when true,
327  rejects sources according to the minPix/minChannels requirements
328  before the merging and growing is done. This defaults to false (so
329  default behaviour is as before). Thanks to Tobias Westmeier for
330  suggesting this and the above improvement.
331
332* There is also a new parameter flagTwoStageMerging. When true (the
333  default), newly-detected objects are merged with the first close
334  object in the list. When false, they are simply added to the end of
335  the list. In both cases, a second round of merging is done to pick
336  up all close matches.
337
338* The way the beam is specified in the absence of the BMAJ/BMIN/BPA
339  parameters has been slightly changed. The use of "beamSize" was
340  somewhat ambiguous (it meant the number of pixels covered by the
341  beam, not the FWHM), so new parameters have been introduced. The
342  beam can be specified by beamArea (which replaces beamSize), or
343  beamFWHM (the FWHM of the beam in pixels). If beamFWHM is provided,
344  it will be used, else the beamArea will be.
345
346* There is a new parameter searchType that allows the user to choose
347  how the searching is done. The behaviour up till now has been to
348  search the 2D channel maps one at a time to build up the 3D
349  detections. This is the default, and can be obtained by setting
350  searchType=spatial. Alternatively, you can set searchType=spectral
351  to do the searching in the 1D spectra.
352
353* Profiling of the code has led to a few changes that have sped up
354  some calculations, particularly those involving the median. Rather
355  than using std::sort, the code now simply uses std::nth_element,
356  which shows a substantial speedup factor.
357
358* When using the objectList parameter, the maps will now only show the
359  selected objects, rather than all detected objects.
360
361* There is a new verification test that looks at the "growing"
362  functionality.
363
364Bug fixes:
365
366* When saving a mask FITS file, there was an occasional memory leak
367  that caused Duchamp to crash (only when the cube was suitably
368  large). This has been fixed.
369
370* The median calculation had a bug that led to an incorrect value
371  being reported. The difference will not be great (the element of an
372  array being reported as the median was out by one), but could be
373  noticeable depending on the dataset.
374
375* The MW range parameters did not respond to the use of a
376  subsection. They should refer to channel ranges in the original
377  cube, and changes have been made to the code to reflect this.
378
379* There was an occasional error in writing out FITS keywords, due to
380  the way comment strings were intialised. This has been improved and
381  made more robust.
382
383* Testing with gcc-4.3.2 showed a number of compilation warnings &
384  errors, which have been fixed.
385
386* The Gaussian smoothing kernel was too small - it is now made big
387  enough to encompass all possible values. It was also being
388  incorrectly scaled. This had no functional effect on the detection
389  of objects, but for other uses was incorrect. This has been
390  rectified.
391
392* Some outputs that weren't caught by the verbose flag now are.
393
394
395Code development notes:
396
397  The following notes only apply if you want to use the code as a
398  library, rather than the stand-alone package. These have generally
399  come out of the ASKAP software development. If you just use the
400  stand-alone package, they won't be a concern.
401
402* The API for storing detected objects has changed, making it more
403  robust from a programming point of view.
404
405* Additional versions of some functions (and the Gaussian smoothing
406  class) have been added - this has come out of the ASKAP development
407  work.
408
409* A number of instances of "unsigned int" being used for loops have
410  been changed to iterators or "size_t". There shouldn't be any
411  functional change from this -- the code should just build more
412  cleanly.
413
414* The Detection class now records whether the parameters have been
415  set, and unsets that flag when objects are merged. This allows
416  external programs to track which objects have been merged and deal
417  with them appropriately.
418
419
420Duchamp version 1.1.8 (2009/07/20)
421----------------------------------
422
423Improvements:
424
425* The final source list can now be sorted on one of a range of object
426  parameters: x-value, y-value, z-value, ra, dec, vel, w50, iflux,
427  pflux (integrated and peak flux respectively), or snr. Simply use
428  the new input parameter sortingParam. Its default value is vel, so
429  the default behaviour stays the same.
430
431* The FDR method involves calculating how many pixels are
432  correlated. This previously involved multiplying the beam area by 2,
433  implying that 2 neighbouring channels were correlated. This factor
434  is now a user-selectable parameter, so that if you know how many
435  spectral channels should be correlated, you can set the parameter
436  FDRnumCorChan and the FDR calculation will be done correctly.
437
438* Screen output now has more information on the growing threshold when
439  the cube statistics are printed.
440
441
442Bug fixes:
443
444* When reading in previous detections, there was a bug causing a
445  segmentation fault. This has been fixed and made more robust.
446
447* There was an apparent bug when writing out the reconstructed cubes
448  to a FITS file that caused the new file to be very large. Changing
449  the cfitsio function used has fixed this.
450
451* Using NCP projections near the equator could result in a large
452  stream of error messages, due to coordinate transformations being
453  done in blank regions outside of the acceptable range of world
454  coordinates. This has been fixed.
455
456* The function that grows objects had a major bug, that stopped it
457  working when flagAdjacent=false. This has been fixed and should work
458  fine.
459
460* The input parameter spectraTextFile was not being read
461  properly. This has been remedied.
462
463* The units for the integrated flux were wrong for some types of pixel
464  flux units. This has been fixed.
465
466* A minor bug caused incorrect calculation of the 20% widths.
467
468* All calls to uint were changed to unsigned int, as some compilers
469  had problems recognising that they were the same thing.
470
471* src/fitsHeader.cc was missing an include statement for stdlib.h,
472  again causing some compilers to fail.
473
474
475Duchamp version 1.1.7 (2009/01/16)
476----------------------------------
477
478Improvements:
479
480* The mask file that can be created with flagMask can now specify
481  which object a given detected pixel belongs to, by taking the object
482  ID as its value. This applies if you set flagMaskWithObjectNum=true
483  (it defaults to false, which means each detected pixel has the value
484  1). Non-detected pixels still have the value 0.
485
486* The names of the output FITS files are now able to be specified
487  directly by input parameters. These will override the default names
488  that Duchamp provides.
489
490* The Karma annotation file now specifies a font - previously it did
491  not and so was at the mercy of whatever was specified in other
492  annotation files being viewed at the same time. Please let me know
493  if the font I've chosen (lucidasans-12) is not suitable.
494
495* The default flag, used in the absence of 'E', 'S' or 'N', is now '-'
496  rather than blank. This will ensure that each column in the results
497  file has something in it.
498
499* The configure script is somewhat more robust to non-standard
500  systems, particularly when the X11 libraries are in unusual
501  places. It can also deal with the slightly different way that
502  wcslib-4.3 installs itself.
503
504* The installation part of the Makefile was altered so that the
505  symbolic link Duchamp that points to Duchamp-1.1.7 does not have the
506  absolute path encoded in it. This will enable the files to be moved
507  around at a later date.
508
509Bug Fixes:
510
511* Fixed a bug that caused problems when the flux units were less than
512  5 characters long (e.g. 'K')
513
514* The new velocity width values were not getting proper headers in the
515  VOTable output. This has been remedied.
516
517* Minor improvements to help non-pgplot use
518
519* Minor fix in WCS functionality.
520
521* Minor fixes to the FitsHeader class.
522
523* Improved FITS data reading, that doesn't require a nullarray
524  array. This will reduce memory usage.
525
526* Improved subsection parsing, particularly for non-standard WCS
527  systems.
528
529* Improving the memory allocation of the Cube class.
530
531* Fixed a bug that meant the growthThreshold values were sometimes
532  being lost.
533
534* Improving the way the growing and merging is done, particularly in
535  the case of non-adjacent merging.
536
537* Fixed a scaling bug in the GaussSmooth function.
538
539Other:
540
541* Numerous changes to the code that do not affect how Duchamp itself
542  runs, but were necessary for code development with the Duchamp
543  libraries (primarily as a part of the ASKAP code development).
544
545
546Duchamp version 1.1.6 (2008/09/08)
547----------------------------------
548
549New features:
550
551* The velocity widths can now be quoted in two additional ways: the
552  full width at 50% of the peak flux (ie. the FWHM) and the full width
553  at 20% of the peak flux (w_50 and w_20 respectively). The w_50
554  parameter is now quoted in the results printed to the screen
555  (instead of w_VEL), but all three are written to the results file.
556
557* The facility now exists to bypass the searching step and go straight
558  to the output stage using a list of previously detected objects. The
559  list is extracted from a previously-created log file. The user can
560  change the way the spectra are plotted (e.g. choose to plot
561  integrated rather than peak spectra), and select a subset of the
562  detected objects to be printed to individual postscript files. Use
563  the input parameter usePrevious (set to true) to use this facility,
564  and provide a list of objects using the objectList parameter
565  (eg. 1,2,4,7-10,15)
566
567* A new parameter "growthThreshold" has been created, that goes with
568  "threshold". If you manually specify a threshold, and you wish to
569  grow detections, you need to manually specify a growthThreshold (in
570  flux units).
571
572Bug fixes:
573
574* Cubes that do not have a spectral axis (such as Stokes cubes) were
575  causing memory leaks and crashing, due to some assumptions of the
576  existence of the spectral axis. These bugs have been fixed, so that
577  Duchamp will run on such cubes, although the WCS transforms may not
578  always work appropriately. Please let me know via bug reports of
579  difficulties you encounter!
580
581* The growing function has had a few bugs fixed and is now more
582  efficient.
583
584* The FDR method was previously overriding any manually-specified
585  threshold. This has been fixed so that if the user specifies
586  threshold=X, then the FDR method is not called.
587
588* Some undefined things were happening with textual output when no
589  objects were detected. This has been fixed.
590
591* Memory leaks caused by FITS WCS irregularities have been fixed
592  (specifically, if a FITS header had NAXIS=2 but more than two WCS
593  axes were defined).
594
595
596
597Duchamp version 1.1.5 (2008/03/20)
598----------------------------------
599
600Bug fixes:
601* A major bug was discovered in the calculation of integrated
602  fluxes. The flux was incorrectly being weighted by the spatial size
603  of the detection, resulting in a (often considerable) overestimation
604  of the integrated flux. This has been corrected.
605
606* The beam size calculation was also incorrect, being out by a factor
607  of 1/ln(2).
608
609* The "make install" command previously failed to work properly if the
610  library had not been created. This has been fixed so that the
611  installation works even if there is no library file.
612
613New Features:
614* You can now provide a new flux unit string, so that the pixel fluxes
615  will be converted to these new fluxes before any calculations are
616  done. This is done with the newFluxUnits input parameter. The new
617  units should be directly convertible from the old ones (given by the
618  BUNIT header -- e.g. if BUNIT = Jy/beam, you can give newFluxUnits
619  as mJy/beam, but not mJy).
620
621* You can now specify the precision used in the output files for flux,
622  velocity and S/N ratio columns, via new input parameters precFlux,
623  precVel and precSNR. These will apply to the VOTable output as well
624  as the text table and information on the spectral plots.
625
626* The Karma annotation files previously provided just a circle at the
627  spatial location of the detected object. You now have the option to
628  draw a border around the detected pixels (in a similar manner to the
629  PGPLOT moment maps). Karma does not do this perfectly, so the lines
630  may not line up exactly with the pixel borders, so the option
631  remains to keep using the circles. This is controlled by the
632  parameter "annotationType", with possible options being "borders"
633  (the default) or "circles".
634
635* There is a new function getMetadata() that loads the header and WCS
636  data, but no pixel information. This is only of interest if you are
637  doing code development with the Duchamp library -- it does not
638  affect the running of Duchamp itself.
639
640Duchamp version 1.1.4 (2008/02/15)
641----------------------------------
642
643Most of the changes incorporated in this update result from ASKAP code
644development (that makes use of some of the Duchamp functionality),
645but do not actually affect the way Duchamp runs (other than, in a
646couple of cases, making it a bit faster). Changes that are
647relevant for regular users of Duchamp are as follows:
648
649* A new option has been made available (thanks to Mary Putman et al
650  for the suggestion!) to save the spectra of all detected objects in
651  a text file. This has the form of one column for the spectral
652  coordinates and one column of fluxes for each object. This is
653  enabled by the input parameters flagTextSpectra and spectraTextFile.
654
655* Slight changes to the output include he formatting of the way the
656  detected objects are written to the log file. This meant the
657  verification files were also updated. Also the cube statistics are
658  not printed to the results file if a user threshold is specified (as
659  they are not calculated!).
660
661* The determination of beam size has been made more reliable, as has
662  the BUNIT determination.
663
664* Fixed a bug that meant source merging was not done for 2D images
665  (not usually a problem, as the source-finding takes care of this,
666  but came up in parallelisation development).
667
668Duchamp version 1.1.3 (2008/01/08)
669----------------------------------
670
671A minor update that just improves the build process. The location of
672the wcslib include files was not specified correctly, so that some
673machines were unable to find them. The configure script has been fixed
674so that the compilation should work in all cases.
675
676Some minor changes have also been made to the code in the PixelMap
677directory, and to param.cc, to fix warnings or errors that may appear
678in compilation.
679
680However, there are no changes to the actual workings of the code in
681this minor version, so if you have a working version of Duchamp 1.1.2
682there is probably no reason to update.
683
684Duchamp version 1.1.2 (2007/12/07)
685----------------------------------
686
687Compilation:
688
689* The compilation had problems working correctly when the cfitsio and
690  wcs libraries were not in standard locations. The configure script
691  has been tweaked to make sure the libraries are found.
692
693* Problems with compiling in the absence of pgplot have been fixed,
694  and it should now compile fine.
695
696* Improved the configure script for use with Mac OS X and gfortran.
697
698
699Bugfixing:
700
701* A major problem with the text-based output was fixed. The widths of
702  the columns were not being calculated correctly, and were running
703  together. This should no longer be the case.
704
705* There was a bug in the way Duchamp interpreted spectral axes,
706  particularly AIPS types such as FELO-HEL, leading to incorrect
707  scaling of the velocity axis. This has been fixed, so that
708  non-standard types are now dealt with appropriately and the velocity
709  scaling should be accurate.
710
711* Another bug meant the beam correction was always applied to the
712  integrated spectra, regardless of whether the flux units involved
713  the beam (e.g. Jy/beam). This has been fixed. Note that the values
714  of the integrated flux presented in the results table were always
715  calculated correctly.
716
717* Other minor changes to improve the speed and efficiency, especially
718  of the merging code.
719
720New user parameters:
721
722* There is now the possibility to save a FITS file that contains a
723  mask array, indicating where the detected objects are. This is
724  governed by the flagOutputMask parameter.
725
726* The header information that is printed at the start of the results
727  file -- that is, the list of parameters, the statistics of the cube,
728  and the number of detections -- can now be written to a separate
729  file. Set the flagSeparateHeader parameter to true or 1 and it will
730  go to the file indicated by the headerFile parameter (the default is
731  duchamp-Results.hdr).
732
733* A "scaleMax" parameter has been added, so that the wavelet
734  reconstruction can include only a specified range of scales (for
735  instance, if one wants to search just for narrow features such as
736  absorption lines).
737
738* A "flagRobustStats" parameter has been added. This allows the user
739  to elect to use robust statistics (median and median absolute
740  deviation from the median) to estimate the noise parameters. If set
741  to false, the mean and standard deviation will be used. The default
742  value is true.
743
744New Feature:
745
746* When Duchamp is run on a 2D image (i.e. with no spectral output), it
747  now produces a postscript file that shows information each
748  individual source plus the image cutout of the source. This is
749  similar to the spectral output file, but without the spectra. The
750  filename is given by the spectraFile parameter.
751
752* Ability to save a FITS file containing a mask array, with pixel
753  values of 1 for pixels in a detected object, and 0 elsewhere.
754
755
756Duchamp version 1.1.1 (2007/07/20)
757------------------------------------
758
759Minor update to the release, with just a few changes:
760
761* The verification script was generating diffs due to differences in
762  the precision of some calculations when done on different machines.
763  One of these was the integrated flux, and this has been corrected so
764  that the same value should appear on all machines.
765  The verification script has also changed so that just the number of
766  sources and their positions are tested, as well as the
767  logfiles. This should avoid the reporting of spurious differences.
768
769* The grey-scale used in the image cutouts (in the spectral plots) has
770  been inverted. This makes it consistent with the scale used in the
771  maps, and hopefully easier to see when printing.
772
773* The configure/make system has been altered to make it easier to
774  install, and to specify different external libraries. In particular,
775  it is now not necessary to have PGPLOT installed on your system to
776  run Duchamp. In the absence of PGPLOT, Duchamp will run as normal,
777  just without any graphical output. Some source code needed to be
778  restructured to make this work. Please read the README file, or the
779  Installation appendix in the User's Guide, for complete details.
780
781* The time of completion is now written to the log file, so that the
782  user can have some idea of how long the program took.
783
784
785Duchamp version 1.1 (2007/05/17)
786------------------------------------
787
788Changes made since the last numbered release:
789
790EFFICIENCY OF EXECUTION:
791
792* Major change to the way detections are stored. They are now recorded
793  via run-length encoding in the x-y plane, and stored as collections
794  of channel-maps. This improves the memory usage and the speed of
795  execution (particularly for routines such as the merging
796  routines). The memory usage has also been improved by changing the
797  way the list of detected objects is stored.
798
799SMOOTHING:
800
801* Addition of ability to spatially smooth the cube before
802  searching. This is done with a Gaussian kernel, the size and shape
803  of which can be specified by the user, using the parameters kernMaj
804  & kernMin (which are the FWHM for the major & minor axes -- if
805  kernMin is not given it is assumed to match kernMaj) and kernPA (the
806  position angle of the major axis).  There is a new parameter
807  smoothType that distinguishes this method from the spectral (Hanning)
808  smoothing already in use.
809
810DETECTIONS:
811
812* Changed the way detection is done -- now only search in 2D images,
813  keeping all detections (ie. no minimum-pixel requirement). This
814  should be the same as doing both the 1D and 2D searches with the
815  minimum pixel requirement in, but simpler. The rejection is done at
816  the end after all the merging and growing has been done. This leads
817  to a larger number of intermediate detections (ie.  the number
818  before merging), which may increase the run-time slightly, although
819  this is offset by the changes to the efficiency (see above).
820
821* Increased the range over which objects can be grown to match the
822  threshSpatial & threshVelocity parameters, as well as rejecting new
823  pixels based on the BLANK and MW tests.
824
825INPUTS:
826
827* There has been a subtle change in the way BLANK pixels are dealt
828  with. To enable trimming of BLANK pixels, users should use the new
829  parameter flagTrim. The parameters flagBlankPix and blankPixVal are
830  no longer accessible via the parameter file -- warning messages are
831  given if they are provided in the parameter file.
832
833* Added ability to specify a statsec, so that the statistics are only
834  calculated from a given subsection of the cube, rather than the full
835  cube.
836
837* The subsections can also specify the ability to remove a border of n
838  pixels from the edges of an axis (ie. n from the start and from the
839  end) -- as requested in Trac ticket #5.
840
841* Changed the default value of the reconDim parameter from 3 to 1, and
842  the default value of the flagATrous parameter to false.
843
844OUTPUTS:
845
846* The output now includes three different estimates of the "centre" of
847  the detection: peak location, centroid (flux-weighted), or average.
848
849* Some slight improvements to the user feedback during execution.
850
851* Spectral output now includes the baseline, if one was fitted. The
852  format of the plot headers is slightly changed, to stop information
853  falling off the edge of the page.
854
855* Added command-line option -x to directly disable the PGPLOT X-window
856  map plotting. This overrides the setting of flagXOutput in the
857  parameter file.
858
859* Fallback spectral units (in case the FITS header fails to record
860  them) changed to SPC, but code has been fixed so that any problem
861  should be caught before this becomes an issue.
862
863BUGFIXES:
864
865* Fixed bug (Trac ticket #4) that meant the velocity units were not
866  correctly dealt with. Duchamp should be more robust for different
867  velocity axis setups now.
868
869* Fixed bug (ticket #9) with the VOTable output, where the tags were
870  not closing.
871
872* Fixed a bug that meant the integrated fluxes were not calculated
873  properly -- no allowance was made for the number of spatial
874  pixels. This should also work properly for 2D images now as well.
875
876* Fixed minor bug in calculating the flux threshold from the p-value
877  threshold when the FDR method is used. This did not affect the
878  actual thresholding, merely the quoted flux threshold in the results
879  file. Also, the MW channels are kept out of the p-value threshold
880  determination if they are being flagged.
881
882
883
884Duchamp version 1.0.7 (2006/11/22)
885----------------------------------
886
887One new feature, and some improvements and bug-fixes.
888
889* The user can now elect to Hanning-smooth the spectral direction of
890  the cube before the searching is done. This can be done instead of the
891  wavelet reconstruction (although if both are selected, the
892  reconstruction takes precendence). This functionality is switched on
893  and off by the flagSmooth parameter, and the width of the Hanning
894  kernel is governed by the hanningWidth parameter.
895
896* The smoothed array can also be written to a FITS file and read in at
897  the start in the same manner as the reconstructed array.
898
899* Several improvements have been made to the memory
900  management. Several memory leaks were fixed, and the memory usage of
901  Duchamp is considerably less, which will be a help for running it on
902  large cubes. The speed of some routines has also been improved, by
903  removing unnecessarily repetitive loops.
904
905* The sorting routines have been changed, and should now be slightly quicker
906  (and a bit more robust).
907
908* Enabled the scale bar (plotted on the 0th moment map in the spectral
909  output file) to be able to deal with much finer spatial scales (as
910  may be expected from VLBI data).
911
912* Other minor changes to code and presentation.
913
914
915Duchamp version 1.0.6 (2006/11/01)
916----------------------------------
917
918Some incremental improvements and changes to the code:
919
920* The statistics used for the detection are now calculated once for
921  the entire cube, rather than being calculated separately for each
922  channel map and spectrum. This results in a uniform threshold value
923  across the entire cube. The way the statistics calculations are
924  implemented in the code was changed, using a new statistics class.
925
926* This means we can report a peak signal-to-noise value for each
927  detection. This is given in the text-based output, as well as in the
928  summary information above the spectral output for each source.
929
930* The user can now elect not to have the moment map displayed in a
931  PGPlot window, through the use of the new flagXOutput
932  parameter. This should help in cases of running Duchamp on a dumb
933  terminal (with no X-display capabilities), or as part of a batch
934  script where the continual popping-up of the PGPlot window might
935  become annoying.
936
937* There are different Flags reported for the case of an object lying on
938  the spatial edge and spectral edge of the cube (E and S respectively).
939
940* A few minor bug fixes:
941  - When providing a subsection, any axes with the range given by *
942    had the pixel values offset incorrectly. This has been fixed.
943  - Negative searches now correctly invert the reconstructed array
944    when it is read in from a file.
945
946* Other minor changes to the code, to improve memory handling and
947  remove redundant arrays and variables.
948
949
950Duchamp version 1.0.5 (2006/09/06)
951----------------------------------
952
953Since version 1.0, there have been a range of mostly minor
954changes. These are generally bug-fixes, thanks to feedback from a
955number of users, as well as a few extra features.
956
957Here is a summary of the changes since version 1.0:
958
959USER INTERFACE:
960
961* The user is now able to enter the value of flag parameters in the
962  parameter file as strings, as well as integers (so one can enter
963  true or 1 to give the same result).
964
965GRAPHICAL OUTPUT:
966
967* Added a new function that draws the edge of the blank region on the
968  maps and image cutouts. There is a new flag parameter --
969  drawBlankEdges -- that controls the use of this function (the
970  default is true).
971  Also, the edge of the field is plotted on the image cutouts as a
972  yellow line. This distinguishes it from the purple blank pixel
973  dividing line.
974
975* The size of the tick mark in the image cutout (indicating the
976  spatial scale of the image) is now adaptable, so that it shows the
977  scale that best fits the image (ranging from 1 arcsec to 15
978  degrees). (This is a change from the previous standard length of 15
979  arcmin.)
980
981* The zoomed spectrum had its flux scale determined by all points in
982  the zoom box,including those flagged by the Milky Way (MW)
983  range. This has been fixed so that any MW channels will not affect
984  the flux scaling, potentially improving the appearance of objects
985  detected close in velocity to strong MW emission or absorption.
986
987* Improved Karma Annotation file output, so that files without good
988  WCS are dealt with appropriately.
989
990TEXT-BASED OUTPUT:
991
992* The position resolution in the IAU-format names has been increased,
993  so that distinct objects close to one another are more readily
994  distinguished.  Overlapping sources at different velocities may
995  still have the same name, but this is a more general problem.
996
997* The presentation of the results in tabular format has been improved,
998  so that the precision of faint sources is sufficient to show their
999  flux, for instance, is not 0. This also applies to the information
1000  presented above the spectral plots.
1001
1002* Improved error and warning reporting, so that the formatting and
1003  presentation of information is clearer.
1004
1005FITS I/O RELATED:
1006
1007* The previous version was did not deal well with 4 (or greater)
1008  dimensions in the FITS file -- the memory allocation was badly done,
1009  and the axes needed to be in a particular order. This has been
1010  fixed, so that the FITS I/O routines can now handle any sized FITS
1011  file, with axes in any order (as long as there are two spatial and one
1012  spectral axes).
1013
1014* When the FITS file does not have a rest frequency defined, there is
1015  no way to transform the frequency axis into velocity. In this case,
1016  the spectral axis is left as frequency, and put into units of MHz.
1017
1018* If the beam size is not indicated by the FITS header (through the
1019  BMAJ and BMIN keywords), the user can manually specify the size
1020  through the parameter file (using parameter beamSize).
1021
1022* The FITS I/O routines make use of the command fits_file_exists. This
1023  was introduced into the CFITSIO package from version 2.5, so Duchamp
1024  will only work with this version or higher. Version 3+ is preferred.
1025
1026OTHER BUG-FIXING:
1027
1028* Several bugs related to the blank-pixels were dealt with.
1029  The correct values were not propagating through correctly, meaning
1030  that the default values were being used. Also, there was the
1031  potential for blank pixels to be detected when the FDR method was
1032  used. Finally, the image cutout displays occasionally did not
1033  obscure the blank pixels. All these problems should be fixed now.
1034
1035* The FDR setup routine was occasionally failing with a segmentation
1036  fault, due to it accessing unallocated memory. This has been fixed.
1037
1038* Fixed bugs that affected the way 2D images (rather than 3D cubes)
1039  were dealt with. The minChannels test failed when minChannels=0.
1040
1041* If there is no beam information in the FITS header, the user can now
1042  enter a beam size as a parameter.
1043
1044* Other minor fixes to the code, to improve its structure and
1045  robustness.
1046
1047
1048Duchamp version 1.0 (2006/06/30)
1049--------------------------------
1050
1051The first full release version.
1052
1053Several major changes compared to previous version:
1054
1055* Incorporated a FitsHeader class, to keep track of header information
1056  (including WCS) from the FITS file. All units-related calculations
1057  are done with this class, such as conversion to velocity, and
1058  pixel-to-WCS conversion.  The main advantage of this is improved
1059  speed of execution, more adaptable to different units, and improved
1060  code (easier for me!).
1061
1062* Able to select the dimensionality of the reconstruction, rather than
1063  just doing it in 3-D.
1064
1065* Robust text output, so that the columns are sized correctly given
1066  the data being written, and the relevant units (derived from the
1067  FITS header) are included in the output table.
1068
1069* Improved the way the reconstructed image is saved and read back in,
1070  with a better naming scheme and a better set of new FITS header
1071  keywords. The MW headers are now not written.
1072
1073Other, more minor changes include:
1074
1075* MW channels now not set to 0, but just ignored for searching
1076  purposes (and for the scaling of the spectral output -- they are
1077  plotted though and highlighted with a hashed box).
1078
1079* The scale bar in the moment map in the spectral output file has its
1080  length indicated.
1081
1082* Improved VOTable output, with more information about the way Duchamp
1083  was run.
1084
1085* Made sure all statistics calculations in the reconstruction are
1086  based on robust (ie. median) methods.
1087
1088Also moved to an autoconf based configure/make compilation, for ease of use.
1089
1090
1091Duchamp version 0.9.2 (2006/06/27)
1092--------------------------------
1093
1094* Added the use of warning flags for detections: for either edge
1095  location or negative enclosed flux.
1096
1097* A new command line option added so that the user can specify a FITS
1098  file to be searched with the default parameters, rather than giving
1099  a full parameter file.
1100
1101* The scale bar on the moment cutout now works in general case, using
1102  an iterative process to get to the correct length.
1103
1104* RA axes are now done correctly on the full-scale maps.
1105
1106* Various coding fixes, improving the readability of the code,
1107  particularly regarding the declaration of temporary variables.
1108
1109* Changed the way the time of execution is called : now using time.h
1110  functions, rather than a system call.
1111
1112* Changed output in Detection/outputDetection.cc for the pixel
1113  information of detected objects (the info printed above the spectral
1114  plots) -- now print out number of voxels in object, and have
1115  improved the formatting a little.
1116
1117* Improved the functionality of saving the FITS file containing the
1118  reconstructed array. Added ability to save and read back in the MW
1119  parameters when saving a reconstructed cube (if a reconstructed cube
1120  has different MW range to that being considered, it would cause
1121  problems). ReadRecon now just returns a FAILURE and will reconstruct
1122  the cube anew.
1123
1124* Changed default value of flagGrowth to 0.
1125
1126
1127
1128Duchamp version 0.9 (2006/05/05)
1129--------------------------------
1130
1131First public release version.
Note: See TracBrowser for help on using the repository browser.