memory issue with duchamp

[qnguyen@…]

Dear Matthew,

Currently I have been running duchamp on molecular line data and I encounter a problem. For small cube or higher thresold, duchamp worked fine. However, for an extraction that results more than 500 sources, it crashed (see below). Do you have an idea why? I suspect this has something to do with memory or so. Could you help me to figure it out?

Thank you very much,

Quang,

Dimensions of FITS file: 2881x481x493x1
Reading data ... 
 About to allocate 2.54763GB of which 2.54504GB is for the image
Done. Data array has dimensions: 2881x481x493
Opened successfully.

---- Parameters ----
Image to be analysed.............................[imageFile]  =  COGAL_all_interp_reproject.fits
Intermediate Logfile...............................[logFile]  =  duchamp-Logfile_1K.txt
Final Results file.................................[outFile]  =  duchamp-Results_1K.txt
Spectrum file..................................[spectraFile]  =  duchamp-Spectra_1K.ps
0th Moment Map...................................[momentMap]  =  duchamp-MomentMap_1K.ps
Detection Map.................................[detectionMap]  =  duchamp-DetectionMap_1K.ps
Display a map in a pgplot xwindow?.............[flagXOutput]  =  true
Saving mask cube?...........................[flagOutputMask]  =  false
Saving 0th moment to FITS file?........[flagOutputMomentMap]  =  false
Saving 0th moment mask to FITS file?..[flagOutputMomentMask]  =  false
Saving baseline values to FITS file?....[flagOutputBaseline]  =  false
------
Type of searching performed.....................[searchType]  =  spatial
Trimming Blank Pixels?............................[flagTrim]  =  false
Searching for Negative features?..............[flagNegative]  =  false
Area of Beam................................................  =  No beam
Removing baselines before search?.............[flagBaseline]  =  false
Smoothing data prior to searching?..............[flagSmooth]  =  false
Using A Trous reconstruction?...................[flagATrous]  =  false
Using Robust statistics?...................[flagRobustStats]  =  true
Using FDR analysis?................................[flagFDR]  =  false
Detection Threshold..............................[threshold]  =  0.5
Minimum # Pixels in a detection.....................[minPix]  =  5
Minimum # Channels in a detection..............[minChannels]  =  5
Minimum # Voxels in a detection..................[minVoxels]  =  10
Growing objects after detection?................[flagGrowth]  =  false
Using Adjacent-pixel criterion?...............[flagAdjacent]  =  true
Max. velocity separation for merging........[threshVelocity]  =  5
Reject objects before merging?.......[flagRejectBeforeMerge]  =  true
Merge objects in two stages?...........[flagTwoStageMerging]  =  true
Method of spectral plotting.................[spectralMethod]  =  sum
Type of object centre used in results..........[pixelCentre]  =  centroid
--------------------

Commencing search in cube...
  Using flux threshold of: 0.5
  Searching... Found 68943.          
  Updating detection map... Done.
Done. Intermediate list has 30023 objects.
Merging and Rejecting...  Done.                      
Final object count = 525
Calculating final parameters and setting flags...  |#                   |terminate called after throwing an instance of 'std::bad_alloc'
  what():  std::bad_alloc
Aborted (core dumped)

[MatthewWhiting]

Gday Quang,

Thanks for your feedback. It's not completely obvious exactly what is causing the problem here - the error message indicates it is coming from a failure to allocate memory, but there are a few candidates for which allocation is doing it.

I'm actually going to release an updated version very soon. You could try that (there are a few performance improvements to the code that may help), but if that doesn't help I could happily look at the actual data and try and replicate the problem (that would help me track down what is going wrong). Feel free to email me if you like.

How big are your objects? Could you attach the logfile, perhaps, as that will give me an idea of their size (possibly gzipped - it may be large..). Also, how large is the memory on the machine you are running this on?

Cheers, Matt

[MatthewWhiting]

Quang provided a FITS cube to test with.

Fault still occurring in v1.6.

Cause identified as a memory leak in creating the blank pixel masks used in the parameterisation. Principal fix was applied in [1380].

Patched version 1.6.1 will be released very soon.

[MatthewWhiting]

1.6.1 was released last week.

Closing this ticket but please re-open if the problem reoccurs.