Opened 10 years ago
Closed 10 years ago
#226 closed defect (fixed)
memory issue with duchamp
Reported by: | Owned by: | MatthewWhiting | |
---|---|---|---|
Priority: | normal | Milestone: | Release-1.6.1 |
Component: | other | Version: | 1.5 |
Severity: | normal | Keywords: | |
Cc: |
Description (last modified by )
Dear Matthew,
Currently I have been running duchamp on molecular line data and I encounter a problem. For small cube or higher thresold, duchamp worked fine. However, for an extraction that results more than 500 sources, it crashed (see below). Do you have an idea why? I suspect this has something to do with memory or so. Could you help me to figure it out?
Thank you very much,
Quang,
Dimensions of FITS file: 2881x481x493x1 Reading data ... About to allocate 2.54763GB of which 2.54504GB is for the image Done. Data array has dimensions: 2881x481x493 Opened successfully. ---- Parameters ---- Image to be analysed.............................[imageFile] = COGAL_all_interp_reproject.fits Intermediate Logfile...............................[logFile] = duchamp-Logfile_1K.txt Final Results file.................................[outFile] = duchamp-Results_1K.txt Spectrum file..................................[spectraFile] = duchamp-Spectra_1K.ps 0th Moment Map...................................[momentMap] = duchamp-MomentMap_1K.ps Detection Map.................................[detectionMap] = duchamp-DetectionMap_1K.ps Display a map in a pgplot xwindow?.............[flagXOutput] = true Saving mask cube?...........................[flagOutputMask] = false Saving 0th moment to FITS file?........[flagOutputMomentMap] = false Saving 0th moment mask to FITS file?..[flagOutputMomentMask] = false Saving baseline values to FITS file?....[flagOutputBaseline] = false ------ Type of searching performed.....................[searchType] = spatial Trimming Blank Pixels?............................[flagTrim] = false Searching for Negative features?..............[flagNegative] = false Area of Beam................................................ = No beam Removing baselines before search?.............[flagBaseline] = false Smoothing data prior to searching?..............[flagSmooth] = false Using A Trous reconstruction?...................[flagATrous] = false Using Robust statistics?...................[flagRobustStats] = true Using FDR analysis?................................[flagFDR] = false Detection Threshold..............................[threshold] = 0.5 Minimum # Pixels in a detection.....................[minPix] = 5 Minimum # Channels in a detection..............[minChannels] = 5 Minimum # Voxels in a detection..................[minVoxels] = 10 Growing objects after detection?................[flagGrowth] = false Using Adjacent-pixel criterion?...............[flagAdjacent] = true Max. velocity separation for merging........[threshVelocity] = 5 Reject objects before merging?.......[flagRejectBeforeMerge] = true Merge objects in two stages?...........[flagTwoStageMerging] = true Method of spectral plotting.................[spectralMethod] = sum Type of object centre used in results..........[pixelCentre] = centroid -------------------- Commencing search in cube... Using flux threshold of: 0.5 Searching... Found 68943. Updating detection map... Done. Done. Intermediate list has 30023 objects. Merging and Rejecting... Done. Final object count = 525 Calculating final parameters and setting flags... |# |terminate called after throwing an instance of 'std::bad_alloc' what(): std::bad_alloc Aborted (core dumped)
Change History (4)
comment:1 Changed 10 years ago by
Component: | Building/Installation → other |
---|---|
Milestone: | → Release-1.7 |
Status: | new → assigned |
Type: | enhancement → defect |
comment:2 Changed 10 years ago by
Description: | modified (diff) |
---|
comment:3 Changed 10 years ago by
Milestone: | Release-1.7 → Release 1.6.1 |
---|
Quang provided a FITS cube to test with.
Fault still occurring in v1.6.
Cause identified as a memory leak in creating the blank pixel masks used in the parameterisation. Principal fix was applied in [1380].
Patched version 1.6.1 will be released very soon.
comment:4 Changed 10 years ago by
Resolution: | → fixed |
---|---|
Status: | assigned → closed |
1.6.1 was released last week.
Closing this ticket but please re-open if the problem reoccurs.
Gday Quang,
Thanks for your feedback. It's not completely obvious exactly what is causing the problem here - the error message indicates it is coming from a failure to allocate memory, but there are a few candidates for which allocation is doing it.
I'm actually going to release an updated version very soon. You could try that (there are a few performance improvements to the code that may help), but if that doesn't help I could happily look at the actual data and try and replicate the problem (that would help me track down what is going wrong). Feel free to email me if you like.
How big are your objects? Could you attach the logfile, perhaps, as that will give me an idea of their size (possibly gzipped - it may be large..). Also, how large is the memory on the machine you are running this on?
Cheers, Matt