Duchamp crashes

[andrew.walsh@…]

Hey Matty, just downloaded version 1.5 of Duchamap, as the old version crashed on me with a new file I am processing. Unfortunately, this version crashes as well. I suspect it might be because the fits file is 14GB, but am not sure. Here is a log of what I did:

awalsh@awalsh:~/THOR/1667$ cat duckmaps.param 
imageFile 1667.sn.fits
logFile logfile.txt
outFile results.txt
spectraFile spectra.ps
Subsection [*,*,*]
flagSubsection true
flagKarma true
minPix  15
minChannels  2
minVoxels 26
snrCut 3.0
flagAdjacent true
threshVelocity 1
awalsh@awalsh:~/THOR/1667$ Duchamp -v
Duchamp version 1.5
awalsh@awalsh:~/THOR/1667$ Duchamp -p duckmaps.param 
Opening image: 1667.sn.fits[1:2880,1:2880,1:460]
Dimensions of FITS file: 2880x2880x460
Reading data ... 
 About to allocate 14.229GB of which 14.2136GB is for the image
Done. Data array has dimensions: 2880x2880x460
Opened successfully.

---- Parameters ----
Image to be analysed.............................[imageFile]  =  1667.sn.fits[1:2880,1:2880,1:460]
Intermediate Logfile...............................[logFile]  =  logfile.txt
Final Results file.................................[outFile]  =  results.txt
Karma annotation file............................[karmaFile]  =  duchamp-Results.ann
Saving mask cube?...........................[flagOutputMask]  =  false
Saving 0th moment to FITS file?........[flagOutputMomentMap]  =  false
Saving 0th moment mask to FITS file?..[flagOutputMomentMask]  =  false
Saving baseline values to FITS file?....[flagOutputBaseline]  =  false
------
Type of searching performed.....................[searchType]  =  spatial
Trimming Blank Pixels?............................[flagTrim]  =  false
Searching for Negative features?..............[flagNegative]  =  false
Area of Beam................................................  =  No beam
Removing baselines before search?.............[flagBaseline]  =  false
Smoothing data prior to searching?..............[flagSmooth]  =  false
Using A Trous reconstruction?...................[flagATrous]  =  false
Using Robust statistics?...................[flagRobustStats]  =  true
Using FDR analysis?................................[flagFDR]  =  false
SNR Threshold (in sigma)............................[snrCut]  =  3
Minimum # Pixels in a detection.....................[minPix]  =  15
Minimum # Channels in a detection..............[minChannels]  =  2
Minimum # Voxels in a detection..................[minVoxels]  =  26
Growing objects after detection?................[flagGrowth]  =  false
Using Adjacent-pixel criterion?...............[flagAdjacent]  =  true
Max. velocity separation for merging........[threshVelocity]  =  1
Reject objects before merging?.......[flagRejectBeforeMerge]  =  false
Merge objects in two stages?...........[flagTwoStageMerging]  =  true
Method of spectral plotting.................[spectralMethod]  =  peak
Type of object centre used in results..........[pixelCentre]  =  centroid
--------------------

Commencing search in cube...
  Calculating the cube statistics... terminate called after throwing an instance of 'std::bad_alloc'
  what():  std::bad_alloc
Aborted (core dumped)
awalsh@awalsh:~/THOR/1667$ 

Later. Andrew xxx

[MatthewWhiting]

Hi Andrew,

Apologies - have just found this ticket and can't remember if I spoke to you about this (may have been lost in the pre-christmas madness...)

Anyway, what seems to be happening is that it runs out of memory in the statistics-calculation stage - while it was able to read the FITS image, it then makes a mask indicating the valid pixels, and I suspect this is causing it to run out of memory.

I'm actually just about to release a new version that includes a small fix for this - if you don't have any blank pixels or flagged channels, and you aren't using a specific subsection for stats calculations, then it won't make this mask. That should help get past this point (although you may strike further problems depending on how many sources you find).

Give the new one a try (will be available shortly, once I've finalised documentation etc). If it is still a problem, you may have to split the cube up using the subsection parameter.

Matt.

[MatthewWhiting]

Tracking note: relevant change was in [1371].