Opened 14 years ago

Last modified 14 years ago

#87 assigned task

Enable sequential reading/searching of large images

Reported by: MatthewWhiting Owned by: MatthewWhiting
Priority: normal Milestone: Release-2.0
Component: Searching Version: 1.1.9
Severity: normal Keywords:
Cc:

Description

While proper distributed processing is not yet available in Duchamp, is should be easy to implement a pseudo-distributed search, where each channel map is read and searched individually, then the sources combined in the usual way.

Use this ticket to record work needed to produce this option - it would be useful for searching large images that do not fit in memory.

Change History (3)

comment:1 Changed 14 years ago by MatthewWhiting

Status: newassigned

comment:2 Changed 14 years ago by MatthewWhiting

Have attempted to implement something along these lines (subCubeIter.x in my devel directory). There are several problems:

  • The sources need to have their voxels extracted and saved for use in the WCS parameter calculations (a la the askapsoft approach)
  • It crashes when doing the plots - these functions currently require access to the full image array, which is empty in this case.

Changes needed:

  • Make use of vectors of voxels for detected sources
  • Alternative way of doing maps & spectra, that is oriented towards individual sources and either reads the data directly from the image or takes a vector of voxels

comment:3 Changed 14 years ago by MatthewWhiting

[724] has a new function for Object3D that returns a pixelset (vector list of voxels) with the flux values filled in from an array (that is given as an argument).

This should do what is required for the first dot point above.

Note: See TracTickets for help on using tickets.