Spatial & spectral searches do not agree

[MatthewWhiting]

We get different results if we use searchMethod = spectral or spatial!

This should not be the case.

As an example, here are the results from running a 3sigma search on a 3D-reconstructed version of the standard H201 hipass test:

• spectral search: found 372; intermediate list 372; final list 20
• spatial search: found 8477; intermediate list 8477; final list 38

Both flagRejectBeforeMerge=false and flagTwoStageMerging=false

[MatthewWhiting]

If I turn off the reconstruction, and do normal searching to a flux threshold (0.07 Jy/beam) then I get the same results for both (subject to single differences in the last decimal place of the velocity for two sources.

This might suggest it is the reconSearch tasks that are doing something odd. But the giant tests, that first threw up this issue were done without reconstruction, so that might not be the full story.

[MatthewWhiting]

(Note on previous result - the number of intermediate sources differs, but that is probably expected.)

[MatthewWhiting]

Looking at the mask cubes that are produced, with no reconstruction they are identical, but the original case with reconstruction has quite a few differences.

[MatthewWhiting]

Potential solution found. The spectral recon search was setting the minimum size to be minChannels, whereas the regular spectral search was using 1.

I've changed [1007] the recon search to use 1 as well -- we apply the size criteria at the merging stage, so we want everything included here.

Appears to work for the HIPASS case, with the original test described above now giving consistent results for spectral & spatial. Now to test with the giant case.

[MatthewWhiting]

OK, getting a bit closer. That solution didn't have any effect (it really only affected the dimension lengths, which are not big enough to be changed by size_t/int/long).

Instead, it appears the MW flagging is being applied. The input parset has flagMW=false (and the output reports it as such), but it does provide minMW and maxMW. Perhaps the checking functions are not responding to the flag correctly?

It does seem that isInMW could be incorrectly flagging, but fixing this doesn't solve it.

[MatthewWhiting]

Solved in [1010]. The problem originated in the fact that the spectral searches have an spatial-sized array doPixel to see whether there are any good (ie. not blank and not MW) channels in that spectrum.

The problem was that the test for blankness only ever looked at the first channel, so if that was blank the entire spectrum was ignored. And this is the case for some sources in BiQing?'s cube (the one that I was looking at for the test).

Fixing this gives consistent results between spectral and spatial searches, so I think we can close this ticket.