Changes/atlod

atlod
Item	Complexity	Research	Milestone	Time
Sampler Statistics	easy	none	6	1d
StorageOfSYSCALData	moderate	some	6	2*5d
ApplyingXYPhaseCorrections	easy	little	6	5d
DataSizeLimits?	easy	little	6	2d
FlaggingWithinATLOD - basics	easy	little	6	5d
MoreSelectionOptions?	easy	little	6	5d
IdentifyParentWindow?	moderate	some	8	2*5d
FlaggingWithinATLOD - autoflagging	moderate	some	9	4*5d

SYSCAL Record Changes

The SYSCAL records will change from their current form.

SamplerStatistics?

we no longer use these. So we need to avoid checking for them with CABB-type data.

StorageOfSYSCALData

Now that there are no sampler statistics, we need a different way of carrying along Tsys? and XYPhase? data than we have at present. At present there are 24 scalar values of Tsys (6 antennae * 2 polarizations * 2 IFs) written into every integration, and 12 (6 ant. * 2 IFs) of XYPhase.[BR] [BR] In the new system there will likely be a spectrum of each, in the form of autocorrelations. Tsys is formed from the XX and YY polarisations of the autocorrelations, with the noise diode on and off; XYPhase from the XY (or its conjugate, YX) with the diode on and off.

Deciding what to do here in terms of data representation requires discussions with WarwickWilson?. Making changes here has flow on effects to other tasks that use Tsys or XYPhase.

ApplyingXYPhaseCorrections

At present atlod applies the XYPhase during filling. Nothing else (except maybe atxy) uses them. So we can compute the corrections, apply them and drop the XYPhase information, during ATLOD? May need to order the RPFITS data stream so that autocorrelations come before cross-correlations (and after SYSCAL records), so that the corrections for the cross-correlation can be computed before reaching the cross-correlations. We will need to carry Tsys info along but may be able to do it more compactly than storing the autocorrelations themselves.[BR]

[BR]

Research required: The phase values computed on a per channel basis will be noisy, and applying them will just introduce noise. We need to understand how much smoothing across frequency will be needed. Need to do some experiments to understand the sequence of events in reading in a file.

DataSizeLimits?

Research required: need to check for size limits on the data dimensions - number of windows, channels, etc

XYPhaseCalibration?

instead of a single (instrumental) XY phase term, we will now get all the autocorellations. This has the potential to greatly inflate the data volume.

IdentifyParentWindow?

Each spectral window output by the correlator can have some ZoomBands? defined. We think that it will be necessary to keep track of each window's parent window. There should only be 0 or 1 parents.

Research needed: Whether it is necessary to keep track of the children of a particular band is unclear.

FlaggingWithinATLOD?

atlod does a qualitatively different type of flagging to other parts of Miriad; it drops data from the output instead of just marking it as bad.

There are two possible types of flagging-related changes.

• basic things like removal of birdies - there are likely to be some new ones.[BR] Research needed: locations of birdies; will there be new types of flags on scans that indicate bad data?
• Possibly using the autocorrelations? to flag data affected by interference.

MoreSelectionOptions?

People will want to atlod just a subset of the data, not the whole thing. This requires more selection options than is available with 'ifsel'.

Research required: need to list out the types of datasets that are likely to be written and then decide what the "sub-selectable" elements are.

Wiki Home