Changes/atlod

atlod
Item	Complexity	Research		Time
Sampler Statistics	easy	none	7	1d
Storage Of SYSCAL Data	moderate	some	old6	2*5d
ApplyingXYPhaseCorrections	easy	little	old6	5d
Data Size Limits	easy	little	old6	2d
FlaggingWithinATLOD - basics	easy	little	old6	5d
MoreSelectionOptions?	easy	little	old6	5d
IdentifyParentWindow?	moderate	some	old8	2*5d
Flagging Within ATLOD - autoflagging	moderate	some	old9	4*5d

Milestone links: 7, 9, 10

SYSCAL Record Changes

The SYSCAL records will change from their current form, see SyscalDataFormats for an attempt to summarize the current state for the ATNF Telescopes.

SamplerStatistics?

Not present in CABB data; do not execute flagging or "samcorr" code.

tickets: #22, #23

StorageOfSYSCALData

ticket #46

AT correlator data format holds 24 scalar values of Tsys (6 antennae * 2 polarizations * 2 IFs) written into every integration, and 12 (6 ant. * 2 IFs) of XYPhase.

CABB data will include autocorrelation for both ON and OFF phases of the switched noise diode.
ATLOD will calculate

Tsys(f) = OFF(f) * Tcal(f) / ( ON(f) - OFF(f) )

for each of the XX and YY polarization products. In the equation above 'f' is the frequency.
Tcal(f) will be included in the RPFITS file once per scan.
ATLOD will also calculate

XYPhase(f) = arg( XYPhase_ON(f) - XYPhase_OFF(f) )

ATLOD will record in the Miriad data TsysXX(f), TsysYY(f) and XYPhase(f). Input option 'noauto' will determine whether autocorrelation spectra are kept.
Deciding what to do here in terms of data representation requires discussions with WarwickWilson?. Making changes here has flow on effects to other tasks that use Tsys or XYPhase.

ApplyingXYPhaseCorrections

ATLOD applies the XYPhase correction, but retains the XYPhase(f) spectrum in the output. A few tasks (eg atxy and varplt ) use these data.

Research needed: The phase values computed on a per channel basis will be noisy, and applying them will just introduce noise. We need to understand how much smoothing across frequency will be needed. Need to do some experiments to understand the sequence of events in reading in a file.
Research needed: Does ATLOD work in a streaming mode? (LiveData? has a simple implementation of this - when you hit EOF, wait for a bit and then give up)

Data Size Limits

Research needed: check for size limits on the data dimensions - number of windows, channels, etc

• Limit on number of channels is ok, edit maxchan in maxdim.h. The total number of simultaneous channels from CABB could be as high as 74k. The main limits on CABB output rates are set by the maximum rate data can be written to disk, so we need to wait to see what will be used in practice. Currently maxchan is set to about 32k.
• Limit on number of spectral windows or if_chains is set by max_if=16 in rpfits.inc. This should be ok for most observations, but CABB could produce up to 34 simultaneous spectra (2 continuum bands with up to 16 zoom bands each). The constant ATIF in atlod.h sets the maximum number of IFs atlod can cope with, it has been increased from 4 to 16 to match the rpfits limit. As of Dec 2010 the max_if limit has been increased to 48 in atlod and rpfits.

Identify Parent Window

Each spectral window output by the correlator can have some ZoomBands? defined. Miriad will keep track of each window's parent window. There will be 0 or 1 parents.

Research needed: Determine whether it is necessary to keep track of the children of a particular band.

• We can use the if_chain variable to keep track of this, all zoom bands should have the same value for if_chain as the main continuum band. Comparing the if_chain and the time stamps for the data is sufficient to identify the parent window.

Flagging Within ATLOD

atlod does not copy bad data to its output. This makes flagging in ATLOD qualitatively different to other parts of Miriad. Other editing tasks in Miriad set flags in a flag table to mask out bad data.

For CABB data, ATLOD's birdie flagging will change:

• a different set of birdie frequencies (to be determined)
• removal of every second frequency channel will not be needed.

Research needed

• What are the birdie frequencies in the new system?
  • There may be birdies and there will definitely be known external RFI that can be masked out.
• Can data be flagged automatically by ATLOD using autocorrelations?? (Could we use algorithms from 'pieflag'?)
  • Probably not - it's hard to distinguish genuine spectral lines from RFI, it might be possible for continuum observations.

DEC 2010: CABB specific birdie and rfi flagging is in place for fixed birdies and rfi. Variable rfi can be flagged with mirflag, pgflag or uvflag.

More Selection Options

Miriad will need the capability to inspect an RPFITS file in order to determine the identity of the various spectral windows in the file.

MarkCalabretta? wrote the 'rpfhdr' utility which does most of this.

• rpfhdr will give you the frequency table for each scan, atlod followed by uvindex gives a detailed listing of the frequency setups present.
• the ifsel selection in atlod lets you select the cabb continuum bands 1 or 2 for loading. At present there is no way to just load a single zoom band.

Research needed: How are zoom bands to be represented in the RPFITS data format?

• Just like current multiple simultaneous IFs, except that the if_chain number is set to that of the parent continuum band.

• The ifsel keyword has been extended to allow a number of IFs to be selected.

Changes Home
Wiki Home