Reducing CABB Data

An example

This page will walk through the reduction process for the ToO observation CX167, which was observed on 16 Feb 2009 with the interim CABB system. This system had 5 antenna dual pol, 1 IF at 6/3cm, but not all 2048 continuum channels were working.

First, make sure you have the CABB version of MIRIAD, otherwise this will not work!

To do this on kaputar, make sure that /nfs/atapplic/miriad-cabb/linux64/bin is in your path ahead of any other MIRIAD directories. To check that this is so, in a terminal, type which atlod. It should return /nfs/atapplic/miriad-cabb/linux64/bin/atlod.

Then source /nfs/atapplic/miriad-cabb/MIRRC. You should now be using the CABB version of MIRIAD.

To begin with, the reduction is the same as any other pre-CABB dataset, and we start by using ATLOD to read in the data. We have four files for CX167: 2009-02-16_0658.CX167, 2009-02-16_0701.CX167, 2009-02-16_1651.CX167 & 2009-02-16_1824.CX167.

atlod in=*.CX167 out=cx167.uv ifsel=2 options=birdie

Here, we use ifsel=2 because only 1 IF was available with the interim CABB system and it was written as IF 2.

At this point, we can see what the CABB spectrum looks like with uvspec:

uvspec vis=cx167.uv "select=pol(xx,yy)" interval=10 options=nobase,avall axis=chan,amp device=/xs

We want to calibrate the bandpass. Luckily there is some 1934-638 data, so we calibrate our bandpass with that:

mfcal vis=cx167.uv "select=source(1934-638)" interval=0.1

Giving the same uvspec command as before we now get:

There is some obvious narrowband interference present in the data. With CABB's polyphase filter design, the channels are designed to be very sharp-edged, so most interference will be present only in one channel, even if it is very strong. This should allow us to very effectively excise this RFI with an automated routine.

One option is to use Pieflag, Enno Middleberg's automatic flagging script. It works by looking at each channel in turn through time, and finding outliers in comparison to a channel that is known not to have RFI. This works quite well for continuum datasets from the old correlator, since there are fewer channels. Pieflag loads the entire dataset into memory, which can be quite a strain on a computer that does not have enough memory for the datasets that CABB will produce.

Another option is to use a script made by Jamie Stevens to look for narrowband RFI (which is attached to this page). This script looks for individual channels that deviate from the median by more than an acceptable amount.

To search the data for narrowband RFI, give the command:

cabb_nb_rfi_extractor.pl --doit --select-options "-auto,-ant(6)" --interval 10 --flagscript --vis cx167.uv --sigma 5

The script finds that there is interference in channels 5, 133, 645, 1149, 1150, 1157 and 1669. It makes another script which contains the uvflag commands required to flag these channels, which can then be executed.

After flagging, the bandpasses look like this:

There seems to be higher noise levels towards the edge of the band, especially at the high channel end, so we use uvflag to clip off 50 channels from each side of the bandpass:

uvflag vis=cx167.uv edge=50 flagval=flag

It should also be noted that the RFI detection script only flags RFI channels that persist throughout most of the intervals that are examined. This behaviour can be changed by telling the script to flag each interval separately (using the --nearby option, or by setting the --persistent flag to be a lower number than the default 30%.

The upshot is that at this point there is still some RFI present, ie. channel 513, during the first observations of 1934-638.

The next step is to recalibrate the dataset again after removing most of the RFI.

mfcal vis=cx167.uv "select=source(1934-638)" interval=0.1

Let's look at the calibration of 1934-638.

uvplt vis=cx167.uv "select=-auto,-ant(6),pol(xx,yy),source(1934-638)" axis=real,imag options=nobase device=/xs

It's not bad, but there are enough outliers to warrant a closer look. A time-phase plot shows the problem:

uvplt vis=cx167.uv "select=-auto,-ant(6),pol(xx,yy),source(1934-638)" axis=time,phase options=nobase device=/xs

The time between 07:09:00 and 07:09:10 has bad phases, so we flag this out:

uvflag vis=cx167.uv "select=time(07:09:00,07:09:10)" flagval=flag

and recalibrate as before. The real,imag plot now looks like this:

Not perfect, but it is good enough for now.

The next step is where CABB reduction really differs from that of the old correlator. We need to determine whether we want to attack the dataset as a whole, working with all 2 GHz of bandwidth simultaneously, or do we want to "divide and conquer" by splitting the band into chunks similar to those output by the old correlator and that we know will work fine with MIRIAD?

In this example we will look at how to proceed in both directions, but we will go through the "divide and conquer" (DAC) procedure first.

To begin the DAC procedure, we use the new CABB uvsplit to split the visibility file, not only based on source, but into fixed bandwidth chunks. Since we know that 128 MHz bandwidth works in MIRIAD, we choose this for uvsplit:

uvsplit vis=cx167.uv select=-auto,-ant(6) maxwidth=0.128

This produces visibility sets for 1934-638, 0515-674 and rnovalmc, at centre frequencies (in MHz) of 4540, 4668, 4796, 4924, 5052, 5180, 5308, 5436, 5564, 5692, 5820, 5948, 6076, 6204, 6332 and 6460; a total of 16 bands (16 x 128 MHz = 2048 MHz, the actual bandwidth of CABB).

Let's look at the bandpass for one of the bandwidth chunks of the secondary calibrator 0515-674:

uvspec vis=0515-674.5436 "select=pol(xx,yy)" interval=10 options=nobase,avall axis=chan,amp device=/xs nxy=4,4

The bandpasses look pretty good. Note that although we are using 128 MHz bandwidth, the same as was possible with the old correlator, we have 128 channels across the bandwidth, instead of 32 which was usually found with the old correlator.

We should now calibrate the secondary with mfcal, in the same way that it would have been done with old data.