

\section{Description of various files} \label{sec:files}

In the descriptions that follow, the locations of some files is given as {\tt /home/vlbiobs}, meaning the directory {\tt /home/vlbiobs/astronomy/}{\em mmmyy}{\tt /}{\em project} or one of its subdirectories (this is VLBA-centric).
Here {\em mmmyy} is the month and year of the project's observation (i.e., {\tt jan08}) and project is the full project name, with segment, in lower case, such as {\tt bw088n}.
In what follows, the ``software correlator project directory'' (sometimes ``project directory'') refers to the directory from which software correlation is to proceed.
File names beginning with a period (e.g., {\tt .acb}) represent file name extensions, typically (but not always) to job file bases, such as {\tt job121.000} .
%Examples of many of the following file types for a particular VLBA correlator job are stashed at \url{http://www.aoc.nrao.edu/~wbrisken/NRAO-DiFX-1.1/} .






% .aapd -----------------------------------------------------------------------

\subsection{.aapd} \label{sec:aapd}

The program {\tt apd2antenna} (see Sec.~\ref{sec:apd2antenna}) takes the {\tt .apd} file (Sec.~\ref{sec:apd} created by {\tt difx2fits} (Sec.~\ref{sec:difx2fits}) and performs least-squares fits to reference the phase, delay, and rate measurements to a specified reference antenna.

The first line in the file is {\tt obscode:} followed by the observation code, e.g., {\tt MT831} .

Each subsequent line has the same format with the following fields:

\begin{center}
\begin{tabular}{l l l}
\hline
Key & Units/allowed values & Comments \\
\hline
{\it MJD}            & integer $\ge 1$    & MJD day number corresponding to line \\
{\it hour}           & $\ge 0.0$, $< 24.0$ & hour within day \\
{\it source name}    & string             & name of source; no spaces allowed \\
{\it antenna number} & integer $\ge 1$    & antenna table index for first antenna \\
{\it antenna name}   & string             & name of antenna 1; no spaces allowed \\
{\it n}$_{\mathrm{BBC}}$  & integer $\ge 1$    & number of baseband channels, $n_{\mathrm{BBC}}$ \\
& & The next four columns are repeated $n_{\mathrm{BBC}}$ times \\
\hline
{\it delay}         & ns                 & the fringe fit delay \\
{\it phase}         & degrees            & phase of fringe fit peak \\
{\it rate}          & Hz                 & the fringe fit rate \\
\hline
\end{tabular}
\end{center}





% .abp ------------------------------------------------------------------------

\subsection{.abp} \label{sec:abp}

When run with the {\tt --bandpass} option, {\tt difx2fits} will create a {\tt .bandpass} file.
This file can be run through {\tt bp2antenna} (Sec.~\ref{sec:bp2antenna} to convert the baseline-based bandpasses to antenna-based bandpasses.

It is possible that line comments, starting with \#, are present in the file.

The first line of the file is {\tt obscode:} followed by the project code.

A header line starting with {\tt Bandpass} will signify the start of the data for one antenna for one baseband channel.
Such a line has exactly eight space-separated fields:
\begin{enumerate}
  \item The keyword {\tt Bandpass}
  \item Zero-based antenna number for the antenna
  \item Uppercase antenna code for the antenna
  \item Baseband channel number (zero-based)
  \item Number of points in the bandpass, $N_{\rm point}$
  \item Signed Sum of Local Oscillator (SSLO) for the baseband channel frequency (MHz)
  \item Baseband channel bandwidth (MHz)
  \item Polarization (typically {\tt R}, {\tt L}, {\tt H}, or {\tt V})
\end{enumerate}

Following a header line should be the specified number, $N_{\rm point}$, of data lines, each with three space-separated values:
\begin{enumerate}
  \item Sky frequency (MHz)
  \item Real part of bandpass at this frequency
  \item Imaginary part of bandpass at this frequency
\end{enumerate}

Note that the bandpass is represented as the measurements.
The correction factor to flatten the bandpass would be the complex-valued reciprocal of these measurements.








% .acb ------------------------------------------------------------------------

\subsection{.acb} \label{sec:acb}

When generation of sniffer output files is not disabled, each {\tt .FITS} file written by {\tt difx2fits} will be accompanied by a corresponding {\tt .acb} file. 
This file contains auto-correlation spectra for each antenna for each source.
In order to minimize the output data size, spectra for the same source will only be repeated once per 15 minutes.
The file contains many concatenated records.
Each record has the spectra for all baseband channels for a particular antenna and has the following format.  
Note that no spaces are allowed within any field.
Values in {\tt typewriter} font without comments are explicit strings that are required.

\begin{center}
\begin{tabular}{l l l l}
\hline
Line(s) & Value & Units & Comments \\
\hline
1 & {\tt timerange:} & & \\
  & {\it MJD}    & integer $\ge 1$    & MJD day number corresponding to line \\
  & {\it start time} & string & e.g., {\tt 13h34m22.6s} \\
  & {\it stop time}  & string & e.g., {\tt 13h34m52.0s} \\
  & {\tt obscode:} & & \\
  & {\it observe code} & string & e.g., MT831 \\
  & {\tt chans:} & & \\
  & {\it n}$_{\mathrm{chan}}$ & $\ge 1$ & number of channels per baseband channel \\
  & {\tt x} & & \\
  & {\it n}$_{\mathrm{BBC}}$ & $\ge 1$ & number of baseband channels \\
\hline
2 & {\tt source:} & & \\
  & {\it source name} & string & e.g., {\tt 0316+413} \\
  & {\tt bandw:} & & \\
  & {\it bandwidth} & MHz & baseband channel bandwidth \\
  & {\tt MHz} & & \\
\hline
3 to 2+$n_{\mathrm{BBC}}$ & {\tt bandfreq:} & & \\
  & {\it frequency} & GHz & band edge (SSLO) frequency of baseband channel \\
  & {\tt GHz polar:} & & \\
  & {\it polarization} & 2 chars & e.g. {\tt RR} or {\tt LL} \\
  & {\tt side:} & & \\
  & {\it sideband} & {\tt U} or {\tt L} & for upper or lower sideband \\
  & {\tt bbchan:} & & \\
  & {\it bbc} & {\tt 0} & Currently not used but needed for conformity \\
\hline
3+$n_{\mathrm{BBC}}$ to & {\it antenna number} & $\ge 1$ & antenna table index \\
2+$n_{\mathrm{BBC}}(n_{\mathrm{chan}}+1)$  & {\it antenna name} & string & \\
  & {\it channel number} & $\ge 1$ & $= \mathrm{chan} + (\mathrm{bbc}-1) \cdot n_{\mathrm{chan}}$ for chan, bbc $\ge 1$ \\
  & {\it amplitude} & $\ge 0.0$ & \\
\hline
\end{tabular}
\end{center}

\noindent
The above are repeated for each auto-correlation spectrum record.
This file can be plotted directly with {\tt plotbp} or handled more automatically with {\tt difxsniff}.






% .apc ------------------------------------------------------------------------

\subsection{.apc} \label{sec:apc}

This file type is nearly identical to the better known {\tt .apd} file; the name acronym refers to Amplitude Phase Channel.
The amplitude, phase, and rate for the brightest channel is determined for each IF for each solution interval.
When generation of sniffer output files is not disabled, each {\tt .FITS} file written by {\tt difx2fits} will be accompanied by a corresponding {\tt .apc} file. 
This file contains {\em channel-based} fringe fit solutions typically every 30 seconds for the entire experiment.
These solutions are not of calibration quality but are sufficient for use in evaluating the data quality.

The first line in the file is the observation code, e.g., {\tt MT831} .

Each subsequent line has the same format with the following fields:

\begin{center}
\begin{tabular}{l l l}
\hline
Key & Units/allowed values & Comments \\
\hline
{\it MJD}           & integer $\ge 1$    & MJD day number corresponding to line \\
{\it hour}          & $\ge 0.0$, $< 24.0$ & hour within day \\
{\it source number} & integer $\ge 1$    & source table index \\
{\it source name}   & string             & name of source; no spaces allowed \\
{\it ant1 number}   & integer $\ge 1$    & antenna table index for first antenna \\
{\it ant2 number}   & integer $\ge 1$    & antenna table index for second antenna \\
{\it ant1 name}     & string             & name of antenna 1; no spaces allowed \\
{\it ant2 name}     & string             & name of antenna 2; no spaces allowed \\
{\it n}$_{\mathrm{BBC}}$  & integer $\ge 1$    & number of baseband channels, $n_{\mathrm{BBC}}$ \\
& & The next four columns are repeated $n_{\mathrm{BBC}}$ times \\
\hline
{\it channel}       & $\ge 1$, $\le n_\mathrm{chan}$            & the strongest channel \\
{\it amplitude}     & $\ge 0.0$          & the amplitude of the peak channel \\
{\it phase}         & degrees            & phase of the peak channel \\
{\it rate}          & Hz                 & the channel phase rate \\
\hline
\end{tabular}
\end{center}






% .apd ------------------------------------------------------------------------

\subsection{.apd} \label{sec:apd}

When generation of sniffer output files is not disabled, each {\tt .FITS} file written by {\tt difx2fits} will be accompanied by a corresponding {\tt .apd} file. 
This file contains Amplitude, Phase, Delay (hence the name) and rate results from fringe fit solutions typically every 30 seconds for the entire experiment.
These solutions are not of calibration quality but are sufficient for use in evaluating the data quality.

The first line in the file is the observation code, e.g., {\tt MT831} .

Each subsequent line has the same format with the following fields:

\begin{center}
\begin{tabular}{l l l}
\hline
Key & Units/allowed values & Comments \\
\hline
{\it MJD}           & integer $\ge 1$    & MJD day number corresponding to line \\
{\it hour}          & $\ge 0.0$, $< 24.0$ & hour within day \\
{\it source number} & integer $\ge 1$    & source table index \\
{\it source name}   & string             & name of source; no spaces allowed \\
{\it ant1 number}   & integer $\ge 1$    & antenna table index for first antenna \\
{\it ant2 number}   & integer $\ge 1$    & antenna table index for second antenna \\
{\it ant1 name}     & string             & name of antenna 1; no spaces allowed \\
{\it ant2 name}     & string             & name of antenna 2; no spaces allowed \\
{\it n}$_{\mathrm{BBC}}$  & integer $\ge 1$    & number of baseband channels, $n_{\mathrm{BBC}}$ \\
& & The next four columns are repeated $n_{\mathrm{BBC}}$ times \\
\hline
{\it delay}         & ns                 & the fringe fit delay \\
{\it amplitude}     & $\ge 0.0$          & the amplitude of fringe fit peak \\
{\it phase}         & degrees            & phase of fringe fit peak \\
{\it rate}          & Hz                 & the fringe fit rate \\
\hline
\end{tabular}
\end{center}








% .bandpass -------------------------------------------------------------------

\subsection{.bandpass} \label{sec:filebandpass}

When run with the {\tt --bandpass} option, {\tt difx2fits} will create a {\tt .bandpass} file.
The data in this file is created after applying the results of the fringe fit process and then averaging over all data.
This option should only be used when it is expected that all of the data being processed is on a source strong enough for valid fringe fit solutions.
The contents of the file complex-valued bandpasses determined on each of the baselines.
In its current form (2023/04/24) only cross-correlations are considered, but the option remains open to write real-valued autocorrelations as well.

It is possible that line comments, starting with \#, are present in the file.

The first line of the file is {\tt obscode:} followed by the project code.

A header line starting with {\tt Bandpass} will signify the start of the data for one baseline for one baseband channel.
Such a line has exactly ten space-separated fields:
\begin{enumerate}
  \item The keyword {\tt Bandpass}
  \item Zero-based antenna number for first antenna
  \item Zero-based antenna number for second antenna
  \item Uppercase antenna code for first antenna
  \item Uppercase antenna code for second antenna
  \item Baseband channel number (zero-based)
  \item Number of points in the bandpass, $N_{\rm point}$
  \item Signed Sum of Local Oscillator (SSLO) for the baseband channel frequency (MHz)
  \item Baseband channel bandwidth (MHz)
  \item Polarization (typically {\tt R}, {\tt L}, {\tt H}, or {\tt V})
\end{enumerate}

Following a header line should be the specified number, $N_{\rm point}$, of data lines, each with three space-separated values:
\begin{enumerate}
  \item Sky frequency (MHz)
  \item Real part of bandpass at this frequency
  \item Imaginary part of bandpass at this frequency
\end{enumerate}

Note that the bandpass is represented as the measurements.
The correction factor to flatten the bandpass would be the complex-valued reciprocal of these measurements.




% .binconfig ------------------------------------------------------------------

\subsection{.binconfig} \label{sec:binconfig}

The {\tt .binconfig} file is a file created by the user of DiFX and referenced by the {\tt .input} file to specify pulsar options.
The file uses the standard DiFX input file format and has the following parameters:

\begin{center}
\begin{tabular}{l l l}
\hline
Key & Units/allowed values & Comments \\
\hline
NUM POLYCO FILES      & integer $\ge 1$ & Number of polyco files to read ({\em nPoly}) \\
                      &                 & The next row is duplicated {\em nPoly} times \\
POLYCO FILE {\em p}   & string          & Name of {\em p}$^{th}$ polynomial file \\
NUM PULSAR BINS       & integer $\ge 1$ & Number of pulse bins to create ({\em nBin}) \\
SCRUNCH OUTPUT        & boolean         & Sum weighted bins?  If not, write all bins \\
                      &                 & The next rows are duplicated {\em nBin} times \\
BIN PHASE END {\em b} & float 0.0-1.0   & Pulsar phase where bin ends \\
BIN WEIGHT {\em b}    & float $\ge 0.0$ & Weight to use when scrunching \\
\hline
\end{tabular}
\end{center}

The start of one bin is equal to the end of the previous bin; bins wrap around phase 1.0.
The BIN PHASE END parameters must be listed in ascending phase order.
See Sec.~\ref{sec:pulsars} for example usage of {\tt .binconfig} files.






% .bootstrap -------------------------------------------- INCOMPLETE ----------

\subsection{.bootstrap} \label{sec:bootstrapfile}

The {\tt difxbuild} installer program begins its process by building an environment based on the contents of a {\tt .bootstrap} file.
In the simplest case only three parameters are required ({\tt version}, {\tt headnode}, and {\tt difxbase}), however installations can be customized through the use of several other parameters.
The {\tt .bootstrap} file is a text file containing zero or one {\em key = value} statements on each line.
Comments begin with a \#.

The parameters specified can include:

\begin{itemize}
\item {\tt version}: which version of difx to install.
Currently supported values are DIFX-DEVEL, DIFX-2.1, and DIFX-2.2.
The {\tt DIFX\_VERSION} environment varialbe will reflect this value.
This parameter is required.

\item {\tt headnode}: the computer that will be singled out as the head node.
The {\tt DIFX\_HEAD\_NODE} environment variable will reflect this value.
This parameter is required.

\item {\tt difxbase}: the top level directory for DiFX software.
DiFX-version-independent files will be placed directly beneath this directory.
By default DiFX version specific files will be installed in a subdirectory of this (see information about the root parameter below).
It is okay (and encouraged) to use the same difxbase for all installed versions as this allows common third-party software to be used.
This parameter is required.

\item {\tt root}: the base directory for DiFX version/label specific files for the primary platform.
Secondary platforms will use the same but with a provided extension (see altplatformX below).
If not provided, this parameter will default to {\em difxbase} (or {\em label} if specified).

\item {\tt ipproot}: path to the base of the Intel Performance Primitives library.
This is IPP version dependent and may require a bit of trial and error to get right.
If this is set to  none  then an IPP-free DiFX will be installed.
This requires FFTW to be installed.
Each architecture can have its own ipproot value.
{\tt ipproot} specifies the the default; architecture-dependent overrides are specified with a parameter such as {\tt ipprooti686} or {\tt ipprootx86\_64}.

\item {\tt label}: a label used to identify an installation of DiFX.
By default it is set equal to the specified version.
Setting it to an alternate value allows multiple installations of the same DiFX version to be later identifiable.
The {\tt DIFX\_LABEL} environment variable will reflect this value.

\item {\tt calcserver}: the computer to send RPC model requests to.
If not specified, this will default to the value of the headnode parameter.
The {\tt CALC\_SERVER} environment variable will reflect this value.

\item {\tt cflags}: default c and c++ compiler flags to use.
If not specified, the default of {\tt -O2 -Wall -march=core2} will be used.

\item {\tt pathextra}: extra binary search paths to add the the {\tt PATH} environment variable that is set in the {\tt setup\_difx} script.

\item {\tt ldextra}: extra paths to be added to the {\tt LD\_LIBRARY\_PATH} environment variable that is set in the {\tt setup\_difx} script.

\item {\tt wrapper}: an optional wrapper program that can be used to spawn {\tt mpifxcorr}.
This value gets coded into the {\tt runmpifxcorr.}{\em label} launcher script.
For example, {\tt valgrind} could be used as the wrapper program if memory leek checking is desired.
Use this parameter with caution.

\item {\tt mca}: parameters to add to the {\tt /etc/openmpi-mca-params.conf} file.
If not provided, no such file will be created.
This can be useful to include or exclude certain network interfaces.
You can set this on a per platform basis.
To do this, for example, set {\tt mcai686} and {\tt mcax86\_64} separately.

\item {\tt primaryarch}: Normally bootstrapping needs to be done on a machine running on the primary architecture.
If {\tt primaryarch} is set, the bootstrapping step can be run on any machine.
This should be set to {\tt i686}, {\tt x86\_64}, or whatever {\tt "uname -m"} returns on the primary architecture.

\item {\tt altplatform}{\em X}: Here {\em X} is a number from 1 to 9.
This parameter gives a sub-label to each non-primary platform.
Examples might be {\tt SDK8} and {\tt SDK9} for Mark5 units using two different Conduant library versions.
For each specified alternate platform the following three additional parameters are needed \ldots

\begin{itemize}
\item {\tt altplatform}{\em X}{\tt arch}: The CPU architecture, as determined by {\tt "uname -m"}, that this platform is based upon.

\item {\tt altplatform}{\em X}{\tt host}: A representative computer making use of this platform.
This is used when spawning a parallel build process.

\item {\tt altplatform}{\em X}{\tt test}: A bash conditional expression used to determine if the computer running the {\tt setup\_bash} script belongs to this platform.
An example is: {\tt x`pkg-config --modversion streamstor` < "x9.0"}

\end{itemize}
\end{itemize}







% .cablecal -------------------------------------------------------------------

\subsection{.cablecal} \label{sec:cablecal}

Cable calibration is used to measure the electrical pathlength of the oscillator signals being sent from the control building to the antenna vertex room.
The VLBA was designed to minimize pathlength variations over time, but inevitably some temperature change and stretching due to antenna motion is to be expected.
At the VLBA, the program {\tt db2cc} is used to pull cable cal from the VLBA monitor database and format it in a manner that can be read by {\tt difx2fits}.
There will be one file per antenna which is given filename of {\it exp}.{\it stn}.cablecal, where {\it exp} is the experiment code and {\it stn} is the antenna station code.

The {\tt .cablecal} files are text format files.
Comments within the file start with \#.
Valid data lines have four space-separated columns of data:
\begin{itemize}
  \item station code (in capital letters, usually two characters long)
  \item MJD timestamp
  \item integration period for the measurement (measured in seconds, or zero if not specified)
  \item the cable cal round-trip pathlenght (measured in picoseconds)
\end{itemize}







% cal.vlba --------------------------------------------------------------------

\subsection{cal.vlba} \label{sec:cal}

Monitor data that gets attached to FITS files is extracted by {\tt tsm} into a file called {\em project}{\tt cal.vlba} where {\em project} is the name of the project, i.e., {\tt bg167} or {\tt bc120a}.
A single file contains the monitor data for all VLBA antennas, maybe also including GBT, Effelsberg and Arecibo, for the duration of the project.
The file is left in {\tt /home/vlbiobs} and is compressed with {\tt gzip} after some time to save disk space, resulting in additional file extension {\tt .gz}.
A program called {\tt vlog} (sec \S\ref{sec:vlog}) reads this file and produces files called {\tt flag}, {\tt pcal}, {\tt tsys}, and {\tt weather} in the software correlator project directory.
This file type can be read by AIPS task ANTAB.







% .calc -----------------------------------------------------------------------

\subsection{.calc} \label{sec:calc}

The main use of the {\tt .calc} file is to drive the geometric model calculations but this file also serves as a convenient place to store information that is contained in the {\tt .fx} file but not in the {\tt .input file} and is needed for {\tt .FITS} file creation.
In the DiFX system, one {\tt .calc} file is created by {\tt vex2difx} (\S\ref{sec:vex2difx}) for each {\tt .input} file.
This file is read by {\tt calcif2}) (\S\ref{sec:calcif2}) to produce a tabulated delay model, $u, v, w$ values, and estimates of atmospheric delay contributions.

In brief, the parameters in this file that are relevant for correlation include time, locations and geometries of antennas, pointing of antennas (and hence delay centers) as a function of time and the Earth orientation parameters relevant for the correlator job in question.
Additional parameters that are stuffed into this file include spectral averaging, project name, and information about sources such as calibration code and qualifiers.
In the NRAO application of DiFX, source names are faked in the actual {\tt .input} file in order to allow multiple different configurations for the same source.
A parameter called {\em realname} accompanies each source name in the {\tt .calc} file to correctly populate the source file in {\tt .FITS} file creation.

The syntax of this file is similar to that of the {\tt .input} file.
The file consists entirely of key-value pairs separated by a colon.
The value column is not constrained to start in column 21 as it is for the files used by {\tt mpifxcorr}.
There are five sections in the {\tt .calc} file; these sections are not separated by any explicit mark in the file.

The first section contains values that are fixed for the entire experiment and at all antennas; all data in this section is scalar.
In the following table, all numbers are assumed to be floating point unless further restricted.
The keys and allowed values in this section are summarized below.
Optional keys are identified with a $\star$.
Deprecated keys that will likely be removed in an upcoming version are identified with an $\times$.

\begin{center}
\begin{tabular}{l l l}
\hline
Key & Units/allowed values & Comments \\
\hline
JOB ID             & integer $\ge 1$& taken from {\tt .fx} file \\
\Oa{JOB START TIME}     & MJD + fraction & start time of original {\tt .fx} file \\
\Oa{JOB STOP TIME}      & MJD + fraction & end time of original {\tt .fx} file \\
\Oa{DUTY CYCLE}		& float $\le 1$ & fraction of the job contained within scans \\
OBSCODE            & string         & observation code assigned to project \\
\Oa{SESSION}            & short string   & session suffix to OBSCODE, e.g., {\tt A} or {\tt BE} \\
\Oa{DIFX VERSION}       & string         & version of correlator, e.g. {\tt DIFX-1.5} \\
\Oa{DIFX LABEL}         & string         & name of correlator install, e.g. {\tt DIFX-WALTER} \\
%\Da{SUBJOB ID}          & integer $\ge 0$& subjob id assigned by {\tt job2difx} (\S\ref{sec:job2difx}) \\
%\Da{SUBARRAY ID}        & integer $\ge 0$& subarray id assigned by {\tt job2difx} \\
VEX FILE	   & string         & dir/filename of vex file used to create the job \\
START MJD          & MJD + fraction & start time of this subjob \\
START YEAR         & integer        & calendar year of START MJD \\
START MONTH        & integer        & calendar month of START MJD \\
START DAY          & integer        & day of calendar month of START MJD \\
START HOUR         & integer        & hour of START MJD \\
START MINUTE       & integer        & minute of START MJD \\
START SECOND       & integer        & second of START MJD \\
\Oa{SPECTRAL AVG}       & integer $\ge 1$& number of channels to average in FITS creation \\
\Oa{START CHANNEL}      & integer $\ge 0$& start channel number (before averaging) \\
\Oa{OUTPUT CHANNELS}    & integer $\ge 1$& total number of channels to write to FITS \\
                   & $> 0.0 , < 1.0$& fraction of total channels to write to FITS \\
\Oa{TAPER FUNCTION}     & string         & currently only {\tt UNIFORM} is supported \\
\hline
\end{tabular}
\end{center}

The second section contains antenna(telescope) specific information.
After an initial parameter defining the number of telescopes, there are {\em nTelescope} sections (one for each antenna), each with the following six parameters.
Lowercase {\em t} in the table below is used to indicate the telescope index, an integer ranging from 0 to {\em nTelescope} - 1.
Note that in cases where units are provided under the Key column, these units are actually part of the key.

\begin{center}
\begin{tabular}{l l l}
\hline
Key & Units/allowed values & Comments \\
\hline
NUM TELESCOPES               & integer $\ge 1$& number of telescopes ({\em nTelescope}). \\
&& The rows below are duplicated {\em nTelescope} times. \\
\hline
TELESCOPE {\em t} NAME       & string & upper case antenna name abbreviation \\
TELESCOPE {\em t} MOUNT      & string & the mount type: altz, equa, xyew, or xyns \\
TELESCOPE {\em t} OFFSET (m) & meters & axis offset in meters \\
TELESCOPE {\em t} X (m)      & meters & X geocentric coordinate of antenna at date \\
TELESCOPE {\em t} Y (m)      & meters & Y geocentric coordinate of antenna at date \\
TELESCOPE {\em t} Z (m)      & meters & Z geocentric coordinate of antenna at date \\
\Oa{TELESCOPE} {\em t} SHELF & string & shelf location of module to correlate \\
\hline
\end{tabular}
\end{center}

The third section contains a table of sources.
Sources are indexed from the following section describing the scans.

\begin{center}
\begin{tabular}{l l l}
\hline
Key & Units/allowed values & Comments \\
\hline
NUM SOURCES            & integer $\ge 1$ & number of sources ({\em nSource}) \\
&& The rows below are duplicated {\em nSource} times. \\
\hline
SOURCE {\em s} NAME    & string  & name of source (possibly renamed from {\tt .vex} \\
SOURCE {\em s} RA      & radians & J2000 right ascension \\
SOURCE {\em s} DEC     & radians & J2000 declination \\
SOURCE {\em s} CALCODE & string  & usually upper case letters or blank \\
SOURCE {\em s} QUAL    & integet $\ge 0$ & source qualifier \\
\hline
\end{tabular}
\end{center}


The fourth section contains scan specific information.
Except for one initial line specifying the number of scans, {\em nScan}, this section is composed of nine parameters per scan.
Each parameter is indexed by {\em s} which ranges from 0 to {\em nScan} - 1.

\begin{center}
\begin{tabular}{l l l}
\hline
Key & Units/allowed values & Comments \\
\hline
NUM SCANS              & integer $\ge 1$ & number of scans ({\em nScan}) \\
&& The rows below are duplicated {\em nScan} times. \\
\hline
SCAN {\em s} IDENTIFIER & string & name of the scan (not of the source) \\
SCAN {\em s} START (S) & seconds  & start time of scan, relative to job start time \\
SCAN {\em s} DUR (S) & seconds  & duration of scan \\
SCAN {\em s} OBS MODE NAME & string & reference to {\tt .input} file configuration \\
SCAN {\em s} UVSHIFT INTERVAL (NS) & time to integrate before doing uv shifts (used mainly for multi-phase-center observing) \\
SCAN {\em s} AC AVG INTERVAL (NS) & averaging interval for export of fast-dump spectra (used for VFASTR) \\
SCAN {\em s} POINTING SRC & integer $\ge 1$ & source table index identifying pointing center of scan \\
SCAN {\em s} NUM PHS CTRS & integer $\ge 1$ & number of phase centers ({\em nPC}) \\
&& The rows below are duplicated {\em nPC} times. \\
SCAN {\em s} PHS CTR {\em p} & integer $\ge 1$ & index to the source table \\
\hline
\end{tabular}
\end{center}

The fifth section contains Earth orientation parameters (EOP).
Except for one initial line specifying the number of days of EOPs, {\em nEOP}, this section is composed of five parameters per day of sampled EOP values.
Each parameter is indexed by {\em e} which ranges from 0 to {\em nEOP} - 1.

\begin{center}
\begin{tabular}{l l l}
\hline
Key & Units/allowed values & Comments \\
\hline
NUM EOP                    & integer $\ge 1$ & number of tabulated EOP values ({\em nEOP}) \\
&& The rows below are duplicated {\em nEOP} times. \\
\hline
EOP {\em e} TIME (MJD)     & MJD + fraction  & time of sample; fraction almost always zero \\
EOP {\em e} TAI\_UTC (sec) & integer seconds & leap seconds accrued at time of job start \\
EOP {\em e} UT1\_UTC (sec) & seconds         & UT1 - UTC \\
EOP {\em e} XPOLE (arcsec) & arc seconds     & X coordinate of polar offset \\
EOP {\em e} YPOLE (arcsec) & arc seconds     & Y coordinate of polar offset \\
\hline
\end{tabular}
\end{center}

The next (completely optional) section has a table for positions and velocites of spacecraft.
Each spacecraft is indexed by {\em s} and each row thereof by {\em r}.

\begin{center}
\begin{tabular}{l l l}
\hline
Key & Units/allowed values & Comments \\
\hline
\Oa{NUM SPACECRAFT}             & integer $\ge 0$ & number of spacecraft ({\em nSpacecraft}) \\
&& Everything below is duplicated {\em nSpacecraft} times. \\
\hline
SPACECRAFT {\em s} NAME    & string          & name of spacecraft \\
SPACECRAFT {\em s} ROWS    & integer $\ge 1$ & number of data rows, {\em nRow}$_s$ for spacecraft {\em s} \\
&& The row below is repeated {\em nRow}$_s$ times. \\
\hline
SPACECRAFT {\em s} ROW {\em r} & 7 numbers & tabulated data; see below \\
\hline
\end{tabular}
\end{center}

Each data vector of data consists of seven double precision values: time (mjd), $x$, $y$, and $z$ (meters), and $\dot{x}$, $\dot{y}$, and $\dot{z}$ (meters per second).
These values should be separated by spaces.

The final section identifies the files to be produced.

\begin{center}
\begin{tabular}{l l l}
\hline
Key & Units/allowed values & Comments \\
\hline
IM FILENAME        & string         & dir/filename of {\tt .im} file to create \\
FLAG FILENAME      & string         & dir/filename of {\tt .flag} file to create \\
\hline
\end{tabular}
\end{center}




% .difx/ ----------------------------------------------------------------------

\subsection{.difx/} \label{sec:difx}

The SWIN format visibilities produced by {\tt mpifxcorr} are written to a directory with extension {\tt .difx}.
Three kinds of files can be placed in this directory as described below.

Note that the formats and naming conventions of these files is not guaranteed to stay unchanged from version to version of DiFX, and hence it is not recommended to rely on these files for archival purposes.

\subsubsection{Visibility files} \label{sec:difxvisibilities}

The bulk of the output from {\tt mpifxcorr} is usually in the form of a binary visibility file.
Usually there will be a single visibility file in this directory, but there are three ways in which multiple files may be produced: 1.\ a restart of the correlation, 2.\ if there are multiple phase centers, and 3.\ if there are multiple pulsar bins.

The visibility files are systematically named in the form: {\tt DIFX\_}{\em day}{\tt \_}{\em sec}{\tt .s}{\em src}{\tt .b}{\em bin}, where {\em day} is the 5 digit integer MJD of the start of visibilities, {\em sec} is a zero-padded 6 digit number of seconds since the MJD midnight, {\em src} is a 4 digit zero-padded integer specifying the phase center number (starting at 0), and {\em bin} is a 4 digit zero-padded integer specifying the pulsar bin number (starting at 0).

These files contain visibility data records.
Each record contains the visibility spectrum for one polarization of one baseband channel of one baseline for one integration time.  
Each starts with a binary header and is followed by binary data.

The format of the header is shown in the table below.

\begin{center}
\begin{tabular}{llll}
\hline
Key & data type & units & comments \\
\hline
baseline number		& int	&	& $= (a_1+1)*256 + (a_2+1)$ for $a_1, a_2 \ge 1$ \\
day number		& int	& MJD	& date of visibility centroid \\
seconds			& double& sec	& vis.\ centroid seconds since beginning of MJD \\
config index		& int	&$\ge 0$& index to {\tt .input} file configuration table \\
source index		& int	&$\ge 0$& index to {\tt .calc} file scan number \\
freq index		& int	&$\ge 0$& index to {\tt .input} frequency table \\
antenna 1 polarization	& char  & {\tt R}, {\tt L}, {\tt X}, {\tt Y} & \\
antenna 2 polarization	& char  & {\tt R}, {\tt L}, {\tt X}, {\tt Y} & \\
pulsar bin number	& int	&$\ge 0$& \\
visibility weight	& double& $\ge 0.0$ & data weight for spectrum; typically $\sim 1$ \\
$u$			& double& meter & $u$ component of baseline vector \\
$v$			& double& meter & $v$ component of baseline vector \\
$w$			& double& meter & $w$ component of baseline vector \\
\hline
\hline
\end{tabular}
\end{center}


Note that for both the header and the data, the endianness is native to the machine running {\tt mpifxcorr}, and there are currently no provisions for processing such files on a machine of different endianness.

Following the end-of-line mark for the last header row begins binary data in the form of (real, imaginary) pairs of 32-bit floating point numbers.
The {\tt .input} file parameter {\tt NUM CHANNELS} indicates the number of
complex values to expect.  
In the case of upper sideband data, the first reported channel is the ``zero frequency'' channel, that is its sky frequency is equal to the value in the frequency table for this spectrum.  
The Nyquist channel is not retained.
For lower sideband data, the last channel is the ``zero frequency'' channel.
That is, in all cases, the spectrum is in order of increasing frequency and the Nyquist channel is excised.


\subsubsection{Pulse cal data files} \label{sec:difxpulsecal}

Pulse calibration data can be extracted by {\tt mpifxcorr}.
Extraction is configured on a per-antenna basis.
Data for each antenna is written to a separate file; if correlation is restarted, an additional pulse cal data file will be written.

The pulse cal data files are systematically named in the form: {\tt PCAL\_}{\em day}{\tt \_}{\em sec}{\tt \_}{\em ant}, where {\em day} is the 5 digit integer MJD of the start of visibilities, {\em sec} is a zero-padded 6 digit number of seconds since the MJD midnight, and {\em ant} is the 1 or 2 letter antenna name in capital letters.
There is potential for these text files to have very long lines (more than 10,000 bytes) when many pulse cal tones are extracted.

For DiFX versions 2.3 and earlier the data format was exactly the same as documented in \S\ref{sec:pcal}.
This old version will be considered ``version 0''.

The data format being used now is similar in spirit but more convenient for {\tt mpfixcorr} to produce and for {\tt difx2fits} and {\tt difx2mark4} to digest leading to broader support (in theory complete) of the various polarization, frequency, and sideband combinations allowed by DiFX.
The data format is as follows:

Comment lines begin with an octothorpe (\#).
The first few lines of comments may contain machine-readable information in the following format:
\begin{verbatim}
# DiFX-derived pulse cal data
# File version = 1
# Start MJD = 
# Start seconds = 
# Telescope name = 
\end{verbatim}

Data lines always contain 6 fixed-size fields:
\begin{enumerate}
\item {\em antId} : Station name abbreviation, e.g., {\tt LA}
\item {\em day} : Time centroid of measurement (MJD, including fractional portion)
\item {\em dur} : Duration of measurement (days)
\item {\em datastreamId} : The datastream index of for this data.
\item {\em nRecBand} : Number of recorded baseband channels
\item {\em nTone} : (Maximum) number of pulse cal tones detected per band per polarization
\end{enumerate}
Following these fields is a variable-length arrays of numbers.
This array contains the pulse cal data and consists of {\em nRecBand*nTone} groups of four numbers.
The groups are arranged in ascending record band index (slow index) and ascending tone number (fast index) where the tone number increases away from the reference frequency; not sure what happens with dual-sideband complex!
The first member of this group is the tone frequency (MHz), or -1 to indicate there was not a measurment.
The second member of this group is the polarization, one of {\tt R}, {\tt L}, {\tt X} or {\tt Y}.
The third and fourth are respectively the real and imaginary parts of the tone measured at the given sky frequency.


\subsubsection{Switched power files} \label{sec:difxswitchedpower}

{\tt mpifxcorr} can be used to extract switched power from individual antennas.
Extraction is configured on a per-datastream (usually the same as per-antenna) basis.
Data for each data stream is written to a separate file; if correlation is restarted, an additional set of switched power files will be started.

The switched power files are systematically named in the form: {\tt SWITCHEDPOWER\_}{\em day}{\tt \_}{\em sec}{\tt \_}{\em ds}, where {\em day} is the 5 digit integer MJD of the start of visibilities, {\em sec} is a zero-padded 6 digit number of seconds since the MJD midnight, and {\em ds} is the datastream index as set in the {\tt .input} file.
The test lines in these files can be long (more than 1000 bytes).

The format of these files is as follows.
Each line of the file represents all measurements made on one datastream at over one integration period.
The lines contain the following columns:
\begin{enumerate}
\item {\em mjdstart} : The start of the integration period in mjd+fraction
\item {\em mjdstop} : The end of the integration period in mjd+fraction
\end{enumerate}
These are then followed by 4 numbers for each recorded channel:
\begin{enumerate}
\item $P_\mathrm{on}$ : power in the ``on'' state
\item $\sigma_{P_\mathrm{on}}$ : uncertainty of the power in the ``on'' state
\item $P_\mathrm{off}$ : power in the ``off'' state
\item $\sigma_{P_\mathrm{off}}$ : uncertainty of the power in the ``off'' state
\end{enumerate}
The magnitudes of the numbers are meaningless but their ratios have meaning.
Text after a comment character (\#) are ignored.





% .difxlog --------------------------------------------------------------------

\subsection{.difxlog} \label{sec:dotdifxlog}

The {\tt difxlog} program (\S\ref{sec:difxlogprogram}) captures {\tt DifxAlertMessage} and {\tt DifxStatusMessage} message types that are sent from an ongoing software correlation process and writes the information contained within to a human readable text file.
One line of text is produced for each received message.
The first five columns contain the date and time in {\em ddd MMM dd hh:mm:ss yyyy} format (e.g., {\tt Wed Apr 22 12:48:41 2009}).
The sixth column contains a word describing the contents of the remainder of the line:  Options are:
\begin{itemize}
\item[] {\tt STATUS} : The status of the process is described
\item[] {\tt WEIGHTS} : The playback weights for each antenna are listed
\item[] {\em other} : This word represents an alert severity level (one of {\tt FATAL}, {\tt SEVERE}, {\tt ERROR}, {\tt WARNING}, {\tt INFO}, {\tt VERBOSE} and {\tt DEBUG}) and is followed by the alert message itself.
\end{itemize}





% .difxspeed ------------------------------------------------------------------

\subsection{.speed} \label{sec:speed}

The program {\tt difxspeed} (\S\ref{sec:difxspeed}) runs a set of performance benchmarks described by a {\tt .speed} file as documented here.
This file is a text file containing various parameters.
There are 5 required parameters:
\begin{center}
\begin{tabular}{ll}
\hline
Parameter & Comments \\
\hline
{\tt datastreams} & comma separated list of nodes on which to run datastream processes  \\
{\tt cores} & comma separated list of nodes on which to run core processes \\
{\tt antennas} & list of 1 or 2 letter antenna names to process \\
{\tt nThread} & one or more values listing number of threads to use (see below) \\
{\tt vex} & vex file to use as descriptor of observation \\
\hline
\end{tabular}
\end{center}

The value of {\tt nThread} applies to all core processes; if multiple comma-separated values are specified, these will result in additional runs of benchmarking.

Other parameters that can be specified, either as single values or as arrays to be used in full combination with all other value arrays, include:

\begin{center}
\begin{tabular}{ll}
\hline
Parameter & Comments \\
\hline
{\tt nAnt} & number of antennas (in order of listed antennas, starting with first listed) \\
{\tt nCore} & number of core processes to start, using in order {\tt cores} \\
{\tt tInt} & integration time (seconds) \\
{\tt specRes} & spectral resolution (MHz) \\
{\tt fftSpecRes} & resolution of transform \\
{\tt xmacLength} & cross-multiply stride size \\
{\tt strideLength} & fringe rotation stride size \\
{\tt numBufferedFFTs} & number of FFTs to process in one go \\
{\tt toneSelection} & pulse cal tone selection \\
\hline
\end{tabular}
\end{center}

\noindent
Notes: 
\begin{enumerate}
\item {\tt datastreams} and {\tt cores} lists can repeat hostnames.
\item Some combinations of parameters is illegal; at the moment it is up to the user to ensure all combinations of values are allowed.
\item Additional parameters can be easily added to the program on demand.
\end{enumerate}


% .difxspeed.out --------------------------------------------------------------

\subsection{.speed.out} \label{sec:speed.out}

The output of {\tt difxspeed} (\S\ref{sec:difxspeed}) is a file usually ending in {\tt .speed.out}.
The first many lines are comments describing to a human reader the fixed parameters of the benchmark trials and a table describing the meanings of the columns of the uncommented data lines that follow.
In summary, the columns in the lines that follow are:

\begin{center}
\begin{tabular}{ll}
\hline
Column(s) & value(s) \\
\hline
1 to $N$ & values of variable parameters as described by comments above \\
$N+1$ & The average execution time of all trials run with the combination of parameters \\
$N+2$ & The RMS scatter in execution time \\
$N+3$ to $N+2+R$ & List of execution times from all trials \\
\hline
\end{tabular}
\end{center}

In the above table, $N$ is the number of parameters taking on multiple values and $R$ is the number of times {\tt difxspeed} was run.



% $DIFX_MACHINES --------------------------------------------------------------

\subsection{\$DIFX\_MACHINES} \label{sec:difxmachines}

This section describes the format of a file used through DiFX-2.2.
For more recent versions please see documentation on the DiFX wiki \url{http://www.atnf.csiro.au/vlbi/dokuwiki/doku.php/difx/start/} .

Environment variable {\tt DIFX\_MACHINES} should point to a file containing a list of machines that are to be considered elements of the software correlator.
Program {\tt genmachines} (\S\ref{sec:genmachines}) uses this file and information within a {\tt .input} file to populate the {\tt .machines} file needed by {\tt mpifxcorr}.
Because usually only one node in a cluster has direct access to a particular Mark5 module (or data from that module), the ordering of computer names in the {\tt .machines} file is important.
Rows in the {\tt \$DIFX\_MACHINES} file contain up to three items, the last one being optional.
The first column is the name of the machine.
The second column is the number of processes to schedule on that machine (typically the number of CPU cores).
The third column is a 1 if the machine is a Mark5 unit and 0 otherwise.
If this column is omitted, the machine will be assumed to be a Mark5 unit if the first 5 characters of the computer name are {\tt mark5}, and will be assumed not to be otherwise.
Comments in this file begin with an octothorpe (\#).
Lines with fewer than two columns (after excision of comments) are ignored.







% .dir ------------------------------------------------------------------------

\subsection{.dir} \label{sec:dir}

Reading directory information off Mark5 modules can take a bit of time (measured in minutes usually).
Since the same modules are often accessed multiple times, the directories are cached in {\tt \$MARK5\_DIR\_PATH/} .
In this directory, there will be one file per module that has been used, named {\em VSN}{\tt .dir}, where {\em VSN} is the volume serial number of the module, e.g., NRAO$-$023.
The format of these files is as follows:
The first line contains three fields: {\em VSN}, the number of scans on the module, {\em nScan}, and either {\tt A} or {\tt B} indicating the last bank the module was installed in.
At the end of this line the characters {\em RT} can be added (by hand) which will cause the modules to be accessed using {\em Real-Time} mode which is tolerant of missing or bad disks within the module.
Then there are {\em nScan} rows containing information about each scan, each with 11 columns.
Values are floating point unless otherwise noted.
\begin{center}
\begin{tabular}{l l l}
\hline
Key & Units/allowed values & Comments \\
\hline
Start byte & 64-bit integer bytes & offset of the scan on the Mark5 module \\
Length & 64-bit integer bytes & length of the scan \\
Start day & integer MJD & the modified Julian day of the scan start \\
Start time & integer seconds & the scan start time \\
Frame num & integer & frame number since last second tick \\
Frames per sec & integer & number of frames per second \\
Scan Duration & seconds & the duration of the scan \\
Frame size & integer bytes & the length of one data frame, including headers \\
Frame offset & integer bytes & the offset to the start of the first entire frame \\
Tracks & integer & the number of data tracks \\
Format & integer & 0 for VLBA format, 1 for Mark4 format, 2 for Mark5B \\
Name & string & scan name, usually including the project code and station \\
Extra info & string(s) & see below \\
\hline
\end{tabular}
\end{center}

After the name of the scan additional free-form text can appear.
These extra parameters can be machine parsable.
The only use of this as of this writing is to indicate the thread ids for VDIF data.
This will always be formatted as {\tt Threads=} followed by a comma separated list of thread ids without any spaces.  For example: {\tt Threads=0,640,256,896} .







% .filelist -------------------------------------------------------------------

\subsection{.filelist} \label{sec:filelist}

When using the {\tt filelist} parameter in an {\tt ANTENNA} section of a {\tt .v2d} (\S\ref{sec:v2d}), the list of data files to correlate are stored in a text file.
This is a text file containing data lines and optionally comments.
Any text after the comment character (\#) is ignored.
A data line consists of a filename (must have complete path as can be used to find the file on the datastream node for this antenna) and optionally a start time and stop time.
Start and stop times can be expressed in any of the formats supported by {\tt vex2difx}.





% .FITS -----------------------------------------------------------------------

\subsection{.FITS} \label{sec:FITS}

The {\tt .FITS} files discussed here are produced by {\tt difx2fits}.
They aim to conform to the same table structures as the FITS-IDI files produced by the VLBA correlator.
The format is described in AIPS Memo 102, ``The FITS Interferometry Data Interchange Format'', however, this memo is a bit out of date and the data structures described are not in exact agreement with those made by the VLBA correlator; in all cases the format of data produced by the VLBA hardware correlator is favored where the two disagree.
The tables in these FITS files have a nearly 1 to 1 relationship with the tables that are seen within AIPS, though their two letter abbreviations differ.
The following tables are produced by {\tt difx2fits}:
\begin{center}
\begin{tabular}{l l}
\hline
Table & Description \\
\hline
AG & The array geometry table \\
SU & The source table \\
AN & The antenna table \\
FR & The frequency table \\
ML & The model table \\
CT & The correlator (eop) table \\
MC & The model components table \\
SO & The spacecraft orbit table \\
UV & The visibility data table \\
FG & The flag table \\
TS & The system temperature table \\
PH & The phase calibration table (pulse cals and state counts) \\
WR & The weather table \\
GN & The gain curve table \\
GM & The pulsar gate model table \\
\hline
\end{tabular}
\end{center}
Not all of these tables will always be written.







% .fitslist -------------------------------------------------------------------

\subsection{.fitslist} \label{sec:fitslist}

A {\tt .fitslist} file is written by {\tt makefits} and contains the entire list of {\tt .FITS} files for the correlator pass.
Due to the different constraints of the correlation process and the FITS-IDI format, the number of resultant FITS files may be greater or less than the number of jobs.
This file type is used by {\tt difxarch} to ensure that all of the correlated output ends up in the archive.
The file is composed of two parts: a header line and one line for each {\tt .FITS} file.
The header line consists of a series of {\em key=value} pairs.  
Each {\em key} and {\em value} must have no whitespace and no whitespace should separate these words from their connecting {\tt =} sign.
While any number of key-value pairs may be specified, the following ones (which are case sensitive) are expected to be present:
\begin{enumerate}
\item {\tt exper} : the name of the experiment, including the segment code
\item {\tt pass} : the name of the correlator pass
\item {\tt jobs} : the name of the {\tt .joblist} file used by {\tt makefits}
\item {\tt mjd} : the modified Julian day when {\tt makefits} created this file
\item {\tt DiFX} : the version name for the DiFX deployment (the value of {\tt \$DIFX\_VERSION} when {\tt vex2difx} was run)
\item {\tt difx2fits} : the version of {\tt difx2fits} that was run
\end{enumerate}
Each additional line contains information for one {\tt .FITS} file of the correlation pass.
These lines contain three fields:
\begin{enumerate}
\item {\em archiveName} : the name of the file that will get injected into the archive (see \S\ref{sec:archive})
\item {\em fileSize} : the size of the file in MB
\item {\em origName} : the name of the file as produced by {\tt difx2fits} (via {\tt makefits})
\end{enumerate}







% .flag -----------------------------------------------------------------------

\subsection{.flag} \label{sec:dotflag}

The program {\tt vex2difx} may write a {\tt .flag} file for each {\tt .input} file it creates.
This file is referenced from the {\tt .calc} file.
This flag file is used by {\tt difx2fits} to exclude nonsense baselines that might have been correlated.
Data from nonsense baselines can occur in DiFX output when multiple subarrays are coming and going.
The flag file instructs {\tt difx2fits} to drop these data during conversion to FITS-IDI.
The format of this text file is as follows.
The first line contains an integer, $n$, which is the number of flag lines that follow.
The next $n$ lines each have three numbers: $MJD_1$, $MJD_2$ and $ant$.
The first two floating point numbers determine the time range of the flag in Modified Julian Days.
The last integer number is the antenna number to flag, a zero-based index corresponding to the {\tt TELESCOPE} table of the corresponding {\tt .input} file.


% .antId.flag -----------------------------------------------------------------------

\subsection{.{\textless}antId{\textgreater}.flag} \label{sec:dotantflag}

A series of files called {\tt .{\textless}antId{\textgreater}.flag} are created when program {\tt vlog} operates on the {\tt cal.vlba} file.
These files contain lists of antenna-based flags generated by the on-line system that should be propagated into the FITS FL table.
These flag files contains two kinds of lines.
Comment lines begin with an octothorpe (\#) and contain no vital information.
Flag lines always consist of exactly 5 fields:
\begin{enumerate}
\item {\em antId} : Station name abbreviation, e.g., {\tt LA}; also part of the file name.
\item {\em start} : Beginning of flagged period (day of year, including fractional portion; or Modified Julian Days)
\item {\em end} : End of flagged period (day of year, including fractional portion; or Modified Julian Days)
\item {\em recChan} : Record channel affected; -1 for all record channels, otherwise a zero-based index
\item {\em reason} : Reason for flag, enclosed in single quotes, truncated to 24 characters
\end{enumerate}
The flag rows are sorted by start time. The flags are propagated into the FL table. Visibility data are not altered.
Special VLBA reason codes recognized by {\tt difx2fits} are as follows: $'recorder'$, the flag entry will be ignored and is not propagated into FITS, and $'observing system idle'$, the value of $MJD_2$ is ignored and replaced by the ending MJD of the observation.


% .channelflags -----------------------------------------------------------------------

\subsection{.channelflags} \label{sec:dotchannelflags}

User scripts or the program {\tt vex2difx} may write a {\tt .channelflags} file for each {\tt .input} file.
{\tt Difx2fits} will propagate the user flags of bad spectral channels into the FITS flag table.
This file contains two kinds of lines.
Comment lines begin with an octothorpe (\#) and contain no vital information.
Flag lines always consist of 7 fields:
\begin{enumerate}
\item {\em antId} : Station name abbreviation, e.g., {\tt LA}
\item {\em start} : Beginning of flagged period (Day of Year or Modified Julian Day; including fractional portion)
\item {\em end} : End of flagged period (Day of Year or Modified Julian Day; including fractional portion)
\item {\em freqIndex} : The DiFX frequency to flag, a zero-based index corresponding to a frequency in the {\tt FREQ TABLE} of the corresponding {\tt .input} file.
\item {\em startCh} : The first spectral channel to flag, a zero-based index.
\item {\em endCh} : The last spectral channel to flag, a zero-based index.
\item {\em reason} : Reason for flag, enclosed in single quotes, truncated to 24 characters.
\end{enumerate}
{\tt Difx2fits} translates the DiFX frequency $freqIndex$ to the corresponding FITS IF, and flags the specified channels in {\em all polarizations} of that IF.
These flags are propagated into the FL table. Visibility data are not altered.


% flag ------------------------------------------------------------------------

\subsection{flag} \label{sec:flag}

A file called {\tt flag} is created when program {\tt vlog} operates on the {\tt cal.vlba} file.
The file contains a list of antenna-based flags generated by the on-line system that should be propagated into the FITS FL table.
The file is an experiment-wide flag file and effectively a concatenation of {\tt .{\textless}antId{\textgreater}.flag} files.
This file contains two kinds of lines.
Comment lines begin with an octothorpe (\#) and contain no vital information.
Flag lines always consist of exactly 5 fields:
\begin{enumerate}
\item {\em antId} : Station name abbreviation, e.g., {\tt LA}
\item {\em start} : Beginning of flagged period (Day of Year or Modified Julian Day; including fractional portion)
\item {\em end} : End of flagged period (Day of Year or Modified Julian Day; including fractional portion)
\item {\em recChan} : Record channel affected; -1 for all record channels, otherwise a zero-based index
\item {\em reason} : Reason for flag, enclosed in single quotes, truncated to 24 characters
\end{enumerate}
The flag rows are sorted first by antenna, and then start time.
Special VLBA reason codes recognized by {\tt difx2fits} are as follows: {\em 'recorder'}, the flag entry will be ignored and is not propagated into FITS, and {\em 'observing system idle'}, the value of $MJD_2$ is ignored and replaced by the ending MJD of the observation.


% .fx ----------------------------------------------------- OBSOLETE ----------
%
%\subsection{.fx} \label{sec:fx}
%
%The program {\tt cjobgen} is used to create job scripts (with filename extension {\tt .fx}) for use with the VLBA hardware correlator.
%Although the functionality of {\tt cjobgen} will eventually be replaced, it is convenient to use the {\tt .fx} files it creates in the interim as they contain all the information required to calculate a delay model and drive the software correlator.
%These files are converted by {\tt job2difx} (\S\ref{sec:job2difx}) to produce {\tt .input} files to control correlation and {\tt .calc} files to drive the delay model generator.
%Note that there will not in general be a 1 to 1 relationship between {\tt .fx} files and {\tt .input} or {\tt .calc} files due to combination of multiple passes and splitting due to frequency changes.
%The program {\tt getjobs} (\S\ref{sec:getjobs}) can be used to copy these files from their original location in {\tt /home/vlbiobs}.







% .im -------------------------------------------------------------------------

\subsection{.im} \label{sec:im}

The {\tt .im} file contains polynomial models used by {\tt difx2fits} in the creation of {\tt FITS} files.
After a header that is similar to that of a {\tt .rate} file, the contents are organized hierarchically with scan number, sub-scan interval, and antenna number being successively faster-incrementing values.
The keys and allowed values in this section are summarized below:
Note that the values of the delay polynomials in this file have the opposite sign as compared to those generated by CALC and those stored in {\tt .FITS} files.
Keys preceded by $\star$ are optional.
Note that all polynomials are expanded about their {\tt MJD, SEC} start time and use seconds as the unit of time.

\begin{center}
\begin{tabular}{l l l}
\hline
Key & Units/allowed values & Comments \\
\hline
\Oa{CALC SERVER}        & string         & name of the calc server computer used \\
\Oa{CALC PROGRAM}       & integer        & RPC program ID of the calc server used \\
\Oa{CALC VERSION}       & integer        & RPC version ID of the calc server used \\
START YEAR         & integer        & calendar year of START MJD \\
START MONTH        & integer        & calendar month of START MJD \\
START DAY          & integer        & day of calendar month of START MJD \\
START HOUR         & integer        & hour of START MJD \\
START MINUTE       & integer        & minute of START MJD \\
START SECOND       & integer        & second of START MJD \\
POLYNOMIAL ORDER   & 2, 3, 4 or 5   & polynomial order of interferometer model {\em order} \\
INTERVAL (SECS)    & integer        & interval between new polynomial models \\
ABERRATION CORR    & $\left\{\begin{array}{l}\mbox{\tt UNCORRECTED}\\\mbox{\tt APPROXIMATE}\\\mbox{\tt EXACT}\end{array}\right.$ & level of $u, v, w$ aberration correction \\
NUM TELESCOPES               & integer $\ge 1$& number of telescopes ({\em nTelescope}) \\
&& The row below is duplicated {\em nTelescope} times. \\
\hline
TELESCOPE {\em t} NAME       & string & upper case antenna name abbreviation \\
\hline
NUM SCANS          & integer $\ge 1$ & number of scans ({\em nScan}). \\
&& Everything below is duplicated {\em nScan} times. \\
\hline
SCAN {\em s} POINTING SRC & string & name of source used as pointing center \\
SCAN {\em s} NUM PHS CTRS  $\ge 1$ & number of phase centers this scan ({\em nPC}$_\mathit{s}$) \\
&& Everything below is duplicated {\em nPC}$_\mathit{s}$ times. \\
SCAN {\em s} PHS CTR {\em p} SRC  & string & name of source defining this phase center \\
SCAN {\em s} NUM POLY  & $\ge 1$ & number of polynomials covering scan ({\em nPoly}$_\mathit{s,p}$) \\
&& Everything below is duplicated {\em nPoly}$_\mathit{s,p}$ times. \\
\hline
SCAN {\em s} POLY {\em p} MJD  & integer $\ge 0$ & the start MJD of this polynomial \\
SCAN {\em s} POLY {\em p} SEC  & integer $\ge 0$ & the start sec of this polynomial \\
&& Everything below is duplicated {\em nTelescope} times. \\
ANT {\em a} DELAY (us) & {\em order}+1 numbers & terms of delay polynomial \\
ANT {\em a} DRY (us)   & {\em order}+1 numbers & terms of dry atmosphere \\
ANT {\em a} WET (us)   & {\em order}+1 numbers & terms of wet atmosphere \\
\Oa{ANT} {\em a} AZ  & {\em order}+1 numbers & azimuth polynomial (deg) \\ 
\Oa{ANT} {\em a} EL GEOM & {\em order}+1 numbers & geometric (encoder) elevation (deg) \\ 
\Oa{ANT} {\em a} EL CORR & {\em order}+1 numbers & refraction corrected elevation (deg) \\ 
\Oa{ANT} {\em a} PAR ANGLE & {\em order}+1 numbers & parallactic angle (deg) \\ 
ANT {\em a} U (m)      & {\em order}+1 numbers & terms of baseline $u$ \\ 
ANT {\em a} V (m)      & {\em order}+1 numbers & terms of baseline $v$ \\ 
ANT {\em a} W (m)      & {\em order}+1 numbers & terms of baseline $w$ \\ 
\hline
\end{tabular}
\end{center}






% .input ----------------------------------------------------------------------

\subsection{.input} \label{sec:input}

This section describes the {\tt .input} file format used by {\tt mpifxcorr} to drive correlation.
Because NRAO-DiFX 1.0 uses a non-standard branch of {\tt mpifxcorr} some of the data fields will differ from those used in the official version, either in parameter name or in the available range of values.
Currently the parameters must be in the order listed here.
To get the most out of this section it is advisable to look at an actual file while reading.
An example file is stashed at \url{http://www.aoc.nrao.edu/~wbrisken/NRAO-DiFX-1.1/} .
In the tables below, numbers are assumed to floating point unless otherwise stated.

Note that the input file format has undergone a few minor changes since NRAO-DiFX version 1.0.

\subsubsection{Common settings table}

Below are the keywords and allowed values for entries in the common settings table.
This table begins with header 
\begin{itemize}
\item[] {\tt \verb+# COMMON SETTINGS ##!+} 
\end{itemize}
This is always the first table in a {\tt .input} file.

\begin{center}
\begin{tabular}{l l l}
\hline
Key & Units/allowed values & Comments \\
\hline
CALC FILENAME      & string & name and full path to {\tt .calc} file \\
CORE CONF FILENAME & string & name and full path to {\tt .threads} file \\
EXECUTE TIME (SEC) & integer seconds & observe time covered by this {\tt .input} file \\
START MJD          & integer MJD & start date \\
START SECONDS      & integer seconds & start time \\
ACTIVE DATASTREAMS & integer $\ge 2$ & number of antennas ({\em nAntenna}) \\
ACTIVE BASELINES   & integer $\ge 1$ & number of baselines to correlate ({\em nBaseline}) \\
VIS BUFFER LENGTH  & integer $\ge 1$ & the number of concurrent integrations to allow \\
OUTPUT FORMAT      & boolean & always {\tt SWIN} here \\
OUTPUT FILENAME    & string & name of output {\tt .difx} directory \\
\hline
\end{tabular}
\end{center}

Typically, $\mathit{nBaseline} = \mathit{nAntenna} \cdot (\mathit{nAntenna}-1)/2$.  Autocorrelations are not included in this count.

\subsubsection{Configurations table}

Below are the keywords and allowed values for entries in the configurations table.
This table begins with header 
\begin{itemize}
\item[] {\tt \verb+# CONFIGURATIONS ###!+}
\end{itemize}
Two indexes are used for repeated keys.  
The index over datastream (antenna) is {\em d}, running from 0 to {\em nAntenna} - 1 and the index over baseline is {\em b}, running from 0 to {\em nBaseline} - 1.

\begin{center}
\begin{tabular}{l l l}
\hline
Key & Units/allowed values & Comments \\
\hline
NUM CONFIGURATIONS & integer $\ge 1$ & number of modes in file ({\em nConfig}) \\
\hline
CONFIG NAME        & string & name of configuration \\
INT TIME (SEC)     & seconds & integration time \\
SUBINT NANOSECONDS & nanosec & amount of time to process as one subintegration \\
GUARD NANOSECONDS  & nanosec $\ge 0$ & amount of extra data to send for overlap \\
FRINGE ROTN ORDER  & int & 0 is post-FFT, 1 is delay/rate, \ldots \\
ARRAY STRIDE LENGTH & int & used for optimized fringe rotation calculations \\
XMAC STRIDE LENGTH  & int & number of channels to cross multiply in one batch (must evenly divide into number of channels) \\
NUM BUFFERED FFTS   & int & number of FFTs to cross-multiply in one batch \\
WRITE AUTOCORRS    & boolean & enable auto-correlations; {\em TRUE} here \\
PULSAR BINNING     & boolean & enable pulsar mode \\
PULSAR CONFIG FILE & string & ({\em only if BINNING is True}) see \S~\ref{sec:binconfig} \\
PHASED ARRAY       & boolean & set to FALSE (placeholder for now) \\
DATASTREAM {\em d} INDEX & integer $\ge 0$ & DATASTREAM table index, starting at 0 \\
BASELINE {\em b} INDEX   & integer $\ge 0$ & BASELINE table index, starting at 0 \\
\hline
\end{tabular}
\end{center}

\subsubsection{Rule table} \label{table:rule}

The rule tables describes which configuration will be applied at any given time.
Usually this filters on scan attributes such as source, but can also be done in a time-based manner (start and stop times).
An time for which no configuration matches will not be correlated.
If more than one rule matches a given time, they must all refer to the same configuration.

This table begins with header
\begin{itemize}
\item[] {\tt \verb+# RULES ############!+}
\end{itemize}
The table below uses {\em r} to represent the rule index, which ranges from 0 to {\em nRule} - 1.
\begin{center}
\begin{tabular}{l l l}
\hline
Key & Units/allowed values & Comments \\
\hline
RULE {\em r} CONFIG NAME & string & name to associate with this rule \\
\Oa{SOURCE} & string & source to match \\
\Oa{SCAN ID} & string & scan name to match \\
\Oa{CALCODE} & string & cal code to match \\
\Oa{QUAL} & string & source qualifier to match \\
\Oa{MJD START} & string & earliest time to match \\
\Oa{MJD STOP} & string & latest time to match \\
\hline
\hline
\end{tabular}
\end{center}

\subsubsection{Frequency table} \label{table:freq}

Below are the keywords and allowed values for entries in the frequency table which defines all possible sub-bands used by the configurations in this file.
Each sub-band of each configuration is mapped to one of these through a value in the datastream table (\S\ref{table:datastream}).
Each entry in this table has three parameters which are replicated for each frequency table entry.
This table begins with header 
\begin{itemize}
\item[] {\tt \verb+# FREQ TABLE #######!+}
\end{itemize}
The table below uses {\em f} to represent the frequency index, which ranges from 0 to {\em nFreq} - 1 and {\em t} to represent pulse cal tone index, which ranges from 0 to {\em nTone}$_f$.

\begin{center}
\begin{tabular}{l l l}
\hline
Key & Units/allowed values & Comments \\
\hline
FREQ ENTRIES & integer $\ge 1$ & number of frequency setups ({\em nFreq}) \\
\hline
FREQ (MHZ) {\em f} & MHz & sky frequency at band edge \\
BW (MHZ) {\em f} & MHz & bandwidth of sub-band \\
SIDEBAND {\em f} & {\tt U} or  {\tt L} & net sideband of sub-band \\
NUM CHANNELS {\em f} & integer $\ge 1$ & initial number of channels (FFT size, {\em nFFT}, is twice this) \\
CHANS TO AVG {\em f} & integer $\ge 1$ & average this many channels before generating output spectra) \\
OVERSAMPLE FAC. {\em f}  & integer $\ge 1$ & total oversampling factor of baseband data \\
DECIMATION FAC. {\em f}  & integer $\ge 1$ & portion of oversampling to handle by decimation \\
PHASE CALS {\em f} OUT & integer $\ge 0$ & number of phase cals to produce ({\em nTone}$_f$)\\
&& The row below is duplicated {\em nTone}$_f$ times. \\
PHASE CAL {\em f}/{\em t} INDEX & integer & tone number of band \\
\hline
\end{tabular}
\end{center}

\subsubsection{Telescope table}

Below are the keywords and allowed values for entries in the telescope table which tabulates antenna names and their associated peculiar clock offsets, and the time derivatives of these offsets.
Much of the other antenna-specific information is stored in the datastream table (\S\ref{table:datastream}).
Each datastream of each configuration is mapped to one of these through a value in the datastream table.
Each entry in this table has three parameters which are replicated for each telescope table entry.
This table begins with header 
\begin{itemize}
\item[] {\tt \verb+# TELESCOPE TABLE ##!+}
\end{itemize}
The table below uses {\em a} to represent the antenna index, which ranges from 0 to {\em nAntenna} - 1 and {\em c} to represent clock coefficient, ranging from 0 to {\em nCoeff}$_a$.

\begin{center}
\begin{tabular}{l l l}
\hline
Key & Units/allowed values & Comments \\
\hline
TELESCOPE ENTRIES & integer $\ge 1$ & number of antennas ({\em nAntenna}) \\
\hline
TELESCOPE NAME {\em a} & string & abbreviation of antenna name \\
CLOCK REF MJD {\em a} & double & date around which the following polynomial is expanded \\
CLOCK POLY ORDER {\em a} & int $\ge 0$ & polynomial order of telescope clock model ({\em nCoeff}$_a$\\
CLOCK COEFF {\em a}/{\em c} & $\mu$sec/sec$^c$ & clock model polynomial coefficient \\
\hline
\end{tabular}
\end{center}


\subsubsection{Datastream table} \label{table:datastream}

The datastream table begins with header 
\begin{itemize}
\item[] {\tt \verb+# DATASTREAM TABLE #!+}
\end{itemize}
The table below uses {\em f} to represent recorded frequencies, which ranges from 0 to {\em nFreq} - 1.
A second index, {\em z}, is used to iterate over zoom bands, ranging from 0 to {\em nFreq} - 1.
A third index, {\em i}, is used to cover the range 0 to $\mathit{nBB}$ - 1, where the total number of basebands is given by $\mathit{nBB} \equiv \sum_f \mathit{nPol}_f$.
In the DiFX system, all sub-bands must have the same polarization structure, so $\mathit{nBB} = \mathit{nFreq} \cdot \mathit{nPol}$.
This index is reused for the zoom bands in an analogous manner.

\begin{center}
\begin{tabular}{l l l}
\hline
Key & Units/allowed values & Comments \\
\hline
DATASTREAM ENTRIES & integer $\ge 1$ & number of antennas ({\em nDatastream}) \\
DATA BUFFER FACTOR & integer $\ge 1$ &  \\
NUM DATA SEGMENTS  & integer $\ge 1$ &  \\
\hline
TELESCOPE INDEX & integer $\ge 0$ & telescope table index of datastream \\
TSYS & Kelvin & if zero (normal in NRAO usage), don't scale data by {\em tsys} \\
DATA FORMAT & string & data format \\
QUANTISATION BITS & integer $\ge 1$ & bits per sample \\
DATA FRAME SIZE & integer $\ge 1$ & bytes in one frame(or file) of data \\
DATA SAMPLING & string & {\tt REAL} or {\tt COMPLEX} \\
DATA SOURCE & string & {\tt FILE} (see \S\ref{sec:datafiles}), {\tt MODULE} for Mark5 playback, or {\tt FAKE} for benchmarking mode \\
FILTERBANK USED & boolean & currently only {\tt FALSE} \\
PHASE CAL INT (MHZ) & int & pulse cal comb frequency spacing, or 0 if no pulse cal tones \\
NUM RECORDED FREQS & integer $\ge 0$ & number of different frequencies recorded for this datastream \\
\hline
REC FREQ INDEX {\em f} & integer $\ge 0$ & index to frequency table \\
CLK OFFSET {\em f} (us) & $\mu$sec & frequency-dependent clock offset \\
FREQ OFFSET {\em f} (us) & $\mu$sec & frequency-dependent LO offset \\
NUM REC POLS {\em f} & 1 or 2 & for this recorded frequency, the number of polarizations \\
\hline
REC BAND {\em i} POL & {\em R} or {\em L} & polarization identity \\
REC BAND {\em i} INDEX & integer $\ge 1$ & index to frequency setting array above; {\em nBB} per entry \\
\hline
NUM ZOOM FREQS & integer $\ge 0$ & number of different zoom bands set for this datastream \\
\hline
ZOOM FREQ INDEX {\em z} & integer $\ge 0$ & index to frequency table \\
NUM ZOOM POLS {\em z} & 1 or 2 & for this recorded frequency, the number of polarizations \\
\hline
ZOOM BAND {\em i} POL & {\em R} or {\em L} & polarization identity \\
ZOOM BAND {\em i} INDEX & integer $\ge 1$ & index to frequency setting array above; {\em nBB} per entry \\
\end{tabular}
\end{center}

\subsubsection{Baseline table}

In order to retain the highest level of configurability, each baseline can be independently configured at some level.
This datastream table begins with header 
\begin{itemize}
\item[] {\tt \verb+# BASELINE TABLE ###!+}
\end{itemize}
The baseline table has multiple entries, each one corresponding to a pair of antennas, labeled {\tt A} and {\tt B} in the table.
For each of {\em nBaseline} baseline entries, {\em nFreq} sub-bands are processed, and for each a total of {\em nProd} polarization products are formed.
Indexes for each of these dimensions are {\em b}, {\em f} and {\em p} respectively, each starting count at 0.
Within the DiFX context, all baselines must have the same {\em nFreq} and {\em nProd}, though this is not a requirement of {\tt mpifxcorr} in general.
Each of the {\em nFreq} sub-bands specifies a pair of (possibly identical) datastream table bands to correlate.
The global frequency table index of that product is by default identical to some index that one band out of the datastream table band pair ultimately refers to.
Under DiFX~2.7 that target frequency can be specified explicitly and is interpreted as a spectral placement directive -- which target frequency
the sub-band shall contribute data to. One or more sub-bands can refer to the same target frequency index.

\begin{center}
\begin{tabular}{l l l}
\hline
Key & Units/allowed values & Comments \\
\hline
BASELINE ENTRIES & integer $\ge 1$ & number of entries in table, {\em nBaseline} \\
\hline
D/STREAM A INDEX {\em b} & integer $\ge 0$ & datastream table index of first antenna \\
D/STREAM B INDEX {\em b} & integer $\ge 0$ & datastream table index of second antenna \\
NUM FREQS {\em b} & integer $\ge 1$ & number of frequencies on this baseline, {\em nFreq$_\mathit{b}$} \\
(TARGET FREQ {\em b} & integer $\ge 0$ & index to frequency table) \\
POL PRODUCTS {\em b}/{\em f} & integer $\ge 1$ & number of polarization products, {\em nProd$_\mathit{b}$} \\
D/STREAM A BAND {\em p} & integer $\ge 0$ & index to frequency array in datastream table \\
D/STREAM B BAND {\em p} & integer $\ge 0$ & same as abovem, but for antenna {\tt B}, not {\tt A} \\
\hline
\end{tabular}
\end{center}

\subsubsection{Data Table}

In the following table, {\em d} is the datastream index, ranging from 0 to {\em nDatastream} - 1 and {\em f} is the file index ranging from 0 to {\em nFile$_\mathit{d}$}.

\begin{center}
\begin{tabular}{l l l}
\hline
Key & Units/allowed values & Comments \\
\hline
D/STREAM {\em d} FILES & integer $\ge 1$ & number of files {\em nFile$_\mathit{d}$} associated with datastream {\em d} \\
FILE {\em d}/{\em f} & string & name of file or module associated with datastream {\em d} \\
\hline
\end{tabular}
\end{center}

For datastreams reading off Mark5 modules, {\em nFile} will always be 1 and the filename is the {\em VSN} of the module being read.







% .joblist --------------------------------------------------------------------

\subsection{.joblist} \label{sec:joblistfile}

A single {\tt .joblist} file is written by {\tt vex2difx} (\S\ref{sec:vex2difx}) as it produces the DiFX {\tt .input} (and other) files for a given correlator pass.
This file contains the list of jobs to run and some versioning information that allows improved accountability of the software versions being used.
This file us used by {\tt difxqueue} and {\tt makefits} to ensure that a complete set of jobs is accounted for.
The file is composed of two parts: a header line and one line for each job.
The header line consists of a series of {\em key=value} pairs.  
Each {\em key} and {\em value} must have no whitespace and no whitespace should separate these words from their connecting {\tt =} sign.
While any number of key-value pairs may be specified, the following ones (which are case sensitive) are expected to be present:
\begin{enumerate}
\item {\tt exper} : the name of the experiment, including the segment code
\item {\tt v2d} : the {\tt vex2difx} input file used to produce the jobs of this pass
\item {\tt pass} : the name of the correlator pass
\item {\tt mjd} : the modified Julian day when {\tt vex2difx} created this file
\item {\tt DiFX} : the version name for the DiFX deployment (the value of {\tt \$DIFX\_VERSION} when {\tt vex2difx} was run)
\item {\tt vex2difx} : the version of {\tt vex2difx} that was run
\end{enumerate}
Each additional line contains information for one job in the pass.
The columns are:
\begin{enumerate}
\item {\em jobName} : the name/prefix of the job
\item {\em mjdStart} : the observe start time of the job
\item {\em mjdStop} : the observe stop time of the job
\item {\em nAnt} : the number of antennas in the job
\item {\em maxPulsarBin} : the maximum number of pulsar bins to come from any scan in this job (usually zero)
\item {\em nPhaseCenters} : the maximum number of phase centers to come from any scan in this job (usually one)
\item {\em tOps} : estimated number of trillion floating point operations required to run the job
\item {\em outSize} : estimated FITS file output size (MB)
\end{enumerate}
Usually the comment character {\tt \#} followed by a list of station codes is appended to the end of each line.








% .jobmatrix ------------------------------------------------------------------

\subsection{.jobmatrix} \label{sec:jobmatrixfile}

As of version~2.6 of {\tt difx2fits} a file with extension {\tt .jobmatrix} will be written for each {\tt .FITS} file created.
This file is meant as a summary for human use and as such does not have a format that should be considered fixed.
The file contains a 2-dimensional map (antenna number vs.\ time) of which jobs contributed to the {\tt .FITS} file.





% .lag ------------------------------------------------------------------------

\subsection{.lag} \label{sec:lag}

Program {\tt zerocorr} (\S\ref{sec:zerocorr} produces lag output in a format documented here.
There is one line of output per lag.
Each line has 7 columns as per the following table:
\begin{center}
\begin{tabular}{ll}
\hline
Line & Contents \\
\hline
1 & Channel (spectral point) number (counts from zero) \\
2 & Time lag (sec) \\
3 & Real value of the lag function \\
4 & Imaginary value of the lag function \\
5 & Amplitude \\
6 & Phase \\
7 & Window function (weight at this lag) \\
\hline
\end{tabular}
\end{center}



% .log ------------------------------------------------------------------------

\subsection{.log} \label{sec:log}

When generation of sniffer output files is not disabled, each {\tt .FITS} file written by {\tt difx2fits} will be accompanied by a corresponding {\tt .log} file. 
This file contains a summary of the contents of that {\tt .FITS} file.
It is analogous to the {\tt logfile.lis} file produced by the old {\tt FITSsniffer} program.
This file is free-form ASCII that is intended for viewing by human eyes, and is should not be used as input to any software as the format is not guaranteed to remain constant.







% .machines -------------------------------------------------------------------

\subsection{.machines} \label{sec:machines}

The {\tt .machines} file is used by {\tt mpirun} to determine which machines will run {\tt mpifxcorr}.
This is a text file containing a list of computers, one to a line possibly with additional options listed, on which to spawn the software correlator process.
As a general rule the MPI rank, a unique number for each process that starts at 0, are allocated in the order that the computer names are listed.
This general rule can break down in cases where the same computer name is listed more than once; the behavior in this case depends on the MPI implementation being used.
MPI rank 0 will always be the manager process.
Ranks 1 through {\em nDatastream} will each be a datastream process.
Additional processes will be computing (core) processes.
If more processes are specified for {\tt mpirun} with the {\tt -np} option than there are lines in this file, the file will be read again from the top, so the processes will be assigned in a cyclic fashion (again, this depends somewhat on the MPI implementation and the other parameters passed to {\tt mpirun}; for DiFX with OpenMPI, this assumes {\tt --bynode} is used).
If the program {\tt startdifx} is used to start the correlation process, the number of processes to start is determined by the number of lines in this file.
If wrapping to the top of this file is desired, dummy comment lines (beginning with \#) can be put at the end of the {\tt .machines} file to artificially raise the number of processes to spawn.
Within DiFX, this file is typically produced by {\tt genmachines}.
Keep in mind that this file is directly read by the MPI execution program {\tt mpirun} and the format of the file may differ depending on the MPI implementation that you are using.
With OpenMPI appending {\tt slots=1 max-slots=1} to the end of each line ensures that a single instance of {\tt mpifxcorr} is run on that machine.
If both a datastream process and a core process are to be run on the same computer, then using options {\tt slots=1 max-slots=2} might be appropriate.









% .mark4list ------------------------------------------------------------------

\subsection{.mark4list} \label{sec:mark4list}

A {\tt .mark4list} file is written by {\tt makemark4} and contains the entire list of {\tt gzip} compressed file sets (ending in {\tt .mark4.tar.gz}) for the correlator pass.
Due to the different constraints of the correlation process and the Mark4 format, the number of resultant compressed file sets may be greater or less than the number of jobs.
This file type is used by {\tt difxarch} to ensure that all of the correlated output ends up in the archive.
The file is composed of two parts: a header line and one line for each compressed file set.
The header line consists of a series of {\em key=value} pairs.  
Each {\em key} and {\em value} must have no whitespace and no whitespace should separate these words from their connecting {\tt =} sign.
While any number of key-value pairs may be specified, the following ones (which are case sensitive) are expected to be present:
\begin{enumerate}
\item {\tt exper} : the name of the experiment, including the segment code
\item {\tt pass} : the name of the correlator pass
\item {\tt jobs} : the name of the {\tt .joblist} file used by {\tt makemark4}
\item {\tt mjd} : the modified Julian day when {\tt makemark4} created this file
\item {\tt DiFX} : the version name for the DiFX deployment (the value of {\tt \$DIFX\_VERSION} when {\tt vex2difx} was run)
\item {\tt difx2mark4} : the version of {\tt difx2mark4} that was run
\end{enumerate}
Each additional line contains information for one compressed file set of the correlation pass.
These lines contain three fields:
\begin{enumerate}
\item {\em archiveName} : the name of the file that will get injected into the archive (see \S\ref{sec:archive})
\item {\em fileSize} : the size of the file in MB
\item {\em origName} : the name of the file as produced by {\tt difx2mark4} (via {\tt makemark4})
\end{enumerate}





% .oms ------------------------------------------------------------------------

\subsection{.oms} \label{sec:oms}

A {\tt .oms} file is written by {\tt sched} and contains machine (and human) readable information that is useful in setting up correlator jobs.
In the case of the VLBA DiFX correlator, program {\tt oms2v2d} (\S\ref{sec:oms2v2d}) uses this file to prepare a template {\tt .v2d} file (\S\ref{sec:v2d}) that contains some information not available in the vex file, such as intended integration time and number of channels.








% .params ---------------------------------------------------------------------

\subsection{.params} \label{sec:params}

A file with extension {\tt .params} is written by {\tt vex2difx} (\S\ref{sec:vex2difx}) when it is provided with the {\tt -o} option.
This file is a duplicate of the {\tt .v2d} file that was supplied but with all unspecified parameters listed with the defaults that they assumed.
The format is exactly the same as the {\tt .v2d} files; see \S\ref{sec:v2d} for documentation of the format.
The {\tt .params} file can be used as a legal {\tt .v2d} file if necessary.







% pcal ------------------------------------------------------------------------

\subsection{pcal} \label{sec:pcal}

A file called {\tt pcal} is created when program {\tt vlog} operates on the {\tt cal.vlba} file.
This file contains three measurements: the cable length calibration, pulse calibration, and state counts.
This file contains two kinds of lines.
Comment lines begin with an octothorpe (\#) and contain no vital information.
Data lines always begin with 9 fields describing the content of that data line:
\begin{enumerate}
\item {\em antId} : Station name abbreviation, e.g., {\tt LA}
\item {\em day} : Time centroid of measurement (MJD or day of year, including fractional portion)
\item {\em dur} : Duration of measurement (days)
\item {\em cableCal} : Cable calibration measurement (picoseconds)
\item {\em nPol} : Number of polarizations with measurements 
\item {\em nBand} : Number of sub-bands with measurements 
\item {\em nTone} : Number of pulse cal tones detected per band per polarization, possibly zero 
\item {\em nState} : Number of state count states measured per band per polarization, possibly zero
\item {\em nRecChan} : Number of record channels at time of measurement ($\le$ \em{nPol * nBand})
\end{enumerate}
Following these nine fields are two variable-length arrays of numbers.
The first variable-length field is the pulse cal data field consisting of {\em nPol*nBand*nTone} groups of four numbers.
The first member of this group is the recorder channel number (zero-based) corresponding to the measurement.
The second member of this group is the tone sky frequency (MHz).
The third and fourth are respectively the real and imaginary parts of the tone measured at the given sky frequency.
The order in which the groups are presented (in `C' language array syntax, as used throughout this document) is $[${\em nPol}$][${\em nBand}$][${\em nTone}$]$.
Note that if there are fewer than {\em nPol*nBand} record channels, the record channel will be $-1$ for some groups.
The second variable-length field is the state count data.
For each band of each polarization, {\em nState} + 1 values are listed.
The first number is the record channel number or -1 if that polarization/band combination was not observed or monitored.
The remainder contain state counts.
{\em nState} can be either 0 or $2^{\mathit nBit}$, where {\em nBit} is the number of quantization bits.
The order in which these groups are listed is $[${\em nPol}$][${\em nBand}$]$.







% .polyco ---------------------------------------------------------------------

\subsection{.polyco} \label{sec:polycofile} 

A polyco file contains a single polynomial for pulse phase that is valid for a fraction (up to 100\%) of a job file.
An additional numeric suffix is appended to the filename specifying the polynomial index for a particular {\tt .pulsar} file that shares the same base name.
The format of the file is the same as a Tempo pulsar file \cite{tempo}.








% .shelf ----------------------------------------------------------------------

\subsection{.shelf} \label{sec:shelf}

The vex file format (see \S\ref{sec:dotvex} and references within) does not have a formal slot to record the shelf location of media, so {\tt db2vex} stashes the shelf location in a separate file.
This information is critical for the correlator operators to know where to find modules for a project and for analysts preparing correlator jobs to know if media have arrived.
The {\tt .shelf} file is used by {\tt vex2difx} when writing {\tt .calc files}.
It can also be used as input to {\tt getshelf}.
The file format is very simple.
One row is used for each module that was used in the observation.
Typically rows are sorted in the same order as antennas in the {\tt .input} file.
The comment character is {\tt \#} -- any text following this character on a line is ignored.
Each line contains 3 white-space separated columns:
\begin{enumerate}
\item {\em antId} : The typically 2 letter station abbreviation
\item {\em vsn} : The volume serial number of the media (e.g., {\tt NRAO-123})
\item {\em shelf} : The shelf location, which can be any string without whitespace (e.g., {\tt BD89}), or {\tt none} if the media is not at the correlator 
\end{enumerate}







% .threads --------------------------------------------------------------------

\subsection{.threads} \label{sec:threads}

The {\tt .threads} file tells {\tt mpifxcorr} how many threads to start on each processing node.
Within DiFX, this file is typically produced by {\tt genmachines}.
The {\tt.threads} file has a very simple format.
The first line starts with {\tt NUMBER OF CORES:}.
Starting at column 21 is an integer that should be equal to the number of processing nodes ({\em nCore}) specified in the corresponding {\tt .machines} file.
Each line thereafter should contain a single integer starting at column 1.
There should be {\em nCore} such lines.








% tsys ------------------------------------------------------------------------

\subsection{tsys} \label{sec:tsys}

A file called {\tt tsys} is created when program {\tt vlog} operates on the {\tt cal.vlba} file.
This text file contains measurements of the system temperature and name of receiver for each baseband channel.
This file contains two kinds of lines.
Comment lines begin with an octothorpe (\#) and contain no vital information.
Data lines always contain 4 fixed-size fields:
\begin{enumerate}
\item {\em antId} : Station name abbreviation, e.g., {\tt LA}
\item {\em day} : Time centroid of measurement (day of year or mjd, including fractional portion)
\item {\em dur} : Duration of measurement (days) or zero if not known
\item {\em nRecChan} : Number of baseband channels recorded 
\end{enumerate}
Following these 4 fields are {\em nRecChan} pairs of values, one for each baseband channel.
The first element of each pair is the system temperature (in K) and the second is the receiver name (e.g., {\tt 4cm}, or {\tt 7mm}).

This format should not be confused with switched power files produced by {\tt mpifxcorr} (see \S\ref{sec:difxswitchedpower}).






% weather ---------------------------------------------------------------------

\subsection{weather} \label{sec:weather}

A file called {\tt weather} is created when program {\tt vlog} operates on the {\tt cal.vlba} file.
This file contains tabulated values of various meteorological measurements.
This file contains two finds of lines.
Comment lines begin with an octothorpe (\#) and contain no vital information.
Data lines always contain 9 fixed-size fields:
\begin{enumerate}
\item {\em antId} : Station name abbreviation, e.g., {\tt LA}
\item {\em day} : Time of measurement (MJD or day of year, including fractional portion)
\item {\em T} : Ambient temperature (Centigrade)
\item {\em P} : Pressure (mbar)
\item {\em dewPoint} : Dew point (Centigrade)
\item {\em windSpeed} : Wind speed (m/s)
\item {\em windDir} : Wind direction (degrees E of N)
\item {\em precip} : Accumulated rain since UT midnight (cm)
\item {\em windGust} : Maximum wind gust over collection period (m/s)
\end{enumerate}








% .wts ------------------------------------------------------------------------

\subsection{.wts} \label{sec:wts}

When generation of sniffer output files is not disabled, each {\tt .FITS} file written by {\tt difx2fits} will be accompanied by a corresponding {\tt .wts} file. 
This file contains statistics of the data weights, typically dominated by the completeness of records as determined by the data transport system, over a typically 30 second long period.

The first line is simply the observe code, e.g., {\tt MT831} .

Each additional line in the file is a complete record for a given antenna for a given interval, containing information for each baseband channel separately.
The format of these lines is as follows:

\begin{center}
\begin{tabular}{l l l}
\hline
Key & Units/allowed values & Comments \\
\hline
{\it MJD}                & integer $\ge 1$     & MJD day number corresponding to line \\
{\it hour}               & $\ge 0.0$, $< 24.0$ & hour within day \\
{\it antenna number}     & $\ge 1$             & antenna table index \\
{\it antenna name}       & string              & \\
{\it n}$_{\mathrm{BBC}}$ & $\ge 1$             & Number of baseband channels \\
{\it mean weight}        & $\ge 0.0$           & This column repeated $n_{\mathrm{BBC}}$ times \\
{\it min weight}         & $\ge 0.0$           & This column repeated $n_{\mathrm{BBC}}$ times \\
{\it max weight}         & $\ge 0.0$           & This column repeated $n_{\mathrm{BBC}}$ times \\
\hline
\end{tabular}
\end{center}

\noindent
This file can be used directly with plotting program {\tt plotwt} or used more automatically with {\tt difxsniff}.









% .v2d ------------------------------------------------------------------------

\subsection{.v2d} \label{sec:v2d}

The {\tt .v2d} file is used to specify correlation options to {\tt vex2difx} and adjust the way in which it forms DiFX input files based on the {\tt .vex} file.
The {\tt .v2d} file consists of a number of global parameters that affect the way that jobs are created and several sections that can customize correlation on a per-source, per mode, or per scan basis.  
All parameters (those that are global and those that reside inside sections) are specified by a parameter name, the equal sign, and one value, or a comma-separated list of values, that cannot contain whitespace.  
Whitespace is not required except to keep parameter names, values, and section names separate.  
All parameter names and values are case sensitive except for source names and antenna names.  
The \# is a comment character; any text after this on a line is ignored.

Most parameters are one of the following types:
\begin{itemize}
\item {\bf bool} : A boolean value that can be True or False.  Any value starting with {\tt 0}, {\tt f}, {\tt F}, or {\tt -} will be considered False and otherwise True.
\item {\bf float} : A floating point number.  Can be of the forms: {\tt 1.23}, {\tt 1.2e-4}, {\tt -12.6}, {\tt 4}
\item {\bf int} : An integer.
\item {\bf string} : Any sequence of printable(non-whitespace) characters.  Certain fields require strings of a maximum length or certain form.
\item {\bf date} : A date field; see below.
\item {\bf array} :  Array can be of any of the four above types and are indicated by enclosing brackets, e.g., $[$int$]$.  The empty list is indicated with $[ ]$ which is usually implied to be all-inclusive.
\end{itemize}

All times used in {\tt vex2difx} are in Universal Time and are internally represented as a double precision value.
The integer part of this value is the date corresponding to 0$^h$ UT.
The fractional part, when multiplied by 86400, gives the number of seconds since 0$^h$ UT.
Note that this format does not allow one to specify the actual leap second if one occurs on that day.
Several date formats are supported:
\begin{itemize}
\item {\bf Modified Julian Day} : A decimal MJD possibly including fractional day.  E.g.: {\tt 54345.341944}
\item {\bf Vex time format} : A string of the form: {\tt 2009y245d08h12m24s}
\item {\bf VLBA-like format} : A string of the form: {\tt 2009SEP02-08:12:24}
\item {\bf ISO 8601 format} : A string of the form: {\tt 2009-09-02T08:12:24}
\end{itemize}

Global parameters can be specified one or many per line such as:

\noindent
{\tt maxGap = 2000 \# seconds}

\noindent
or

\noindent
{\tt mjdStart = 52342.522 mjdStop=52342.532}

The following parameter names are recognized:

\begin{center}
\begin{tabular}{l l l l l}
\hline
Name & Type & Units & Defaults & Comments \\
\hline
vex              & string &       &              & filename of the vex file to process; {\bf this is required} \\
mjdStart         & date   &       & obs.\ start  & discard any scans or partial scans before this time \\
mjdStop          & date   &       & obs.\ stop   & discard any scans or partial scans after this time \\
break            & date   &       &              & list of MJD date/times where jobs are forced to be broken \\
minSubarray      & int    &       & 2            & don't make jobs for subarrays with fewer antennas than this \\
maxGap           & float  & sec   & 180          & split an observation into multiple jobs if there are \\
                 &        &       &              & correlation gaps longer than this number \\
tweakIntTime	 & bool   &       & False	 & adjust (up to 40\%) int.\ time to ensure int.\ blocks per send \\
singleScan       & bool   &       & False        & if True, split each scan into its own job \\
singleSetup      & bool   &       & True         & if True, allow only one setup per job; True is required \\
                 &        &       &              & for FITS-IDI conversion \\
maxLength        & float  & sec   & 7200         & don't allow individual jobs longer than this amount of time \\
minLength        & float  & sec   & 2            & don't allow individual jobs shorter than this amount of time \\
maxSize          & float  & MB    & 2000         & max FITS-IDI file size to allow \\
dataBufferFactor & int    &       & 32           & the {\tt mpifxcorr} DATABUFFERFACTOR parameter \\
nDataSegments    & int    &       & 8            & the {\tt mpifxcorr} NUMDATASEGMENTS parameter \\
jobSeries        & string &       &              & the base filename of {\tt .input} and {\tt .calc} files to be \\
                 &        &       &              & created; defaults to the base name of the {\tt .v2d} file \\
startSeries      & int    &       & 20           & the default starting number for jobs created \\
sendLength       & float  & sec   & 0.262144     & roughly the amount of data to send at a time from \\
                 &        &       &              & datastream processes to core processes \\
antennas         & $[$string$]$ &  & $[ ]$ = all & a comma separated list of antennas to include in correlation \\
baselines        & $[$string$]$ &  & $[ ]$ = all & a comma separated list of baselines to correlate; see below \\
padScans         & bool   &       & True         & insert non-correlation scans in recording gaps to prevent \\
                 &        &       &              & {\tt mpifxcorr} from complaining \\
invalidMask      & int    &       & 0xFFFF       & this bit-field selects which flag conditions are considered \\
                 &        &       &              & when writing flag file: 1=Recording, 2=On source, 4=Job \\
		 &        &       &              & time range, 8=Antenna in job \\
visBufferLength  & int    &       & 32           & number of visibility buffers to allocate in mpifxcorr \\
overSamp         & int    &       &              & force all basebands to use the given oversample factor \\
mode             & string &       & normal       & mode of operation; see below \\
threadsFile      & string &       &              & overrides the name of the threads file to use \\
nCore            & int    &       &              & with nThread and machines, cause a {\tt .threads} file to be made \\
nThreads         & int    &       &              & number of threads per core in {\tt .threads file} \\
machines         & $[$string$]$ & &              & comma separated list of machines to use as processors \\
                 &        &       &              & first is head node, then datastreams, then cores \\
maxReadSize      & int    & bytes & 25000000     & maximum number of bytes to read at a time \\
minReadSize      & int    & bytes & 10000000     & minimum number of bytes to read at a time \\
\hline
\hline
\end{tabular}
\end{center}

The {\em baselines} parameter supports the wildcard character {\tt *} an individual antenna name, or lists of antenna names separated by {\tt +} on each side of a hyphen ({\tt -}).
Multiple baseline designators can be listed.
Examples:
\begin{itemize}
\item {\tt A1-A2} : Only correlate one baseline
\item {\tt A1-A2, A3-A4} : Correlate 2 baselines
\item {\tt *-*} : Correlate all baselines
\item {\tt A1-*} {\em or} {\tt *-A1} : Correlate all baselines to antenna A1
\item {\tt A1+A2-*} : Correlate all baselines to antenna A1 or A2
\item {\tt A1+A2-A3+A4+A5} : Correlate 6 baselines
\end{itemize}

A source section can be used to change the properties of an individual source, such as its position or name.
In the future this is where multiple correlation centers for a given source will be specified.
A source section is enclosed in a pair of curly braces after the keyword SOURCE followed by the name of a source, for example

\begin{Verbatim}[commandchars=\|\[\]]
  SOURCE 3C273
  {
    |bfit[source parameters go here]
  }
\end{Verbatim}

\noindent
or equivalently

\begin{Verbatim}[commandchars=\|\[\]]
  SOURCE 3c273 { |bfit[source parameters go here] }
\end{Verbatim}

\begin{center}
\begin{tabular}{l l l l l}
\hline
Name			& Type		& Units & Defaults	& Comments \\
\hline
ra			&		& J2000 &		& right ascension, e.g., {\tt 12h34m12.6s} or {\tt 12:34:12.6} \\
dec			&		& J2000 &		& declination, e.g., {\tt 34d12'23.1}" or {\tt 34:12:23.1} \\
name			& string	&       &		& new name for source \\
calCode			& char		&       & ' '		& calibration code, typically {\tt A}, {\tt B}, {\tt C} for calibrators, \\
			&		&	&		& {\tt G} for a gated pulsar, or blank for normal target \\
naifFile		& string	&       &		& name of leap seconds file (e.g., {\tt naif0010.tls} for ephemeris \\
ephemObject		& string	&       &		& name or number of object in ephemeris file \\
ephemFile		& string	&       &		& path of ephemeris file (either {\tt .bsp} or {\tt .tle} format \\
doPointingCentre	& bool		&	& true		& Whether the pointing centre should be correlated \\
			&		&	&		& (only ever turned off for multi-phase center) \\
addPhaseCentre		& string	&       &		& contains info on a source to add; see below \\
\hline
\hline
\end{tabular}
\end{center}

To add additional phase centers, add one or more ``addPhaseCentre'' parameters to the source setup.
In the parameter, the RA and dec must be provided.
A name and/or calibrator code can be added as well.
For example: {\tt addPhaseCentre=name@1010-1212/RA@10:10:21.1/Dec@-12:12:00.34} .


An antenna section allows properties of an individual antenna, such as position, name, or clock/LO offsets, to be adjusted. 
Note that the ``late'' convention is used in {\em clockOffset} and {\em clockRate}, unlike the ``early'' convention used in the {\tt .vex} file itself (see \S\ref{sec:clockconventions}).

\begin{center}
\begin{tabular}{l l l l l}
\hline
Name		& Type		& Units 	& Defaults	& Comments \\
\hline
name		& string	&       	&		& new name to assign to this antenna \\
polSwap		& bool		&       	& False		& swap the polarizations (i.e., {\tt L} $\Leftrightarrow$ {\tt R}) for this antenna \\
clockOffset	& float		& us		& vex value	& overrides the clock offset value from the vex file \\
clockRate	& float		& us/s		& vex value	& overrides the clock offset rate value from the vex file \\
clockEpoch	& date		&		& vex value	& overrides the epoch of the clock rate value; must be present \\
		&		&		&		& present if clockRate or clockOffset parameter is set \\
deltaClock	& float		& us		& 0.0		& adds to the clock offset (either the vex value or the \\
		&		&		&		& clockOffset above  \\
deltaClockRate	& float		& us/s		& 0.0		& adds to the clock rate (either the vex value or the \\
		&		&		&		& clockRate above \\
X		& float		& m		& vex value	& change the X coordinate of the antenna location \\
Y		& float		& m		& vex value	& change the Y coordinate of the antenna location \\
Z		& float		& m		& vex value	& change the Z coordinate of the antenna location \\
format		& string	&		&		& force format to be one of VLBA, MKIV, Mark5B, S2, or \\
		&		&		&		& one of the VDIF types \\
file		& $[$string$]$	&		& (none)	& a comma separated list of data files to correlate \\
filelist	& string	&		&		& a filename listing files for the DATA TABLE (see \S\ref{sec:filelist}) \\
networkPort	& int		&		&		& The eVLBI network port to use for TCP/UDP. A non-number indicates \\
		&		&		&		& raw mode attached to an ethernet interface. Both force NETWORK media \\
		&		&		&		& type in {\tt .input} file. \\
windowSize	& int		&		&		& TCP window size in kilobytes for eVLBI. Set to $<0$ in bytes for UDP \\
UDP\_MTU	& int		&		&		& Same as setting windowSize to negative of value. \\
		&		&		&		& For raw mode, the number of bytes to strip from ethernet frame. \\
vsn		& string	&		&		& override the Mark5 Module to be used \\
zoom 		& string	&		&		& uses the global zoom configuration with matching name for \\
		&		&		&		& this antenna; {\tt zoom=Zoom1} will match ZOOM block called {\tt Zoom1} \\
addZoomFreq	& string	&		&		& adds a zoom band with specified freq/bw as shown: \\
		&		&		&		& freq@1810.0/bw@4.0$[$/specAvg@4$][$/noparent@false$]$ \\
freqClockOffs	& $[$float$]$	& microsec	&		& adds clock offsets to each recorded frequency using the format: \\
		&		&		&		& {\tt freqClockOffs=}{\em f1,f2,f3,f4}; must be same length as \\
		&		&		&		& number of recorded freqs, first value must be zero \\
loOffsets 	& $[$float$]$	& Hz 		&		& adds LO offsets to each recorded frequency using the format: \\
		&		&		&		& {\tt loOffsets=}{\em f1,f2,f3,f4}; must be same length as \\
		&		&		&		& number of recorded freqs. \\
tcalFreq	& int		& Hz		& 0 		& enables switched power detection at specified frequency \\
phaseCalInt	& int		& MHz 		& 1		& zero turns off phase cal extraction, positive value is \\
		&		&		&		& the interval between tones to be extracted \\
toneGuard	& float		& MHz 		& 0.125 of bw	& when using toneSelection {\em smart} or {\em most} don't select tones \\
		&		&		&		& within this range of band edge, if possible \\
toneSelection & string  &		& {\tt smart}	& tone selection algorithm; see below \\
sampling	& string	&		& {\tt REAL}	& set to {\tt COMPLEX} for complex sampled data \\
fake		& bool		&		& False		& enable a fake data source \\
\hline
\hline
\end{tabular}
\end{center}

Possible values of ``tone Selection'' are:
\begin{center}
\begin{tabular}{ll}
smart	& write the 2 most extreme tones at least toneGuard from band edge (default) \\
vex 	& write the tones listed in the vex file to FITS \\
none 	& don't write any tones to FITS \\
all 	& write all extracted tones to FITS \\
ends 	& write the 2 most extreme tones to FITS \\
most 	& write all tones not closer than toneGuard to band edge \\
\end{tabular}
\end{center}

Setup sections are enclosed in braces after the word SETUP and a name given to this setup section.
The setup name is referenced by a RULE section (see below).
A setup with the special name {\tt default} will be applied to any scans not otherwise assigned to setups by rule sections.
If no setup sections are defined, a setup called {\tt default}, with all default parameters, will be implicitly created and applied to all scans.
The order of setup sections is immaterial.

\begin{center}
\begin{tabular}{l l l l l}
\hline
Name		& Type		& Units	& Defaults	& Comments \\
\hline
tInt		& float		& sec	& 2		& integration time \\
FFTSpecRes	& float		& MHz	& 0.125		& frequency resolution of FFT \\
specRes		& float		& MHz	& 0.5		& output freq res (must be mult.\ of FFTSpecRes \\
nChan		& int		&	& 16		& number of channels per spectral window; must be $5^m \cdot 2^n$ \\
specAvg		& int		&	& 1		& how many channels to average together after correlation \\
fringeRotOrder	& int		&	& 1		& fhe fringe rotation order: 0=post-F, 1=linear, 2=quadratic \\
strideLength	& int		& 	& 16		& number of channels to ``stride'' for fringe rotation, etc. \\
xmacLength	& int		&	& 128		& number of channels to ``stride'' for cross-multiply accumulations \\
numBufferedFFTs & int		&	& 1		& number of FFTs to do in a row for each datastream, before XMAC \\
doPolar		& bool		&	& True		& correlate cross hands when possible \\
postFFringe	& bool		&	& False		& do fringe rotation after FFT? \\
binConfig	& string	&	& none		& if specified, apply this pulsar bin config file to this setup \\
freqId		& $[$int$]$	&	& $[ ]$ = all	& frequency bands to correlate \\
outputBandwidth & float\textbar{}'auto' & MHz & n/a & Target bandwidth when auto determining outputbands, or, \\
addOutputBand   & $[$string$]$          &     & n/a & Add an outputband with explicit placement and bandwidth \\
	            &                       &     &     & freq@1810.0/bw@4.0  \\
\hline
\hline
\end{tabular}
\end{center}

Note that either ``FFTSpecRes'' and ``specRes'' can be used, or ``nChan'' and ``specAvg'' can be used, but the two sets cannot be mixed.

Zoom channels can be configured in a special section and referenced from ANTENNA sections to minimize complexity of the {\tt .v2d} file.
Each ZOOM section has a name and one or more ``addZoomFreq'' parameters, with the same format as they would have in the ANTENNA block.

Output bands can be defined in the SETUP block in DiFX~2.7 (OUTPUTBAND block in DiFX~2.8) via ``outputBandwidth'' or alternatively ``addOutputBand''.
Permitted ``outputBandwidth'' values are 'auto' to auto-determine the bandwidth, or, an explicit bandwidth as a decimal value in MHz.
Band placement is automatic. For explicit placement and bandwith instead use ``addOutputBand''.

Earth Orientation Parameter (EOP) data can be provided via one or more EOP sections.
EOP data should be provided either in the {\tt .v2d} file or in the vex file, but not both.
Normally the vex file would be used to set EOP values, but there may be cases (eVLBI?) that want to use the vex file from {\tt sched} without any modification.
Like ANTENNA and SOURCE sections, each EOP section has a name.
The name must be in a form that can be converted directly to a date (see above for legal date formats).
Conventional use suggests that these dates should correspond to 0 hours UT; deviation from this practice is at the users risk.
There are four parameters that should all be set within an EOP section:

\begin{center}
\begin{tabular}{l l l l l}
\hline
Name		& Type	& Units		& Defaults	& Comments \\
\hline
tai\_utc	& float	& sec		&		& TAI minus UTC; the leap-second count  \\
ut1\_utc	& float	& sec		&		& UT1 minus UTC; Earth rotation phase \\
xPole		& float	& arcsec	&		& X component of spin axis offset \\
yPole		& float	& arcsec	&		& Y component of spin axis offset \\
\hline
\hline
\end{tabular}
\end{center}

A rule section is used to assign a setup to a particular source name, calibration code (currently not supported), scan name, or vex mode.
The order of rule sections {\em does} matter as the order determines the priority of the rules.
The first rule that matches a scan is applied to that scan.
The correlator setup used for scans that match a rule is determined by the parameter called ``setup''.
A special setup name {\tt SKIP} causes matching scans not to be correlated.
Any parameters not specified are interpreted as fully inclusive.
Note that multiple rule sections can reference the same setup section.
Multiple values may be applied to any of the parameters except for ``setup''.
This is accomplished by comma separation of the values in a single assignment or with repeated assignments.
Thus

\begin{verbatim}
  RULE rule1
  {
    source = 3C84,3C273
    setup = BrightSourceSetup
  }
\end{verbatim}

\noindent
is equivalent to

\begin{verbatim}
  RULE rule2
  {
    source = 3C84 3C273
    setup = BrightSourceSetup
  }
\end{verbatim}

\noindent
is equivalent to

\begin{verbatim}
  RULE rule3
  {
    source = 3C84
    source = 3C273
    setup = BrightSourceSetup
  }
\end{verbatim}

The names given to rules (e.g., rule1, rule2 and rule3 above) are not used anywhere (yet) but are required to be unique.

\begin{center}
\begin{tabular}{l l l l l}
\hline
Name	& Type		& Units	& Comments \\
\hline
scan	& $[$string$]$	&	& one or more scan name, as specified in the vex file, to select with this rule \\
source	& $[$string$]$	&	& one or more source name, as specified in the vex file, to select with this rule \\
calCode	& $[$char$]$	&	& one or more calibration code to select with this rule \\
mode	& $[$string$]$	&	& one or more modes as defined in the vex file to select with this rule \\
setup	& string	&	& The name of the SETUP section to use, or SKIP if this rule describes scans \\
	&		&	& not to correlate \\
\hline
\hline
\end{tabular}
\end{center}

Note that source names and calibration codes reassigned by source sections are not used.
Only the names and calibration codes in the vex file are compared.

There are currently two modes of operation supported by {\tt vex2difx}.  
The mode used in the vast majority of situations is called {\tt normal} and is the default if none is specified.
Currently one alternative mode, {\tt profile}, is supported.
This mode is useful for generating pulse profiles that would be useful for pulsar gating, scrunching, and binning.
The difference compared to normal mode is that the standard autocorrelations are turned off and instead are computed as if they are cross correlations.
This allows multiple pulsar bins to be stored. 
No formal cross correlations are performed. 
To be useful, one must create and specify a {\tt .binconfig} file and select only the pulsar(s) from the experiment.

See \url{http://www.atnf.csiro.au/vlbi/dokuwiki/doku.php/difx/vex2difx} for more complete information and examples.








% .xcb ------------------------------------------------------------------------

\subsection{.xcb} \label{sec:xcb}

When generation of sniffer output files is not disabled, each {\tt .FITS} file written by {\tt difx2fits} will be accompanied by a corresponding {\tt .xcb} file. 
This file contains cross-correlation spectra for each antenna for each baseline.
In order to minimize the output data size, spectra for the same source will only be repeated once per 15 minutes.
The file contains many concatenated records.
Each record has the spectra for all baseband channels for a particular baseline and has the following format which is very similar to that of the {\tt .acb} files. 
Note that no spaces are allowed within any field.
Values in {\tt typewriter} font without comments are explicit strings that are required.

\begin{center}
\begin{tabular}{l l l l}
\hline
Line(s) & Value & Units & Comments \\
\hline
1 & {\tt timerange:}          &                    & \\
  & {\it MJD}                 & integer $\ge 1$    & MJD day number corresponding to line \\
  & {\it start time}          & string             & e.g., {\tt 13h34m22.6s} \\
  & {\it stop time}           & string             & e.g., {\tt 13h34m52.0s} \\
  & {\tt obscode:}            &                    & \\
  & {\it observe code}        & string             & e.g., {\tt MT831} \\
  & {\tt chans:}              &                    & \\
  & {\it n}$_{\mathrm{chan}}$ & $\ge 1$            & number of channels per baseband channel \\
  & {\tt x}                   &                    & \\
  & {\it n}$_{\mathrm{BBC}}$  & $\ge 1$            & number of baseband channels \\
\hline
2 & {\tt source:}             &                    & \\
  & {\it source name}         & string             & e.g., {\tt 0316+413} \\
  & {\tt bandw:}              &                    & \\
  & {\it bandwidth}           & MHz                & baseband channel bandwidth \\
  & {\tt MHz}                 &                    & \\
\hline
3 to 2+$n_{\mathrm{BBC}}$ 
  & {\tt bandfreq:}           &                    & \\
  & {\it frequency}           & GHz                & band edge (SSLO) frequency of baseband channel \\
  & {\tt GHz polar:}          &                    & \\
  & {\it polarization}        & 2 chars            & e.g., {\tt RR} or {\tt LL} \\
  & {\tt side:}               &                    & \\
  & {\it sideband}            & {\tt U} or {\tt L} & for upper or lower sideband \\
  & {\tt bbchan:}             &                    & \\
  & {\it bbc}                 & {\tt 0}            & Currently not used but needed for conformity \\
\hline
3+$n_{\mathrm{BBC}}$ to 
  & {\it ant1 number}    & $\ge 1$            & number of first antenna \\
2+$n_{\mathrm{BBC}}(n_{\mathrm{chan}}+1)$  
  & {\it ant2 number}    & $\ge 1$            & \\
  & {\it ant1 name}      & string             & \\
  & {\it ant2 name}      & string             & \\
  & {\it channel number}      & $\ge 1$            & $= \mathrm{chan} + (\mathrm{bbc}-1) \cdot n_{\mathrm{chan}}$ for chan, bbc $\ge 1$ \\
  & {\it amplitude}           & $\ge 0.0$          & \\
  & {\it phase}               & degrees            & \\
\hline
\end{tabular}
\end{center}

\noindent
The above are repeated for each cross correlation spectrum record.
This file can be plotted directly with {\tt plotbp} or handled more automatically with {\tt difxsniff}.








% .ved .skd .vex.obs .skd.obs -------------------------------------------------

\subsection{.vex, .skd, .vex.obs, \& .skd.obs} \label{sec:dotvex}

The vex (Vlbi EXperiment) file \cite{vex} format is a standard observation description format used globally for scheduling observations and for driving the correlation thereof.
The original vex file for an experiment is typically created by {\tt sched} or {\tt sked}. 
In the former case (the case used by most astronomical VLBI), the vex file has the unfortunate file extension {\tt .skd}; in the later, the file extension is usually the less confusing {\tt .vex} .
These two vex formatted files contain only observation-scheduling based information.
A small amount of information based on the actualities of the observation are added by {\tt db2vex}, producing a new vex file with an additional file extension {\tt .obs} .
Please see vex documentation external to this manual for more information.



% .vis ------------------------------------------------------------------------

\subsection{.vis} \label{sec:vis}

Program {\tt zerocorr} (\S\ref{sec:zerocorr} produces visibility output in a format documented here.
There is one line of output per generated spectral point.
Each line has 8 columns as per the following table:
\begin{center}
\begin{tabular}{ll}
\hline
Line & Contents \\
\hline
1 & Channel (spectral point) number (counts from zero) \\
2 & Frequency relative to first spectral channel (Hz) \\
3 & Real value of the visibility \\
4 & Imaginary value of the visibility \\
5 & Amplitude \\
6 & Phase \\
7 & Autocorrelation of the first datastream (real only) \\
8 & Autocorrelation of the second datastream (real only) \\
\hline
\end{tabular}
\end{center}




% .zerocorr -------------------------------------------------------------------

\subsection{.zerocorr} \label{sec:zc}

Program {\tt zerocorr} (\S\ref{sec:zerocorr} is a simple cross correlator.
It is limited to correlating one visibility spectrum from one baseband channel and can only make use of a constant offset delay model.
The section documents the file that drives this program.
This file consists of 17 lines of text.
The first 7 lines describe properties of the data from the first antenna and the following (7) lines describe properties of the second antenna as follows:
\begin{center}
\begin{tabular}{ll}
\hline
Line & Contents \\
\hline
1 (8) & Input baseband data file name \\
2 (9) & Data format (e.g., Mark5B-2048-16-2) \\
3 (10) & Input sub-band to process (0-based index) \\
4 (11) & Offset into the file (bytes) \\
5 (12) & Size of FFT to perform over input bandwidth ($2\times n_{\rm chan}$) \\
6 (13) & First channel (spectral point) to correlate \\
7 (14) & Number of channels to correlate (negative indicates LSB) \\
\hline
\end{tabular}
\end{center}

The last three lines dictate the output data files and the number of FFTs to process:

\begin{center}
\begin{tabular}{ll}
\hline
Line & Contents \\
\hline
15 & Name of output visibility ({\tt .vis}; \S\ref{sec:vis}) file \\
16 & Name of output {\tt .lag} (\S\ref{sec:lag}) file \\
17 & Number of FFTs to process (if -1, run on entire input files) \\
\hline
\end{tabular}
\end{center}

