\section{The DiFX correlator}

This document is centered around the NRAO installation of the DiFX \cite{difx} software correlator and its supporting software.
Much of the contents here applies to other installations of DiFX as well, but keep in mind that not a lot of effort is made to generalize these instructions.
Fig.~\ref{fig:block} shows the general data flow-path within the DiFX software correlator system.

\begin{figure}[h]
\begin{center}
\resizebox{5in}{!}{\includegraphics{swcorr_vex_diagram_ver3-crop}}
\caption[blockdiagram]{
{\em The DiFX software correlator block diagram as implemented for the VLBA.}
\label{fig:block}
}
\end{center}
\end{figure}

Past, present, and future versions of DiFX as packaged and used by VLBA operations are described in the following subsections.

% \subsubsection{Supported data formats} \label{sec:formats}


\subsection{NRAO-DiFX 1.0}

Versions 1.0 and 1.1 based correlation on the VLBA hardware correlator job scripts -- the {\tt .fx} files.
This ensures a compatibility period during which both correlators can produce visibilities with expectations of functionally identical results, a feature critical for validation.
This strategy also minimizes the required software effort at its earliest phases.
Version 1.0 came with the following features:
\begin{enumerate}
\item A complete path from {\tt .fx} job scripts to {\tt .FITS} files
\item A command-line only interface
\item Documentation (you are reading it now)
\item Support for VLBA and Mark IV formats
\item Correlation directly off Mark5 modules
\item Support for all projects types except those using special modes, such as pulsars, space VLBI, and near field objects
\item Spectral and time resolution bounded only by practicality
\end{enumerate}
While this version should handle most observations, fast frequency switching and geodesy experiments will produce a large number of output FITS files which may be annoying to observers and the archive.
Version 1.0 was available on February 6, 2008.

\subsection{NRAO-DiFX 1.1}

Version 1.1 builds on version 1.0 and adds the following features:
\begin{enumerate}
\item Used version of {\tt mpifxcorr} that has gone through code merge with the official version
\item Blanking of data replaced by headers (Mark4 format only)
\item Proper data weights
\item Initial Mark5B support
\item Support for oversampled data through decimation
\item Multicast status information for GUI interface
\item Correlation of moving and near field objects
\item Concatenation of multiple output files into a single or multiple FITS-IDI file(s)
\item Better support for jobs with multiple configuration tables
\item Playback off Mark5 modules with missing disks
\item Support for Amazon based Mark5 units
\item Completely replaced the ``Makefile'' system with better integrated alternative
\item Generation of delay model polynomials rather than tables, more like VLBA HW correlator
\item $u, v, w$ values are derived from the delay model (and hence include corrections for aberration, near field observations, and other subtle effects) and are evaluated when writing the FITS file
\item DiFX version accountability
\item Validation of data frames prior to decoding
\item Data evaluation (``sniffing'') built into FITS converter
\end{enumerate}
This version was released on September 3, 2008.
%Features new to version 1.1 are marked with \difxoneone.

\subsubsection{Bugs fixed}

Here are listed some of the more important bug fixes:
\begin{enumerate}
\item The clock offset was used with the wrong sign in the IM table.
\item Printed precision of some important numbers (RA and Dec) was increased.
\item Autocorrelations were ordered incorrectly for observations with a single polarization.
\item The Mark4 format decoder had a 1 day off bug.
\item The Mark4 format decoder had a 64$\times${\it fanout} sample timing offset.
\item Several causes of crashes were fixed; no known crashes remain.
\item Missing VLBA monitor data was handled badly.
\item Due to OpenMPI peculiarity, some processing nodes would get most or all of the work in some cases, which cause the work being done on other nodes to be ignored.  This was fixed by looking for results in a round-robin manner.
\item Integrations that contain data from two adjacent scans are stripped when writing FITS files.
\item Allow FITS files larger than 2GiB in size.
\end{enumerate}

\subsubsection{Known problems}

Known bugs as of the NRAO-DiFX 1.1 release:
\begin{enumerate}
\item The last couple (typically 2) integrations of a job (not a scan) tend to have low weight due to a premature termination of data processing.
\end{enumerate}

\subsection{DiFX 1.5.0}

With DiFX 1.5.0 comes a name change.
Past releases of this series have been known as ``NRAO-DiFX''.
The DiFX community has been largely receptive to the NRAO additions in support of {\tt mpifxcorr} and it was decided that dropping the ``NRAO'' was appropriate.
In some cases the term ``VLBA DiFX'' or ``VLBA DiFX 1.5'' may be mentioned.
These are simply the deployment of DiFX 1.5.0 for the VLBA correlator with some VLBA specific features.
Note that the name given to the VLBA deployment of DiFX is formally ``VLBA DiFX''.

Version 1.5.0 will start allowing correlation of experiments that cannot be represented by {\tt .fx} files and will be based on vex files.
Version 1.5.0 builds on version 1.1 and adds the following features:
\begin{enumerate}
\item Support for using a wide variety of vex files as the basis for correlation.
\item Native ephemeris-based object trajectories are supported.
\item Pulsar gating is supported.
\item Pulsar binning is supported, but not cleanly yet.
\item A graphical user interface is available for correlator operators.
\item The multicast system is fully implemented and is used monitor and control correlation and other operations.
\item Mark5B formatted data, including its 2048 Mbps extension, is supported.
\item The VLBA DiFX Operations Plan \cite{opsplan} is implemented, including interface to the VLBA archive.
\end{enumerate}
Non-NRAO users of DiFX 1.5.0 will still be able to use the tools provided but may not be able to take full advantage of the database back-end without some customization; it is the aim of this document to point out cases where the database is required.
Many of the programs described in previous versions of this document will be upgraded or overtaken by more capable replacements.

Release of DiFX 1.5.0 was announced on June 25, 2009.

%Features new to the 1.5 series are marked with \difxonefive.

\subsubsection{Bugs fixed}

Here are listed some of the more important bug fixes:
\begin{enumerate}
\item A rounding issue in {\tt mpifxcorr} occasionally caused the wrong source's UVWs to be assigned.
\item Lower side band data would come out of the sniffer portion of {\tt difx2fits} with the wrong sign for phase, rate, and delay.
\item Different rounding was used to generate start times for {\tt .input} and {\tt .calc} files.
There are no severe consequences of this issue.
\item Scaling in pulsar gating has been made more sane.
\end{enumerate}

\subsection{DiFX 1.5.1}

DiFX 1.5.1 is mostly a bug fix update to version 1.5.0, but with a few new features.
The new features include:
\begin{enumerate}
\item Option to force job breaks (with the break parameter) has been added to {\tt vex2difx}
\item Time/date formats other than decimal MJD are now accepted by {\tt vex2difx}
\item Specification of data files to correlate (rather than Mark5 units) is supported in {\tt vex2difx}
\item Specification of network parameters in {\tt vex2difx} to allow correlation of eVLBI projects
\item {\tt difx2fits} produces a new output file with suffix {\tt .jobmatrix} provides the user with a better idea of the mapping of jobs into {\tt .FITS} files
\item A {\tt vex2difx} mode for generating DiFX files useful for determining pulsar phase has been added
\item EOP values can now be provided within the {\tt .v2d} file
\item Upcoming FITS-IDI keyword WEIGHTYP populated
\item Zero-weight data is not written from {\tt mpifxcorr}
\item New utility {\tt checkdir} to look for oddities in Mark5 module directory files
\end{enumerate}

\subsubsection{Bugs fixed}

Here are listed some of the more important bug fixes:
\begin{enumerate}
\item Concatenation of jobs in the creation of {\tt .FITS} files does the right thing for cases where the antenna subsets change and where antenna reordering is done.
\item The Pulsar Gate Model (GM) {\tt .FITS} file table is now correctly populated for pulsar observations.
\item Autocorrelations are written for each pulsar bin
\item The FXCORR simulator mode of {\tt vex2difx} now selects the correct reference time for antenna clock offsets.
\item A work-around for a Streamstor problem has been added that should improve reliability in Mark5 module correlation when a change in bank is needed.
\item The sign of clock offsets in vex files has been reversed to follow the vex standard
\item Jobs are split at leap seconds
\item LBA data formats are handled more correctly in {\tt vex2difx}
\item The model generator ({\tt calcif2}) now respects polynomial parameters interval and order given on the command line.
\end{enumerate}

DiFX~1.5.1 was made available via subversion on Sep 8, 2009.

\subsection{DiFX 1.5.2}

DiFX~1.5.2 is mostly a bug fix update to version 1.5.1, but with a few new features.
This version of DiFX comes with the following components (and versions): calcif2 (1.1), calcserver (1.2), difx\_db (1.12; NRAO-only), difx2fits (2.6.1), difx2profile (0.1), difxio (2.12.1), difxmessage (0.7), mark5access (1.3.3), mk5daemon (1.2), mpifxcorr (1.5.2), vex2difx (1.0.2), and vis2screen (0.1).
The new features include:
\begin{enumerate}
%\item Primitive support for single-thread VDIF format data
\item Support unmodulated VLBA format data with new pseudo-format ``VLBN''
\item {\tt mpifxcorr} now warns when difxmessage is in use so the user knows why no messages appear on the screen
\item New utility {\tt difxcalculator} in the {\tt difxio} package
\item eVLBI support within {\tt vex2difx}
\item Vastly improved real-time correlation monitoring
\item New utility {\tt diffDiFX.py} to compare two DiFX output files
\item Improved and more consistent error messages (and some of them are now documented!)
\item {\tt vex2difx} now operates in strict parsing mode by default
\item Additional user feedback to indicate suspicious or bad {\tt .polyco} and {\tt .v2d} files
\item {\tt calcif2} warns if any NaNs or Infs are produced
\item Clock adjustments are easier now with {\em deltaClock} and {\em deltaClockRate} parameters in the {\tt vex2difx} antenna settings
\end{enumerate}

\subsubsection{Bugs fixed}

\begin{enumerate}
\item Improve timestamp precision (thanks to John Morgan)
\item The {\tt vlog} program (used at NRAO only, I think) misparsed the pulse cal information in some cases
\item Fixed memory leak in {\tt difx2fits} when combining a large number of jobs
\item Improved FXCORR simulation mode in {\tt vex2difx}
\item Mark5 directory reading systematically generates unique names for all scans even when two scans have the same name
\item Improve reporting of Mark5 errors during playback and change alert severity to be more appropriate
\item Don't overblank certain Mark4 modes (thanks to Sergei Pogrebenko for the bug report)
\item Vex `data valid' period now properly respected
\item Vex clock table tolerance issue corrected
\item Changes in Mark5 mode should be safer (note that currently {\tt vex2difx} never exercises multiple modes in a single job)
\item When making the cross spectrum sniffer plots, respect the reference antenna
\item Improved pulsar polynomial file error checking is performed
\item Amplitude-phase-delay (APD) sniffer plots always have refant first when multiple refants are supplied
\item Project name should now appear on sniffer APD plots
\item Mark5 units now send status information even when no playback is occuring (eliminating the incorrect {\tt LOST} state issue as displayed in the DOI)
\end{enumerate}

\subsubsection{Known problems}

\begin{enumerate}
\item Extensive use in VLBA operations has shown that occasional data dropouts of one or more antenna, sometimes in a quasi-repeatable manner, affect completeness of some jobs.  
It is not clear exactly what the cause is at this point, however its cure is a high priority.
\item Loss of a few FFTs of data will occur in rare circumstances.
\item Clock accountability is poor when jobs containing multiple clock models for antennas are combined.
\end{enumerate}

DiFX~1.5.2 was made available via subversion on Jan 20, 2010.

\subsection{DiFX~1.5.3}

DiFX~1.5.3 is mainly intended as a bug fix update to version~1.5.2, though some new features have made their way into the codebase.
This version of DiFX comes with the following components (and versions): calcif2 (1.3), calcserver (1.3), difx\_db (1.13; NRAO-only), difx2fits (2.6.2), difx2profile (0.2), difxio (2.12.2), difxmessage (7.2), mark5access (1.3.4), mk5daemon (1.3), mpifxcorr (1.5.3), vex2difx (1.0.3), and vis2screen (0.2).
Many changes are motivated by issues found running DiFX full time in Socorro.

The new features include:
\begin{enumerate}
\item Mark5 directory ({\tt .dir}) files can contain {\tt RT} on the top line to indicate the need to play back using {\em Real-Time} mode.
\item {\tt difxqueue} (NRAO only) now takes an optional parameter specifying the staging area to use.
\item New Mark5 diagnostic programs ({\tt vsn} and {\tt testmod}) introduced to wean off the use of the {\tt Mark5A} program.
\item {\tt mk5daemon} can now mount and dismount USB and eSATA disks through mk5commands.
\item {\tt mk5cp} now makes the destination directory if it doesn't exist.
\item {\tt mk5daemon} will now warn if free disk space is getting low.
\item {\tt db2vex} (NRAO only) now allows field station logs to be provided.  As of now, only media VSNs are extracted.
\item Playback off Mark5 units has been made more robust with better error reporting.
\item New utility {\tt m5fold} that can be used to look at repeating signals in baseband data total power (e.g., switched power)
\item {\tt vex2difx} now supports job generation in cases where upper side band was observed at one antenna and lower sideband at another.
\end{enumerate}

\subsubsection{Bugs fixed}

\begin{enumerate}
\item Don't unnecessarily drop any FFTs of data.
\item Sub-integrations longer than one second could cause integer overflows.
\item Fix bug in {\tt vex2difx} where jobs were not split at clock breaks.
\item {\tt difx2fits} was guilty of incorrect clock accoutability after a clock change at a station when merging multiple jobs.  Worked around by not allowing such jobs to merge.
\item {\tt db2vex} (NRAO only) warns when more than one clock value is found for an antenna.
\item Mark5 unit bank switches now routinely call {\tt XLRGetDirectory()} to work around a newly discovered bug in the StreamStor software.
\item A couple possible memory leaks in the mark5access library were fixed (thanks Alexander Neidhardt and Martin Ettl).
\item Lots of compiler warnings quashed (mostly of the ``unused return value'' kind).
\item Olaf Wuchnitz found two FITS file writing problems in {\tt difx2fits} dating back to code inherited from FXCORR!
\item Two more digits are retained for the time and one more digit is retained for amplitude information in the {\tt .apd} and {\tt .apc} sniffer files.
\item Some bugs related to replacement of special characters by ``entities'' in XML messages are fixed.
\item New traps are in place in many places to catch string overruns.
\item Fix for writing {\tt .calc} files with more than one ephemeris driven object.
\item {\tt vex2difx} would get {\em very} slow due to constantly sorting a list of events.  Now this list is only sorted when necessary, drastically speeding it up.
\item The {\tt RCfreqId} parameter in the difxdatastream structure (in difxio) was used with two different meanings that are normally the same.  Cases where they differred caused exceptions.  Fixed in difxio and difx2fits. (Thanks to Randall Wayth for leading to the discovery)
\item {\tt difx2fits} would assign a bogus {\tt .jobmatrix} filename when not running the sniffer.
\item {\tt vex2difx} could get caught in an infinite loop when making jobs where two disk modules had zero time gap.
\item {\tt difx2fits} used a bad config index when making the puslar GM table when multiple configs were present.
\item Within {\tt mpifxcorr} an extra second was added to the validity period for polycos to ensure no gap in coverage.
\item {\tt mk5dir} would add correct the date improperly for Mark4 formats after beginning of 2010.
\item Lots of fixes for building FITS files out of a subset of baseband recorded channels.
\item FITS files now support antennas with differing numbers of quantization bits.
\item Lots of Mac OS/X build issues fixed.
\end{enumerate}

DiFX 1.5.3 was released on April 16, 2010.

\subsection{DiFX~1.5.4}

DiFX~1.5.4 is likely the last 1.5 series formal release of DiFX, though an additional release could be made if demand is there.

The new features include:
\begin{enumerate}
\item {\tt difx2fits} can now produce FITS files with only a subset of the correlated sub-bands.
\item {\tt difx2fits} can be instructed to sniff on an arbitrary timescale.
\item The {\tt makefits} wrapper for {\tt difx2fits} now respects a -B option for phase bin selection.
\item {\tt difxio} has improved checking that prevents merging of jobs with incompatible clocks.
\item {\tt difxio} now maintains a separate clockEpoch parameter for each antenna.
\item {\tt difxStartMessage} now contains DiFX version to run, allowing queued jobs to be run under different DiFX versions.
\item The curses utilities {\tt mk5mon} and {\tt cpumon} now catch exceptions and can be resized without infecting the terminals they are run in.
\item New sub-library called mark5ipc added that provides a semaphore lock for Mark5 units.
\item The {\tt testdifxmessagereceive} utility can now filter on message types.
\item Support for SDK9 throughout (e.g., in {tt mpifxcorr}, {\tt mk5daemon}, and other utilites).
\item Support for new Mark5 module directory formats (Haystack Mark5 memo 81).
\item Several new Mark5 utilities to make up for Mark5A functionality that will not longer be available: {\tt vsn}, {\tt testmod}, {\tt recover}, {\tt m5erase}.
\item {\tt mk5cp} can now copy data based on byte range.
\item Many programs directly talking to the StreamStor card of Mark5 units use WATCHDOG macros for improved diagnostics when problems occur.
\item More protection against incomplete polyco files added to {\tt mpifxcorr} (Note: should add this to {\tt vex2difx} as well).
\item The GUI can now spawn different DiFX versions at will through the use of difxVersion parameter in the DifxStartMessage and wrapper scripts.
\item {\tt difx2fits} can now convert LSB to USB for matching purposed.  When used, all LSB sub-bands must have corresponding USB sub-bands on one or more other antenna.
\item {\tt mark5access}-based utilities (e.g., {\tt mp5spec}) can now read from stdin.
\item New utility {\tt mk5cat} can send data on a Mark5 module to stdout.
\end{enumerate}

\subsubsection{Bugs fixed}

\begin{enumerate}
\item Only alt-az telescopes received the correct model.  Fixed.  Note that CALC and FITS-IDI don't have a good match between their sets of allowed mount types.
\item {\tt difx2fits} now properly propagates quantization bits on a per antenna basis.
\item Logic errors in {\tt difxio} would confuse {\tt difx2fits} in cases where different antennas use different frequency setups.  Fixed.
\item Weights are blanked in {\tt difx2fits} prior to populating each record, preventing screwy weights for unused sub-bands.
\item {\tt vex2difx} would sometimes hang or not converge on job generation.  Fixed.
\item {\tt vex2difx} now doesn't assume source name is same as vex source def identifier.
\item {\tt mpifxcorr} generated corrupted weights and amplitudes when post-FFT fringe rotation was done.  Fixed.
\end{enumerate}

\subsection{Known bugs}

\begin{enumerate}
\item Tweak Integration Time feature of {\tt vex2difx} often does the wrong thing.
\end{enumerate}

DiFX 1.5.4 was released on October 12, 2010.

\subsection{DiFX 2.0.0}

DiFX 2.0.0 is based on an upgraded {\tt mpifxcorr} that breaks {\tt .input} file compatibility with the 1.0 series.
This new version will allow more flexible correlation of mis-matched bands and correlation at multiple phase centers along with general performance improvements.
Development of the 2.0 capabilities will occur in parallel with the 1.0 series features.

\subsubsection{New features}

\begin{enumerate}
\item Pulse cal extraction in {\tt mpifxcorr}.
\item Massive multi-phase center capabilitiy.
\item New utitility {\tt zerocorr} added.
\item External pulse cal extraction utility {\tt m5pcal} added.
\item DiFX output format is all-binary, meaning speed and disk savings
\end{enumerate}

\subsection{Known bugs}

\begin{enumerate}
\item Zoom band support has multiple problems.
\end{enumerate}

DiFX 2.0.0 was released on October 12, 2010.

\subsection{DiFX 2.0.1}

DiFX 2.0.1 is a bug fix and clean-up version in response to numerous improvements to DiFX 2.0.0.
There are a number of new features as well.

\subsubsection{New features}

\begin{enumerate}
\item New utility {\tt checkmpifxcorr} to validate DiFX input files
\item Switched power detection in {\tt mpifxcorr}
\item Early multi-thread VDIF format support
\item RedHat RPM file generation for some packages (can extend to others on request)
\item Improvements to method of selecting which pulse cal tones get propagated to FITS
\item Initial complex sampling support
\item Improved locking mechanism for direct mark5 access (using IPC semaphores; difxmessage)
\end{enumerate}

\subsubsection{Bug fixes}

\begin{enumerate}
\item Fix model accountability bug in difx2fits when combining jobs
\item Numerous fixes for zoom bands (in {\tt mpifxcorr}, {\tt vex2difx} \& {\tt difx2fits})
\item Native Mark5 has improved stability for cranky modules
\item Numerous fixes for DiFX-based phase cal extraction (mostly in {\tt difx2fits}, mostly for multi-job)
\item Fractional bit correction for a portion of lower sideband data got broken in difx 2.0.0. Fixed.
\item Migrate {\tt difxcalculator} to DiFX 2; was not complete for DiFX 2.0.0
\end{enumerate}

DiFX 2.0.1 was released on June 24, 2011.

\subsection{DiFX 2.1}

\subsubsection{New features}

\begin{enumerate}
\item Mark5-based correlation: easy access to S.M.A.R.T.\ data (can be viewed with getsmart)
\item Mark5-based correlation: emit multicast message containing drive statistics after each scan
\item VSIS interface added to mk5daemon
\item Support for non power-of-2 FFT lengths
\item New utilities: {\tt mk5map} (limited functionality), {\tt fileto5c}, {\tt record5c}
\item Remote running of {\tt vex2difx} from {\tt mk5daemon}
\item Multithread VDIF support enabled for the data sources FILE and MODULE, including stripping of non-VDIF packets
\item New features added to existing utilities:
\begin{itemize}
\item {\tt mk5cp}: copy without reference to a module directory
\item {\tt mk5cp}: ability to send data over ssh connection
\item vsn: get SMART data from disk drives
\end{itemize}
\item e-Control source code analysis (Martin Ettl, Wettzell)
\item Restart of correlation is now possible
\item {\tt difx2fits}: -0 option to write minimal number of visibilities to FITS
\item {\tt difx2fits}: write new RAOBS, DECOBS columns in source table
\item tweakIntTime option to {\tt vex2difx} has been re-enabled
\item {\tt diffDiFX.py} can now cope with two files that don't have exactly the same visibilities (i.e., some visibilities are missing from one file)
\item {\tt plotDiFX.py} and {\tt plotDynamicSpectrum.py} now have better plotting and more options
\item New FAKE datastream type for performance testing
\item Espresso, a lightweight system for managing disk-based correlation, has been added to the DiFX repository. 
\item Option to correlate only one polarization has been added.
\item {\tt mk5dir} can now produce {\tt .dir} file information for VDIF formatted data.
\item Add NRAO's sniffer plotters to the repository.
\end{enumerate}

\subsubsection{Bug fixes}

\begin{enumerate}
\item LBA format data now scaled roughly correctly (removing the need for large ACCOR corrections).
\item There was a bug when xmaclength was $>$ nfftchan for pulsar processing. This has been corrected.
\item guardns was incorrectly (overzealously) calculated in mpifxcorr.
\item {\tt Mk5DataStream::calculateControlParams: bufferindex>=bufferbytes} bug fixed.
\item Low weight reads could result in uninitialized memory; fixed.
\item Streamstor {\tt XLRRead()} bug work-around installed several places (read at position 0 before reading at position $> 0$). This is thought not to be needed with Conduant SDK 9.2 but the work-around has no performance impact.
\item Fix to pulse calibration data ordering for LSB or reordered channels.
\item Pulse cal amplitude now divided by pulse cal averaging time in seconds.
\item Pulse cal system would cause crash if no tones in narrow channel. Fixed.
\item Zoom band support across mixed bandwidths (see caveat below).
\item Fix for spurious weights at end of jobs (untested\ldots)
\item Mixed 1 and 2 bit data are handled more cleanly
\item mpifxcorr terminates correctly for all short jobs.  Previously it hung for jobs with a number of subints between nCores and $4 \times$ nCores
\item Correctly scale cross-correlation amplitudes for pulsar binning when using {\tt TSYS} $>0$ (accounts for varying number of samples per bin c.f.\ nominal)
\item Lower side-band pulse cal tones had sign error.  Fixed.
\end{enumerate}


DiFX 2.1 was released on May 25, 2012.


\subsection{DiFX 2.1.1}

DiFX 2.1.1 was a minor patch release to fix a scaling issue with autocorrelations of LBA-format data in mpifxcorr.

DiFX 2.1.1 was committed as a patch to DiFX 2.1 on June 7, 2012.

\subsection{DiFX 2.2}

\subsubsection{New features}

\begin{enumerate}
\item {\tt calcif2}: ability to estimate delay polynomial interpolarion errors
\item Support for a ``label'' identifier for a local version of DiFX that will help discriminate exact version used.
\item Faster Mark5 directory reading
\item Faster VDIF corner turning through customized bit shifting functions
\item {\tt mpifxcorr} can now be built without Intel Integrated Performance Primitives, though resulting in a slower correlator.
\item {\tt vdifio}: several new VDIF manipulation and processing utilities added: {\tt vmux}, {\tt vsum}, {\tt vdifd}, {\tt vdifspec}, {\tt vdiffold}, {\tt vdifbstate}
\item {\tt difxbuild}: a new installation program
\item {\tt difxspeed}: a program to benchmark and help optimize DiFX
\end{enumerate}

\subsubsection{Bug fixes}

\begin{enumerate}

\item Mutex locking bugfix for very short jobs
\item Prevent MODE errors when a datastream runs out of data well before the end of a job
\item {\tt calcif2}: fix azimuth polynomial generation in case of wrap
\item Fix for FITS file generation for mixed sideband correlation
\item {\tt difx2fits} now uses appropriate gain tables for S and X band in S/X experiments (Thanks to James Miller-Jones for reporting)
\item {\tt difx2fits}: correct pcal, weather, tsys and flag data for observations crossing new year
\item Fixed scaling of autocorrelations for LBA format data
\item 0.5 ns wobble in delays for 2 Gbps Mark5B data fixed
\item Fix bug preventing subintegrations longer than 1 second. Now 2 seconds is allowed (this limit comes from signed integer number of nanoseconds).
\item Weights corrected in cases where two setups differening only by pcal setup were correlated against each other
\item Quashed data and weight echos that would occur for about 1 integration at the beginning of each scan for datstreams that ran out of data before end of job.
\item The multicast (diagnostic) weights were low or zero in case of frequency selection (zoom band or freqId selection). Fixed.
\item {\tt mark5access}: fix (non)blocking issue when receiving data from {\em stdin}


\end{enumerate}


DiFX 2.2 was released on June 12, 2013.



\subsection{DiFX 2.3}

\subsubsection{New features}

\begin{enumerate}

\item mpifxcorr: LO offsets are now corrected in the time domain when fringe rotation is also done in the time domain (the usual mode), allowing considerably larger LO offsets without decorrelation
\item mpifxcorr: Working polarization dependent delay and phase offsets
\item mpifxcorr: Experimental linear2circular conversion
\item mpifxcorr: Complex Double sideband (RDBE/Xcube) sampling support (Note: things are not perfect here; wait for 2.4 for real use)
\item mpifxcorr: new file/Mark5 based VDIF/Mark5b datastream (faster and more robust)
\item mpifxcorr: implement work-around for buggy kernel-driver combinations; Mark5 read sizes >20 MB now allowed
\item utilities: some new command line tools for Mark5B and VDIF files (vsum, mk5bsum, vmux, mk5bfix)
\item new options for passing calibration (Walter B: memo forthcoming)
\item Hops updated to version 3.9

\end{enumerate}

\subsubsection{Bug fixes}

\begin{enumerate}

\item mpifxcorr: Datasteam buffer send size now calculated correctly for complex sampled data
\item mpifxcorr: Avoid very rare bug where combination of geometric delay and data commencing mid-subint meant one invalid FFT might be computed
\item mpifxcorr: multicast weights are now computed correctly for mixed-sideband correlation
\item mpifxcorr: fixed bug where some autocorrelations were not saved in a mixed-sideband correlation
\item mpifxcorr: fixed bug where send size could be computed incorrectly by 1-2 bytes for
\item Mark4/VLBA/Mark5B/VDIF formats, potentially resulting in very small amounts of data loss

\end{enumerate}

DiFX 2.3 was released on January 18, 2014.


\subsection{DiFX 2.4}

\subsubsection{New Features}

\begin{enumerate}

\item mpifxcorr
\begin{itemize}
\item Support a FAKE correlation mode for multi-threaded VDIF.
\item The mpifxcorr produced PCAL files have had a format change that allows unambiguous interpretation across all use cases.
\item Add network support (TCP, UDP and Raw Ethernet) for multi-threaded VDIF:
1.\  TCP and UDP variants not tested yet;
2.\  raw Ethernet variant is used for the VLITE project.
\item Support updated Mark5 module directories.
\item Better checking that Mark5 data being processed matches what is expected.
\item Improved Mark5B decoding:
1.\ Mark5B data streams are now filtered for extra or missing data;
2.\ packets with invalid bit (actually the TVG bit) set replace missing data;
3.\ this means any valid Mark5B data with the TVG bit set will not correlate.
\item Information about each Mark5 unit used in ``native mode'' is emitted at start of jobs so it can be logged.
\item Ultra-low frame rate VDIF data was affected by allowing a long ``sort window'' in the VDIF multiplexer.  
This has been reduced to 32 frames and seems to work fine for all bandwidths now.
\end{itemize}

\item difx2fits
\begin{itemize}
\item Slightly improved compliance with the FITS-IDI convention:
1.\ invalid Tsys values become NaN, not 999;
2.\ populate {\tt DELTAT} keyword in ModelComps table.
\end{itemize}

\item difxio
\begin{itemize}
\item Support for X/Y polarization correlation.  Many fundamental issues with       
  linear polarization remain though: 
1.\ this does not support in a meaningful way Linear*Circular correlations;
2.\ there is a terminology gap in many bits of software and file formats that confuses X/Y with H/V polarization bases;
3.\ the intent of this support is for short baselines (VLITE).
\end{itemize}

\item mark5access
\begin{itemize}
\item Support for ``d2k'' mode in Mark5B format (swapped sign and mag bits).
\item fixmark5b() function fixed for case that fill pattern is seen at the 1 second transition.
\item Make use of the TVG bit as an ``invalid frame'' indicator for Mark5B data.
\item m5bstate: support complex sampled data
\end{itemize}

\item mk5daemon
\begin{itemize}
\item New utility mk5putdir: reads a binary file and replaces a Mark5 directory with it.
\item mk5dir: when reading the directories, saves a copy of the binary representation in case it is needed later (perhaps via mk5putdir)
\item Reworked mark5 module directory support, including support for many new variants of the directory format.
\item mk5erase will save a ``conditioning report'' to {\tt \$MARK5\_CONDITION\_PATH} if that environment variable is set.
\end{itemize}

\item vdifio
\begin{itemize}
\item Fairly large change to the API.  Please read the ChangeLog for details.
\end{itemize}

\item vex2difx
\begin{itemize}
\item Respect the record enable bit in the SCHED block.  If that value is 0 no correlation will be attempted for that antenna.
\item Bug fixes preventing some LSB/zoom bands from being correlated.
\item Complex data type and number of bits are now read from vex file.
\end{itemize}

\end{enumerate}

\subsubsection{Bug fixes}

\begin{enumerate}

\item A ``jitter'' of 0.5~ns when using 2Gbps Mark5B format was fixed.  A fix was back-ported to DiFX 2.3.
\item A similar jitter was corrected for high frame rate VDIF (problem identified by the VLITE project)
\item Fix case of intermittant fringes that was due to incorrect assumption about the sizeof(unsigned long): 32 bits on a 32-bit system vs. 64 bits on a 64-bit system.
Some other variable types were changed for long term type safety
\item Fix off-by-one in correlation using LSB and zoom bands together.

\end{enumerate}

\subsection{DiFX 2.5.1}

\subsubsection{New features}

\begin{enumerate}

\item Innitial support for correlating Mark6. This is still much a work in progress.
\item Multiple datastreams per antenna supported via {\tt vex2difx}
\item New delay model program: difxcalc11.? No longer requires calcserver.
\item Support for more than 6 days of EOP values.
\item ``Union mode'' in difx2fits allows merging of correlation output that uses different setups. Some restrictions apply. Designed for GMVA and RadioAstron use.
\item Improved VDIF support: wider range of bits/threads, support for multi-channel, multi-thread VDIF, support for complex multi-thread VDIF
\item Support for new VDIF Extended Data Version 4 which is useful for multiplexed VDIF data. See: \url{http://vlbi.org/vdif/docs/edv4description.pdf}
\item Python bindings for vdifio and mark5access
\item mpifxcorr: per-thread weights implemented
\item Automatic selection of arraystride by mpifxcorr if set to zero; this is done per-datastream.? Very useful for correlation of ALMA data or others with non-standard sample rates.
\item Automatic selection of xmacstride by mpifxcorr if set to zero
\item Automatic selection of guardns by mpifxcorr if set to zero
\item mpifxcorr can now operate with unicast messages instead of multicast. Useful in some situations where multicast is not
supported.
\item New ``dirlist'' module/file directory listing format. 
\item {\tt mk5cp} append mode to resume interrupted copy
\item ALMA support in HOPS: non-power-of-two FFTs, up to 64 freq.\ channels, full linear/circular/mixed polarization support
\item HOPS improvemetns for VGOS through improved manual phase cal support
\item New package: polconvert. Used to post-correlation convert from linear to circular polariations
\item New package: autozoom. Helps a user develop {\tt .v2d} file content when setting up complicated zoom band configurations.
\item New package: datasim: generate baseband data suitable for simulated correlation
\item Improved error reporting in many places

\end{enumerate}

\subsubsection{Bug fixes}

\begin{enumerate}

\item fix for incorrect reporting of memory use by mpifxcorr (needed longer int sizes)
\item dataweights would sometimes be incorrect after abrupt ending of data from a datastream.
\item FITS-IDI files produced by difx2fits more standards compliant; fix problem that caused AIPS task VBGLU to fail.
\item Several segfaults across a number of programs/utils now are caught and provide useful feedback. 

\end{enumerate}

\subsubsection{Caveats}

\begin{enumerate}

\item Various changes made between DiFX 2.4 and 2.5 are not API-compatible. Please don't mix packages from these two releases.  If you have non-DiFX software that links against the DiFX libraries, be sure to recompile them. A small number of changes may result in need to restructure such code.
\item Unlike previous DiFX releases, each tagged version will be its own SVN copy. If the number of minor releases within the 2.5 series gets large, some (reversible) pruning of the SVN repository may occur.  There has been some debate about the best tagging strategy: bring any strong opinions to the Bologna meeting, where further changes to release and tagging policies can be discussed if needed.

\end{enumerate}


\subsection{DiFX 2.5.2}

\subsubsection{Bug fixes}

\begin{enumerate}

\item Fixes for the HOPS 'rootid' rollover.  The new rootcode is a conventional base-36 timestamp in seconds from the start of the new epoch (zzzzzz in the old epoch).  This will last until 2087.
\item Major fixes/improvements to PolConvert for use with ALMA by the EHTC and GMVA.

\end{enumerate}


\subsection{DiFX 2.5.3}

\subsubsection{Updates}

\begin{enumerate}

\item genmachines
\begin{itemize}
  \item r8264 genmachines and mark6 datastream updates
  \item r8357 add mark6 activity message to mark6 datastream
  \item r8409 allow multiple nodes to serve as datastream nodes for FILE based-data in the same location
\end{itemize}
\item hops 3.19 new features
\begin{itemize}
  \item increased the number of allowed frequency ``notches'' to ridiculous levels
  \item an “ad hoc” data flagging capability to allow improved time / channel data selection for fringing
  \item a capability to dump all the information on the fringe plots into ascii files for “roll your own” plotting
  \item removed obsolete {\tt max\_parity}
  \item introduced {\tt min\_weight} (to discard APs with very little correlated data in support)
  \item vex2xml, a program that converts VEX (v1.5) into XML to allow easy parsing via standard XML parsers.
  \item added {\tt type\_222} to save control file contents, enabled by keyword {\tt gen\_cf\_record}.
\end{itemize}
    polconvert to v1.7.5 (mostly minor bug fixes and robustifications)
\end{enumerate}

DiFX 2.5.3 was released on March 28, 2019.

\subsection{DiFX 2.5.4}

\subsubsection{New features}

\begin{enumerate}
\item HOPS updated to 3.22
\item Vex2difx and difxio
\begin{itemize}
  \item permit H and V polarization labels (but need trunk difx2fits if these are to be propagated into “enhanced” FITS-IDI)
  \item support the v2d parameter 'exhaustiveAutocorrs'
  \item support Mark6 MSNs i.e. MSNs that contain '\%'
\end{itemize}
\item Difx2mark4
\begin{itemize}
  \item support PCal data from multi-datastream correlation
  \item option '-e expt\_nr' additionally propagates the experiment number into the generated root files
  \item backported the options -w for mixed-bandwidth data, and -g for filtering freq groups
\end{itemize}
\item Genmachines updated to support Mark6 host auto-detection
\item Includes copies of more recent utilities: packHops.py, distFourfit.py, fplot2pdf, plotResiduals.py
\end{enumerate}

\subsubsection{Bug fixes}

\begin{enumerate}
\item Mark6 native playback fixed to support other than VLBA VDIF recording
\item Vdifio
\begin{itemize}
  \item fix excessively long station name printout in printVDIFheader
  \item fix to mk6gather to not crop a scan
  \item resync code extended with further VDIF frame sizes that are common in VGOS
\end{itemize}
\item Mpifxcorr fix for complex data (VDIFC) PCal extraction for bands with other than 16 tones
\item Mpifxcorr fix for IVS-related 5/10MHz pcal issue specific to those channels where first pcal is at baseband 2.01 MHz
\item Difx2mark4
\begin{itemize}
  \item fix count of total bands and polarizations for multi datastream datasets, solving an issue in HOPS post-processing
  \item fix parallactic angle calculation error in first t303 record
  \item fix to permit Tab characters in VEX
\end{itemize}
\item Mpifxcorr no longer segfaults after ``exiting gracefully'' message upon missing data files
\item Tkinter add-on package renamed from tkinter to kinter\_difx to avoid name collision
\item Install-difx now works under Python 3 and supports the option –withmark6meta
\item Startdifx no longer piles up difxlog processes
\end{enumerate}

\subsubsection{Caveats}

Support for Complex VDIF is incomplete in 2.5.3 and 2.5.4, both treat Complex VDIF LSB as USB data. This handling was retained in 2.5.4
for VGOS compatibility reasons in order to have all-LSB(-mislabeled) data products, and avoid postprocessing issues with
mixed USB and LSB data products. One should not correlate any actual Complex VDIF LSB recordings - this produces no fringes.
The issue affects only VDIFC (complex VDIF), not VDIF nor other formats.

DiFX 2.5.4 was released on August 27, 2021.


\subsection{DiFX 2.5.5}

\subsubsection{New features}

\begin{enumerate}
\item genmachines extended to support Mark6 with multiple expansion chassis
\item update polconvert scripts
\end{enumerate}

\subsubsection{Bug fixes}

\begin{enumerate}
\item genmachines fix to support December i.e. doy 335 and later
\item vex2difx
\begin{itemize}
  \item fix freqClockOffs and loOffsets parameters to not expect more values than recorded frequencies
  \item removed obsolete warning about 10 MHz PCal not being supported
\end{itemize}
\end{enumerate}

DiFX 2.5.5 was released on October 24, 2022.

\subsection{DiFX 2.6.1}

\subsubsection{New features}

\begin{enumerate}
\item Improved VDIF support
\begin{itemize}
  \item Increased robustness in processing VDIF data with many gaps
  \item Improvements in processing VDIF with frame sizes very different from 5000 bytes
  \item  New in-line reordering functionality via vdifreader…() functions; allows operation on more highly skewed VDIF files
\end{itemize}
\item mpifxcorr {\tt .input}, {\tt .calc}, {\tt .threads}, and pulsar files are now only read by the head node
\item mpifxcorr can be provided a new stop time via a DifxParameter message; results in clean shutdown at that time.
\item mpifxcorr can extract pulse cals with tone spacing smaller than 1 MHz
\item Support for Intel Performance Primitives version > 9 (specfically IPP 2018 and 2019)
\begin{itemize}
  \item These newer IPP versions are more readily available than earlier versions
\end{itemize}
\item Improved support for Mark6 playback
\begin{itemize}
  \item Mark6 activity messages in difxmessage
  \item Support in genmachines with updated mk5daemon
  \item Support playback of Mark5B data on Mark6
  \item New and improved mark6 utilities
\end{itemize}
\item difx2fits: populate antenna diameters and mount types for antennas known to the difxio antenna database
\item difx2fits: in verbose mode, explain why files are being split
\item difx2fits: new options for merging correlator jobs run with different clock models
\item vex2difx: new parameter {\tt exhaustiveAutocorrs} can be used to generate cross-hand autocorrelations even when the two polarizations for an antenna come from different datastreams
\item difx2mark4: support multiple bandwidths in one pass
\item hops: to rev 3.19 (see notes on 2.5.3 above for details on several new and useful features)
\item polconvert: to rev 1.7.5 (see notes on 2.5.3 above for details)
\end{enumerate}

\subsubsection{Bug fixes}

\begin{enumerate}
\item mpifxcorr: Retry on NFS open errors of kind: ``EAGAIN Resource temporarily unavailable''
\item mpifxcorr: Fix weight issue when the parameter {\tt nBufferedFFTs} $ > 1$
\item startdifx/genmachines: Fixes for cases when multiple input files are provided
\item Python 2 scripts now explicitly call python2
\item vex2difx: allow up to 32 IFs (was 4) and warn when this is exceeded
\item vex2difx: support units in the clock rate (e.g., usec/sec); in general support time in the numerators.
\item Sun RPC is on its way out; support for ``tirpc'' added to calcif2 and calcserver
\end{enumerate}

\subsubsection{Caveats}

\begin{enumerate}
\item Moved “mark6gather” functions from vdifio to mark6sg; this changes the order of dependencies!
\item Various changes made between DiFX 2.5 and 2.6 are not API-compatible. Please don't mix packages from these two releases. If you have non-DiFX software that links against the DiFX libraries, be sure to recompile them. A small number of changes may result in need to restructure such code.
\item There is some suspicion that correlation of very narrow bandwidth VDIF modes on Mark6 media can result in premature termination of datastreams.
\item The {\tt .threads} file must now exist; previously (before the change to only have manager read these files), a missing {\tt .threads} file would cause each core process instance to have a single thread.
\item difx\_monitor won't compile with IPP $\ge 9$
\end{enumerate}

DiFX 2.6.1 was released on August 28, 2019.


\subsection{DiFX~2.6.2}

\subsubsection{New features}
\begin{enumerate}
\item Python parseDiFX package added
\end{enumerate}

\subsubsection{Updates}
\begin{enumerate}
\item HOPS updated to version 3.21
\item PolConvert updated to version 1.7.8
\item Former FTP access to CDDIS servers changed to FTP-SSL (geteop.pl)
\end{enumerate}

\subsubsection{Bug fixes}
\begin{enumerate}
\item mpifxcorr: Fix correlation of Complex LSB data, restore fringes. Note: DiFX 2.5.x and 2.6.1 treated Complex LSB as if Complex USB, while Trunk prior to r9647 05aug2020 treated LSB nearly correctly except for a off-by-one channel bug
\item difx2mark4: Fix seg-fault in createType3s.c when a station has only a single entry in the PCAL file
\item difx2mark4: Remove unneeded debugging statement (calling d2m4\_pcal\_dump\_record())
\item difx2mark4: Update createType3s.c to add support for DiFX PCAL files generated from station data where each data-stream thread resides in a separate file (multi-datastream support). This separates the code reading the PCAL files from the code filling the type-3 records so that tone records from multiple data streams can be merged before populating the type-309s
\item difx2mark4: Update createType3s.c to remove support for DiFX version-0 PCAL files
\item difx2mark4: Add support for 10 MHz p-cal tone spacing (needed by VGOS at Yebes)
\item difx2mark4: Significantly increase hardcoded array sizes (difx2mark4.h: NVRMAX 8M, MAX\_FPPAIRS 10k, MAX\_DFRQ 800) as required for EHT2018
\item difx2mark4: Fix a bounds check, permit tabs in VEX file
\item difx2fits: Fix FITS PH table having missing or superfluous pcal records when one correlates multi-datastream antennas, or not all recorded frequencies, or multiple zooms per recorded frequency
\item mark6gather: Fix poor weights in native Mark6 correlation for VDIF frame sizes not equal to 5032 bytes
\item difxio: Fix PCal tone frequency rounding bug on some platforms
\item difxio: Cope with recorded bands that lack PCal tones, e.g., 200 MHz PCal spacing of KVN with say 32 MHz recorded bands
\item calc11: Dave Gordon provided ocean loading params at EHT stations
\item calc11: Increased the number of field rows supported in .calc files
\item vex2difx: Fix internal merge of SamplingType (real, complex) when info found in VEX and/or v2d file
\item Minor changes to oms2v2d and vexpeek
\item More IPP versions supported
\item Minor issues with vis2screen fixed
\item Fixed build failure with gcc defaults
\item Python3 support in many/most places
\end{enumerate}

\subsubsection{Caveats}
\begin{enumerate}
\item difx2mark4: Some LSB-LSB baselines do not get converted in mixed-sideband correlation setups (DiFX 2.6.1, 2.6.2); if affected, use difx2mark4 2.5.3 with –override-version. A bugfix is pending for DiFX 2.6.3 later this year.
\item difx2mark4: Performance regression with p-cal files, conversion of p-cal data may take noticeably longer than before
\item calcserver and difxcalc11: With the latest versions of gfortran (10.1 or newer) you will need to uncomment the line with -fallow-argument-mismatch line in the environment setup in order to compile. Users who do this should be alert to possible issues.
\end{enumerate}

DiFX 2.6.2 was released on September 11, 2020.


\subsection{DiFX~2.6.3}

This version has never been officially released.

\subsection{DiFX~2.7.1}

This version has never been officially released.
The 2.7 series was used by the Event Horizon Telescope.
The 2.7 series is a feature branch derived from DiFX~2.6.2 with support added for outputbands.


\subsection{DiFX~2.8.1}

The DiFX~2.8 series collected updates and fixes from many DiFX users and is the first version universally usable on VGOS, EHT, and ``traditional'' VLBI data.
Because the 2.7 series was never formally released, the notes below include all changes since the 2.6 series.

\subsubsection{New features}
\begin{enumerate}
\item Support for appending contiguous subbands together to create outputbands
\item Initial support for vex2
\item Complete conversion to Python 3
\item Support for CODIF format
\item mark5access programs: error output to stderr to allow piping
\item Support for IPP 2019 series
\item Experimental support for Vienna Mapping Functions in calcif2
\item Some features (within mpifxcorr and vex2difx) that provide additional options for real-time correlation
\item Espresso modification to work with singularity (or docker) image
\end{enumerate}

\subsubsection{Updates}
\begin{enumerate}
\item HOPS updated to version 3.24
\item PolConvert updated to version 2.0.3
\item Several new VDIF decoders and corner turners introduced to widen range of support
\item Improved support for more than 2 bits per sample (see \url{https://library.nrao.edu/public/memos/vlba/up/VLBASU_52.pdf})
\item Mark6: support for larger number of expansion units
\item Add a few more stations to ocean loading tables within difxcalc
\item Several new options in the tabulatedelays program
\item difxio programs (e.g., vex2difx and difx2fits) can support project codes up to 24 characters long (was 7)
\end{enumerate}

\subsubsection{Bug fixes}
\begin{enumerate}
\item difx2mark4: fix a parallactic angle calculation bug
\item vdifmux() function had some logic errors causing bad performance in gappy data; fixed.
\item In subarray cases difx2fits could provide incorrect pulse cal values; fixed.
\item mark5access bug fix to prevent crash
\item mpifxcorr: fix bug affecting data weights when nBufferedFFTs $> 1$ and datastreams weight $< 1$
\item mpifxcorr could end early due to bug in receiving a DifxParameter message; fixed.
\item some of the vdif python utilities (e.g., vdifd) had errors in parsing the command line: nbit and offset were swapped
\end{enumerate}

\subsubsection{Caveats}
\begin{enumerate}
\item Outputbands support can only work on one frequency setup at a time.
\item The DiFX-2.8 series may be the last to support Mark5 recordings. 
\end{enumerate}

Many additional non-user-visible improvements were made to the code as well as many small user-visible improvements that do not warrant specific mention.
The {\tt Changelog} files that are packaged with most of the DiFX software modules contain more detailed lists of code changes.
DiFX 2.8.1 was released on XXX 2023.
A wide array of testing has been done and this version is considered ready to be adopted by all DiFX users.


%%%%  Future releases:
%
%  \section{DiFX <version>}
%
%  \subsection{New features}
%     \begin{enumerate}
%     \item <Description>
%     \end{enumerate}
% 
%  \subsection{Bug fixes}
%     \begin{enumerate}
%     \item <component>: <Description of fix>
%     \end{enumerate}
%  \subsection{Caveats}
%     \begin{enumerate}
%     \item <Description of caveat>
%     \end{enumerate}
%
%  \section{Features left to implement}
%
% DiFX <version> was released on <Month Day, Year>.
%
%%%%



\section{Features left to implement}

Here is a list of other features to add to DiFX that are not directly tied to any particular version:
\begin{enumerate}
\item Support for K5 format
\item Pulsar bins with proper output format
\item Space VLBI support
\end{enumerate}


\subsection{DiFX and AIPS}

Only one task in AIPS, {\tt FITLD}, has to deal with the telescope/correlator specific aspect of the FITS-IDI files that the VLBA correlator and DiFX generate.
The FITS-IDI variant of FITS was first documented in AIPS Memo 102 \cite{aips102}, and more recently in AIPS Memo 114 \cite{aips114}, which will be generally available shortly.
It has been modified for better support support of DiFX FITS output.
In general, these changes make {\tt FITLD} less telescope specific so the resulting FITS-IDI files from any DiFX installation should be highly compatible with AIPS.
Several changes have been made to the 31DEC08 AIPS as a result of DiFX testing:
\begin{enumerate}
\item Correction for digital {\it saturation} in auto-correlations is disabled for DiFX FITS files.  See \cite{sci12} for some details on this correction which is not needed for DiFX data.
\item Support for FITS-IDI files greater than 2~GiB in size.
\item Weather table was not populated properly.
\item FITS files with multiple UV tables would generate incomplete GEODELAY columns in CL tables (not relevant to DiFX).
\end{enumerate}
It is recommended that your AIPS installation be kept up to date.

With the following exceptions, data reduction of DiFX correlated data should be identical to that of VLBA hardware correlator data.
This includes the continued use of {\tt DIGICOR=1} in {\tt FITLD} and the use of {\tt ACCOR} as you would have for the hardware correlator.
The exceptions are:
\begin{enumerate}
\item Use of {\tt FXPOL} to correct data ordering in the case of {\em half} polar (e.g., {\tt RR} and {\tt LL} products) is no longer needed.
\item Use of {\tt VBGLU} to concatenate data sets in the case of 512~Mbps observations is no longer needed.
\item Data is usually combined into a single FITS-IDI file with proper calibration data attached, usually implying that {\tt TBMRG} is not needed to properly concattenate calibration data.  This makes DiFX FITS-IDI data similar to the {\em pipeline-processed} VLBA data that was made available to users of the hardware correlator with the difference being that the original FITS-IDI format is retained, keeping file sizes typically 25\% smaller. 
\end{enumerate}
These changes should make data processing easier in almost all circumstances.
