Version 3.1.0 ------------- * Fix for datastream to antenna mapping when not sorting antennas * oms2v2d: add mode to generate .v2d file that is useful for use with vdifsim * oms2v2d: do not require DIFX_BASE to be set * Fix stale pointer to source after manipulating multi-phase-center bits Version 3.0.0 ------------- * Version bump prior to DiFX 2.8 branching * vexpeek: --scanfreqs option Version 2.99.3 ~~~~~~~~~~~~~~ * Merge of vex2 branch into trunk, Jan 27, 2022, from revision 10279 of https://svn.atnf.csiro.au/difx/applications/vex2difx/branches/vex2 * vex2: parse equip, equip_set, and equip_info from $DAS * add some sample vex2 files for the VLBA (other groups encouraged to add vex2 samples as well!) * add option to oms2v2d to override a station format from external file * introduce feature (a bit of a hack) to allow correlation of recorded interlaced VDIF data with non 2^n channels * in cases where it is known (e.g., vex2-based files) and the same across antennas, populate the "RX NAME" field in the .input file * increase maximum project code length to 24 (from 7) * Support for output band configuration * Note: fanout_def is now required for every data format, even VDIF. See https://www.atnf.csiro.au/vlbi/dokuwiki/doku.php/difx/formatdef for more info. Version 2.99.2 ~~~~~~~~~~~~~~ * Support Nasmyth antenna mounts * Propagate nBit from track_frame_format * Modernize configure.ac a bit Version 2.99.1 ~~~~~~~~~~~~~ * vex2 branch created on Aug 2, 2021 from rev. 10089 * Updated Goddard/NVI Vex parser to version supporting vex2 * Significant updates to vexdatamodel/ source to better match vex2 * General trend to move more of the .v2d / .vex merge to applycorrparams * vex2difx.cpp getting streamlined a bit * Break up some large functions into smaller units * vex2 concepts that are now supported in vex2difx (that were not previously supported) - $BITSTREAMS - $DATASTREAM - $CLOCK with up to 4 terms - $SCHED blocks specifying multiple correlation centers - $SOURCE entities that are non-sidereal - $EXPER section with separately-specified experiment code/segment * As of Oct 27, 2021 (ver 2.99.1) sufficient testing has been performed successfully to designate this SVN snapshot as a beta version, leading up to a fully vex2 supported ver 3.0. Version 2.7.0 ~~~~~~~~~~~~~ * Post DiFX-2.6 * vexpeek: add some new options for combining kinds of output * more explicitly catch cases where the number of provided channels is less than the number of recorded channels for a given format. * difxspeed: allow "doPolar" parameter to be set * python3 conversion: difxspeed, mk62v2d, oms2v2d, vlog * remove padCalcScans script * Add a new match type for file formats CODIF and CODIFD * Added support for CODIF format in vex files (at least, in the vex2difx parser) * Don't quit if unused antenna's filelist is absent * In ANTENNA {} now can set ds_filelist to a list of filelists, each one creating its own DATASTREAM * oms2v2d: some new functionality to assist VLBA eVLBI * vex2difx: support override of data frame size with frameSize parameter of DATASTREAM * vexpeek: new verbose mode * oms2v2d: fix crash in some modes if no data for project is found * vexpeek: option to view station coordinates * oms2v2d: use "difxcalc" for VLBA observing with sched 11.6 and newer * Support of polarization labels H and V is added. (Commit 9636 2020.07.30) * Function isSX() added -- true if all modes in a file are pure S/X * Add option to vexpeek to print list of sources - List is indexed by "vex def" name. Assigned source name(s), if present, will be listed after a colon on the source line. - RA and Dec listed in radians as second and third columns. * vexpeek: extend "doTime" option to apply to scan lists * oms2v2d: some improvements for special cases, don't print unneeded zeros * oms2v2d: add "profile" mode * oms2v2d: some improvements for real-time correlation * Restore capability to override a non-VDIF format with a VDIF format * Implement "threadsAbsent" DATASTREAM parameter to specify VDIF threads that were identified in the .vex file but not in the recorded data * Implement "threadsIgnore" DATASTREAM parameter to specify VDIF threads that are present but should not be correlated * Add vex2v2d -- a tool to create a skeleton (and possibly final) .v2d file based on the contents of a .vex file * vexpeek: new scan summary mode: --scans2; add --scans2 twice to command line for nChan, nBit, Bandwidth info * oms2v2d: force max length to be 3200 (sec) * vex2difx: fix bug preventing "canonical VDIF" setups from using other than 2 bits per sample * Fix: Constructor of VexStream: initialize format() Version 2.6.0 ~~~~~~~~~~~~~ * New DATASTREAM parameter tSys that passes through into .input files * vexpeek: - new --scans option produces a listing of scans in the VEX file, useful to help select suitable scans for clock searches in early correlation - new --modules option produces listing of media in the TAPELOG_OBS section * Improved parsing of P-cal $IF if_def entries (vex1.5) - parse P-cal base frequency; allows a shifted P-cal - finer P-cal spacing; 'float' replaces 'int' in vex2difx and .input - Note: P-cal base frequency is not yet supported in .input file * ANTENNA parameter polConvert can be set to true to indicate desire to perform polarization basis conversion on this antenna. This will eliminate a warning that is not needed in this case. * Added parameter exhaustiveAutocorrs to allow correlation of cross-hand autocorrelations when polarisations are in different datastreams * Support units in the clock rate (e.g., usec/sec). This change generalizes unit support to allow time in numerators. * Increase maximum number of allowed IFs from 4 to 32, and catch cases where this gets exceeded. * Add mk62v2d script and call from vex2script to support mark6 prework to .v2d * Fix indexing bug in baseline table preparation. Affected zoom band projects where only a subset of antennas is multi-datastream. * Python scripts explicitly call python2 * difxspeed: allow benchmarking on real data * remove calc2xyz: was on a dead-end development path * Version for DiFX-2.6, Mar 4, 2019 Version 2.5.0 ~~~~~~~~~~~~~ * Users can now supply comments in one or more COMMENT { } blocks * In the ANTENNA { } blocks, one can specify machine = to force a datastream process to be run on a particular machine for a particular antenna. * NOTE: the global "machines" parameter must be set, from which missing antenna machine assignments and the usual head / core processor assignments will be drawn. * outPath global parameter added: all .difx/ output gets put here rather than ./ * Split vex2difx.cpp into smaller pieces (in progress) * If baseband data files are provided via file lists and some jobs have missing data, exlude antennas as necessary and write a "nodata" file * Allow phase cal interval of 200 MHz (KVN) without warning since extraction works * Fix guardNS calculation (finally!)... - Needed to consider bits per sample and number of baseband channels - Swiftly moving sources and/or antennas could still trigger this problem * leap second file now optional for spice kernel evaluation. - If not provided, the EOPs from within the DifxInput structure will be used * Major restructuring of code to facilitate multiple datastream support - vex2 support is being anticipated - Break out each vex table structure into its own .h/.cpp file pair - Vex tables remain const except through access from VexData structure - Load vex file without reference to the CorrParams structure (.v2d file) - Data (files, modules, network info) are tied to a VexAntenna structure - Multiple data streams (VexStream objects) are referenced from setups * Don't export the vex library anymore. It causes some trouble * Some other configure.ac / Makefile.am cleanups * If EXPER section is missing or incomplete, let user know and if forced, make assumptions. * If vex file does not specify a record channel ordering (e.g., through TRACKS), an ordering is determined via sorting channels by their names * Improve uniformity of classification of messages to users. This is how they should be chosen: - Note : Indication that some (usually safe) assumptions are being made because of incomplete information. - Warning : Indication that something doesn't seem right. Will proceed if forced. - Error : A problem with input data that cannot be ignored. - Developer error : A case where the program detected internal self inconsistency. This implies a bug in the code. * Support mark6 media * Allow guardNS = 0 (default) --> mpifxcorr choses the smallest reasonable value * Allow strideLength = 0 (default) --> mpifxcorr chooses a pretty good value for you (per datastream) * Allow xmacLength = 0 (default) -> mpifxcorr chooses a pretty good value for you * Don't allow changing from or to VDIF format with the .v2d file since channel order is defined differently for these two types. * Stations defaulting to use of Canonical VDIF threads changed to VLBA antennas, VLA, and GBT only (Arecibo and Effelsberg removed). Can be changed through environment variable CANONICAL_VDIF_USERS * Move dataSource into VexStream object. Not as simple as it seemed... - Allow "source=none" to be set for up to n-1 streams of an antenna - If all datastreams of an antenna have no data, better to exclude through the global antennas list. * Make use of recorder number in TAPELOG_OBS to automatically generate multiple datastreams for simple cases. * Move vlog from calcif2; this script is nearly obsolete but still might have some value beyond the time that calcif2 lasts. * LBA modes: S2_data_source = VLBA -> LBAVSOP mode * new global parameter delayModel -> .calc file DELAY MODEL parameter * Move to use of dirlist library for parsing file listings. This allows both old-style and new-style file lists to be used. * It is now deprecated (but still legal) to supply file lists without start/stop times associated with each file. This capability may vanish at some point. * Fix long standing issue w.r.t. (Mark5) media change detection: media change would never split a job. * Option sortAntennas in .v2d file (default=true) now allows prevention of sorting (set sortAntennas=false). Use of this feature requires an antenna list to be provided in the .v2d file. * Remove dangerous option for antenna partial matching in .v2d file * Check for duplicate channels and warn if found. * Check for unsupported polarizations * Version for DiFX-2.5, May 16, 2017 Version 2.4.0 ~~~~~~~~~~~~~ * Support axis offset argument for ANTENNAs * Adapt to string-based networkPort in support of raw sockets * Detect mismatched bandwidths and issue warning * Fix smart tone selection for phase cal interval != 1 MHz * Make tone selection default the same whether an ANTENNA section exists or not (default is Smart) * EB and AR to use VLBA Cannonical Thread Ids for VDIF * Respect record enable bit in SCHED section (enable column must be > 0 to retain the data) * Bugfix for allocating bands to baselines for some combinations of zoom bands and LSB * Reading complex type and #bits from vexfile * Version for DiFX-2.4, Nov 4, 2014 Version 2.3.0 ~~~~~~~~~~~~~ * Suport for VDIFL added * Move some NRAO-specific utilities outside this package * vexpeek: new --format and --diskusage options * Version for DiFX-2.3, Dec 20, 2013 Version 2.2.0 ~~~~~~~~~~~~~ * vexpeek: --bands mode * Support writing of .machines file * Improved support for FAKE * Verify input files do not have DOS end of line characters * Populate difxLabel in difxio->.calc and in .joblist * Generate systematic threadId * Verify all or none of the ephemeris parameters are provided * c++11 compliance * Don't allow subintNS greater than about 1 second (to ward off MODE errors) * Remove "using namespace std" from .h files; goal is to remove them from .cpp files in time * Track oversample value in channel structure; move checking downstream to more relevant code. * Add a few features (ephemDeltaT, ephemStellarAber, ephemClockError) useful in exploring spacecraft ephemeris but probably not useful for operations * change zoom band definition to always be USB (specify lower edge freq) * Reorder Mark5B tracks to accomodate bitstream masks * Consider two sources within 1 mas of each other to be the same * Added separate mjdStart and mjdStop parameters for each antenna * Version for DiFX-2.2, May 31, 2013 Version 2.1.0 ~~~~~~~~~~~~~ * Allowed zoom bands to be used with profile mode also * Version for DiFX-2.1, May 14, 2012 Version 2.0.2 ~~~~~~~~~~~~~ * Bug fix: when sorting events, round to nearest 0.1 second * Bug fix: order of operations in computing zoom channels: don't overflow integers * Change parameters to be better optimized for larger bandwidths (at marginal, if any, impact to small bandwidths) * Reimplement tweak integration time * Fix zoom bands when statting with different bandwidths. WFB gets a beer for this one! * Start implementation of "global zoom bands" which can either be referenced from an ANTENNA or a SETUP * Distinction between recorded and non-recorded channels is now more clear. Only recorded channels will be correlated. * No more VexFormat class. Its contents were moved into VexSetup. * vex2script: better job allocating RDBE personality and choosing whether or not to record * set xmacLength and strideLength automatically for powers of 2 and powers of 5 FFT sizes * Can set up "fake" correlations Version 2.0.1 ~~~~~~~~~~~~~ * add -v option to vexpeek * move most strcpy to snprintf and make use of parameterized string lengths in difxio * add recordEnable map to VexScan. * fix back-to-back scan handling when singleScan = False * fix media changes that occur immediately before a scan change * don't error-out if an ANTENNA block does not correspond to an entry in antennas list * Follow refs to CLOCK entries; fall back on unreffed clocks only if no other clocks found first * Follow refs to TAPELOG_OBS entries; fall back on unreffed VSNs only if no other VSNs found first * Support proper vex epoch for SITE; allow mjd to continue to work * Generalize vexDate just a bit to allow for the posibility of vexdate without all date fields * Add sampling parameter to ANTENNA confuration * When cross-pol enabled, allow correlation of R-L even if one antenna doesn't have both polarizations * Try different subintNS until the readsize becomes reasonable (probably some tuning to be done still) * Propagate .vex file phase cal extraction into .input file* Pass tcalFreq from .v2d to .input * Support several different modes of pulse cal tone selection * Allow different numbers of bits on different antennas within same Mode (Thanks Jasper Horrell for the bug report) * Catch a few vex problems before segfaulting (thanks Chris Phillips for the bug report) * Fix bug estimating the output size of the correlation * Change the nChan input parameter to mean the number of post-averaged channels - Clarified in several places in the code whether nChan is nInputChan or nOutputChan * FIXMEs are now #warnings * Default pulse cal tone extraction mode is smart: that is take the two most extreme tones per band that are not within the outer 1/8 of the band if possible. * Allow source names with underscores * Fix bug calculating spectral averaging for zoom bands * Ephemeris driven objects: use linear interpolation with a default interval of 15 seconds. Results are much smoother. * Version for DiFX 2.0.1. Tagged June 12, 2011 / updated June 16 Version 2.0 ~~~~~~~~~~~ * Jumped because in the mean-time difx-1.5 branch had gone to 1.1 * Includes the following corrections merged in from the 1.5 branch - Zero length scans are ignored - Warn if no clock values are applied on a job/antenna basis - oms2v2d: correct use of POLARIZATION - oms2v2d: respect TWEAK_INT_TIME parameter - oms2v2d: trap missing FFT_SIZE and SPECTRAL_AVERAGE parameters * Also bugfixes related to variable initialisation * Adds support for multiple phase centres * Requires difxio 3.0 * Add support for VLBN "format" -- that is VLBA with no data modulation * Use strict parsing by default * Add deltaClock and deltaClockRate * More user warnings * Further fix to generated flags when an antenna is not in a job * oms2v2d: trap missing FFT_SIZE and SPECTRAL_AVERAGE parameters * Actually respect clock breaks! * Fix some logic to avert some unneeded warnings * Don't require DIFX_VERSION to be set * Check that all supplied antenna, source, scan and mode names are in vex file * Huge speed-up by not constantly sorting events * Fix convergence problem when there is no time gap between modules * Force v2d file to start with a letter * oms2v2d: warn if both .skd and .vex are found; default to .vex * Source names were confused: no distinction was made between the vex "def" name and the source name. Fixed. * Check for sources with illegal names * Improved function/variable naming to indicate explicitly where some def names are used * Vex loading error messages are slightly more useful * Adapt to difxio changes where data source info was moved from antenna into datastream * Support default ANTENNA setup in .v2d file * Support new namings in difxio. recChan -> recBand * Fix year definition ( was correct in difx 1.5 for quite some time ) * Better default some difx2.0 specific parameters * Don't write out rules which are not used in a given job * improve checking of subintNS and averaging intervals * prevent a crash when source RA and decs are changed * add vexfile to .calc * Version for DiFX-2.0.0 Version 0.3 ~~~~~~~~~~~ * Post DiFX-1.5 * Mark5B support * format override support * Pulsar profile mode Version 0.2 ~~~~~~~~~~~ * Make several classes subclasses of VexInterval to simplify things * Compute "Computational load" -- an estimate of the number of numerical operations required. * Initial (untested) support for ephemeris-driven targets * Estimate output data size / allow job splitting based on job size * Version for DiFX-1.5 Version 0.1 ~~~~~~~~~~~ * simplified difx input file output * add SOURCE blocks to .v2d files * allow colon separated RAs and Decs in the config file * implement ANTENNA blocks * write SCAN_GAP scans instead of no scans * start time and stop time are no longer expanded to full length of all included scans * write flag file that is compliant with difx2fits * add oms2v2f to this package * implement maxLength (seconds) * don't allow white-space separated lists Version 0.0 ~~~~~~~~~~~ * Initial version