ssh -N -L [LOCALPORT]:[WHAT FIREWALL CALLS DIFX HEADNODE]:[CONNECTION PORT] USER\@FIREWALL
- USER\@FIREWALL is how you would log into the firewall from your local machine.
- CONNECTION PORT is the TCP connection port used by guiServer on the head node of the DiFX cluster.
- WHAT FIREWALL CALLS DIFX HEADNODE is the node name of the head node as the firewall sees it (what you would use if logging into the head node from the firewall).
- LOCALPORT is a port on the machine where the GUI is being run.
- \ref difxHost "DiFX Host" is the host name of the "head node" of the DiFX cluster - where guiServer should be running. The DiFX host name should be whatever the machine on which the GUI is run calls the DiFX head node - i.e. a "ping" of this host name from the GUI host should be successful.
- \ref difxControlPort "Control Port" is the port number at which the GUI will repeatedly attempt to make a connection to guiServer if such a connection does not exist. The control port should match the \ref startingGuiServer "port number" given by guiServer when it is started.
- \ref runWithDifxVersion "Run w/DiFX Version" is the version of DiFX software that will be run by the GUI. This version does not need to match that of the GUI or guiServer, however it does need to be installed on the DiFX cluster.
- \ref difxExecuteScript "DiFX Execute Script" is a script on the DiFX processing nodes that is used to execute all DiFX and mpi commands. The script defines environment variables and performs any other necessary setup before running things. Most of the time the script selected automatically by the GUI should be fine.
- \ref workingDirectory "Working Directory" is the path under which new directories are stored for DiFX experiments that the user creates using the GUI. The "user" running guiServer needs to have write permission in this location or none of this will work.
- An Experiment is usually bound to a single data set (one or more scans) collected over a specific time span - the results of a single observing session for instance. It can contain any number of "Passes" (including zero).
- A Pass is used to contain a single analysis of a subset of the data within an Experiment. Often Experiments contain a "Clock Pass" run on a few scans to generate the time delays for each involved antenna, and a "Production Pass" run on all scans with those time delays in place.
- Within each Pass is a series of Jobs, each controlling the processing of at least one scan.
- Setting up a location to do the processing
- Creating a .v2d file to go with the .vex file
- Running vex2difx and calcif2 to create .input and .im files
This will bring up the "Create New Experiment" window:
IMAGE
The purpose of this window is to allow the user to tailor a new experiment to meet their needs. It creates a "working" directory for the new experiment, allows the specification of data sources, and puts all relevant DiFX files (.v2d, .input, etc.) in the working directory from which they can be run (either through the GUI or by hand). Experimentation with different GUI settings while creating an experiment is not dangerous as the original .vex file and data files are not moved or altered in any way. If you mess up, delete the experiment and try again.
\subsection namingTheExperiment Naming the New Experiment and Putting It Somewhere \subsection gettingVexFileContent Getting .vex File Content For a DiFX Experiment to be created and run, it must begin with a .vex file. Vex files should be provided with observed data and imported into the GUI where they are parsed to determine the structure of jobs that will be created, data sources that are necessary, and other things. \subsubsection changingVexFileContent Editing .vex File Content Once you have obtained .vex data from some source, the data are displayed in the ".vex File Editor" panel. This panel provides a (rather rudimentary) text editor that can be used to edit the .vex data by hand. The final edited text is used in the .vex file assigned to your created experiment, a copy of which is put in your working directory. For any changes to apply, you must click the "Parse Content" button. Some care should be taken in directly editing .vex data as it is trivial to corrupt the .vex to the point where it can't be used (the DiFX operational paradigm says that users should never need to do this), but editing this content does not alter the original source .vex file, only the final .vex file associated with the experiment - so playing around with things is not permanently harmful.
\subsection correlationTuningParameters Correlation Tuning Parameters The Correlation Tuning Parameters section includes values that can be changed to adjust the quality of the correlation results, and/or the total time processing takes. Adjustments to many of these values is something of an art in itself, and the details of what things do and what their "best" values should be is not covered here (some talks at DiFX Users Meetings have covered the subject - slides can be viewed here). Each item has an associated "apply" check box. If this box is not checked, no instructions regarding the item will be put in the .v2d file and vex2difx will be allowed to pick its own defaults. Unless you know what you are doing, don't check the apply box - let vex2difx pick the values! The GUI has default values for all items but they are not based on anything - they are essentially placeholders. The default values that are picked by vex2difx are far better. \subsection stationsAndDataSources Stations and Data Sources Each antenna involved in the observations described by the .vex data triggers the creation of a panel in the "Stations" section with the two-letter code station/antenna code used as a panel title. Each station panel contains a number of sub-sections: at least one Data Stream; Antenna; Site; and Settings. \image html stationsSection.png In most cases users will only be concerned with changing the Data Stream settings. This section tells DiFX where the data for a particular station/antenna can be found. Because filling out the Data Source section can be tedious, the DiFX GUI provides ways of pre-defining all Data Source settings for a station/antenna based on known and/or likely locations. See \ref antennaDefaults "Antenna Defaults" in the Settings Window "Job Creation Settings" section. \subsubsection stationsSettings Station Settings
The Settings section contains settings for Tone, Phase
Calibration
Interval and Delta Clock. The Delta Clock value is often
gleaned
by running a "Clock Pass" on a subset of the experiment's data
(see
some sort of explanation here).
.v2d and .vex files that
are created as part of the new experiment, so selections are part of the
experiment creation process. The \ref experimentScanSelection "Scan Selection"
section of the Experiment Editor can be used at any time to view the scans that will be
included in the experiment when it is created as well as which stations are used for
each.
\image html experimentEditor_scanSelection.png
Some of the scan and station selection controls can work at
cross-purposes - effectively they provide more than one way to cause
a scan or a station to be used. When a conflict occurs, the GUI
will give the most recent command precedence (if, for instance, a
command is given that a scan be included in the final experiment
when a previous command eliminated the scan, the GUI will include the
scan).
Eliminating Stations in the "Source" .vex Data
Stations can be eliminated from individual scans by putting a "-1" in the
"code" column within the appropriate "scan" section in the "source" .vex
data. When the GUI encounters the "-1", it will remove the
station from the scan. This duplicates hardware correlator
behavior. Starting with the .vex file snippet below, the
final .vex file will not include the station "Bd" because of the "-1" in
the final column.
\code
scan 128-1703;
start = 2014y128d17h03m34s;
mode = GEOSX8N.8F;
source = 1846+322;
station = Bd : 0 sec : 20 sec : 0 ft : 1A : &n : -1;
station = Ho : 0 sec : 20 sec : 0 ft : 1A : &n : 1;
station = Kk : 0 sec : 20 sec : 0 ft : 1A : &cw : 1;
station = Ny : 0 sec : 20 sec : 0 ft : 1A : &ccw : 1;
station = Ts : 0 sec : 20 sec : 0 ft : 1A : &cw : 1;
endscan;
\endcode
For changes of this type to be recognized, the .vex file must be
parsed by the \ref experimentEditor "Experiment Editor". You edit the .vex file
in the \ref vexFileEditor ".vex File Editor" or you can use your favorite text editor.
If you do not want the GUI to pay attention to the "-1" code in this
way, un-check the \ref eliminateStations "Eliminate Stations With \"-1\" Code" box in the Settings menu.
Eliminating
Stations in the "Stations" Section
The "Stations" section is primarily set up to change parameters
related to each antenna involved in an experiment, and to select
the
data sources associated with them (see above). However it
also
includes a check box that can be used to completely remove each
station
from the experiment. Any scans that no longer have enough stations
to
form a baseline (i.e. less than two) will be eliminated.
IMAGE
The "Scan/Station Timeline" Editor
The "Scan/Station Timeline" section provides a visual map of all
scans and the stations used in them in a timeline. It allows
the
selection/deselection of individual stations within scans or the
inclusion of data from different stations based on time.
IMAGE
Somewhat more complex explanation here.
Selecting by Source Using the "Sources"
Editor
The "Sources" section shows all sources and the stations used to
observe them. It allows sources to be selected and
deselected,
and stations to be selectively used or eliminated from sources.
IMAGE
- All sources included in the .vex file are listed. Boxes show which sources are observed with which stations.
- Hover over the name of a source to produce a tooltip that includes information about the source as well as the names of the scans used to observe it and the stations used for each of those scans (stations marked in red have been eliminated).
- Hover over the boxes to produce a tooltip that includes which
scans use the associated station on the associated source.
A scan
that appears in red text has been eliminated - either explicitly
or
because it lacks sufficient stations to form a baseline.
- Use check boxes to add or eliminate a source. When a source is removed, all scans associated with it are eliminated from the final experiment. When a source is added, all scans associated with it are put into the final experiment (assuming they have sufficient stations to form a baseline).
- Click on the boxes to add or eliminate a station from the
observations of a given source. Scans will be added or
eliminated
from the final experiment based on whether changes give them
enough
stations to form a baseline.
The Sources section is something of a work in progress, and not something anyone uses at the USNO, so it is a little confused at this point as to what it wants to be. It was developed originally with the idea that astronomical observers would be interested in sources (in geodesy they are uninteresting). Suggestions are welcome.
Selecting Specific Scans With the "Scan
Selection" Editor
At any time in the scan/station selection process, the "Scan
Selection" editor will show which scans will be included in the
final
experiment (included scans are green, scans not included are
gray). It allows the user to make selections on a
scan-by-scan
basis by clicking on individual scans, or by turning all scans on
or
off using the "Select All" and "Clear All" buttons.
IMAGE OF SCAN SELECTION PANEL WITH LABELS HERE
The Scan Selection Editor includes a "Time Limits" plot that
shows
all scans from the original .vex file as a time sequence (again,
scans
in green are included, those in gray are not included). The
mouse
wheel can be used to "zoom in" on different time limits, and the
red
and blue triangles can be grabbed and dragged to limit the final
experiment in time. This widget is somewhat redundant with
the
Scan/Station Timeline Editor, but it may be useful to someone.