Meetings/20080613

Regular catchup, 02 6246-3169 Usual time - 12:00

Agenda

• status reports.

Minutes

Some useful questions were asked in this meeting

• What is the CABB commissioning plan and where do we fit in? We didn't know the answer.

• what is our release process? we didn't know at first.

"release" means "merge changes into main line version".
A release comprises a defined set of new functionality, as per the ​plan.

• Given that a "release" is pretty final, we need to do plenty of testing before release, both for bugs in the new code and especially for regressions. How will we do that?

• The modified code needs to be available to testers to run, well ahead of release time. Because some of the changes have a widespread impact (eg changing maxdim.inc) we should have a completely distinct set of binaries, in a directory like /nfs/atapplic/miriad-cabb.

Update: 2008-06-20 how about /nfs/atapplic/miriad-test, which could have general use? Fine, as long as only one set of changes is being tested at any one time (ie miriad-cabb, then miriad mainline, but not both together).

• We should probably also provide "snapshot" tarballs in the ​FTP directory for people who want to try the code on their laptop for example. Probably a structure that replicates the existing one, but in a separate miriad-cabb directory under software.

• We need regression testers. After some discussion we arrived at the idea that the test space can be partitioned on miriad task program boundaries. Changes to some programs impacts particular areas of scientific application - eg uvspec is more relevant to people doing spectral line observations. The main areas of scientific application are in the table below. These are the roughly defined areas - the precise bounds of each area are defined by a list of miriad tasks. (We should compile those lists in full detail)

Science area	Tester	Programs
General	anyone	atlod, gpcal, mfcal, invert, uvflag, blflag, uvplt, cgdisp, histo
GRB brightness measurements	Wieringa	imfit
HI synthesis of galaxies	Koribalski (LVHIS)	mosmem?
HI synthesis of wide areas	McClure-Griffiths	mosmem?
Masers	Voronkov
Continuum imaging (mosaicing)	Feain	linmos
mm line mapping	Brooks?
Zeeman splitting	?
Calibration	Wieringa?	early, sigest, planets, paraplot

(please add more as you think of them)

The "testers" named are people whom we could approach for data sets and processing scripts that we could co-opt for testing with. Their expertise may be useful if we need to decide whether some changes we detect are significant or not.

• How do we detect regressions?

In principle, differencing images made with the two suites should be enough. The differences should be exactly zero.

In practice there may be least-significant bit differences, so we will have to devise ways of comparing and deciding when a difference is significant.

We will also need to build up ways of detecting where in the processing chain the divergence in results occurred.

• We should do the "zeroth level" tests -
  • same machine, separately compiled codes
  • different machine, same (mainline) code

Actions

• Vince & Naomi to discuss CABB commissioning plan with Jim Caswell

Update 2008-06-13
It turns out Jim is planning to use real observations as the "commissioning" process. Most of the tests of the correlator that one would want to do in a formal commissioning period will already be done by Warwick and co in their test procedures.

So it appears our task is to look at the schedule and check for tasks that are likely to be needed, and work on those with higher priority.

• Vince to talk to Tim Cornwell about approaches to regression testing in ASKAP
• Mark to prepare his GRB data processing scripts for use as the first set of regression test scripts.
• Vince to discuss release process and setup of test code with Mark Calabretta

Participants

• Dave, MarkW, Vince

Meetings Home