Developer Meetings

Regular Meetings

DevMeet20101126

2010 Developer Meeting

July 28 - August 7, 2010 at NAOJ (Mitaka, Tokyo)

Goals

• merge the asap trunk and alma branches
• create development plan which includes CASS and ALMA
• Document design
• Improve testing

Attendees

Daisuke Iono	NAOJ
Wataru Kawasaki	NAOJ
George Kosugi	NAOJ
Takeshi Nakazato	NAOJ
Malte Marquarding	CSIRO
Masao Saito	NAOJ
Kana Sugimoto	NAOJ
Tak Tsutsumi	NRAO

Schedule

Jul 28 (at ALMA building, 3F, Room 307)

Overview
planning sessions for the rest of the meeting to set goals

9:30 - 12:00

• welcome/opening remarks​ - Tak
• overview and current status of ASAP
  • Status of ASAP and ASAP development at CASS - Malte ASAP2010.pdf​
  • CASA-ASAP overview (including requirements, CASA-ASAP specific functionality)​ - Takeshi
  • Recent merging of ASAP3.0 to alma branch​ - Kana

12:00 - 13:00 lunch break

13:00 - 15:00

• set goals and make a plan for the merging work sessions

• set goals for the discussion on planning for next one year (make some bullet points,update this wiki page, etc)

15:00 - 15:15 coffee/email break

15:15 - 17:00

• set goals for the discussion on long term planning (make some bullet points,update this wiki page,etc)

---

Jul 29-30 and Aug 2-5 (in the CASA developers' office, "Shigokan" building)

9:00 - 17:00 -ish

ASAP merge (alma branch -> trunk of ATNF ASAP svn) + code sprints (lead: Malte)

• Starting point would be alma branch updated recently by Kana
• include few more updates from CASA
• Resolve any conflicts, duplicated features
• (move ASAP logger to CASA C++ logger?)
• Documentation (including scantable format description)
• add more testing

Jul 29

Work assignments

• Fillers re-organization (Malte and Takeshi)
• Merge (Kana and Tak)
  • merge current trunk to alma branch
  • investigate how we can merge it back to trunk
• FLAGROW column (Wataru)
  • do some benchmark tests with/without it

Jul 30

9:00 - 10:00 Skype-con with Nick(+Juergen?)

• Continued on the assignments

Aug 2

• The merge essentially completed from alma branch to trunk

Aug 3

• current status and action items
  • merge - done, everyone now commit to trunk (comparable to 'active' in casa)
  • fillers - minor python level changes remain, filler specific info via python dict--requires pylap
  • external
    • external-contains the original atnf version of external was in trunk
    • external-alma contains copy of atnf dir
  • logging - done
  • C++ -ifying (for poly_baseline and auto_poly_baseline, add batch mode parameter)
  • docs
    • need Scantable model doc
    • python style guide
    • Sphinx Guide (including scantable,rcparams, fillers, selector, C++ -> Python interfacing)
  • testing
    • casa side - rebuild, run regular regression scripts
    • need more python unit tests for ASAP classes - assign to the developers
    • move to nosetests
    • supply small data sets from all telescopes that we support for testing

• Discussions on HPC
  • might be useful for average, poly_baseline - testing with parallel_go infrastructure?
  • split the data set on filling based on available resources (cores)
  • how do we split the data?

Aug 5

wrap-up session for the merge

• Identify unfinished/remaining development items, if any and assign them
• work on documentation

---

Aug 6 (at "sho-kaigi-shitsu")

9:00 - 12:00
Discussions: near-term plan on ASAP development coordination

• 9:00-10:00 Skype-con with Nick(+Juergen?) - the meeting status report from us

• Future merge (alma branch->trunk) planning
  • No merging necessary (we all commit to trunk from now on)
  • procedure - make branch/tags off ASAP trunk as necessary
    • use ASAP trunk for CASA active branch
    • might to be better to have external link from CASA active to ASAP trunk (discuss with Wes)
  • how frequent should we schedule merge? (coordinating with release schedules)
  • testing (Malte will set up Hudson for ATNF. Maybe NAOJ CASA group do the same?)

• Development plans (1) (for CASA, CASS?)- some of them may be covered on the first day but may be useful to give some summary here, particularly on filler(s), plotter, flagging)
  • CASS ASAP review by Malte (see ASAP2010.pdf​)

• 11:00-12:00 ASKAP talk by Malte

13:00 - 15:00
Discussions: Scantable vs Measurement Set ("kick-off" discussion)

• Presentation by Takeshi​
• some discussion points:
  • Current usages of scantable
  • Requirements for CASS/CASA
  • Measurement Set def. 2.0: Is it sufficient to store single dish data from relevant telescopes (AT, ALMA, Nobeyama), do we need to modify?
  • Data accessing speed (are there any issues or differences between scantable and MS? i.e. 'flat table' vs more complex table structure)
  • Need some testing with prototype
  • What modifications are needed in order to migrate to MS?
  • ScantableVsMs
• If we can expand scantable and fix some filler issues to include missing meta data, we can probably keep using scantable and may not need to move to MS

• ScantableVsMsDiscussionNotes

• ScantableRedesignProposal

~15:00 - 17:30
Disucssions:near-term plans

• Development plans (2) (CASA side) (joined by Masao Saito, ALMA-J Project Scientist)
  • CASA (ALMA requirements) Takeshi's presentation​

• Proposal of moving some python codes to C++ (benchmark test results) - Wataru

• RequirementsDiscussionNotes

• Dinner

Aug 7 (in CASA developers' office)

10:00 - 13:00
Scantable data format upgrade discussion (Malte, Takeshi, Tak)

Miscellaneous

• HowToImplementNewFiller