Jobs to do

A list of things that need to be done for the Pulsar Archive / VO project.
This list was compiled on June 21 2007, with most items having been discussed in our meeting on Wednesday 20th June.
It has most recently been updated on September 12.
A more detailed list of tasks can be found at DetailedWorkNeeded.

There is also a document that details various aspects of the implementation of the archive and the VO queries: see source:trunk/docs/VOimplementation.pdf​

Job	Person responsible	Comments
Testing of PSRFITS with seek to make sure data is preserved correctly	Dick, George	A pre-requisite for doing the data conversion 20070912: Small discrepancies found in pulse profiles constructed from S70 and PSRFITS versions of same observation. George investigating. Done 20071015
Pre-scrunching of timing data via a cron job	Lucyna	Underway 20070912: we are caught up, task continues as needed
Verification of exactly what goes into the proc_cmd column of the HISTORY table	Dick, Matthew, Lucyna	A minor issue -- see a description of this problem at ProcCmd 20070913: This has been solved and the pvoconvert code adjusted.
Conversion of all 70cm data	Lucyna	Straightforward once we know the conversion is working. 20070921: Underway 20071015: Done
Once converted, incorporate information into mySQL database	Lucyna? Matthew? (minimal effort once conversion is started)	If we wait till all data is converted, this is only one step.
Work out necessary metadata for the 70cm data to go into database -- what fields in the table?	Matthew, Dick, ...	What metadata do we want to make public? How similar to the existing "observations" table do we want it 20070912: Matt has preliminary schema in source:trunk/docs/VOimplementation.pdf​ 20071015: Spec. complete, implementing new schema
Put Multibeam Survey data onto raid array at Parkes	Dick	Underway 20070912: No evidence of this yet.
Make sure the Multibeam survey data is accessible for archive queries	Vince	20070912: waits on data being set up on disk
Add the 70cm data to the mySQL database and make the FITS files available in the web archive	Matthew	We want searches to return survey or timing data without "prejudice", unless the user specifies the type. This means we will need to make changes to the index page, as well as the querying process (to incorporate the different metadata, and possibly provide option of selecting just survey or just timing data). You should be able to download survey data from the list of hits returned, as you can now for timing data. 20071015: sql queries in final testing, must then implement in PHP.
Fix up existing mySQL tables	Matthew	The existing timing data has problems with the way the positions are stored. We should add columns that have RA and Dec in decimal degrees -- this will make doing conesearches and the like easier and consistent with the S70 data 20071015: new columns added to schema, tests in progress
Existing web archive: sort out security issues	Matthew (talk with Arkadi)	Can we use the methods that ATOA/OPAL use? 20071015: still under study.
Existing web archive: front page should still display even if the backend db is unreachable	Matthew/David/Vince?	The page appears to be dependent on a call to mysql_connect() and it falls over badly if that call fails. (ticket: 22)
Existing web archive: organise management of proprietary access	Matthew (talk with Arkadi)	Can we use the methods that ATOA use? 20071015: still under study. New observations should have well-defined project ids (ie association with individual observing project members will be well-defined)
Set up VO cone search service that returns a VOTable	Matthew	This is done for the S70 data, albeit with a kludge to get the S70 data into the database. 20071015: Query code tested and working. New code needs to be installed in the pages driving the user-visible query form.
Register the VO cone search service	Matthew	This might have to wait for the archive website going live.
Make archive website live to outside world	Vince	Obviously this is the final step, once things are working
Convert timer data to PSRFITS	?	Beyond scope of this project? 20070912: Yes, it's out of scope for us

Any comments about these? Feel free to edit, but let me know as well that you've made changes.

Matthew Whiting.