Index: /trunk/Makefile
===================================================================
--- /trunk/Makefile	(revision 65)
+++ /trunk/Makefile	(revision 66)
@@ -145,4 +145,8 @@
 	$(CXX) -o makeFake.x $(TEST)/makeFakeData.o $(HEADER) $(BLOCK) $(PSRFITS) $(UTILS) $(LIBS)
 
+testAngSep : $(TEST)/testAngSepFunctions.cc
+	$(CXX) -c $(TEST)/testAngSepFunctions.cc -o $(TEST)/testAngSepFunctions.o -I/usr/local/pgplot -I$(DUCHAMP_CODE)/src -I $(BASE)
+	$(CXX) -o testAngSep.x $(TEST)/testAngSepFunctions.o $(BASE)/pvo-utils.o -lcpgplot -lpgplot -L/u/whi550/CubeFinder -lpng -L/usr/X11R6/lib -lX11 -lg2c
+
 
 $(USE_UTILS_HH) : $(BASE)/pvo-utils.hh
Index: /trunk/docs/VOrequirements.tex
===================================================================
--- /trunk/docs/VOrequirements.tex	(revision 65)
+++ /trunk/docs/VOrequirements.tex	(revision 66)
@@ -37,4 +37,9 @@
 project progresses.
 
+The scope of this project is just to look at the survey data and
+provide a way of accessing it through the web archive. It will not
+directly cover the existing timing observations, although there
+obviously should be some commonality in the way the data is accessed. 
+
 \section{Expected queries}
 
@@ -56,5 +61,5 @@
 
 \item Accept standard conesearch queries from a VO service.
-\\ \textit{Do not think this is possible at present.}
+\\ \textit{Not possible at present.}
 
 \item All observations within given time range, perhaps combined with
@@ -69,4 +74,12 @@
 \end{itemize}
 
+This is mostly a wish-list. The current project (as at June 2007) will
+involve enabling conesearches to be made on survey data, as well as
+the queries that are currently possible with the on-line
+archive. Conesearches on timing data, particularly with concurrent
+searches on pulsar parameters is probably beyond the scope of what we
+can do.
+
+\newpage
 \section{Work needed}
 
@@ -102,5 +115,12 @@
 for the survey data. 
 
-Note that this will require the database to hold all the survey data!
+The conesearch option should be able to accept a standard-format
+conesearch from a VO service. The details of the implementation of
+such a query are left up to the archive itself, so we have complete
+freedom as to how the querying is done. The results need to be able to
+be presented in a standard (VOTable) fashion -- as well as via the web
+page presumably.
+
+Note that we will require the database to hold all the survey data!
 See the next section for a discussion of this.
 
@@ -137,4 +157,77 @@
 to the data?
 
+\newpage
+\section{Details of implementation}
+
+\subsection{Conesearch}
+
+We want to implement a simple conesearch query that will search the
+survey data. Here is a list of things that need to be present:
+\begin{enumerate}
+\item\label{list-filestore} The PSRFITS files need to be in some
+  central location.
+\item\label{list-metadata} The metadata from these files needs to be
+  stored \eg in an SQL database for easy access and
+  searching. Essential metadata includes: filename, file location, RA,
+  Dec, beam size, time and date, observation frequency, survey, ...
+\item\label{list-queryinput} A web page or web service to accept a
+  query. The conesearch could be entered as an option from the
+  existing archive webpage.
+\item\label{list-querySQL} A way of using the information from the
+  query in a search of the database \ie a simple way of taking the
+  entered RA, Dec, radius and performing the conesearch on the RA,
+  DEC, Beam size columns in the database.
+\item\label{list-queryout} A way of converting the results of the SQL
+  query into a useful format for either displaying in the web browser
+  or returning via the web service.
+\end{enumerate}
+
+The details of \#\ref{list-filestore} are, to a certain extent, fixed,
+with all the data going on the new large disks (once the final details
+of the file conversion have been verified).
+
+Obtaining the metadata should be a straightforward process. I have
+written a program \texttt{pvosummary} that prints many of these basic
+metadata fields, and this could easily be developed for use in a
+script that trawls the data directory. This is essentially what is
+done with the current archive, where \texttt{vap} is used to print the
+details of the timing observations, which are then incorporated into
+the mySQL database.
+
+The metadata listed at \#\ref{list-metadata} is simply the essential
+data that would be required to service the query. Other data is
+probably going to be required: here is the full list of columns from
+the observations table in the mySQL database for the timing
+observations: pulsar\_name, filename, raj\_hour, raj\_minute,
+raj\_second, decj\_degree, decj\_minute, decj\_second, dm, period, bw,
+cnfg, data\_loc, data\_type, freq, inst, MJD, npol, nchan, nbin, nsub,
+rcvr, site, file\_size\_bytes, length, obsrvr, date, ut, gl, gb,
+tsamp, nbeam.
+
+\subsection{Storing positional information}
+
+A brief note on how the positions are stored currently and how they
+should be stored. 
+
+The way the declination information (\eg decj\_degree, decj\_minute,
+decj\_second) is stored is no good, as it will not properly record
+positions like -00:01:23. This is because you cannot distinguish
+between $-0$ and $+0$ in the decj\_degree entry.
+
+The positions should either be stored as decimal degrees (best
+option), or, if the hours/degrees, minutes and seconds must be stored
+separately, also have a decj\_sign column that indicates the sign of
+the declination (equal to +1 or $-1$).
+
+Currently this is done incorrectly even in the ATNF Pulsar
+Database\footnote{See
+http://www.atnf.csiro.au/research/pulsar/psrcat/}. A search ``within
+circular boundary'' with radius of 1 degree from 16:07:00, -00:30:00
+does not return J1607$-$0032, which appears to be 1.05 degrees
+away. This is because the DECJ is returned as +00:32:40.83 (\ie the
+sign of the declination is not preserved). This seems to be a wider
+problem, with \texttt{psrcat} exhibiting the same behaviour.
+
 
 \end{document}
+% LocalWords:  conesearch metadata
Index: /trunk/src/pvoconvert.cc
===================================================================
--- /trunk/src/pvoconvert.cc	(revision 65)
+++ /trunk/src/pvoconvert.cc	(revision 66)
@@ -98,5 +98,8 @@
     }
   }
-  std::cerr << "File " << fname <<" : " << surveyNames[currentSurvey]<<"\n";
+
+  std::string file_loc = get_current_dir_name();
+  file_loc += "/"+fname;
+  std::cerr << "Input file " << file_loc <<" : " << surveyNames[currentSurvey]<<"\n";
   if(currentSurvey == DUD){
     std::cerr << "Exiting...\n";
Index: /trunk/src/tests/globtest.c
===================================================================
--- /trunk/src/tests/globtest.c	(revision 66)
+++ /trunk/src/tests/globtest.c	(revision 66)
@@ -0,0 +1,26 @@
+#include <stdio.h>
+#include <glob.h>
+
+int main(){
+
+  glob_t globbuf;
+  int i;
+
+/*   globbuf.gl_offs = 2; */
+  globbuf.gl_offs = 0;
+/*   glob("*.c", GLOB_DOOFFS, NULL, &globbuf); */
+/*   glob("src/\*.cc", GLOB_DOOFFS, NULL, &globbuf); */
+/*   glob("*.fits", GLOB_DOOFFS, NULL, &globbuf); */
+  glob("*.rf", GLOB_DOOFFS, NULL, &globbuf);
+/*   glob("../\*.c", GLOB_DOOFFS | GLOB_APPEND, NULL, &globbuf); */
+
+  printf("%d\n",globbuf.gl_pathc);
+/*   for(i=0;i<globbuf.gl_pathc;i++) printf("%s\n",globbuf.gl_pathv[i]); */
+  for(i=0;i<globbuf.gl_pathc+globbuf.gl_offs;i++) 
+    printf("%s\n",globbuf.gl_pathv[i]);
+
+/*   globbuf.gl_pathv[0] = "ls"; */
+/*   globbuf.gl_pathv[1] = "-l"; */
+/*   execvp("ls", &globbuf.gl_pathv[0]); */
+
+}
Index: /trunk/src/tests/testAngSepFunctions.cc
===================================================================
--- /trunk/src/tests/testAngSepFunctions.cc	(revision 66)
+++ /trunk/src/tests/testAngSepFunctions.cc	(revision 66)
@@ -0,0 +1,153 @@
+#include <iostream>
+#include <iomanip>
+#include <cpgplot.h>
+#include <stdlib.h>
+#include <time.h>
+#include <math.h>
+#include <Utils/mycpgplot.hh>
+#include <pvo-utils.hh>
+using namespace mycpgplot;
+using namespace PVO;
+double angSepStd(double ra1, double dec1, double ra2, double dec2);
+double angSepHav(double ra1, double dec1, double ra2, double dec2);
+int main()
+{
+  
+//   const int num=10000000;
+
+  srandom(time(0));
+
+  const int num=100000;
+  float std[num],hav[num];
+  float min,max;
+
+  const float ramax = 1.;
+  const float demax = 1.;
+
+  double ra1  = dmsToDec("18:43:22.0")*15.;
+  double dec1 = dmsToDec("89:52:12.3");
+  double ra2  = dmsToDec("18:43:22.0")*15.;
+  double dec2 = dmsToDec("89:52:12.1");
+  
+  std::cout << std::setprecision(10);
+  std::cout << "Std result =\t" << angSepStd(ra1,dec1,ra2,dec2) << "\n";
+  std::cout << "Hav result =\t" << angSepHav(ra1,dec1,ra2,dec2) << "\n";
+  std::cout << "Should get  \t" << 0.2/3600. << "\n";
+
+  for(int i=0;i<num;i++){
+    
+    ra1  = ramax * rand() / (RAND_MAX+1.);
+    dec1 = demax * rand() / (RAND_MAX+1.) - demax/2.;
+    ra2  = ramax * rand() / (RAND_MAX+1.);
+    dec2 = demax * rand() / (RAND_MAX+1.) - demax/2.;
+
+//     float std = angSepStd(ra1,dec1,ra2,dec2);
+//     float hav = angSepHav(ra1,dec1,ra2,dec2);
+
+    std[i] = angSepStd(ra1,dec1,ra2,dec2);
+    hav[i] = angSepHav(ra1,dec1,ra2,dec2);
+
+    if(i==0) min=max=std[i];
+    else{
+      min=std::min(min,std[i]);
+      max=std::max(max,std[i]);
+    }
+    min=std::min(min,hav[i]);
+    max=std::max(max,hav[i]);
+
+    if(fabsf(std[i]-hav[i])>5.e-9){
+      std::cout << "(" << ra1 << "," << dec1<< ") vs. (" << ra2 << "," << dec2 << ")"
+		<< "  " << std[i] << " " << hav[i] << "\n";
+    }
+
+  }
+
+  cpgopen("/vcps");
+  cpgpap(8.,1.);
+  cpgenv(min,max,min,max,0,0);
+  cpglab("Standard way", "Haversine Formula","");
+  cpgpt(num,std,hav,SOLIDCIRCLE);
+  cpgmove(min,min);
+  cpgdraw(max,max);
+
+  float resid[num],rmin,rmax;
+  for(int i=0;i<num;i++){
+    resid[i] = std[i] - hav[i];
+//     if(i==0) rmin=max=resid[i];
+//     else{
+//       rmin=std::min(rmin,resid[i]);
+//       rmax=std::max(rmax,resid[i]);
+//     }
+  }
+  rmin = -1.e-6;
+  rmax = 1.e-6;
+  cpgenv(min,max,rmin,rmax,0,0);
+  cpglab("Standard way", "Standard - Haversine","");
+  cpgpt(num,std,resid,SOLIDCIRCLE);
+  cpgmove(min,0);
+  cpgdraw(max,0);
+
+  for(int i=0;i<num;i++){
+    std[i] = log10(std[i]);
+    hav[i] = log10(hav[i]);
+    resid[i] = std[i] - hav[i];
+//     if(i==0) rmin=max=resid[i];
+//     else{
+//       rmin=std::min(rmin,resid[i]);
+//       rmax=std::max(rmax,resid[i]);
+//     }
+  }
+  cpgenv(log10(min),log10(max),log10(min),log10(max),0,30);
+  cpglab("log10(Standard way)", "log10(Haversine Formula)","");
+  cpgpt(num,std,hav,SOLIDCIRCLE);
+  cpgmove(log10(min),log10(min));
+  cpgdraw(log10(max),log10(max));
+
+  
+  cpgenv(log10(min),log10(max),rmin,rmax,0,10);
+  cpglab("log10(Standard way)", "log10(Standard/Haversine)","");
+  cpgpt(num,std,resid,SOLIDCIRCLE);
+  cpgmove(log10(min),0);
+  cpgdraw(log10(max),0);
+
+  cpgend();
+
+
+}
+
+double angSepStd(double ra1, double dec1, double ra2, double dec2)
+{
+    /**
+     *  This is the "standard" way of calculating the angular separation.
+     */
+    double r1,d1,r2,d2;
+    r1 = ra1  * M_PI / 180.;
+    d1 = dec1 * M_PI / 180.;
+    r2 = ra2  * M_PI / 180.;
+    d2 = dec2 * M_PI / 180.;
+//     std::cerr << cos(r1-r2) << " " << cos(d1) << " " << cos(d2) << " " << sin(d1) << " " << sin(d2) << "\n";
+    double angsep = cos(r1-r2)*cos(d1)*cos(d2) + sin(d1)*sin(d2);
+    
+    return acos(angsep)*180./M_PI;
+}
+
+double angSepHav(double ra1, double dec1, double ra2, double dec2)
+{
+    /**
+     *  This is the way of calculating the angular separation that uses the Haversine formula:
+     *  see http://en.wikipedia.org/wiki/Haversine_formula
+     *  and http://mathforum.org/library/drmath/view/51879.html for derivation.
+     */
+    double r1,d1,r2,d2,diffr,diffd;
+    // put into radians
+    r1 = ra1  * M_PI / 180.;
+    d1 = dec1 * M_PI / 180.;
+    r2 = ra2  * M_PI / 180.;
+    d2 = dec2 * M_PI / 180.;
+    diffr = (r1-r2)/2.;
+    diffd = (d1-d2)/2.;
+//     std::cerr << sin(diffr) << " " << cos(d1) << " " << cos(d2) << " " << sin(diffd) << "\n";
+    double havtheta = sin(diffr)*sin(diffr)*cos(d1)*cos(d2) + sin(diffd)*sin(diffd);
+      
+    return asin(sqrt(havtheta))*2.*180./M_PI;
+}
