Collaboration on Source Catalogue DM, ADQL and SkyNodes

[Mark Taylor]

On Wed, 21 Dec 2005, Pedro Osuna wrote:

> Hi Maria,
> 
> 
> > - How does Catalogue Data Model used look like, especially what is the
> >   common set of attributes and the associated metadata.
> 
> The point is in the (Source) Catalogue Data Model, with emphasis in the
> "Source" part. This one is the one I showed on behalf of the Catalogue
> DM subgroup at our last interop meeting here at ESAC. I attach a pdf
> with the initial proposal, but please use it only for temporal
> reference, as the whole document will be changed (according to
> requirements from Jonathan after the interop meeting).
> 
> > - What are the plans about registration? Will these nodes (Basic?) be
> >   registered and therefore accessible through Open SkyQuery? How many?
> 
> yes, they will. How many, I don't know. In Strasbourg, Inaki and
> Aurelien worked on a couple of them, Tycho-2 and USNOB, but the CDS
> colleagues will work on more.... Francois will answer to this question
> at some point I presume.
> 
> > - The cross-match functionallity. Are you planning on a 2 catalog
> >cross-match or n-catalog cross-match? How does the cross-match function
> >look like?
> 
> n-catalogue cross-match is what we are trying to get at; it will be a
> client based cross-match, and therefore the cross-match function will be
> designed and run at the client side (i.e., servers do not need to worry
> about implementing one specific cross-match or the other). 
> At the current status, the client sends an ADQL to the server to discern
> which type of cross-match it can do with it (whether only positional,
> positional with errors, etc.), and takes the corresponding action.

Pedro,

my STILTS package contains a standalone crossmatcher suitable for
client-side operation which might be of use in implementing the
crossmatching functionality you need.  It is highly flexible 
(can perform matches in N-dimensional space, e.g. sky position only
or sky position plus proximity in one or more flux values),
suitable for matches up to around a million rows or so (could be
extended with some work), and fairly efficient (a recent test 
matching 600,000 vs 700,000 row tables resulting in 2000 matches 
took about 90 sec).  It does not understand ADQL so it's not going
to be a complete solution to your problem, but it may be of use
as a back end matching engine.  The tool currently only 
matches two tables, but I'm planning to provide N-table matching
fairly soon.  STILTS provides this functionality from a command-line
tool (tmatch2), but a public java API is also available for 
programs that want access to it within a JVM.

I'm not sure how you're planning to implement your matching, so
this may not suit your purposes, but I'm happy to discuss it
further if you think it might be of interest.

For more details see:

   http://www.starlink.ac.uk/stilts/sun256/tmatch2.html

Mark

-- 
Mark Taylor   Astronomical Programmer   Physics, Bristol University, UK
m.b.taylor at bris.ac.uk +44-117-928-8776 http://www.star.bris.ac.uk/~mbt/