Collaboration on Source Catalogue DM, ADQL and SkyNodes

[Maria A. Nieto-Santisteban]

Dear all,

> the idea is simply to be able to create clients that can access
> distributed data using ADQL(VOQL) queries, without caring about whether
> the data are in an RDBMS, ODBMS, ascii file on a filesystem, votable or
> whichever the format they might be stored in.

That is the pricinciple of the SkyNodes since the very begining 
Skynodes are implemented using Web services and ADQL is the language 
to access the data. The system behind maybe anything.

> This is the reason why, using this approach, the only thing that the
> server-side (data provider) is obliged to do is to build a translation:
>
>      From				To
>      ----				--
> "my project data model" =====> "IVOA Source Catalogue Data Model" 

This is a good approach as far as the IVOA Source Catalog Data Model 
is recognized znd accepted by data providers. 

> For instance, in the case of the CDS, they have their own language (why
> not?) to access their catalogues (whether in DB or files or whatever)
> called ASU_QUERY. What we've done is to help them create an ADQL-
> understanding-server with a layer to translate from ADQL to their
> ASU_QUERY extracting the relevant Source Catalogue Data Model metadata,
> therefore creating a server that understands "Source Catalogue Data
> Model-aware ADQL queries".

Great so you have set up a service that translates ASU_QUERY into the
still no standard Catalog data model and accepts ADQL queries.  (which by
the way, I hope will be no problem to be upgraded to ADQl 1.0 once this is
accepted as recomendation)  So, where is exactly the meat of all this? I
think is in the way columns in the original catalogs are translated into
the data model and how this tranlation is done. You may think these are
implementation details. But to me the meat is there and I (and probably
many) would like to know this.
 
> Once we have n-servers speaking this "VO Source Catalogue Data Model-
> aware" language (just VOQL over a specific data model), any client will
> be able to call them (with a cone search for instance), get results and
> handle those results.

You don't have to wait. Those n-servers already exist! They are called
Cone Search services which are the most popular, best followed non
standard services in the IVOA. Cone-Search services don't speak ADQL but
you could create client applications that translate the ADQL query into
Cone Search calls (Why Not?)

> One of the applications that could "handle" those results could be a
> cross-match (certainly not the only application that will consume VO
> data through IVOA VOQL language, I hope). This would allow to have
> different clients doing crossmatch, without imposing a specific cross-
> match algorithm.

I still would like to see the performance of that. Few objects, that's
fine. Many objects ... let me object.

> This would demonstrate one face of applicability of the IVOA-VOQL which
> we -the IVOA in general- are after, which should be -in the end- a query
> language to access any type of VO-compatible data, in particular those
> following specific IVOA data models, and regardless of their storage
> mechanism.
>
> (by the way, the system also allows to get data which have not been
> "translated" to Source Catalogue DM. The only problem with those is that
> clients do not necessarily know "what" exactly they are (not what "type
> of" data they are, but "what exactly" they are)).

let's see, the client always knows what exactly the provider has 
as far as it has the right metadata or documentation. How do you think 
Atronomy has been done so far? 

> Regarding cross-match of whole catalogues, we are more interested in
> cone-search type cross-match (identify sources within a certain region)
> but have in mind also a mechanism to handle big cross-matches, whose
> feasibility we are currently studying.

well .. if you are interested in Cone Searches, then don't call them
cross-match. Although they are similar, they are not the same. A
Cross-Match of X arc minute (X>1) region seems to me a bit too relaxed
cross-match filter.

> With respect to specific implementation details, I'd suggest any of
> those questions be sent outside the list, again to not annoy people
> unnecessarily. 

I don't see why what we are discussing should be bothering anybody. Those
who are not interested can always delete the e-mail without reading it. As
I said before, most of the meat of this discussion is in the
implementation details and I think is fair to ask for them and make them
public for public debate which afterall is what these Working groups are.

> I hope this clarifies some of the issues in the different mails. In any
> case, and although I will not be on holidays (bad boy :-(), I would
> suggest people to enjoy theirs and resume discussions back in January.
> Holidays are holy, isn't it?.

Yeap, we better resume the discussion in January, the holidays is not the 
best time to bring up these issues.

Cheers,

Maria


-- 
------------------------------------------------
Maria A. Nieto-Santisteban (nieto at pha.jhu.edu)
Johns Hopkins University
3400 N. Charles St.
Physics & Astronomy Department
Baltimore, MD 21218 (USA)

Tel: 	1 410 516-7679  Fax: 	1 410 516-5096