S*AP and DAL v2

[Doug Tody]

Hi Paul -

On Fri, 16 May 2008, Paul Harrison wrote:

> I thought that I would add my "vision" of how things should/could be...
> 
> The "Simple" access protocols have been a success for the VO - They are simple
> to use and relatively simple to implement. They are perfect for the "what is
> available at that position on the sky" types of query.
> 
> So how to progress, to accommodate the more sophisticated query that people
> now want to make.
> 
> Keep the S*AP simple
> * keep as a synchronous service
> * have parameter based model that can easily be driven by filing in a web form
> - this means easy for the user to understand, and unambiguous for the service
> writer.
> 
> DAL v2 should concern itself with advanced usage;
> * query language driven vs. parameter based.
> * asynchrony
> * interaction with VOSpace
> * interaction with other DAL services to provide "cross matching"

To start with there is more to a DAL service than just the discovery
query; much of data access concerns - well actual data access - where
we subset, filter, transform or otherwise manipulate a dataset at
access time to the generate virtual data the client wants.  This is
most naturally and directly done with a declarative (parameter based)
approach, i.e., make me an image with these attributes.

There is an important role for a QL in discovery where we have
something more closely resembling a database query.  However QL works
better for an actual physical table than for a *data model* where many
of the attributes may lack values.  I think it is worth pursuing QL as
an advanced capability for an S*AP interface but at this point we do
not even know if we can get data providers to provide enough metadata
to make such an approach work.  Hence for most data access we should
continue to rely upon a small number of key parameters which can have
more complex evaluation semantics than is allowable for a formal QL
expression, reserving the QL as an advanced capability.  We can still
have it, after all, if it works.

There is no reason that a fully featured 2ndgen interface cannot
also have a "simple" basic subset of capabilities - many successful
interfaces are of this type (the Web itself, GIS, etc.).  This lowers
the buy-in threshold to the point where one can get users to use
it, and later they can be talked into using a production service
framework which provides much more complete and robust capabilities.
Splitting key interfaces into multiple service interfaces will likely
split the user community as well.  I think this is a very important
point - it is project death to leave the users behind, after a while
you are just talking to yourselves and later your funding gets cut.

It is not correct to say that the parameter-based interfaces are
position oriented only.  While that was true of SCS and SIAV1, which
were just to bootstra VO, it is not true of SSA.  It is little harder
to query on a set of 5-10 or so key attributes than one, particular
if a parameter aproach is used where evaluation semantics can say that
the query constraint is not applied if the service does not support it
(an essential feature for blind distributed queries).

I agree that if the parameter based approach becomes too complex
it is no longer worth it, and we should just use a QL; ultimately
parsed languages do this sort of thing better.  However that is
not an argument to not have both in the same interface.  The simple
parameter approach handles maybe 90% of the common cases very easily,
for both the user and the service implementor, and is easily mapped
to a more powerful back-end (if you have one).  When the user needs
more the advanced capbility is there, and they can just take the time
to learn how to use that instead.

	- Doug