Cube Data Access Layer implementation note

[Markus Demleitner]

Dear DAL,

On Tue, Mar 01, 2016 at 10:39:02AM +0100, François Bonnarel wrote:
>     Facing the multiciplity of protocols and specs which are
>     independant but often work together and have to keep

I still dispute SODA and Datalink are independent, and I've not yet
seen any (technical) argument why or how the could be, but that's
just an aside.

>     consistant, I think it's better to have an independant
>     implementation note which explains how all these things fit
>     together from Discovery to cutouts and maybe more. Apparently

...which of course then is another document we're asking our
implementors to read (obligatory xkcd reference:
https://xkcd.com/927/).  And they'd have to discover they're supposed
to read it before they can expect to understand a standard.

For usability reasons, I'm still strongly against that.

>      Some will have DataLinks and not SODA (only retrieval). Some
>      other will have SODA and no DataLink because no other links
>      available. And some will have both

I see no reason to not "have datalink" -- datalink isn't difficult,
and it already is the SODA autodescription you propose in section 4.4
of your draft.  Why should we write a separate spec and invent
*another* way to describe services?

[...]
>      Some will keep their SIAP1 service and add a DatalInk or/and a
>      SODA layer.

The common theme is that they all have SODA, and so the SODA standard
should contain the information to do what they want.

>             - The note assumes there is no "better" way to go from
>             discovery to  Links and to SODA. There are just ways
>             more adapted to use case so and so...

Well... "better" is a difficult notion.  I don't think I've termed it
"better".  But there definitely is a "basic", clean and simple way
and a more precarious shortcut.  The in-DAL SODA block is a shortcut
that can fail for many interesting use cases.  Again, I now sort of
wish I had never brought it up, because it's causing much more
confusion than it'll ever save time.

>             - Last but not least. SODA is very close to ObsTAP and
>             to SIAV2. It should have been a simple methos of SIAV2.

No, most definitely not.  All other protocols offering access to
datasets require it as well, and historically, the first protocol it
(or its predecessor, which still is very close to it) was used
together with was SSA.

>             It shares the same syntax as SIAV2 and the same model
>             as ObsTAP and SIAV2. We are defining SODA now so we can

No, it does not, at least as far as the model is concerned.  SODA, to
be useful, needs to be model-free[1], which is why it can easily work
for all of SIAv1, SSA, SIAv2, ObsCore, and future protocols we will
fairly certainly define -- that (and the fact we will not want to
touch SODA itself every time we have a new data type in the VO) is
what the three-factor semantics is all about.

>             capabilities. This is autodescription which mlay use
>             the Service descriptor syntac?  We are not facing a
>             custom legacy service where we cannot do anything about
>             the way it works and just have to describe previously
>             defined  parameters

But why go into all that trouble, when Datalink already contains all
the specification we need?

Again, the concern most often voiced to me when I ask people why they
don't take up the VO is: Too many conflicting, unintelligible
standards.  While I can usually convince them it's not *that* bad,
let's not make things worse by defining more than we absolutely need.

       -- Markus


[1] It can be model-aware and should be so (for instance, to
declare coordinate metadata when supporting native cutouts), but
that's for when we finally will have usable data model formalisms.
In case you're curious, my prototypes using atomic parameters, I have
Note-type STC annotations for the parameters -- proper VO-DML-based
annotation technically would be fairly similar.