Datalink Feedback VII: REQUEST

[Patrick Dowler]

Personally, and given the way that CADC services are implemented and 
deployed following a strict "one capability per path" I would be happy 
to do away with REQUEST in DALI. However, this design was a compromise 
that was reached several years ago (around the heated discussions on 
TAP) to try to balance REST vs common practice and implementation 
flexibility. This allows, for example, a VOSI-capabilities document or 
registry records to use the same accessURL with 2+ different standardID 
values. That is a flexibility feature that REQUEST enables.

I don't find the distinction between DataLink and any other capability 
at all compelling. Once you buy into SOA and implement a suite of 
capabilities, there is nothing particularly special about the "links" 
capability that warrants special treatment or exemption. As was pointed 
out below, REQUEST is always completely redundant for anyone who 
implements their capabilities with different paths. That is true for all 
capabilities so this is an issue for DALI-1.1, not DataLink.

As for REQUEST being harmful because of the & in the URLs, that happens 
whenever there are multiple query params in the URL. For the 
single-parametered DataLink links capability that looks like a pain (and 
it can be), but there are also other possible params that come into play 
already: RUNID, RESPONSEFORMAT, etc. And in general services take 
multiple params and you have to deal with &. It's http and & is a 
pain... :-) I do agree that forgetting to include REQUEST because it 
doesn't do anything is a problem, and not an insignificant one. I curse 
our own TAP service whenever I do something quick with curl and forget 
it so that point I agree with entirely.

Options:

1. Remove REQUEST entirely from DALI and this require a different path 
for each capability. Services that require REQUEST would not comply, so 
they would all have to jump a major version to support the new DALI. I 
guess that means DALI should also be 2.0

2. Make REQUEST optional in DALI so that services define values and 
require it only if they have more than one. Same versioning impact as 
above, plus then some specs could be RESTful (different path for each 
capability) and others could be RPCish (different REQUEST value per 
capability)... this seems like a non-option as it is either exactly like 
now, exactly like #1, or inconsistent between specs.

So, if enough people want to go with #1 then I think we put that at the 
top of the list for a DALI update *and* then we decide what to do in 
WD-DataLink. REQUEST: required or removed?


<cadc:hat>
removed from DALI
</cadc:hat>

Pat


On 11/04/14 04:01 AM, Markus Demleitner wrote:
> Dear DAL list:
>
> [Re the vocabulary for the semantics column: I'm working on getting
> opinons on where to keep it and how to maintain it]
>
> This one I've complained about before: I don't think Datalink should
> have REQUEST.  Pat responded to my last bickering in
> http://www.ivoa.net/pipermail/dal/2013-December/006579.html:
>
>> As with RESPONSEFORMAT, the REQUEST parameter is there to facilitate
>> implementers having alternate functions that can be triggered, so it is
>> like a reserved param they can use. It also lets the IVOA add new values
>> later without breaking anything. For example, in TAP we have
>> REQUEST=doQuery but I have considered implementing REQUEST=explainQuery
>> to do, well, exactly what it says.
>>
>> DALI says REQUEST is required in all requests so the description of each
>> capability specifies the value(s). Currently we define one REQUEST value
>> per capability (usually). REQUEST is kind of RPC-ish, while different
>> accessURL for different capabilities is RESTful. Requiring REQUEST means
>> that the RESTful-thinking folks have one boilerplate param to include
>> while the RPC-thinking folks can also comply to the spec. This is only
>> possible and transparent if REQUEST is required, which is why DALI is
>> the way it is.
>
> I find both points unconvincing.  If all this would mean is that
> REQUEST is just plain useless in Datalink, I could just shrug and
> move on.  In Datalink, however, REQUEST is actually harmful, so allow
> me to re-iterate this.
>
> (a) Useless: The point about allowing RPC-ish patterns is IMHO moot.
> First, a pure "RPC-like" implementation of datalink isn't possible,
> as implementors still have to support all the VOSI endpoints (where
> I'm not sure I'd call them RESTful).  Everyone hence needs to parse
> the query path with the current standard, and I somehow don't see
> that future extensions will then go back to parameter-based dispatch
> *in addition to* the path-based dispatch that's probably not going to
> go away.
>
> If someone actually wants to have *custom* extensions using
> parameters, they can of course still do so without everyone having to
> implement a function-less but (with all the case-normalization and
> rejecting going on) still demanding piece of protocol.  I fail to see
> why the standard needs to say anything about that at all, except
> perhaps that some way for standards-compliant clients to discover
> to discover custom extensions should be in place, and that we have with
> the parameter enumeration in the capability element's interface
> element.
>
> As standard uses of REQUEST in general go, 10 years after SIAP WD
> 1.00 came out (where REQUEST apparently started -- 24 May 2004, happy
> anniversary in advance!), I'm still not aware of one: getData?
> Killed by this very standard, and when I lobbied for it as a REQUEST
> value, nobody bothered.  getCapability?  VOSI says it's a separate
> endpoint, so please let's not have two ways of retrieving the same
> data just for the heck of it.  getMetadata?  DALI defines MAXREC=0 to
> effect what it could be doing.  explainQuery?  The next thing to a
> standards draft to cover that functionality is the plan endpoint in
> the TAP implementation note, and there are two prototypes for
> comparable functionality I know of using endpoints, and none that
> would use REQUEST.
>
> Since it after this appears that given a choice, everyone goes for a
> separate endpoints rather than use REQUEST, maybe it shouldn't have
> gone into DALI.  As an excuse for me, I can say that back then
> when I still thought (part of) datalink would be based of
> REQUEST=getData, and I hadn't expected REQUEST to pop up in a
> single-purpose protocol like Datalink.
>
> I'll stop claiming REQUEST is useless when someone shows me a use that
> will go into a future standard with some credibility even after
> considering the fate of REQUEST in SIAP and SSAP.
>
> (b) Harmful: In "full" DAL protocols, REQUEST is just mildly annoying --
> if I had a penny for every time one of the DaCHS deployers complain
> that "my service is broken" and I had to somewhat embarrassedly
> explain that SIAP and SSAP just *always* require REQUEST=queryData
> and I couldn't do anything about it, I probably still couldn't afford
> ice cream in nutritionally relevant quanitites.  I'd not even be
> significantly less stressed.
>
> Datalink is a special case, though, because we're keeping those links in
> database tables in large numbers, hurl them around, possibly in funny
> formats, or people embed them in their HTML pages, they might mail them
> around or even include them in their papers: I fully hope they'll become
> Links To Data.
>
> That means: URL form matters.  And so we're here talking about:
>
> http://data.svc/datalink/links?ID=ivo://svc/data?foo.bar
>
> versus
>
> http://data.svc/datalink/links?ID=ivo://svc/data?foo.bar&REQUEST=getLinks
>
> -- with the two options, after point (a), being exactly equivalent in
> functionality.  The big difference is not length, it's the presence of
> an ampersand.  And ampersands suck everywhere: In TeX, on the shell, in
> XML, whereever.  People will curse us when they discover that for no
> good reason at all, they're seeing the equivalent of
>
> $ curl http://dc.zah.uni-heidelberg.de/datalink/links?ID=foo&REQUEST=getLinks
> [1] 24575
> $ <?xml version='1.0' encoding='utf-8'?>
> <VOTABLE version="1.1" ... ERROR message
> echo $REQUEST
> getLinks
>
> and they have to back and quote things (and maybe un-set the new
> REQUEST shell variable), the equivalent of
>
> $ tex zw
> ...
> ! Misplaced alignment tab character &.
> l.1 ...erg.de/datalink/links?ID=foo&
>                                                    REQUEST=getLinks
>
> and they have to get back and add backslashes, and the equivalent of
>
> $ xmlstarlet val -e zw.html
> zw.html:1.70: EntityRef: expecting ';'
> <a href="http://data.svc/datalink/links?ivo://svc/data?foo.bar&REQUEST=getLinks"
>                                                                        ^
> and they have to go back and add an "amp;".
>
> Note that all these things would have just worked as expected without
> any quoting at all if we could only decide that, whether or not
> REQUEST was a mistake for DALI, it definitely has no place in an
> auxillary protocol like Datalink.
>
> And I don't think the standards police will come after us if we say:
>
>    As Datalink is an auxillary, single-purpose protocol, Datalink
>    services do not support DALI's REQUEST parameter.
>
> Cheers,
>
>          Markus
>

-- 

Patrick Dowler
Canadian Astronomy Data Centre
National Research Council Canada
5071 West Saanich Road
Victoria, BC V9E 2E7

250-363-0044 (office) 250-363-0045 (fax)