Table Link questions

[Tom McGlynn (NASA/GSFC Code 660.1)]

Hi Markus,

I was on vacation last week, so sorry for the delay in responding...  
I'm still pretty confused but I've
tried to clarify my issues a little.  The biggest help would be a 
pointer to active services that use both
the service_def and which have multiple fields used to point to the link.

     Thanks,
     Tom


Markus Demleitner wrote:
> Tom,
>
> On Thu, Aug 13, 2015 at 12:26:10PM -0400, Tom McGlynn (NASA/GSFC Code 660.1) wrote:
>> 1. When we return a data link response we often will have a product for
>> which we have both a URL but which also has subproducts.  So we'd in
>> principle like to fill in both the access_url and the service_def for this
>> field.  Since the standard forbids that instead  we will return two rows.
>> It would be nice to associate these to make it clear that these are
>> effectively two representations of the same product.  Is that possible?
>> Does the ID field do this?
> The ID is constant for all rows pertaining to a given data product.
> There is no way right now to "group" rows that are "more related"
> than "to do with this dataset" (maybe "all calibration files" or so)
> other than giving them identical semantics or use recursive datalink,
> i.e., datalink links within datalink tables.
>
> I personally would only do this if I had large amounts of ancillary
> data that almost nobody is expected to look at.
My hope is to replicate the recursive structure of our data products.  
So if a given product has subproducts
we initially return a link to the product.  If the user is interested in 
drilling down, they can use the the service_def
to get the subproducts for that product.  The ultimate question I'm 
trying to answer is whether the TableLinks support this
model of data products.
>> 2. The syntax of the service_def is completely unclear to me...  Is it
>> intended that this is used in some way to specify the needed fields for a
>> recursive call to the same table link service or is it to be substitute in
>> values in the current table link service?  To clarify what I mean let me
>> give  examples of two possible interpretations.
>>
>>
>> An initial query response might include the following fragments.
>>
>>     <VOTABLE> ...<FIELD ID='rowID' ...> <TR><TD>1234</TD>...</RESOURCE>
>>         <RESOURCE type="meta" utype="adhoc:service">
>>       ... <PARAM name="accessURL" ...value="http://xxx" />
>>        ... <GROUP name="inputParams"><PARAM name=id ...ref="rowID"> ...
> So, this is a table coming back from a DAL service, *not* a datalink
> service, right?

Yes.
>
>> When I invoke   http://xxx?id=1234  I want to get two rows back (see the
>> first question).  The first points to the URL for the observation,
>> the second will request subproducts at the next level in the recursion.
> Ok, this is the datalink call.
Right.
>> These be given by the URL
>>     http://xxx?id=1234&products=sub1,sub2,sub3
> ...and that's now a call to a custom service that just happens to
> share its endpoint with a datalink service.
Did I get this right so far?

Not really "just happens"...  The idea is that the http://xxx URL is a 
generic data product service.  We give it a row
identifier (in practice we'd give both a table and row identifier) and 
it returns data products associated
with that row.  If no product is specified it returns 'root' products.  
But each root product may have one
or more subproducts that I'd like to link to.  Typically the root 
product will be a directory and
the subproducts subdirectories or specific files, but that's not required.



>> Two possibilities from the documentation....
>>
>>     <VOTABLE>... <FIELD="service_def"><FIELD="access_url">...
>> <TR><TD/><TD>http://heasarc/obs/1234</td>..
>> <TR><TD>id=1234&Products=sub1,sub2,sub3</TD><TD/>
>>     </VOTABLE>
> I don't quite get this one, I have to say.  But I guess this is meant
> to be the reponse on the *datalink* endpoint, right?  If so, why
> don't you just give the two URLs and be done with it?

Then I'm not showing the hierarchical structure of the products. There 
can be hundreds of different
product types for a given table.  Returning them all to the user without 
providing any structure is
potentially confusing.  However if using a service_def I can point to 
just the next level of hierarchy
of products then it keeps things comprehensible for the user.

Regardless, the question is what string do I put in the service_def to 
point to some more data products?
Can you provide a complete example?

>> or
>>     <VOTABLE>...<FIELD="service_def" ID="params"><FIELD="access_url">,,,
>> <TR><TD/><TD>http://heasarc/obs/1234</td>..
>> <TR><TD>id=1234&Products=sub1,sub2,sub3</TD><TD/>...
>>         <RESOURCE type="meta" utype="adhoc:service">
>>       ... <PARAM name="accessURL" ...value="http://xxx" />
>>        ... <GROUP name="inputParams"><PARAM name=id ...ref="params"> ...
>>        </VOTABLE>
> These guys (with the adhoc:service utype and the PARAM ref) are for
> embedding into the DAL requests, i.e., in the very first response,
> before any datalink is talked.  They *point to* a datalink service,
> they don't occur in their results.
That's why I say 'the former appears more natural', but I could see 
nothing that precludes
a tablelink response that defines a new table link.
>> The former seems more natural, but there's no place where the syntax is
>> defined --- especially if there are multiple parameters being used.
>> The later is better defined but seems a bit clumsy.  The documentation of
>> the table_def is opaque to me.
>>
>> Is either of these close to what was intended?
> If all you want is return two rows, you don't bother with service_def
> in the datalink response at all -- just return the links with the
> proper semantics.  Service_def only enters the picture if there's
> actually some code that works with, at least effectively,
> non-enumerated parameters (think cutout, where you certainly wouldn't
> want to list all possible cutouts).

There may be hundreds of products associated with the given row.  I want 
to allow the user
to descend that hierarchy in a controlled fashion.  But your words seem 
to be suggesting that
the service_def is not really intended to support this.  I'd still like 
to see some examples though.  Is there
an active service that uses this?

>
>> 3. If I need to use multiple fields in the original VOTable to generate the
>> link what is the ID field?  Is it just f1=v1&f2=v2 or
>> v1,v2 or ....?    Or am I misconstruing the meaning of the ID entirely?
> My original plan had been to require the full ID to be present in the
> table.  But the current standard now lets you, in the adhoc:service
> block, have as many PARAM ref="" things as you want and the client is
> expected to collect those together.  So, you'd just add more PARAMs.
>
> However, again, I suspect that's not the pattern you're looking for here.

No I think it is.   I'm just looking for the specific syntax by which 
these parameters are supposed to be joined to create the ID.
>
>> 4. Is it possible to use a <PARAM> instead of a <FIELD> as the thing
>> referenced in the inputParams?  In practice this can probably be added into
>> the accessURL instead, but if we could use PARAM's then the type="meta"
>> RESOURCE could be exactly the same for all of our queries
>> which would be nice.
> I don't think the current standard says anything about that.  But
> really, my answer would be: For robustness, don't play games and just
> put constant elements into the accessURL.  There'll always be clients
> that don't really expect PARAMs at the other end, and they're not
> necessary either -- keeping constant parts of the accessURL in
> VOTable PARAMs doesn't to me seem to help a lot with clarity,
> robustness, separation of concerns, or anything else desirable.  Or
> am I missing something?
>
> Cheers,
>
>            Markus
>
>
>