Templating in VOTable

[Thomas McGlynn]

Francois suggested that this might be of interest
to the DAL group, and so I've posted it there.
Note that the suggestion below is really just
a sketch of what might be possible, not a real
proposal.
	Tom McGlynn


Francois Bonnarel wrote:

> Dear Tom,
>     At the DAL meeting in Strasbourg , when I proposed to template
> the URL in the acref in SIA, your answer was to propose a templating
> in VOTable cells in general, as a wider solution. I rediscussed this
> recently with François Ochsenbein. We are interested. have you an idea
> of how it could work?
> Cheers
> 
> 
> François
> 


Hi Francois,

Below I'll sketch out a couple of ideas, but the basic concern I have
is that templates are potentially useful in many contexts, so
that whatever we do should probably not be associated with
a single protocol (e.g., SIA), but done more generically.

One approach would be for the VO to assert that all data transmission
protocols at the VO inherit from some base protocol and that templating
would be addressed at this highest level.  E.g., this high level
protocol would allow any XML file to be transferred either as a standard
XML file, or as a combination of a base XML file and an XSLT script.
The script would be run on the base file to produce the standard
file.  This would support templating and a lot more, and leverages
existing standards.  However I'm not sure that we'll be able to
retrofit this into existing protocols.

The other place where it would seem natural to define the templating
mechanism is for VOTables themselves.   Then it naturally transfers
to all protocols that return VOTables.  I'm sure there are
better ways, but here's a suggestion for how it might
look:

    Define a new element <SUB> which is allowed inside
the <RESOURCE> and <TABLE> elements.
<SUB> has KEY and VALUE elements (or attributes I don't really
care).  In any <TD> element that follows a <SUB>, a string
of the form ${nnnn} is replaced by the value of the last SUB
element with the key nnnn (or left unchanged if there
is no such element). This addresses the repetition of long URLs
so that they can be specified once and then given as a short string
Only the parameters that change need be specified fully on each line.

However you were also concerned about the multiplication
of rows required when we had lots of potentially varying
parameters.  We can expand the idea above to handle this.
E.g., let's have:

     <SUB key=baseurl>
         <VALUE>http://skyview.gsfc.nasa.gov/cgi-bin/nnskcall?</VALUE>
     </SUB>
     <SUB group=form key=format>
         <VALUE>GIF</VALUE>
         <VALUE>FITS</VALUE>
         <VALUE>TXT</VALUE>
     </SUB>
     <SUB group=form key=query>
         <VALUE>QUICKLOOK</VALUE>
         <VALUE>DATA</VALUE>
         <VALUE>TEXT</VALUE>
     </SUB>
     <SUB key=survey>
         <VALUE> DSS </VALUE>
         <VALUE> 2MASS-K </VALUE>
         <VALUE> ROSAT</VALUE>
     </SUB>

Now an SIA result from SkyView might have a single line
where the URL and format columns would be:

     <TD> ${baseURL}SURVEY=${SURVEY}&RETURN=${query}&VCOORD=32.13,12.97 </TD><TD>${format}</TD>

There are four different substitutions going on here.  The first is just
the base URL that we had before, but we now also have three other substitutions
which each have three values.  The idea (and perhaps this goes too far), is that
we could allow this to expand to multiple rows in the output.
This example row would be intended to expand to 9 rows.  Since
the format and query fields have the same group key, they are varied together
and must have the same number of elements.  Thus we always generate lines
with where the format is GIF associated with the query being QUICKLOOK.
However the survey key does not have the same group as the others so it
we vary it independently.  So after substitution we get:

      <TD>http://skyview.gsfc.nasa.gov/cgi-bin/nnskcall?SURVEY=DSS&RETURN=QUICKLOOK&VCOORD=32.13,12.97 </TD><TD>GIF</TD>
      <TD>http://skyview.gsfc.nasa.gov/cgi-bin/nnskcall?SURVEY=DSS&RETURN=DATA&VCOORD=32.13,12.97 </TD><TD>GIF</TD>
      <TD>http://skyview.gsfc.nasa.gov/cgi-bin/nnskcall?SURVEY=DSS&RETURN=TEXT&VCOORD=32.13,12.97 </TD><TD>GIF</TD>
      <TD>http://skyview.gsfc.nasa.gov/cgi-bin/nnskcall?SURVEY=2MASS-K&RETURN=QUICKLOOK&VCOORD=32.13,12.97 </TD><TD>GIF</TD>
      <TD>http://skyview.gsfc.nasa.gov/cgi-bin/nnskcall?SURVEY=2MASS-K&RETURN=DATA&VCOORD=32.13,12.97 </TD><TD>FITS</TD>
      <TD>http://skyview.gsfc.nasa.gov/cgi-bin/nnskcall?SURVEY=2MASS-K&RETURN=TEXT&VCOORD=32.13,12.97 </TD><TD>TXT</TD>
      <TD>http://skyview.gsfc.nasa.gov/cgi-bin/nnskcall?SURVEY=ROSAT&RETURN=QUICKLOOK&VCOORD=32.13,12.97 </TD><TD>GIF</TD>
      <TD>http://skyview.gsfc.nasa.gov/cgi-bin/nnskcall?SURVEY=ROSAT&RETURN=DATA&VCOORD=32.13,12.97 </TD><TD>FITS</TD>
      <TD>http://skyview.gsfc.nasa.gov/cgi-bin/nnskcall?SURVEY=ROSAT&RETURN=TEXT&VCOORD=32.13,12.97 </TD><TD>TXT</TD>
      <TD>http://skyview.gsfc.nasa.gov/cgi-bin/nnskcall?SURVEY=DSS&RETURN=QUICKLOOK&VCOORD=32.13,12.97 </TD><TD>GIF</TD>

Any other rows in the table are duplicated. [I left the VCOORD stuff out of the
base URL since I wasn't being very smart, but it does help illustrate
how explicitly specified and templated parameters interact.]

There are probably better ideas for the syntax here, but something
along these lines is what I had in mind.  Of course it's essential
that a user be able to request that the data be expanded before transmission.

This kind of approach which uses a custom substitution scheme is probably
easier to incorporate into existing protocols, but it's less general
than the kind of XSLT based approach.  That's not entirely a
disadvantage.  One can probably make the input files
quite incomprehensible with the first approach.

Just to reiterate in closing, I like  the idea
of templating, I'd just like to see it in a broader context
than the SIA service.

	Regards,
	Tom