More UCD woes

[Markus Demleitner]

Hi Pat,

On Wed, May 04, 2022 at 03:12:23PM -0700, Patrick Dowler wrote:
> I am sure the intent was always to move with later versions of ObsCore, so
> I think it is just that at the time we put the ref to the only existing
> version. The language you quoted is certainly my attempt to just refer and
> not repeat the ObsCore spec. How should we be making references so they are
> forwards compatible e.g [ObsCore-1.0, ObsCore-2.0)? If we could have done
> that, is this an erratum to fix it?

I absolutely agree with the intent; and I think one way to say that
would be something like:

   The response for a successful call to the {query} resource is a
   table resulting from a \verb|SELECT *| against an ivoa.obscore table
   conforming to any minor version of Obscore
   1\footnote{\url{https://ivoa.net/documents/ObsCore}}.  This
   document was written against Obscore 1.0
   \citep{2011ivoa.spec.1028T}.

This may present a bit of a challenge to clients (or, if you will,
validators).  For my s_region UCD problem, that is almost certainly
irrelevant (and I would argue we should never have made UCDs a hard
requirement, as nothing *in the protocol* breaks when they are off;
see also the rich history of UCD-related errata).

Adding fields should also not be a problem, as "Obscore 1" clients
would have to check for their presence anyway and, thanks rich
VOTable metadata, can straightforwardly do that.

Dropping fields, however, may conceivably break clients.  I think if
a client breaks when a non-mandatory field is dropped, the client is
buggy.  However, writing something like the above means we cannot
drop mandatory fields in Obscore without a new major version of
Obscore (or perhaps a long phase of depreciation).  But that would
already be so without SIAP2: We wouldn't want to break queries just
using guaranteed Obscore features.

> Practically, I can think of an way to support both versions
> (ObsCore is a view already) by having an ObsCore10 view with
> associated metadata in tap_schema and then having SIA2 query that
> view instead of ivoa.ObsCore, but it's ugh and I think users would
> want/expect SIA and TAP queries of ivoa.ObsCore to be consistent...
> so I'd rather allow it than do this.

I totally agree.  The price we pay is that the results of SIAP2
queries to different services cannot not trivially be merged in a
conceptual UNION.  But then that was never possible (without
precautions) because you could always have custom fields, and you can
still do it by restricting yourself to mandatory Obscore 1.0 columns
or by extending records with NULLs where necessary.

I'd say something like my text above might work as an erratum (though
I'd like to hear Apps' and Ops' opinions).  On the other hand:
Perhaps a maintainance release of SIA2 would be useful anyway, mainly
to move the thing to ivoatex.  I would also like to contribute a
section on "Registering and Discovering SIA 2 Services".

       -- Markus