Entangled Models [Was: MCT - model document delivery.]

[Markus Demleitner]

Dear Mark,

On Wed, Sep 30, 2020 at 01:43:18PM -0400, CresitelloDittmar, Mark wrote:
> Your arguments always seem to boil down to how it will look/work when
> annotated in a VOTable.

Well, call me a cold utilitarian, but yes, for technologies in
general the question what one can and cannot do with them is very
much in the front of what I'm interested in.

In the present case, I'm worried that by certain choices we will
either fail to provide a way to epoch-transform catalogues for the
third (or fourth, depending on how you count) time or that, if we
manage to get people to adopt what we're desinging, that we're
leading them into something that cannot be evolved, which would then
mean we'd have to start a fourth (or fifth) attempt 10 years down the
road -- which will be even more painful.

So, I'm afraid I have to keep being a pain until I'm confident both
worries are unsubstantiated.

> The models serve a broader audience than that.
> 
> To define isolated models, and then tell providers that
>   * If you annotate model A, you must also annotate to models B and C
>  is

We are telling people "if you have a position, annotate it like this,
regardless of where the position is"; "if you have a value/error
relationship, annotate it like this, regardless of where it comes
from" -- and so on.

If I were an adopter, I think I'd pretty much expect things to work
like this, I have to say.

>   * impossible to validate that you have a usable instance of A.

The tricky part here is "usable".  Once you start to say "usable for
what" you'll notice that having independent models this sort of
validation becomes very much possible (see the examples in
http://mail.ivoa.net/pipermail/dm/2020-September/006126.html -- a
validator can confidently answer the question "will a cube client be
able to make an informed plot of this dataset).

Indeed, such an approach will largely avoid the problem we have in
service validation right now, where lots of services appear to be
invalid but work perfectly in practice (because only some aspect of
them is invalid that the common clients don't need).  More on this,
if you want, in the later TCG discussion.

>   * not going to facilitate interoperability very well

Here, I have to say I'd be curious about your reasoning, because I'd
claim the isolated models are almost a pre-condition of
interoperability on the longer term.  Let's see if we can re-visit
this in the TCG discussion.

> But more importantly, the isolated model approach, especially where
> Coords/Meas is concerned, goes against the expressed goal/direction that
> the data model working group has been working toward for over 10 years.
> 
>    IVOA Architecture Document:
> <http://www.ivoa.net/documents/Notes/IVOAArchitecture/20101123/IVOAArchitecture-1.0-20101123.pdf>
> (2010) - Section 6
>    "Some of the VO Data Models are specific to a type of data collection
> (e.g. Spectrum DM, SSLDM, ObsCoreDM, ObsProvDM, PhotDM, SimDM), while
> others (e.g. *STC,* Units, Utypes, CharDM)* are more foundational and are
> components of the more specific Data Models*. In general, each Data Model
> can be considered as a building block which can be referred from some other
> Data Models."

Well, this was written more than 10 years ago (and before the utypes
tiger team); the outlook what DMs should be doing has changed quite a
bit since then.  More importantly, we have learned a lot since then
about what we can and cannot do in the VO, and in particular about
the pains and difficulties of managing minor version steps -- worse, we
still haven't succeeded in finishing a major version step in a
deployed protocol (except perhaps from Registry Interfaces 1 to
RegTAP, but that's a very special case).

I'd say it's reasonable to reconsider the 2010 stance, in particular if
there (undisputedly, as far as I can see) is the problem of the
update avalanche I've described below the little graph in
http://mail.ivoa.net/pipermail/dm/2020-September/006123.html.
If we keep the notion of the cross-referencing DMs, we will
have to come up with an answer on how to avoid this avalanche, or how
we might be able to manage it in case we need to incompatibly revise
a DM.

Or, perhaps, isolate actual users of the DMs from the DMs and their
changes.  But that sounds like a really painful thing.

> So, in my opinion, this whole discussion is rather moot, unless the
> TCG has decided to abandon this direction and the work done toward
> realizing it, to adopt a different approach.  I say TCG because the
> models support the other groups.. and so affects them as well.
> Given this, I will try to refrain from further contributions to
> this thread.. unless I'm told that we are, in fact, seriously
> considering a change in approach.

Well, fair enough.

I'll happily carry this to the TCG some time next week unless the DM
WG tells me they'd like to do that themselves.

        -- Markus