[utypes] Versioning, open issues

Wed Feb 8 14:15:34 PST 2012

Markus,

> I have to say I largely agree with her.  In particular, I don't think
> we want versions in the utype strings, neither before nor after the
> colon.
>

Where did you see versions in the utype strings? I didn't put versions in
the utype strings, so I don't understand your comment.

>
> As to use cases, I'd like to see more trivial ones added, too; most
> importantly, I'd like to see basically this:
>
> """
> Applications exchange metadata on positions (reference system,
> reference position, epoch and equinox, etc) or similarly structured
> physical quanities (magnitudes, radial velocities, etc) and can
> reconstruct complete instances of classes defined in the data model
> from VOTables.  Applications that do not understand a data model
> easily preserve such data and metadata.
> """
>
>
Why only VOTables? Make an experiment. Take a compliant votable spectrum
1.03 file (e.g. one served by ned, like:
http://vo.ned.ipac.caltech.edu/services/accessSED?REQUEST=getData&TARGETNAME=3c273).
Convert it to FITS using Topcat or STILTS.

Nice, you now have a non-compliant spectrum FITS file, because there is no
way a model-unaware VO application can preserve metadata when converting a
file from fits to votable and vice-versa.

Make another experiment. Take a votable with GROUPs in them (one of your
own or the previous one from NED). Open it with topcat. Save it again as a
VOTable. Nice, you have lost the GROUPs.

Do you get my points now? The current IVOA practices for Data Model
serialization are not interoperable.

Otherwise utypes aren't opaque strings, which I
> think would be bad (that property, and [from my view, unfortunately]
> case-insensitiveness, have been "requirements" for utypes for a long
> time; I don't think we can drop them now, even if we wanted).

I don't know why you keep ignoring my replies... I showed that utypes are
not opaque in the current recommendations :)

An example (and more to come)?

photdm:PhotometryFilter.SpectralAxis.Coverage.Location.Value
spec:Spectrum.Char.SpectralAxis.Coverage.Location.Value

They point to the same concept defined in Characterization. An application
that knows about CharacterizationDM and doesn't know about PhotDM and
SpectrumDM cannot use those utype. Unless you say that they are not opaque
and they are parsable. For instance, you might say that SpectralAxis is a
reserved word that refers to the CharDM::SpectralAxis concept. Then, the
characterization-aware application can *parse* the utype, search for the
SpectralAxis string and build a prefix for the SpectralAxis object
(photdm:PhotometryFilter.SpectralAxis in one case,
spec:Spectrum.Char.SpectralAxis in the other).

Or, more easily, the file itself might declare the prefix and associate it
with the relevant concept (e.g. photdm:PhotometryFilter.SpectralAxis ->
char:SpectralAxis), which is, surprise surprise, my suggestion, if you read
it :)

If you want you can keep stating that utypes are opaque, but the reality is
that they are not.

Not exactly.  Having it in two places makes it easier to ignore it
> when you don't care.

It is in two places in either case:
  - "dm=char-1.0". The dm name is between "=" and "-". The dm version is
after "-"
  - "dmname=char", "dmversion=1.0". The dm name if after "=", the dm
version is after "=" in a different field.

In both cases it is a convention. Since they are equivalent I will agree
with your option and move on. For the rest, you keep ignoring that my
examples described what you describe so, again, I will move on, noting that
we agree 100%.

Example: When I interpret an ObsTAP result, my client may only need
> to know the footprint.  So, it looks for a column with the utype

> obscore:char.spatialaxis.coverage.support.area

Two sets of questions:
1. How is this utype different from the ones in my example?
2. Does this utype refer to a different concept than
spec:char.spatialaxis.coverage.support.area? How can you (a human VO
expert) tell? Can a model-unaware application automatically tell? Consider
the two aforementioned utypes from PhotDM and Spectrum1.03. Can you answer
the same questions?

Markus, you spend most of your email keep trying to convince me that
clients can avoid choking on little differences in data model versions. How
can I convince you that I agree and that this very same concept was part of
my original, awful, email? Do you remember that I was the one
enthusiastically shaking my hands during your talk in Pune?

There's no such list.  It would be nice to have one, though, and I
> could well imagine keeping a list of ivoa-sanctioned data model names
> in the registry.  If we agree to have one, it would be easy to
> maintain it as a StandardKeyEnumeration in the utype standard's
> registry record.

If The Things are universally defined they should be maintained somewhere.
You can't ask a new VO service developer to read all the IVOA documentation
just to get the list of reserved Things. Note that reading all the Data
Model documents wouldn't be enough, since some Things are defined in the
DAL documents (e.g. ssa:).

No -- for figuring out the data model version you don't need any
> network access at all.  You can simply compare the mod:DataModel.URI
> values, and that's it.

And so you need a standard for the versioning string, something along the
lines of "char-1.0".

I'd not be opposed to adding a mod:DataModel.version item if you want
> easier access to the version.

Me neither. Let's do it. I still don't understand the "mod:" part, which is
useless. The client is looking for the instance's Data Model name and
version, but it needs to know the Thing beforehand? Really? And since the
utypes are opaque, and they are fixed and unparsable, once the client knows
the Data Model, all utypes are univocally defined, right? So why would you
need the Thing in the first place?

Operations involving the registry would be exclusively for "Learn
> more about this data model"-type operations directed at the user.
> Requiring network access in such situations is IMHO reasonable.

Again: how does the Char-aware, non-Spectrum-aware, non-Obscore-aware,
non-PhotDM-aware application know that photdm:PhotometryFilter.SpectralAxis
and spec:Spectrum.Char.SpectralAxis are the same thing?
Again: now that Spectral2.0 removes the "Spectrum." prefix from utypes and
change the Thing from spec: to sdm: how will a Spectrum1.3-aware
application know that spec:Spectrum.Char.SpectralAxis and
sdm:Char.SpectralAxis are exactly the same thing?

Well, for me the mail body
> http://www.ivoa.net/pipermail/utypes/2011-December/000019.html mainly
> is about a data modelling process that integrates utypes -- right?

Not only. It provides a way to allow clients to automatically understand
that sdm:Char.SpectralAxis, spec:Spectrum.Char.SpectralAxis
and photdm:PhotometryFilter.SpectralAxis are the same thing.

* Generation of utypes (how do we get from some DM to utypes?)
>

I guess I have said much about this, you can refer to my original email,
amended with Mireille comments.

> * Versioning (how do clients that care figure out the DM?)
>

Again, refer to my email for my opinion and a working prototype for this

> * Typedness (is it ok to only talk about strings in the utype REC?)
>

Again.

> * Serialization of structures of utype-value pairs in VOTable
>  (e.g., would it be cool to push GROUPs as a means for embedding?)
>

GROUPs are not interoperable and require model-awareness to be dynamically
treated. We need a more abstract tool for doing the same thing. I guess I
will ask Mark Taylor (again) to support GROUPs in Topcat. Or, if he doesn't
want to, to explain us all, and not only me, the good reasons why he is
against GROUPs support in Topcat.

> * any other serializations (e.g., do we want to recommend a means for
>  binding together utypes and FITS header keys?)

As I mentioned we *must* do it, because at the moment we can't even do a
trivial roundtripping with a model-unaware application. I have a
suggestion, if anybody wants to hear about it.

Cheers,

Omar.

-- 
Omar Laurino
Smithsonian Astrophysical Observatory
Harvard-Smithsonian Center for Astrophysics
100 Acorn Park Dr. R-376 MS-81
02140 Cambridge, MA
(617) 495-7227
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.ivoa.net/pipermail/utypes/attachments/20120208/a6be667d/attachment-0001.html>