Measurements model

Fri Sep 20 17:13:13 CEST 2019

Markus,

Thanks for looking it over so quickly.. comment early and often!
Some comments in-line.

On Fri, Sep 20, 2019 at 9:10 AM Markus Demleitner <
msdemlei at ari.uni-heidelberg.de> wrote:

> Dear DM,
>
> On Fri, Sep 13, 2019 at 03:50:26PM +0200, Laurent Michel wrote:
> > This is especially true for Meas and Coords; you cannot read one
> > while ignoring the other.  This is why the 2 RFC periods have been
> > both opened today for 6 weeks (21/10).
> >
> > - Meas   https://wiki.ivoa.net/twiki/bin/view/IVOA/MeasRFC
>
> I've had a first read of Measurements (not really Coords, yet), and
> since I probably won't find time to do some actual annotation to data
> I hold until early November, I've posted some first comments on the
> RFC page already.  A few of these points I'd like to bring to the
> mailing list, since I feel there should be some community discussion
> about them, so here's the first installment.
>
> First, I am very much in favour of extending the RFC for this until
> we have the annotation syntax defined, at least at the level of a PR.
>
> True, for DMs the question of what "implemenation" means is always a
> bit tricky. In this particular case, however, it is very clear that
> most people will only properly look at things if they know what they
> will be doing with it. That is particularly true for client authors.
> I'd go as far as to say: I consider the DM implementation-proven if
> there's astropy-affiliated code consuming at least 60% of the model.
>

I'll leave that up to the TCG/Chairs... I've shown the model is
serializable in:
  o VOTable - straight
  o VOTable - annotated with the Mapping syntax
  o XML
All validating against their schema.
It's also been shown that Astropy can invoke instances from a normal
VOTable even with simplistic annotation (ucd, utype, name).. so can be only
easier with any formalized syntax.

>
> Having said that, there are also some more special points on the
> model, the most important of which is: I don't think there is a
> reason to have the subtypes of GenericMeasurement (except perhaps to
> tell discrete variables from continuous ones).  The pertinent
> comments from my RFC response are:
>
> (4) I don't think I quite understand what requirement makes you
> introduce "Time" over "Generic Measure" -- as far as errors are
> concerned, is there anything special about time? Why would I use it
> rather than Generic Measure, which, as far as I can tell, works just
> as well? If it's just about the value representation, I'd much prefer
> if it were left to the serialisation format (like VOTable) -- it's
> always evil if the same information is represented in two different
> ways. Similar considerations apply to position, velocity, and
> polarization (see below, though).
>
> Primarily, there are times where you want to specify the type..
  + Target.position  MUST be a Position, a GenericMeasure does not satisfy
this unless you add in the text that the coordinate MUST refer to a
SpaceFrame, which is an extra step a validator and client must manually
check.
  + TimeSeries MUST have a 'column' of Time data.  same detail as above.
  + If you look at a serialization with just GenericMeasures, its really
hard to know what it is, since you have to interrogate the Frame, which is
somewhat separated (by reference).  Having Position, and then the further
specialized types (eg: EquatorialPosition with ra,dec ) make it very clear
what you have.

> (5) I'm totally against having different classes for coordinates in
> different frames. It makes the model a lot larger without helping
> anything over the simple provision of a frame. And you'll have to say
> what should happen if you annotate a GalacticPosition with an
> equatorial frame. I may be swayed to accept that error modelling is a
> bit different in curvilinear coordindates (spherical, cylindrical,
> whatever) versus plain cartesian. But then the classes should be,
> perhaps, CurvilinearCoordinate versus CartesianCoordinate.
>

Well, the spec says the frame MUST specify a galactic reference frame, so a
decent validator would flag it as invalid.  The alternative is to have a
GenericMeasure with a SpaceFrame.refFrame=ICRS and NO way to know that the
2 coordinate values, in deg, are actually galactic(l,b).

> (7) Polarization -- again, I'd much rather not talk about concrete
> physics in Meas. If you really have to enumerate all the different
> things people can measure, that should be done somewhere else (and
> probably reuse resources like the UAT). Having some special treatment
> for Measurements over discrete domains sounds like a reasonable thing
> to want, though. For that, though, I'd say the diagram in sect. 9 is
> rather confusing. I read it as "Polarization inherits from Measure
> (which has 1:n to Error), and we somehow say n=0 here". To me, it'd
> feel a lot more natural if discrete observables inherited from
> something that doesn't have (numerical) errors in the first place.
> Also, of course, discrete distributions might make sound error
> models, too, so just saying "discrete values have no errors" is
> probably too limiting for a number of interesting use cases.
>

Including discrete types like Polarization states as an extension of
Measure was a conscious choice, which is pretty well justified, and allows
that sort of data to be included in a Cube very easily.

We have polarization data, it needs to be included in a Cube, so the
alternative would be to have a Parent to Measure with no Error, which can
then also be extended to an "AssignedState" type for thinks like
Polarization and other discrete types.  This parent would then be used by
Cube.

With Polarization specifically:
  + Measure has 0..* Error-s, Polarization constrains this to [0]
  + In working Polarization, we focused on the enumerated polarization
states.
     At the time, I did some looking about numerical polarization types,
but they weren't in STC, nor being flagged as interesting, so didn't
include them.
     However, looking at CAOM2 yesterday, I see that it includes the
following in addition to the typical enumeration set:
       POLI  - linear polarized intensity: sqrt(Q^2 + U^2)
       FPOLI - fractional linear polarization: POLI/I
       POLA  - linear polarization angle: 1/2 arctan(U,Q)
       EPOLI - elliptical polarization intensity: sqrt(Q^2 + U^2 + V^2)
       CPOLI -    circular polarization intensity: |V|
       NPOLI - unpolarized intensity: I - EPOLI

     I don't know anything about these, but the descriptions read very
numeric, and presumably would have error.  Migrating from where we are to
include
     those would be a small change to branch Polarization to a numeric type
and a State type. And I'd much prefer to keep them under the same umbrella.

> So -- perhaps I'm missing something big time, but if so, you'd need
> to hit me with a rather solid clue stick.
>
> On the other hand, if I'm not missing anything, I'd say the model
> should just consist of
>
> * Measure.value and Measure.error
> * Error.statError (and perhaps ranError, but there's another comment
>   on this on the RFC page)
> * Error.sysError
> * and some way to model correlated errors (there's yet another
>   comment on that).
>
>
The goal here is to make a model which allows any 'Measure' to be
described, but provides specializations to easily identify and use the most
common types found in our data.  Thus the Position, Time, etc.  The
discussions in recent interops regarding Source properties and such support
this approach.  I distinctly recall that it is a 'disgrace' that we cannot
represent a ProperMotion with current models.  The expectation going
forward is that a SourceProperties model (or other models) would define new
extensions of Measure as needed.
These would, automatically, be usable within a Cube.

I'll take a look at the other comments on the twiki.

Mark
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.ivoa.net/pipermail/dm/attachments/20190920/917af1a1/attachment.html>