SDM evolution roadmap

[CresitelloDittmar, Mark]

Markus,


On Tue, Mar 21, 2023 at 8:26 AM Markus Demleitner <
msdemlei at ari.uni-heidelberg.de> wrote:

> Hi Mark,
>
> On Mon, Mar 20, 2023 at 02:22:10PM -0400, CresitelloDittmar, Mark wrote:
> > The UTypes and groupings look good for the most part.. a couple notes:
> >       ** secondary comment that it'd be easier to read overall if the
> > PARAMref-s had the UTYPEs, but the PARAMs did not.  But they did help me
> > find the match!.
>
> Well, I'll do whatever you (or preferably the spec) tell me :-).
> It's easy to remove the utypes from the PARAMrefs.  Should I?
>


As far as I know, there are no rules for which elements can/should have
UTypes.
The spec doesn't mention/use PARAMrefs at all, but I like what you've got
there.
It's very much like you have GROUP/PARAMref annotation for the flat PARAMs.
I do *not* think you should remove utypes from the PARAMrefs.




> >    o a few PARAMs with UTypes are not included in their respective GROUP.
> >       - "spec:Spectrum.Target.Class"
> >       - "spec:Spectrum.Char.FluxAxis.Ucd"
> >       - "spec:Spectrum.Curation.Publisher"
>
> These are all easy, and I've moved them.
>

Great!


>
> >       - "spec:Spectrum.Data.SpectralAxis.order"
>
> This one is hard, because I can't (easily) put it in the general
> SDM-making code.  How important is it to have it in the groups?  Is
> this a validity issue?
>

Well.. Section 8 of the spec says "the 'Data' element and the 'Point'
elements
are implicitly represented by the table structure itself", but the VOTable
has a GROUP
with utype="spec:Data" containing the data related elements.

I was looking at it from the implementation standpoint.  If I can rely on
your GROUPs to
define the objects/structure, then its much easier.  With even just this
one exception,
I'd have to interpret the GROUPs, and then the individual PARAM/FIELDs to
look for
drop-outs and then figure out where they fit in the hierarcy.



> >   o two errors
> >       - "spec:Spectrum.Target.Redshift" is in Target GROUP, but also in
> > Derived GROUP.
> >         These are different (model-wise), and the Derived value would
> have
> > UType "spec:Spectrum.Derived.Redshift.Value"
>
> Ah.  Yes, that makes sense.  Dropped it from Derived.
>
> >       - "spec:Spectrum.Length" has no value, but the DATA binary stream
> > definitely contains rows.
>
> As long as it's a NULL (rather than a 0 or some other concrete
> length): is that a validity issue?  This is a bit of a special case
> in the echelle spectra.  You see, normally these PARAMs are filled
> from SSA metadata in DaCHS, but for the orders I don't have the
> length in there, which is why this is NULL.
>
> I could do some special and extra code, but that's unattractive as
> long as nobody bothers to look there anyway (and I think clients
> bothering can just look into the data stream, somewhat more easily
> than I can, even).
>
> So, as long as I'm not lying and claim a bad length: is that really an
> error?
>
>
OK.. Table 1 says the default is "(must be derived)", so I suppose a null
value would indicate
that the reader must derive the value from the table.



> > Other comments:
> >   o most of the PARAMs have no value... so while they are there.. they
> > aren't terribly useful.
>
> Yeah; I'd suppress them if I then didn't have to tear down the
> references, too.  Morale: If you're doing things like these groups,
> have a strong reason; references aren't free at all.
>
> >   o UCD discrepancies (no comment on legality/correctness, just
> differences
> > from the spec.)
> >      - "spec:Spectrum.Curation.PublisherDID":  missing specified UCD
>
> Well, the UCD given in SpectrumDM is UCD-invalid. Another case for my
> point that we should stop requiring specific UCDs for columns and
> params in standards.
>
> I now went for meta.ref.ivoid from
> <https://wiki.ivoa.net/twiki/bin/view/IVOA/ObsCore-1_1-Erratum-1>;
> this could be a good opportunity to update this UCD in SpectrumDM,
> too (or, even better, make it explicit that the UCDs given in the
> example are good patterns but not normative).


> >      - "spec:Spectrum.Char.SpatialAxis.Coverage.Bounds.Extent":
> > spec="instr.fov"  file="phys.angSize;instr.fov"
>
> Ah... instr.fov is a again invalid.  See
> <https://wiki.ivoa.net/twiki/bin/view/IVOA/SSA-1_1-Err-2>.  I'll not
> make this UCD-invalid -- perhaps update the spec?
>
> >      - "spec:Spectrum.Length": missing UCD
>
> But that's the case in the spec, too?  And putting meta.number here
> or so indeed doesn't seem helpful.
>
> >      - "spec:Spectrum.Char.FluxAxis.Accuracy.StatError": UCD is missing a
> > bit at end. spec has "em.*" which would presumably be "em.wl" in this
> > dataset but file just has "em"
>
> The ";em" is fine *UCD-wise*.  Whether it's helpful is another
> question, and given the "must be one of" on p. 15 of the 1.1
> document, I'm in violation of the current spec here (I frankly don't
> remember whether I did that on purpose all these years ago).
>

The UCD discrepancies are more for the record than anything.
This is one place where the project scope can creep and cause delays.
We decided at the start that we were not trying to fix all the issues in the
current standard, but focus on adding the RFE elements.

I'll add an Issue to review the UCDs to the repo.


>
> It wouldn't be hard to fix this for this particular case, but in
> DaCHS I'd need to parameterise it so it won't lie when people publish
> spectra over frequency or energy, and that's an actual complication
> *to my adopters* that I frankly would only like to resign into if
> someone promises me they'll write code inspecting these UCDs one of
> these days (and then look for wl, freq, or energy).


> So, does someone?  If not, can we relax the "must" language?
>
> Thanks for the review,
>

No problem!  Thanks for the quick turn-around!.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.ivoa.net/pipermail/dm/attachments/20230321/3cb8c0a8/attachment.htm>