DM Workshop wrap-up
Mark Taylor
m.b.taylor at bristol.ac.uk
Wed Jul 21 18:58:38 CEST 2021
Tom et al.,
I haven't thought deeply about document structure in this context,
but I'd be a bit cautious about mandating where such RESOURCE/VODML
elements could appear; e.g. is it really only ever required to have
a single VODML element in a VOTable document?
One (admittedly non-standard) use case to consider: topcat can save
a session in VOTable format. That takes the form of a single VOTable
document containing all the (perhaps unconnected) tables (down)loaded
or created by the user in the session, with in some cases associated
metadata in RESOURCEs identified by obvious document scope rules
(e.g. sibling elements to the TABLE). If different VODML chunks are
loaded for different input tables, I'd find it hard to serialize
to a VOTable document in this way if it only permitted a single
VODML element.
On the other hand if there is only a single VODML element present
alongside a TABLE or set of linked TABLEs in a given VOTable document,
it's usually pretty straightforward to find it during SAX or DOM
processing, without really needing detailed rules for where to look.
So: if there's a strong case for requiring such rules then go ahead,
but I'd hesitate to impose them just because it looks tidy.
One rule that might be important: require the VODML to appear
earlier in the document than its associated table data, so that
streaming processors can use it to pull out the relevant objects
as they are streamed, rather than having to buffer the XML content.
I don't know if that actually makes sense, but I can imagine it
might do.
Mark
On Wed, 21 Jul 2021, Tom Donaldson wrote:
> Hi All,
>
> Thanks, Mark, for looking into that. This does seem promising. I would like to do a few experiments with our existing parsers/validators to be sure, but it seems good.
>
> If we use this approach I would propose that we give strict guidance on the RESOURCE where VODML content lives. Since RESOURCEs can show up all over the place in a VOTABLE, I think it would be good to mandate things like:
>
>
> * Only one such RESOURCE
> * Only one possible location (e.g., the first RESOURCE)
> * With well-defined scope (most likely it should be a top-level stand-alone RESOURCE, not nested in or containing other RESOURCEs)
> * With type=”meta”
> * With a specific ID and/or name?
>
> Thanks,
> Tom
>
>
> From: <dm-bounces at ivoa.net> on behalf of Laurent Michel <laurent.michel at astro.unistra.fr>
> Date: Wednesday, July 21, 2021 at 11:39 AM
> To: Mark Taylor <m.b.taylor at bristol.ac.uk>
> Cc: "dm at ivoa.net" <dm at ivoa.net>
> Subject: Re: DM Workshop wrap-up
>
> External Email - Use Caution
> Hell all,
>
> This would be a very clean solution.
>
> - Gerard proposed to rephrase the XSD in order to isolate complexTypes from elements which should make easier the validation in the context of a VOTable.
> - We have been using XSD1.1 (with Python) because the merged syntax put many constraints on element attributes. This is a key point of the proposal. I do not think that such rules could be set with XSD1.0
> - The xerces web site does not look worry about this (https://xerces.apache.org/xerces2-j/faq-xs.html<https://urldefense.com/v3/__https:/xerces.apache.org/xerces2-j/faq-xs.html__;!!CrWY41Z8OgsX0i-WU-0LuAcUu2o!nQi4ELtSSzKUpZjOVSCN0rLK20EZWxEHU8Fskrp9js4kx9qVU47SL56je5h6c5iFdQ$>) but I’ve no experience with this code
> - I’ve no idea about how could this fit with volint either.
>
> Best
> Laurent
>
>
>
> On 21 Jul 2021, at 17:02, Mark Taylor <m.b.taylor at bristol.ac.uk<mailto:m.b.taylor at bristol.ac.uk>> wrote:
>
> François,
>
> Yes, I think that use of the xs:any type is the right way to go here.
>
> However, that is already accommodated by the VOTable schema as it stands.
> Quoting from https://www.ivoa.net/xml/VOTable/VOTable-1.4.xsd:<https://urldefense.com/v3/__https:/www.ivoa.net/xml/VOTable/VOTable-1.4.xsd:__;!!CrWY41Z8OgsX0i-WU-0LuAcUu2o!nQi4ELtSSzKUpZjOVSCN0rLK20EZWxEHU8Fskrp9js4kx9qVU47SL56je5i1_43vxg$>
>
> <xs:complexType name="Resource">
> ...
> <!-- Suggested Doug Tody, to include new RESOURCE types -->
> <xs:any namespace="##other" processContents="lax" minOccurs="0" maxOccurs="unbounded"/>
> ...
> </xs:complexType>
>
> which means that the elements from the VODML schema can be included
> within a RESOURCE element in a VOTable document, with no changes
> to the VOTable standard required at all (thanks to Markus for
> pointing this out).
> So something like this should work out of the box:
>
> <VOTABLE version="1.4"
> xmlns="http://www.ivoa.net/xml/VOTable/v1.3<https://urldefense.com/v3/__http:/www.ivoa.net/xml/VOTable/v1.3__;!!CrWY41Z8OgsX0i-WU-0LuAcUu2o!nQi4ELtSSzKUpZjOVSCN0rLK20EZWxEHU8Fskrp9js4kx9qVU47SL56je5h2xvKE0w$>"
> xmlns:dm="http://www.ivoa.net/xml/vodml<https://urldefense.com/v3/__http:/www.ivoa.net/xml/vodml__;!!CrWY41Z8OgsX0i-WU-0LuAcUu2o!nQi4ELtSSzKUpZjOVSCN0rLK20EZWxEHU8Fskrp9js4kx9qVU47SL56je5iiGyvehQ$>">
> <RESOURCE type="meta">
> <dm:VODML ...>
> <MODELS>
> </MODELS>
> ...
> </dm:VODML>
> </RESOURCE>
> <RESOURCE type="results">
> <TABLE>
> ...
> </TABLE>
> </RESOURCE>
> </VOTABLE>
>
> A VOTable document like that validates for me under the VOTable 1.4
> schema (e.g. xmllint -noout -schema VOTable1.4.xsd). At present votlint
> (http://www.starlink.ac.uk/stilts/sun256/votlint.html<https://urldefense.com/v3/__http:/www.starlink.ac.uk/stilts/sun256/votlint.html__;!!CrWY41Z8OgsX0i-WU-0LuAcUu2o!nQi4ELtSSzKUpZjOVSCN0rLK20EZWxEHU8Fskrp9js4kx9qVU47SL56je5iXtd5-Iw$>) does complain:
>
> WARNING: Element in wrong namespace (http://www.ivoa.net/xml/vodml<https://urldefense.com/v3/__http:/www.ivoa.net/xml/vodml__;!!CrWY41Z8OgsX0i-WU-0LuAcUu2o!nQi4ELtSSzKUpZjOVSCN0rLK20EZWxEHU8Fskrp9js4kx9qVU47SL56je5iiGyvehQ$> not http://www.ivoa.net/xml/VOTable/v1.3<https://urldefense.com/v3/__http:/www.ivoa.net/xml/VOTable/v1.3__;!!CrWY41Z8OgsX0i-WU-0LuAcUu2o!nQi4ELtSSzKUpZjOVSCN0rLK20EZWxEHU8Fskrp9js4kx9qVU47SL56je5h2xvKE0w$>)
>
> but that's really a votlint bug that I will fix.
>
> I would certainly favour this approach rather than adding new VOTable
> elements, to avoid unnecessary coupling between the VODML and VOTable
> standards.
>
> Concerning the xsi:type="mapping:VODML-type" attribute:
> as I understand it, that wouldn't work with a schema in the form
> of the one at
>
> https://github.com/ivoa-std/ModelInstanceInVot/blob/master/schema/xsd/merged-syntax.xsd<https://urldefense.com/v3/__https:/github.com/ivoa-std/ModelInstanceInVot/blob/master/schema/xsd/merged-syntax.xsd__;!!CrWY41Z8OgsX0i-WU-0LuAcUu2o!nQi4ELtSSzKUpZjOVSCN0rLK20EZWxEHU8Fskrp9js4kx9qVU47SL56je5jFLJq0uQ$>
>
> since that defines elements directly and not types (Gerard queried
> this stylistic decision during the DM workshop session #4).
>
> So I'm not quite sure how validation of the embedded VODML would
> proceed: either the schema could be rephrased to define types, or
> perhaps XSD validators are able to determine the required contents
> based on the <VODML> element name without requiring an xsi:type.
>
> One other point on the current merged-syntax.xsd: use of XMLSchema 1.1,
> instead of 1.0, makes it harder to validate using some tools.
> Java support at least is not so good for XSD 1.1, so I don't know
> whether votlint would be able to provide XSD validation for VODML
> if it was defined using this schema.
>
> Mark
>
>
>
> On Fri, 16 Jul 2021, BONNAREL FRANCOIS wrote:
>
>
> Hi all,
> Considering the VOTAble schema versioning issue, with the "import" of a
> mapping syntax which may evoluate on its own I wonder i something like the
> following could work.
> 1 ) in next VOTABLE schema version, create a "VODML" tag inside VOTable of
> type "anyType" (minoccurs = 0 of course)
> 2 ) in VOTABLE documents type this VODML tag with an
> xsi:type="mapping:VODML-type" , with "mapping" xmlns defined as the last
> version of the mapping xml schema (can be done with attributes of the VODML
> tag)
> 3 ) as far as I remember this would make the two schemata independant and
> does not require that VOTABLE documents import the mapping schema
>
> Do you think it could work ?
>
> Cheers
> François
>
> Le 15/07/2021 à 11:36, Laurent Michel a écrit :
>
> Dear DM,
>
> Last Tuesday we had our last DM workshop meeting.
> This concluded a fruitful 7-month process whose main conclusions and
> prospects are listed below.
>
> An overview of the work done has been presented by LM
> (https://wiki.ivoa.net/internal/IVOA/Dm2021/MAY2021-ws41.pdf<https://urldefense.com/v3/__https:/wiki.ivoa.net/internal/IVOA/Dm2021/MAY2021-ws41.pdf__;!!CrWY41Z8OgsX0i-WU-0LuAcUu2o!nQi4ELtSSzKUpZjOVSCN0rLK20EZWxEHU8Fskrp9js4kx9qVU47SL56je5jMh_tCrQ$>
> <https://wiki.ivoa.net/internal/IVOA/Dm2021/MAY2021-ws41.pdf<https://urldefense.com/v3/__https:/wiki.ivoa.net/internal/IVOA/Dm2021/MAY2021-ws41.pdf__;!!CrWY41Z8OgsX0i-WU-0LuAcUu2o!nQi4ELtSSzKUpZjOVSCN0rLK20EZWxEHU8Fskrp9js4kx9qVU47SL56je5jMh_tCrQ$>>)
>
> Conclusions in short
> ================
>
> • Models:
> • Meas, Coord, PhotDM, Dataset, Cube and MANGO accepted
> • New use-cases to be investigated
> • X-RAY Astronomy
> • Asteroids, multi-core datasets
> • CTA and MM astronomy : meta-data characterization
>
> • Data Provider / client specific use-cases
> • Need for annotations to help processing spectra
> • Need to associate parameters
> • Need for a simple description of the photometric calibration
> • Need for a simple view on Provenance
> • Annotation on the fly feasible
> • Model-based PyVO API easy to design
>
> • Mapping syntax
> • Divergence between the 2 proposals (VODML mapping and ModelInstanceInVot)
> • Proof of concept for a YAML serialization of model instances
>
> Annotation
> =========
> An important effort has been made over the last 6 weeks to merge the 2
> syntax proposals:
> • see on https://github.com/ivoa-std/ModelInstanceInVot<https://urldefense.com/v3/__https:/github.com/ivoa-std/ModelInstanceInVot__;!!CrWY41Z8OgsX0i-WU-0LuAcUu2o!nQi4ELtSSzKUpZjOVSCN0rLK20EZWxEHU8Fskrp9js4kx9qVU47SL56je5gbe7QepA$>
> <https://github.com/ivoa-std/ModelInstanceInVot<https://urldefense.com/v3/__https:/github.com/ivoa-std/ModelInstanceInVot__;!!CrWY41Z8OgsX0i-WU-0LuAcUu2o!nQi4ELtSSzKUpZjOVSCN0rLK20EZWxEHU8Fskrp9js4kx9qVU47SL56je5gbe7QepA$>>
>
> • 3 items available so far:
>
>
>
> --
> Mark Taylor Astronomical Programmer Physics, Bristol University, UK
> m.b.taylor at bristol.ac.uk<mailto:m.b.taylor at bristol.ac.uk> http://www.star.bristol.ac.uk/~mbt/<https://urldefense.com/v3/__http:/www.star.bristol.ac.uk/*mbt/__;fg!!CrWY41Z8OgsX0i-WU-0LuAcUu2o!nQi4ELtSSzKUpZjOVSCN0rLK20EZWxEHU8Fskrp9js4kx9qVU47SL56je5gnFcabKg$>
>
>
--
Mark Taylor Astronomical Programmer Physics, Bristol University, UK
m.b.taylor at bristol.ac.uk http://www.star.bristol.ac.uk/~mbt/
More information about the dm
mailing list