DM Workshop wrap-up
Gerard Lemson
glemson1 at jhu.edu
Wed Jul 21 17:57:30 CEST 2021
Hi Laurent
I am curious whether XSD 1.1 would be able to express the constraints on VO_DML documents that right now are coded in schematron.
Do you know if they are compatible?
I already once created a refactored "types schema", but that was based on the previous version.
Will create a new one later today and PR it.
Cheers
Gerard
From: dm-bounces at ivoa.net <dm-bounces at ivoa.net> On Behalf Of Laurent Michel
Sent: Wednesday, July 21, 2021 11:39
To: Mark Taylor <m.b.taylor at bristol.ac.uk>
Cc: dm at ivoa.net
Subject: Re: DM Workshop wrap-up
Hell all,
This would be a very clean solution.
- Gerard proposed to rephrase the XSD in order to isolate complexTypes from elements which should make easier the validation in the context of a VOTable.
- We have been using XSD1.1 (with Python) because the merged syntax put many constraints on element attributes. This is a key point of the proposal. I do not think that such rules could be set with XSD1.0
- The xerces web site does not look worry about this (https://xerces.apache.org/xerces2-j/faq-xs.html<https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fxerces.apache.org%2Fxerces2-j%2Ffaq-xs.html&data=04%7C01%7Cglemson1%40jhu.edu%7C804c46a40e744d8405b408d94c5db9de%7C9fa4f438b1e6473b803f86f8aedf0dec%7C0%7C0%7C637624788918461664%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=HqoYCguPIXEkE2hA3xig%2Feq%2B5x6u7KzDP2Qvi3i%2BOgM%3D&reserved=0>) but I've no experience with this code
- I've no idea about how could this fit with volint either.
Best
Laurent
On 21 Jul 2021, at 17:02, Mark Taylor <m.b.taylor at bristol.ac.uk<mailto:m.b.taylor at bristol.ac.uk>> wrote:
François,
Yes, I think that use of the xs:any type is the right way to go here.
However, that is already accommodated by the VOTable schema as it stands.
Quoting from https://www.ivoa.net/xml/VOTable/VOTable-1.4.xsd:<https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.ivoa.net%2Fxml%2FVOTable%2FVOTable-1.4.xsd%3A&data=04%7C01%7Cglemson1%40jhu.edu%7C804c46a40e744d8405b408d94c5db9de%7C9fa4f438b1e6473b803f86f8aedf0dec%7C0%7C0%7C637624788918471661%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=h7qP3ghdD7NIox35yioHOMIzNki66l%2BpW3UJpYolLHE%3D&reserved=0>
<xs:complexType name="Resource">
...
<!-- Suggested Doug Tody, to include new RESOURCE types -->
<xs:any namespace="##other" processContents="lax" minOccurs="0" maxOccurs="unbounded"/>
...
</xs:complexType>
which means that the elements from the VODML schema can be included
within a RESOURCE element in a VOTable document, with no changes
to the VOTable standard required at all (thanks to Markus for
pointing this out).
So something like this should work out of the box:
<VOTABLE version="1.4"
xmlns="http://www.ivoa.net/xml/VOTable/v1.3<https://nam02.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.ivoa.net%2Fxml%2FVOTable%2Fv1.3&data=04%7C01%7Cglemson1%40jhu.edu%7C804c46a40e744d8405b408d94c5db9de%7C9fa4f438b1e6473b803f86f8aedf0dec%7C0%7C0%7C637624788918481649%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=1yrcvCL10NIjIyhUSuca%2BwH76LXG8b7k3Z5XVzkRxhU%3D&reserved=0>"
xmlns:dm="http://www.ivoa.net/xml/vodml<https://nam02.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.ivoa.net%2Fxml%2Fvodml&data=04%7C01%7Cglemson1%40jhu.edu%7C804c46a40e744d8405b408d94c5db9de%7C9fa4f438b1e6473b803f86f8aedf0dec%7C0%7C0%7C637624788918481649%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=HIEuRguR%2BpFPP%2BweFXhsDXBWqwbFUUZpE28p00ij8eA%3D&reserved=0>">
<RESOURCE type="meta">
<dm:VODML ...>
<MODELS>
</MODELS>
...
</dm:VODML>
</RESOURCE>
<RESOURCE type="results">
<TABLE>
...
</TABLE>
</RESOURCE>
</VOTABLE>
A VOTable document like that validates for me under the VOTable 1.4
schema (e.g. xmllint -noout -schema VOTable1.4.xsd). At present votlint
(http://www.starlink.ac.uk/stilts/sun256/votlint.html<https://nam02.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.starlink.ac.uk%2Fstilts%2Fsun256%2Fvotlint.html&data=04%7C01%7Cglemson1%40jhu.edu%7C804c46a40e744d8405b408d94c5db9de%7C9fa4f438b1e6473b803f86f8aedf0dec%7C0%7C0%7C637624788918491644%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=RDkkUNTM1DZL%2BoxGmQuAizOQBaawR6fI7Cm7zNcIpHk%3D&reserved=0>) does complain:
WARNING: Element in wrong namespace (http://www.ivoa.net/xml/vodml<https://nam02.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.ivoa.net%2Fxml%2Fvodml&data=04%7C01%7Cglemson1%40jhu.edu%7C804c46a40e744d8405b408d94c5db9de%7C9fa4f438b1e6473b803f86f8aedf0dec%7C0%7C0%7C637624788918491644%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=mylabw%2BiZ%2BPVK7FfnwZaGpmKkfj5qg3OMQiTWHxfFIY%3D&reserved=0> not http://www.ivoa.net/xml/VOTable/v1.3<https://nam02.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.ivoa.net%2Fxml%2FVOTable%2Fv1.3&data=04%7C01%7Cglemson1%40jhu.edu%7C804c46a40e744d8405b408d94c5db9de%7C9fa4f438b1e6473b803f86f8aedf0dec%7C0%7C0%7C637624788918501642%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=D0PwW93JNz5Cz3l1Pd1nb1xKdFT%2FHAxwolYOMbRyrQQ%3D&reserved=0>)
but that's really a votlint bug that I will fix.
I would certainly favour this approach rather than adding new VOTable
elements, to avoid unnecessary coupling between the VODML and VOTable
standards.
Concerning the xsi:type="mapping:VODML-type" attribute:
as I understand it, that wouldn't work with a schema in the form
of the one at
https://github.com/ivoa-std/ModelInstanceInVot/blob/master/schema/xsd/merged-syntax.xsd<https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fivoa-std%2FModelInstanceInVot%2Fblob%2Fmaster%2Fschema%2Fxsd%2Fmerged-syntax.xsd&data=04%7C01%7Cglemson1%40jhu.edu%7C804c46a40e744d8405b408d94c5db9de%7C9fa4f438b1e6473b803f86f8aedf0dec%7C0%7C0%7C637624788918501642%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=PdCnTSVjPd0%2BOZcK1Kq57lBXsNQ9kkW%2BpTdg288N%2F80%3D&reserved=0>
since that defines elements directly and not types (Gerard queried
this stylistic decision during the DM workshop session #4).
So I'm not quite sure how validation of the embedded VODML would
proceed: either the schema could be rephrased to define types, or
perhaps XSD validators are able to determine the required contents
based on the <VODML> element name without requiring an xsi:type.
One other point on the current merged-syntax.xsd: use of XMLSchema 1.1,
instead of 1.0, makes it harder to validate using some tools.
Java support at least is not so good for XSD 1.1, so I don't know
whether votlint would be able to provide XSD validation for VODML
if it was defined using this schema.
Mark
On Fri, 16 Jul 2021, BONNAREL FRANCOIS wrote:
Hi all,
Considering the VOTAble schema versioning issue, with the "import" of a
mapping syntax which may evoluate on its own I wonder i something like the
following could work.
1 ) in next VOTABLE schema version, create a "VODML" tag inside VOTable of
type "anyType" (minoccurs = 0 of course)
2 ) in VOTABLE documents type this VODML tag with an
xsi:type="mapping:VODML-type" , with "mapping" xmlns defined as the last
version of the mapping xml schema (can be done with attributes of the VODML
tag)
3 ) as far as I remember this would make the two schemata independant and
does not require that VOTABLE documents import the mapping schema
Do you think it could work ?
Cheers
François
Le 15/07/2021 à 11:36, Laurent Michel a écrit :
Dear DM,
Last Tuesday we had our last DM workshop meeting.
This concluded a fruitful 7-month process whose main conclusions and
prospects are listed below.
An overview of the work done has been presented by LM
(https://wiki.ivoa.net/internal/IVOA/Dm2021/MAY2021-ws41.pdf<https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwiki.ivoa.net%2Finternal%2FIVOA%2FDm2021%2FMAY2021-ws41.pdf&data=04%7C01%7Cglemson1%40jhu.edu%7C804c46a40e744d8405b408d94c5db9de%7C9fa4f438b1e6473b803f86f8aedf0dec%7C0%7C0%7C637624788918511632%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=tz4LKSt5AyYtxpQC0TiTsl%2FAWphch6IBgW2XWu3%2Fu2g%3D&reserved=0>
<https://wiki.ivoa.net/internal/IVOA/Dm2021/MAY2021-ws41.pdf<https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwiki.ivoa.net%2Finternal%2FIVOA%2FDm2021%2FMAY2021-ws41.pdf&data=04%7C01%7Cglemson1%40jhu.edu%7C804c46a40e744d8405b408d94c5db9de%7C9fa4f438b1e6473b803f86f8aedf0dec%7C0%7C0%7C637624788918521627%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=BqAbiPgT%2B38qQzBUMkVoePwmYMvbtYAKb372UlYTvH4%3D&reserved=0>>)
Conclusions in short
================
* Models:
* Meas, Coord, PhotDM, Dataset, Cube and MANGO accepted
* New use-cases to be investigated
* X-RAY Astronomy
* Asteroids, multi-core datasets
* CTA and MM astronomy : meta-data characterization
* Data Provider / client specific use-cases
* Need for annotations to help processing spectra
* Need to associate parameters
* Need for a simple description of the photometric calibration
* Need for a simple view on Provenance
* Annotation on the fly feasible
* Model-based PyVO API easy to design
* Mapping syntax
* Divergence between the 2 proposals (VODML mapping and ModelInstanceInVot)
* Proof of concept for a YAML serialization of model instances
Annotation
=========
An important effort has been made over the last 6 weeks to merge the 2
syntax proposals:
* see on https://github.com/ivoa-std/ModelInstanceInVot<https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fivoa-std%2FModelInstanceInVot&data=04%7C01%7Cglemson1%40jhu.edu%7C804c46a40e744d8405b408d94c5db9de%7C9fa4f438b1e6473b803f86f8aedf0dec%7C0%7C0%7C637624788918521627%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=ytc00Qb5Rp9vlyJjMznOaHPlpFjeoYHoSFtbVrVwtqw%3D&reserved=0>
<https://github.com/ivoa-std/ModelInstanceInVot<https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fivoa-std%2FModelInstanceInVot&data=04%7C01%7Cglemson1%40jhu.edu%7C804c46a40e744d8405b408d94c5db9de%7C9fa4f438b1e6473b803f86f8aedf0dec%7C0%7C0%7C637624788918531624%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=7XbwaNytqIOl5hIBVcUjy7vZOf3RX4p48zlj21QnerE%3D&reserved=0>>
* 3 items available so far:
--
Mark Taylor Astronomical Programmer Physics, Bristol University, UK
m.b.taylor at bristol.ac.uk<mailto:m.b.taylor at bristol.ac.uk> http://www.star.bristol.ac.uk/~mbt/<https://nam02.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.star.bristol.ac.uk%2F~mbt%2F&data=04%7C01%7Cglemson1%40jhu.edu%7C804c46a40e744d8405b408d94c5db9de%7C9fa4f438b1e6473b803f86f8aedf0dec%7C0%7C0%7C637624788918531624%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=1n3fN%2FwObTBg0MiUceJ%2FBDXAxKcNanfOvRhTvHLegNU%3D&reserved=0>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.ivoa.net/pipermail/dm/attachments/20210721/014bce01/attachment-0001.html>
More information about the dm
mailing list