DM Workshop wrap-up
Gerard Lemson
glemson1 at jhu.edu
Wed Jul 21 18:56:54 CEST 2021
Hi
It seems there is another component in VOTABLE using an xs:any, namely the anyText type used by DESCRIPTON elements.
But instead of the usage in RESOURCE:
<xs:any namespace="##other" processContents="lax" minOccurs="0" maxOccurs="unbounded"/>
it is used as
<xs:any minOccurs="0" maxOccurs="unbounded" processContents="skip"/>
I don't know if its definition would preclude the possibility of adding a <dm:VODML> element inside?
I have not worked with xs:any at all.
If it would allows that a possible/arguable small advantage could be that it would be used in the single DESCRIPTION element directly following the VOTABLE element, rather than in a RESOURCE.
Gerard
From: dm-bounces at ivoa.net <dm-bounces at ivoa.net> On Behalf Of Tom Donaldson
Sent: Wednesday, July 21, 2021 12:20
To: Laurent Michel <laurent.michel at astro.unistra.fr>; Mark Taylor <m.b.taylor at bristol.ac.uk>
Cc: dm at ivoa.net
Subject: Re: DM Workshop wrap-up
Hi All,
Thanks, Mark, for looking into that. This does seem promising. I would like to do a few experiments with our existing parsers/validators to be sure, but it seems good.
If we use this approach I would propose that we give strict guidance on the RESOURCE where VODML content lives. Since RESOURCEs can show up all over the place in a VOTABLE, I think it would be good to mandate things like:
* Only one such RESOURCE
* Only one possible location (e.g., the first RESOURCE)
* With well-defined scope (most likely it should be a top-level stand-alone RESOURCE, not nested in or containing other RESOURCEs)
* With type="meta"
* With a specific ID and/or name?
Thanks,
Tom
From: <dm-bounces at ivoa.net<mailto:dm-bounces at ivoa.net>> on behalf of Laurent Michel <laurent.michel at astro.unistra.fr<mailto:laurent.michel at astro.unistra.fr>>
Date: Wednesday, July 21, 2021 at 11:39 AM
To: Mark Taylor <m.b.taylor at bristol.ac.uk<mailto:m.b.taylor at bristol.ac.uk>>
Cc: "dm at ivoa.net<mailto:dm at ivoa.net>" <dm at ivoa.net<mailto:dm at ivoa.net>>
Subject: Re: DM Workshop wrap-up
External Email - Use Caution
Hell all,
This would be a very clean solution.
- Gerard proposed to rephrase the XSD in order to isolate complexTypes from elements which should make easier the validation in the context of a VOTable.
- We have been using XSD1.1 (with Python) because the merged syntax put many constraints on element attributes. This is a key point of the proposal. I do not think that such rules could be set with XSD1.0
- The xerces web site does not look worry about this (https://xerces.apache.org/xerces2-j/faq-xs.html<https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Furldefense.com%2Fv3%2F__https%3A%2Fxerces.apache.org%2Fxerces2-j%2Ffaq-xs.html__%3B!!CrWY41Z8OgsX0i-WU-0LuAcUu2o!nQi4ELtSSzKUpZjOVSCN0rLK20EZWxEHU8Fskrp9js4kx9qVU47SL56je5h6c5iFdQ%24&data=04%7C01%7Cglemson1%40jhu.edu%7C95be75cec271439cf12a08d94c64d35d%7C9fa4f438b1e6473b803f86f8aedf0dec%7C0%7C0%7C637624818223683041%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=BzSmrIAlLTqGocz0mQXOgZiltebWxOQblIFa4MhdSg4%3D&reserved=0>) but I've no experience with this code
- I've no idea about how could this fit with volint either.
Best
Laurent
On 21 Jul 2021, at 17:02, Mark Taylor <m.b.taylor at bristol.ac.uk<mailto:m.b.taylor at bristol.ac.uk>> wrote:
François,
Yes, I think that use of the xs:any type is the right way to go here.
However, that is already accommodated by the VOTable schema as it stands.
Quoting from https://www.ivoa.net/xml/VOTable/VOTable-1.4.xsd:<https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Furldefense.com%2Fv3%2F__https%3A%2Fwww.ivoa.net%2Fxml%2FVOTable%2FVOTable-1.4.xsd%3A__%3B!!CrWY41Z8OgsX0i-WU-0LuAcUu2o!nQi4ELtSSzKUpZjOVSCN0rLK20EZWxEHU8Fskrp9js4kx9qVU47SL56je5i1_43vxg%24&data=04%7C01%7Cglemson1%40jhu.edu%7C95be75cec271439cf12a08d94c64d35d%7C9fa4f438b1e6473b803f86f8aedf0dec%7C0%7C0%7C637624818223693035%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=Qz5Bej%2FeVmdYTpU0zzR2TQevt1X0V5Yjq342OehJjpk%3D&reserved=0>
<xs:complexType name="Resource">
...
<!-- Suggested Doug Tody, to include new RESOURCE types -->
<xs:any namespace="##other" processContents="lax" minOccurs="0" maxOccurs="unbounded"/>
...
</xs:complexType>
which means that the elements from the VODML schema can be included
within a RESOURCE element in a VOTable document, with no changes
to the VOTable standard required at all (thanks to Markus for
pointing this out).
So something like this should work out of the box:
<VOTABLE version="1.4"
xmlns="http://www.ivoa.net/xml/VOTable/v1.3<https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Furldefense.com%2Fv3%2F__http%3A%2Fwww.ivoa.net%2Fxml%2FVOTable%2Fv1.3__%3B!!CrWY41Z8OgsX0i-WU-0LuAcUu2o!nQi4ELtSSzKUpZjOVSCN0rLK20EZWxEHU8Fskrp9js4kx9qVU47SL56je5h2xvKE0w%24&data=04%7C01%7Cglemson1%40jhu.edu%7C95be75cec271439cf12a08d94c64d35d%7C9fa4f438b1e6473b803f86f8aedf0dec%7C0%7C0%7C637624818223703028%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=IJ731vhyja3i2gSbSdx0%2FOlWgEzP6Py0eMahV1E0BkA%3D&reserved=0>"
xmlns:dm="http://www.ivoa.net/xml/vodml<https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Furldefense.com%2Fv3%2F__http%3A%2Fwww.ivoa.net%2Fxml%2Fvodml__%3B!!CrWY41Z8OgsX0i-WU-0LuAcUu2o!nQi4ELtSSzKUpZjOVSCN0rLK20EZWxEHU8Fskrp9js4kx9qVU47SL56je5iiGyvehQ%24&data=04%7C01%7Cglemson1%40jhu.edu%7C95be75cec271439cf12a08d94c64d35d%7C9fa4f438b1e6473b803f86f8aedf0dec%7C0%7C0%7C637624818223703028%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=zZ%2Bd5IlYbmWY1Re%2BfocBbAfdTMMpx3hVhLfowDTFS9A%3D&reserved=0>">
<RESOURCE type="meta">
<dm:VODML ...>
<MODELS>
</MODELS>
...
</dm:VODML>
</RESOURCE>
<RESOURCE type="results">
<TABLE>
...
</TABLE>
</RESOURCE>
</VOTABLE>
A VOTable document like that validates for me under the VOTable 1.4
schema (e.g. xmllint -noout -schema VOTable1.4.xsd). At present votlint
(http://www.starlink.ac.uk/stilts/sun256/votlint.html<https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Furldefense.com%2Fv3%2F__http%3A%2Fwww.starlink.ac.uk%2Fstilts%2Fsun256%2Fvotlint.html__%3B!!CrWY41Z8OgsX0i-WU-0LuAcUu2o!nQi4ELtSSzKUpZjOVSCN0rLK20EZWxEHU8Fskrp9js4kx9qVU47SL56je5iXtd5-Iw%24&data=04%7C01%7Cglemson1%40jhu.edu%7C95be75cec271439cf12a08d94c64d35d%7C9fa4f438b1e6473b803f86f8aedf0dec%7C0%7C0%7C637624818223713025%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=rxH9lSdOpBz14%2F61w2%2BXqKl3LEUOzhRgYavm%2BvgXAlc%3D&reserved=0>) does complain:
WARNING: Element in wrong namespace (http://www.ivoa.net/xml/vodml<https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Furldefense.com%2Fv3%2F__http%3A%2Fwww.ivoa.net%2Fxml%2Fvodml__%3B!!CrWY41Z8OgsX0i-WU-0LuAcUu2o!nQi4ELtSSzKUpZjOVSCN0rLK20EZWxEHU8Fskrp9js4kx9qVU47SL56je5iiGyvehQ%24&data=04%7C01%7Cglemson1%40jhu.edu%7C95be75cec271439cf12a08d94c64d35d%7C9fa4f438b1e6473b803f86f8aedf0dec%7C0%7C0%7C637624818223713025%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=RKVZGpzH7X5NoOIUZe3zRi0M8rlLN0FXJZC3RbAYTcU%3D&reserved=0> not http://www.ivoa.net/xml/VOTable/v1.3<https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Furldefense.com%2Fv3%2F__http%3A%2Fwww.ivoa.net%2Fxml%2FVOTable%2Fv1.3__%3B!!CrWY41Z8OgsX0i-WU-0LuAcUu2o!nQi4ELtSSzKUpZjOVSCN0rLK20EZWxEHU8Fskrp9js4kx9qVU47SL56je5h2xvKE0w%24&data=04%7C01%7Cglemson1%40jhu.edu%7C95be75cec271439cf12a08d94c64d35d%7C9fa4f438b1e6473b803f86f8aedf0dec%7C0%7C0%7C637624818223723013%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=4fHc7LtMgplVxiREmgMg9PMozDGJLqhR8IBtBvVvXq8%3D&reserved=0>)
but that's really a votlint bug that I will fix.
I would certainly favour this approach rather than adding new VOTable
elements, to avoid unnecessary coupling between the VODML and VOTable
standards.
Concerning the xsi:type="mapping:VODML-type" attribute:
as I understand it, that wouldn't work with a schema in the form
of the one at
https://github.com/ivoa-std/ModelInstanceInVot/blob/master/schema/xsd/merged-syntax.xsd<https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Furldefense.com%2Fv3%2F__https%3A%2Fgithub.com%2Fivoa-std%2FModelInstanceInVot%2Fblob%2Fmaster%2Fschema%2Fxsd%2Fmerged-syntax.xsd__%3B!!CrWY41Z8OgsX0i-WU-0LuAcUu2o!nQi4ELtSSzKUpZjOVSCN0rLK20EZWxEHU8Fskrp9js4kx9qVU47SL56je5jFLJq0uQ%24&data=04%7C01%7Cglemson1%40jhu.edu%7C95be75cec271439cf12a08d94c64d35d%7C9fa4f438b1e6473b803f86f8aedf0dec%7C0%7C0%7C637624818223723013%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=qVwjm%2F4F0sla74pZx2bgNhZRxlPHvHweSmRezslWYjw%3D&reserved=0>
since that defines elements directly and not types (Gerard queried
this stylistic decision during the DM workshop session #4).
So I'm not quite sure how validation of the embedded VODML would
proceed: either the schema could be rephrased to define types, or
perhaps XSD validators are able to determine the required contents
based on the <VODML> element name without requiring an xsi:type.
One other point on the current merged-syntax.xsd: use of XMLSchema 1.1,
instead of 1.0, makes it harder to validate using some tools.
Java support at least is not so good for XSD 1.1, so I don't know
whether votlint would be able to provide XSD validation for VODML
if it was defined using this schema.
Mark
On Fri, 16 Jul 2021, BONNAREL FRANCOIS wrote:
Hi all,
Considering the VOTAble schema versioning issue, with the "import" of a
mapping syntax which may evoluate on its own I wonder i something like the
following could work.
1 ) in next VOTABLE schema version, create a "VODML" tag inside VOTable of
type "anyType" (minoccurs = 0 of course)
2 ) in VOTABLE documents type this VODML tag with an
xsi:type="mapping:VODML-type" , with "mapping" xmlns defined as the last
version of the mapping xml schema (can be done with attributes of the VODML
tag)
3 ) as far as I remember this would make the two schemata independant and
does not require that VOTABLE documents import the mapping schema
Do you think it could work ?
Cheers
François
Le 15/07/2021 à 11:36, Laurent Michel a écrit :
Dear DM,
Last Tuesday we had our last DM workshop meeting.
This concluded a fruitful 7-month process whose main conclusions and
prospects are listed below.
An overview of the work done has been presented by LM
(https://wiki.ivoa.net/internal/IVOA/Dm2021/MAY2021-ws41.pdf<https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Furldefense.com%2Fv3%2F__https%3A%2Fwiki.ivoa.net%2Finternal%2FIVOA%2FDm2021%2FMAY2021-ws41.pdf__%3B!!CrWY41Z8OgsX0i-WU-0LuAcUu2o!nQi4ELtSSzKUpZjOVSCN0rLK20EZWxEHU8Fskrp9js4kx9qVU47SL56je5jMh_tCrQ%24&data=04%7C01%7Cglemson1%40jhu.edu%7C95be75cec271439cf12a08d94c64d35d%7C9fa4f438b1e6473b803f86f8aedf0dec%7C0%7C0%7C637624818223733012%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=Cg64Kq980PfD%2FzkJX60NR2280Tzk6t%2BYMVfOLS5FvuY%3D&reserved=0>
<https://wiki.ivoa.net/internal/IVOA/Dm2021/MAY2021-ws41.pdf<https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Furldefense.com%2Fv3%2F__https%3A%2Fwiki.ivoa.net%2Finternal%2FIVOA%2FDm2021%2FMAY2021-ws41.pdf__%3B!!CrWY41Z8OgsX0i-WU-0LuAcUu2o!nQi4ELtSSzKUpZjOVSCN0rLK20EZWxEHU8Fskrp9js4kx9qVU47SL56je5jMh_tCrQ%24&data=04%7C01%7Cglemson1%40jhu.edu%7C95be75cec271439cf12a08d94c64d35d%7C9fa4f438b1e6473b803f86f8aedf0dec%7C0%7C0%7C637624818223743006%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=rTiKb0AALhln1WRSVTPpuX%2BC51IttP%2BQvfKogUo%2BHMo%3D&reserved=0>>)
Conclusions in short
================
* Models:
* Meas, Coord, PhotDM, Dataset, Cube and MANGO accepted
* New use-cases to be investigated
* X-RAY Astronomy
* Asteroids, multi-core datasets
* CTA and MM astronomy : meta-data characterization
* Data Provider / client specific use-cases
* Need for annotations to help processing spectra
* Need to associate parameters
* Need for a simple description of the photometric calibration
* Need for a simple view on Provenance
* Annotation on the fly feasible
* Model-based PyVO API easy to design
* Mapping syntax
* Divergence between the 2 proposals (VODML mapping and ModelInstanceInVot)
* Proof of concept for a YAML serialization of model instances
Annotation
=========
An important effort has been made over the last 6 weeks to merge the 2
syntax proposals:
* see on https://github.com/ivoa-std/ModelInstanceInVot<https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Furldefense.com%2Fv3%2F__https%3A%2Fgithub.com%2Fivoa-std%2FModelInstanceInVot__%3B!!CrWY41Z8OgsX0i-WU-0LuAcUu2o!nQi4ELtSSzKUpZjOVSCN0rLK20EZWxEHU8Fskrp9js4kx9qVU47SL56je5gbe7QepA%24&data=04%7C01%7Cglemson1%40jhu.edu%7C95be75cec271439cf12a08d94c64d35d%7C9fa4f438b1e6473b803f86f8aedf0dec%7C0%7C0%7C637624818223743006%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=py%2FhiHvhw0OI%2Fxnk1OJyzVdD%2FLBXvu89S4PjhBhwW3I%3D&reserved=0>
<https://github.com/ivoa-std/ModelInstanceInVot<https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Furldefense.com%2Fv3%2F__https%3A%2Fgithub.com%2Fivoa-std%2FModelInstanceInVot__%3B!!CrWY41Z8OgsX0i-WU-0LuAcUu2o!nQi4ELtSSzKUpZjOVSCN0rLK20EZWxEHU8Fskrp9js4kx9qVU47SL56je5gbe7QepA%24&data=04%7C01%7Cglemson1%40jhu.edu%7C95be75cec271439cf12a08d94c64d35d%7C9fa4f438b1e6473b803f86f8aedf0dec%7C0%7C0%7C637624818223753002%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=nOlguDGGf58WxPYcig6rXsMaDa6Jo%2B2ffiaVfdrsqGQ%3D&reserved=0>>
* 3 items available so far:
--
Mark Taylor Astronomical Programmer Physics, Bristol University, UK
m.b.taylor at bristol.ac.uk<mailto:m.b.taylor at bristol.ac.uk> http://www.star.bristol.ac.uk/~mbt/<https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Furldefense.com%2Fv3%2F__http%3A%2Fwww.star.bristol.ac.uk%2F*mbt%2F__%3Bfg!!CrWY41Z8OgsX0i-WU-0LuAcUu2o!nQi4ELtSSzKUpZjOVSCN0rLK20EZWxEHU8Fskrp9js4kx9qVU47SL56je5gnFcabKg%24&data=04%7C01%7Cglemson1%40jhu.edu%7C95be75cec271439cf12a08d94c64d35d%7C9fa4f438b1e6473b803f86f8aedf0dec%7C0%7C0%7C637624818223753002%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=y96fh9Cu578%2Ff9X69J%2Bt8X4sB21bvXsJ%2FgA%2FIE3rT6g%3D&reserved=0>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.ivoa.net/pipermail/dm/attachments/20210721/7c604473/attachment-0001.html>
More information about the dm
mailing list