<div dir="ltr"><div class="gmail_default" style="font-size:small">note: cross-posting reduced to DAL and semantics</div><div class="gmail_default" style="font-size:small"><br></div><div class="gmail_default" style="font-size:small">In CAOM, I made an ObsCore.dataproduyct_type base vocabulary and then added my own custom child of measurements named <a href="http://www.opencadc.org/caom/DataProductType#catalog">http://www.opencadc.org/caom/DataProductType#catalog</a> (this is literally ObsCore + extensions so if this idea goes ahead we'll have to figure out this vocab refers back to the parent vocab). If one queries caom2 in our tap service you can see that (fully qualified) value mixed in with ObsCore values. In the ObsCore view I am currently limiting the rows to exclude custom (fully qualified) terms for compliance with the current model (1.1). In the event that:<br></div><div class="gmail_default" style="font-size:small"><br></div><div class="gmail_default" style="font-size:small">* ObsCore (1.2) dataproduct_type values are defined by a vocabulary, and</div><div class="gmail_default" style="font-size:small">* providers can extend the vocabulary with custom terms (then fully qualified rather that just the short form when using base terms)</div><div class="gmail_default" style="font-size:small"><br></div><div class="gmail_default" style="font-size:small">I could remove he filtering and allow more entries to appear in ObsCore view. If that doesn't become possible, we would have to decide to (i) leave content as-is or (ii) change those to measurements.</div><div class="gmail_default" style="font-size:small"><br></div><div class="gmail_default" style="font-size:small">Currently CADC has:</div><div class="gmail_default" style="font-size:small"><br></div><div class="gmail_default" style="font-size:small">select dataProductType,count(*) from caom2.Plane group by dataProductType;<br> dataproducttype | count <br>-------------------------------------------------------+----------<br> | 15538972<br> timeseries | 866405<br> spectrum | 7911476<br> cube | 202980<br> <a href="http://www.opencadc.org/caom2/DataProductType#catalog">http://www.opencadc.org/caom2/DataProductType#catalog</a> | 160850<br> eventlist | 88421<br> image | 16076616</div><div class="gmail_default" style="font-size:small"><br></div><div class="gmail_default" style="font-size:small">So, we are using "measurements" but it is not surprising that Markus didn't see it. We only have one child term in use right now.<br></div><div class="gmail_default" style="font-size:small"><br></div><div><div dir="ltr" class="gmail_signature" data-smartmail="gmail_signature"><div dir="ltr"><div><div dir="ltr"><div><div>--<br></div><div>Patrick Dowler<br></div>Canadian Astronomy Data Centre<br></div>Victoria, BC, Canada<br></div></div></div></div></div><br></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Thu, 12 Mar 2020 at 00:01, Slava Kitaeff <<a href="mailto:slava.kitaeff@uwa.edu.au">slava.kitaeff@uwa.edu.au</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div lang="EN-AU">
<div class="gmail-m_-3349331814557714389WordSection1">
<p class="MsoNormal">Hi James et all,<u></u><u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal">Just a quick note on that. It’s a bit of a guess but I’m suspecting that this might be referring to a Measurement Set (MS), which is a specific data format (and a model) used by CASA (and now ASKAPsoft) to store interferometric visibilities.
So, the data type is then must be “visibilities”. To confuse things further, CASA can store continuum images and spectral-line image-cubes in the same MS format as CASA tables. In my view, it’d be ideal to have two things describing a) the nature of data and
b) its specific format (including version).<u></u><u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal">Cheers,<u></u><u></u></p>
<p class="MsoNormal">Slava<u></u><u></u></p>
<p class="MsoNormal" style="background:white none repeat scroll 0% 0%"><span style="font-size:9pt;font-family:"Arial",sans-serif;color:black">________________________________</span><span style="font-size:9pt;font-family:Helvetica;color:black"><u></u><u></u></span></p>
<p class="MsoNormal" style="background:white none repeat scroll 0% 0%"><u></u> <u></u></p>
<p class="MsoNormal" style="background:white none repeat scroll 0% 0%"><b><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:rgb(93,171,206)">Dr Slava Kitaeff</span></b><span style="font-size:9pt;font-family:Helvetica;color:black"><u></u><u></u></span></p>
<p class="MsoNormal" style="background:white none repeat scroll 0% 0%"><span style="font-size:9pt;font-family:"Arial",sans-serif;color:black">Australian SKA Regional Centre Program Manager</span><span style="font-size:9pt;font-family:Helvetica;color:black"><u></u><u></u></span></p>
<p class="MsoNormal" style="background:white none repeat scroll 0% 0%"><span style="font-size:9pt;font-family:Helvetica;color:black"><u></u> <u></u></span></p>
<p class="MsoNormal" style="background:white none repeat scroll 0% 0%"><b><span style="font-size:9pt;font-family:"Arial",sans-serif;color:black">International Centre for Radio Astronomy Research</span></b><span style="font-size:9pt;font-family:Helvetica;color:black"><u></u><u></u></span></p>
<p class="MsoNormal" style="background:white none repeat scroll 0% 0%"><span style="font-size:9pt;font-family:"Arial",sans-serif;color:black">The University of Western Australia</span><span style="font-size:9pt;font-family:Helvetica;color:black"><u></u><u></u></span></p>
<p class="MsoNormal" style="background:white none repeat scroll 0% 0%"><span style="font-size:9pt;font-family:Helvetica;color:black"><u></u> <u></u></span></p>
<p class="MsoNormal" style="background:white none repeat scroll 0% 0%"><b><span style="font-size:9pt;font-family:"Arial",sans-serif;color:black">Astronomy and Space Science</span></b><span style="font-size:9pt;font-family:Helvetica;color:black"><u></u><u></u></span></p>
<p class="MsoNormal" style="background:white none repeat scroll 0% 0%"><span style="font-size:9pt;font-family:"Arial",sans-serif;color:black">CSIRO</span><u></u><u></u></p>
<p class="MsoNormal" style="background:white none repeat scroll 0% 0%"><u></u> <u></u></p>
<p class="MsoNormal" style="background:white none repeat scroll 0% 0%"><span style="font-size:9pt;font-family:"Arial",sans-serif;color:black">Email: </span><span style="font-size:9pt;font-family:"Arial",sans-serif;color:blue"><a href="mailto:slava.kitaeff@icrar.org" target="_blank"><span style="color:rgb(5,99,193)">slava.kitaeff@icrar.org</span></a> </span><span style="font-size:9pt;font-family:"Arial",sans-serif;color:black">or <a href="mailto:slava.kitaeff@csiro.au" target="_blank"><span style="color:rgb(5,99,193)">slava.kitaeff@csiro.au</span></a> </span><span style="font-size:9pt;font-family:Helvetica;color:black"><u></u><u></u></span></p>
<p class="MsoNormal" style="background:white none repeat scroll 0% 0%"><span style="font-size:9pt;font-family:"Arial",sans-serif;color:black">Tel.: (+61) (0) 8 6488 7744 (ICRAR) or (+61) (0) 8 6436 8865 (CSIRO)</span><span style="font-size:9pt;font-family:Helvetica;color:black"><u></u><u></u></span></p>
<p class="MsoNormal" style="background:white none repeat scroll 0% 0%"><span style="font-size:9pt;font-family:"Arial",sans-serif;color:black">Mob.: +61 404 297 414</span><span style="font-size:9pt;font-family:Helvetica;color:black"><u></u><u></u></span></p>
<p class="MsoNormal" style="background:white none repeat scroll 0% 0%"><i><span style="font-size:9pt;font-family:"Arial",sans-serif;color:black"> </span></i><span style="font-size:9pt;font-family:Helvetica;color:black"><u></u><u></u></span></p>
<p class="MsoNormal" style="background:white none repeat scroll 0% 0%"><b><span style="font-size:9pt;font-family:"Arial",sans-serif;color:black">Mailing addresses:</span></b><span style="font-size:9pt;font-family:"Arial",sans-serif;color:black"> </span><span style="font-size:9pt;font-family:Helvetica;color:black"><u></u><u></u></span></p>
<p class="MsoNormal" style="background:white none repeat scroll 0% 0%"><span style="font-size:9pt;font-family:"Arial",sans-serif;color:black">M468, University of Western Australia, 35 Stirling Highway, Crawley WA 6009, Australia, <b>or</b></span><span style="font-size:9pt;font-family:Helvetica;color:black"><u></u><u></u></span></p>
<p class="MsoNormal" style="background:white none repeat scroll 0% 0%"><span style="font-size:9pt;font-family:"Arial",sans-serif;color:black">CSIRO Astronomy and Space Science, PO Box 1130, Bentley WA 6102, Australia</span><span style="font-size:9pt;font-family:Helvetica;color:black"><u></u><u></u></span></p>
<p class="MsoNormal" style="background:white none repeat scroll 0% 0%"><span style="font-size:9pt;font-family:Helvetica;color:black"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:9pt;font-family:"Arial",sans-serif;color:black">ICRAR: Discovering the hidden Universe through radio astronomy<b> </b></span><span style="font-family:"Arial",sans-serif"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:9pt;font-family:"Arial",sans-serif"><a href="http://www.icrar.org" target="_blank"><span style="color:rgb(5,99,193)">www.icrar.org</span></a><span style="color:black;background:white none repeat scroll 0% 0%"> </span></span><a href="http://www.icrar.org/#subscribe" target="_blank"><span style="font-size:9pt;font-family:"Arial",sans-serif;color:rgb(5,99,193)">Subscribe
to our eNewsletter</span></a><span style="font-size:9pt;font-family:"Arial",sans-serif;color:black;background:white none repeat scroll 0% 0%"> </span><a href="http://twitter.com/icrar" target="_blank"><span style="font-size:9pt;font-family:"Arial",sans-serif;color:rgb(5,99,193)">ICRAR on Twitter</span></a><span style="font-size:9pt;font-family:"Arial",sans-serif;color:black;background:white none repeat scroll 0% 0%"> </span><a href="http://www.facebook.com/pages/ICRAR/199692286227" target="_blank"><span style="font-size:9pt;font-family:"Arial",sans-serif;color:rgb(5,99,193)">ICRAR
on Facebook</span></a><u></u><u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
<div style="border-color:rgb(181,196,223) currentcolor currentcolor;border-style:solid none none;border-width:1pt medium medium;padding:3pt 0cm 0cm">
<p class="MsoNormal" style="margin-left:36pt"><b><span style="font-size:12pt;color:black">From:
</span></b><span style="font-size:12pt;color:black"><<a href="mailto:dal-bounces@ivoa.net" target="_blank">dal-bounces@ivoa.net</a>> on behalf of "Dempsey, James (IM&T, Black Mountain)" <James.Dempsey@csiro.au><br>
<b>Date: </b>Wednesday, 11 March 2020 at 5:43 pm<br>
<b>To: </b>Markus Demleitner <<a href="mailto:msdemlei@ari.uni-heidelberg.de" target="_blank">msdemlei@ari.uni-heidelberg.de</a>>, "<a href="mailto:dal@ivoa.net" target="_blank">dal@ivoa.net</a>" <<a href="mailto:dal@ivoa.net" target="_blank">dal@ivoa.net</a>>, "<a href="mailto:registry@ivoa.net" target="_blank">registry@ivoa.net</a>" <<a href="mailto:registry@ivoa.net" target="_blank">registry@ivoa.net</a>>, "<a href="mailto:semantics@ivoa.net" target="_blank">semantics@ivoa.net</a>" <<a href="mailto:semantics@ivoa.net" target="_blank">semantics@ivoa.net</a>><br>
<b>Subject: </b>Re: Vocabularising dataproduct_type<u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:36pt"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:36pt"><span style="font-size:12pt;color:black">Hi Markus,<u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:36pt"><span style="font-size:12pt;color:black"><u></u> <u></u></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:36pt"><span style="font-size:12pt;color:black">We are dealing with a lot of catalogues as well as images, cubes etc. Currently these have a blank dataproduct_type in the CASDA obscore instance as there wasn't anything
we could use in v1.0. We will probably start using measurements soon as it is better than nothing, but it isn't a term that our community uses. Their expectation is to see catalogue in the type, so I'd be keen to see that as term available for use.<u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:36pt"><span style="font-size:12pt;color:black"><u></u> <u></u></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:36pt"><span style="font-size:12pt;color:black">Our catalogues are generally source lists produced from the associated image/cube.<u></u><u></u></span></p>
</div>
<div>
<div>
<p class="MsoNormal" style="margin-left:36pt"><span style="font-size:12pt;color:black"><u></u> <u></u></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:36pt"><span style="font-size:12pt;color:black">Cheers,<u></u><u></u></span></p>
</div>
<div id="gmail-m_-3349331814557714389Signature">
<div name="divtagdefaultwrapper">
<div>
<div>
<div>
<p class="MsoNormal" style="margin-left:36pt"><span style="font-size:10pt">James Dempsey<u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:36pt"><span style="font-size:12pt;background:white none repeat scroll 0% 0%">Senior Developer</span><span style="font-size:10pt"><u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:36pt"><span style="font-size:12pt;background:white none repeat scroll 0% 0%">Information Services Applications</span><span style="font-size:10pt"><u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:36pt"><span style="font-size:10pt">CSIRO Information Management & Technology (IM&T)<u></u><u></u></span></p>
</div>
</div>
</div>
</div>
</div>
</div>
<div class="MsoNormal" style="margin-left:36pt;text-align:center" align="center">
<hr width="100%" size="0" align="center">
</div>
<div id="gmail-m_-3349331814557714389divRplyFwdMsg">
<p class="MsoNormal" style="margin-left:36pt"><b><span style="color:black">From:</span></b><span style="color:black"> <a href="mailto:dal-bounces@ivoa.net" target="_blank">dal-bounces@ivoa.net</a> <<a href="mailto:dal-bounces@ivoa.net" target="_blank">dal-bounces@ivoa.net</a>> on behalf of Markus Demleitner <<a href="mailto:msdemlei@ari.uni-heidelberg.de" target="_blank">msdemlei@ari.uni-heidelberg.de</a>><br>
<b>Sent:</b> Wednesday, 11 March 2020 7:48 PM<br>
<b>To:</b> <a href="mailto:dal@ivoa.net" target="_blank">dal@ivoa.net</a> <<a href="mailto:dal@ivoa.net" target="_blank">dal@ivoa.net</a>>; <a href="mailto:registry@ivoa.net" target="_blank">registry@ivoa.net</a> <<a href="mailto:registry@ivoa.net" target="_blank">registry@ivoa.net</a>>; <a href="mailto:semantics@ivoa.net" target="_blank">semantics@ivoa.net</a> <<a href="mailto:semantics@ivoa.net" target="_blank">semantics@ivoa.net</a>><br>
<b>Subject:</b> Re: Vocabularising dataproduct_type</span> <u></u><u></u></p>
<div>
<p class="MsoNormal" style="margin-left:36pt"> <u></u><u></u></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal" style="margin-left:36pt">Hi Sarah,<br>
<br>
On Tue, Mar 10, 2020 at 03:58:09PM +0000, Sarah Weissman wrote:<br>
> * Would too many things be mapped into "Measurement" to be useful<br>
> from a discovery perspective?<br>
<br>
Given my findings yesterday (no active obscore service right now has<br>
measurements rows), I think we shouldn't sweat this measurements<br>
thing too much at this point -- in Registry, this term would only be<br>
used for (SSAP, for now) services *returning* lists of such<br>
measurements (as they would otherwise return spectra), which seems a<br>
long shot.<br>
<br>
Normal discovery of catalogues and similar will of course proceed as<br>
usual.<br>
<br>
However, I largely agree with Laurent:<br>
<br>
On Tue, 10 Mar 2020 17:42:43 +0100, Laurent Michel wrote:<br>
> I would really prefer the list of dataproduct_type to ensur an ascending<br>
> compatibility with Obscore. I believe it is worth to keep as fas as possible a<br>
> consistance betwenn standards. This might help in many cases to map things<br>
<br>
So, even though I think we've found that the actual definition of<br>
measurement is, well, hard, I'm now rather convinced we shouldn't<br>
drop it. On the other hand, when we follow Marco --<br>
<br>
On Tue, 10 Mar 2020 13:53:09 +0100, Molinaro, Marco wrote:<br>
> Il giorno mar 10 mar 2020 alle ore 13:14 Markus Demleitner <<br>
>> A set of derived measurements obtained from a particular original<br>
>> dataset. The prototypical example would be a list of sources<br>
>> extracted from an image.<br>
><br>
> Does it have to be "original dataset" _singular_?<br>
<br>
and Laurent's previous proposal, we'd end up with:<br>
<br>
A set of derived measurements obtained from original datasets or a<br>
catalog of sources.<br>
<br>
I'd say the "obtained from original datasets" in there doesn't really<br>
mean much (what else could the measurements be derived from?), so<br>
reduced it would work out to<br>
<br>
A set of derived measurements or a catalog of sources.<br>
<br>
Which, as Sarah rightly says, is really vague and all-encompassing.<br>
As I said, I'd normally throw out such a term on grounds of it<br>
colliding with too many other terms without really fitting well into<br>
a hierarchy.<br>
<br>
But again, since it's in obscore, we shouldn't drop it. But since<br>
it, to me, seems ill-defined, perhaps we should be frank and just say:<br>
<br>
Generic tabular data not fitting any of the other terms. Because<br>
of its lack of specificity, this term should generally be avoided,<br>
and new, more precise terms should be introduced instead.<br>
<br>
Can people live with that? If we find good use cases for<br>
"measurements" later, we still can get more precise in this<br>
definition as long as we now say "don't use it, really".<br>
<br>
Back to Sarah:<br>
<br>
On Tue, Mar 10, 2020 at 03:58:09PM +0000, Sarah Weissman wrote:<br>
> * Do we need a separate category for "Model"?<br>
<br>
The future Vocabularies 2 standard is designed to make it easy to add<br>
new terms exactly when they're actually needed. So, when you you<br>
have an SSA or Obscore service returning such Models instances -- or<br>
some future use of this vocabulary needs it --, we can include it.<br>
For now, let's see if we can just stick with what Obscore already<br>
has.<br>
<br>
> * Do you expect that more than one label would be applied to a data<br>
> product? For example (naively) could a "spectral image cube" be<br>
> labeled with "spectrum" and "image" and "cube"?<br>
<br>
This depends on where the terms are being used. In obscore, only one<br>
label per row is possible, and I don't think that can be changed in a<br>
backwards-compatible way; we could, however, in some future version<br>
of obscore, introduce a multi-valued "other_dataproduct_types" in the<br>
style of EPN-TAP -- but that's a different discussion.<br>
<br>
In what I'm drafting for SimpleDALRegExt, SSA services can be<br>
annotated with zero or more dataproductType elements (current<br>
internal draft: <a href="http://docs.g-vo.org/SimpleDALRegExt.pdf" target="_blank">http://docs.g-vo.org/SimpleDALRegExt.pdf</a> or on<br>
volute). More on this later on the Registry list.<br>
<br>
Thanks,<br>
<br>
Markus<u></u><u></u></p>
</div>
</div>
</div>
</div>
</blockquote></div>