<html>
  <head>
    <meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
  </head>
  <body text="#000000" bgcolor="#FFFFFF">
    <p>Hi all, </p>
    <p>Two words on this.</p>
    <p>A ) Measurements<br>
    </p>
    <p> I remember the discussion among authors when we added (to the
      initial 1.0 list) the "measurements" term among authors of ObsCore
      1.1 . Indeed I think the idea to add something like "catalog" was
      given by Daniel DUrand from CADC because they had this use case
      that Pat  is exposing.</p>
    <p>There was however a reason  why we didn't choose to add the
      dataproduct-type "catalog".</p>
    <p>     ObsCore/ObsTAP is for discovery of datasets which have time,
      spatial, spectral and polarisation axes. Selection on the ObsCore
      parameters is not sufficient for catalogs with plenty of other
      parameters which are directly queried via TAP (or even in the
      registry). So we had to find another word for these specific
      tables extracted from the datasets in order to not let beleive
      that ObsCore is a discovery model for any kind of catalog. Hence
      "measurements"</p>
    <p>     HiPS was different because they are really creating HiPS for
      ANY KIND of catalog !!! <br>
    </p>
    <p>     So I think we should keep "measurements" but not with the
      negative definition "Generic tabular data not fitting any of the
      other terms.  Because<br>
        of its lack of specificity, this term should generally be
      avoided,  and new, more precise terms should be introduced
      instead" any of the others will fit I think and yes we have to 
      keep ascendant compatibility with obsCore. This is pretty
      important for interoperability<br>
    </p>
    <p>B ) TimeSeries and :SED. hierarchies<br>
    </p>
    <p>     This is maybe more critical. the definition in ObsCore which
      has been reproduced in Markus' list seems to exclude TimeSeries of
      spectra or Images or whatever thing which is not a varying scalar.
      I don't remember the reason but where do we put spectrochronogram
      then ? Do we create subtypes of spectrum or cube ?</p>
    <p>      The question of subtypes and "parent" terms seems to be
      open. In Obscore there was no hierarchy in the terms. But in
      Markus list there is sed as a child from spectrum. I think some
      people complained about using dataproduct_subtype for such things
      and prefer to let this subtype Obscore field available for free
      strings.</p>
    <p>       Imagine in the future (next version of ObscORe) we decide
      to have ObsCore vocabulary for dataproduct_type externally defined
      as the VOcabulary list we are discussing here. This could be a way
      to easilly extend ObsCore vocabulary for this specific
      dataproduct_type attribute.<br>
    </p>
    <p>        We may imagine have "spectrochronogram" and "sed" as
      children elements of spectrum. "timecube" or "spectralcube" as
      children from cubes. <br>
    </p>
    <p>        This will be clear in the vocabulary page but ObsCore
      will manage that at the same level in dataproduct_type (exactly
      like we already have sed and spectrum in parallel today)</p>
    <p>       Thoughts ?</p>
    <p>C ) Miscelaneous.</p>
    <p>      If this vocabulary is to be used in various contexts (and
      indeed it is) why do we link it to SimpleDALRegExt ? Vocabularies
      2.0 is proposing to manage vocabularies as endorsed notes. Why
      don't we do it this way and refer it from  SimpleDAL, ObsCore,
      DataLink, etc ...</p>
    <p>      The MS discussion indeed shows we need to add new
      refinements in formats conveyed by the "media types". Does one
      exist for Measurement sets already ?</p>
    <p>Cheers</p>
    <p>François<br>
    </p>
    <p>        <br>
    </p>
    <p><br>
    </p>
    <div class="moz-cite-prefix">Le 12/03/2020 à 16:20, Patrick Dowler a
      écrit :<br>
    </div>
    <blockquote type="cite"
cite="mid:CAFK8nrr7JRsj_G3UH9MFWBoT_xnbCAxJra7ipcWor6Y4Bsn8yw@mail.gmail.com">
      <div dir="ltr">
        <div class="gmail_default">note: cross-posting reduced to DAL
          and semantics</div>
        <div class="gmail_default"><br>
        </div>
        <div class="gmail_default">In CAOM, I made an
          ObsCore.dataproduyct_type base vocabulary and then added my
          own custom child of measurements named <a
            href="http://www.opencadc.org/caom/DataProductType#catalog"
            moz-do-not-send="true">http://www.opencadc.org/caom/DataProductType#catalog</a>
          (this is literally ObsCore + extensions so if this idea goes
          ahead we'll have to figure out this vocab refers back to the
          parent vocab). If one queries caom2 in our tap service you can
          see that (fully qualified) value mixed in with ObsCore values.
          In the ObsCore view I am currently limiting the rows to
          exclude custom (fully qualified) terms for compliance with the
          current model (1.1). In the event that:<br>
        </div>
        <div class="gmail_default"><br>
        </div>
        <div class="gmail_default">* ObsCore (1.2) dataproduct_type
          values are defined by a vocabulary, and</div>
        <div class="gmail_default">* providers can extend the vocabulary
          with custom terms (then fully qualified rather that just the
          short form when using base terms)</div>
        <div class="gmail_default"><br>
        </div>
        <div class="gmail_default">I could remove he filtering and allow
          more entries to appear in ObsCore view. If that doesn't become
          possible, we would have to decide to (i) leave content as-is
          or (ii) change those to measurements.</div>
        <div class="gmail_default"><br>
        </div>
        <div class="gmail_default">Currently CADC has:</div>
        <div class="gmail_default"><br>
        </div>
        <div class="gmail_default">select dataProductType,count(*) from
          caom2.Plane group by dataProductType;<br>
                              dataproducttype                    |
           count   <br>
-------------------------------------------------------+----------<br>
                                                                 |
          15538972<br>
           timeseries                                            |  
          866405<br>
           spectrum                                              |
           7911476<br>
           cube                                                  |  
          202980<br>
           <a
            href="http://www.opencadc.org/caom2/DataProductType#catalog"
            moz-do-not-send="true">http://www.opencadc.org/caom2/DataProductType#catalog</a>
          |   160850<br>
           eventlist                                             |  
           88421<br>
           image                                                 |
          16076616</div>
        <div class="gmail_default"><br>
        </div>
        <div class="gmail_default">So, we are using "measurements" but
          it is not surprising that Markus didn't see it. We only have
          one child term in use right now.<br>
        </div>
        <div class="gmail_default"><br>
        </div>
        <div>
          <div dir="ltr" class="gmail_signature"
            data-smartmail="gmail_signature">
            <div dir="ltr">
              <div>
                <div dir="ltr">
                  <div>
                    <div>--<br>
                    </div>
                    <div>Patrick Dowler<br>
                    </div>
                    Canadian Astronomy Data Centre<br>
                  </div>
                  Victoria, BC, Canada<br>
                </div>
              </div>
            </div>
          </div>
        </div>
        <br>
      </div>
      <br>
      <div class="gmail_quote">
        <div dir="ltr" class="gmail_attr">On Thu, 12 Mar 2020 at 00:01,
          Slava Kitaeff &lt;<a href="mailto:slava.kitaeff@uwa.edu.au"
            moz-do-not-send="true">slava.kitaeff@uwa.edu.au</a>&gt;
          wrote:<br>
        </div>
        <blockquote class="gmail_quote">
          <div lang="EN-AU">
            <div class="gmail-m_-3349331814557714389WordSection1">
              <p class="MsoNormal">Hi James et all,</p>
              <p class="MsoNormal"> </p>
              <p class="MsoNormal">Just a quick note on that. It’s a bit
                of a guess but I’m suspecting that this might be
                referring to a Measurement Set (MS), which is a specific
                data format (and a model) used by CASA (and now
                ASKAPsoft) to store interferometric visibilities. So,
                the data type is then must be “visibilities”. To confuse
                things further, CASA can store continuum images and
                spectral-line image-cubes in the same MS format as CASA
                tables. In my view, it’d be ideal to have two things
                describing a) the nature of data and b) its specific
                format (including version).</p>
              <p class="MsoNormal"> </p>
              <p class="MsoNormal">Cheers,</p>
              <p class="MsoNormal">Slava</p>
              <p class="MsoNormal"><span>________________________________</span><span></span></p>
              <p class="MsoNormal"> </p>
              <p class="MsoNormal"><b><span>Dr Slava Kitaeff</span></b><span></span></p>
              <p class="MsoNormal"><span>Australian SKA Regional Centre
                  Program Manager</span><span></span></p>
              <p class="MsoNormal"><span> </span></p>
              <p class="MsoNormal"><b><span>International Centre for
                    Radio Astronomy Research</span></b><span></span></p>
              <p class="MsoNormal"><span>The University of Western
                  Australia</span><span></span></p>
              <p class="MsoNormal"><span> </span></p>
              <p class="MsoNormal"><b><span>Astronomy and Space Science</span></b><span></span></p>
              <p class="MsoNormal"><span>CSIRO</span></p>
              <p class="MsoNormal"> </p>
              <p class="MsoNormal"><span>Email: </span><span><a
                    href="mailto:slava.kitaeff@icrar.org"
                    target="_blank" moz-do-not-send="true"><span>slava.kitaeff@icrar.org</span></a> </span><span>or <a
                    href="mailto:slava.kitaeff@csiro.au" target="_blank"
                    moz-do-not-send="true"><span>slava.kitaeff@csiro.au</span></a> </span><span></span></p>
              <p class="MsoNormal"><span>Tel.: (+61) (0) 8 6488 7744
                  (ICRAR) or (+61) (0) 8 6436 8865 (CSIRO)</span><span></span></p>
              <p class="MsoNormal"><span>Mob.: +61 404 297 414</span><span></span></p>
              <p class="MsoNormal"><i><span> </span></i><span></span></p>
              <p class="MsoNormal"><b><span>Mailing addresses:</span></b><span> </span><span></span></p>
              <p class="MsoNormal"><span>M468, University of Western
                  Australia, 35 Stirling Highway, Crawley WA 6009,
                  Australia, <b>or</b></span><span></span></p>
              <p class="MsoNormal"><span>CSIRO Astronomy and Space
                  Science, PO Box 1130, Bentley WA 6102, Australia</span><span></span></p>
              <p class="MsoNormal"><span> </span></p>
              <p class="MsoNormal"><span>ICRAR: Discovering the hidden
                  Universe through radio astronomy<b> </b></span><span></span></p>
              <p class="MsoNormal"><span><a href="http://www.icrar.org"
                    target="_blank" moz-do-not-send="true"><span>www.icrar.org</span></a><span>    </span></span><a
                  href="http://www.icrar.org/#subscribe" target="_blank"
                  moz-do-not-send="true"><span>Subscribe to our
                    eNewsletter</span></a><span>    </span><a
                  href="http://twitter.com/icrar" target="_blank"
                  moz-do-not-send="true"><span>ICRAR on Twitter</span></a><span>   </span><a
href="http://www.facebook.com/pages/ICRAR/199692286227" target="_blank"
                  moz-do-not-send="true"><span>ICRAR on Facebook</span></a></p>
              <p class="MsoNormal"> </p>
              <p class="MsoNormal"> </p>
              <div>
                <p class="MsoNormal"><b><span>From:
                    </span></b><span>&lt;<a
                      href="mailto:dal-bounces@ivoa.net" target="_blank"
                      moz-do-not-send="true">dal-bounces@ivoa.net</a>&gt;
                    on behalf of "Dempsey, James (IM&amp;T, Black
                    Mountain)" <a class="moz-txt-link-rfc2396E" href="mailto:James.Dempsey@csiro.au">&lt;James.Dempsey@csiro.au&gt;</a><br>
                    <b>Date: </b>Wednesday, 11 March 2020 at 5:43 pm<br>
                    <b>To: </b>Markus Demleitner &lt;<a
                      href="mailto:msdemlei@ari.uni-heidelberg.de"
                      target="_blank" moz-do-not-send="true">msdemlei@ari.uni-heidelberg.de</a>&gt;,
                    "<a href="mailto:dal@ivoa.net" target="_blank"
                      moz-do-not-send="true">dal@ivoa.net</a>" &lt;<a
                      href="mailto:dal@ivoa.net" target="_blank"
                      moz-do-not-send="true">dal@ivoa.net</a>&gt;, "<a
                      href="mailto:registry@ivoa.net" target="_blank"
                      moz-do-not-send="true">registry@ivoa.net</a>" &lt;<a
                      href="mailto:registry@ivoa.net" target="_blank"
                      moz-do-not-send="true">registry@ivoa.net</a>&gt;,
                    "<a href="mailto:semantics@ivoa.net" target="_blank"
                      moz-do-not-send="true">semantics@ivoa.net</a>"
                    &lt;<a href="mailto:semantics@ivoa.net"
                      target="_blank" moz-do-not-send="true">semantics@ivoa.net</a>&gt;<br>
                    <b>Subject: </b>Re: Vocabularising dataproduct_type</span></p>
              </div>
              <div>
                <p class="MsoNormal"> </p>
              </div>
              <div>
                <p class="MsoNormal"><span>Hi Markus,</span></p>
              </div>
              <div>
                <p class="MsoNormal"><span> </span></p>
              </div>
              <div>
                <p class="MsoNormal"><span>We are dealing with a lot of
                    catalogues as well as images, cubes etc. Currently
                    these have a blank dataproduct_type in the CASDA
                    obscore instance as there wasn't anything we could
                    use in v1.0. We will probably start using
                    measurements soon as it is better than nothing, but
                    it isn't a term that our community uses. Their
                    expectation is to see catalogue in the type, so I'd
                    be keen to see that as term available for use.</span></p>
              </div>
              <div>
                <p class="MsoNormal"><span> </span></p>
              </div>
              <div>
                <p class="MsoNormal"><span>Our catalogues are generally
                    source lists produced from the associated
                     image/cube.</span></p>
              </div>
              <div>
                <div>
                  <p class="MsoNormal"><span> </span></p>
                </div>
                <div>
                  <p class="MsoNormal"><span>Cheers,</span></p>
                </div>
                <div id="gmail-m_-3349331814557714389Signature">
                  <div name="divtagdefaultwrapper">
                    <div>
                      <div>
                        <div>
                          <p class="MsoNormal"><span>James Dempsey</span></p>
                        </div>
                        <div>
                          <p class="MsoNormal"><span>Senior Developer</span><span></span></p>
                        </div>
                        <div>
                          <p class="MsoNormal"><span>Information
                              Services Applications</span><span></span></p>
                        </div>
                        <div>
                          <p class="MsoNormal"><span>CSIRO Information
                              Management &amp; Technology (IM&amp;T)</span></p>
                        </div>
                      </div>
                    </div>
                  </div>
                </div>
              </div>
              <div class="MsoNormal">
                <hr width="100%">
              </div>
              <div id="gmail-m_-3349331814557714389divRplyFwdMsg">
                <p class="MsoNormal"><b><span>From:</span></b><span> <a
                      href="mailto:dal-bounces@ivoa.net" target="_blank"
                      moz-do-not-send="true">dal-bounces@ivoa.net</a>
                    &lt;<a href="mailto:dal-bounces@ivoa.net"
                      target="_blank" moz-do-not-send="true">dal-bounces@ivoa.net</a>&gt;
                    on behalf of Markus Demleitner &lt;<a
                      href="mailto:msdemlei@ari.uni-heidelberg.de"
                      target="_blank" moz-do-not-send="true">msdemlei@ari.uni-heidelberg.de</a>&gt;<br>
                    <b>Sent:</b> Wednesday, 11 March 2020 7:48 PM<br>
                    <b>To:</b> <a href="mailto:dal@ivoa.net"
                      target="_blank" moz-do-not-send="true">dal@ivoa.net</a>
                    &lt;<a href="mailto:dal@ivoa.net" target="_blank"
                      moz-do-not-send="true">dal@ivoa.net</a>&gt;; <a
                      href="mailto:registry@ivoa.net" target="_blank"
                      moz-do-not-send="true">registry@ivoa.net</a> &lt;<a
                      href="mailto:registry@ivoa.net" target="_blank"
                      moz-do-not-send="true">registry@ivoa.net</a>&gt;;
                    <a href="mailto:semantics@ivoa.net" target="_blank"
                      moz-do-not-send="true">semantics@ivoa.net</a> &lt;<a
                      href="mailto:semantics@ivoa.net" target="_blank"
                      moz-do-not-send="true">semantics@ivoa.net</a>&gt;<br>
                    <b>Subject:</b> Re: Vocabularising dataproduct_type</span>
                </p>
                <div>
                  <p class="MsoNormal"> </p>
                </div>
              </div>
              <div>
                <div>
                  <p class="MsoNormal">Hi Sarah,<br>
                    <br>
                    On Tue, Mar 10, 2020 at 03:58:09PM +0000, Sarah
                    Weissman wrote:<br>
                    &gt; * Would too many things be mapped into
                    "Measurement" to be useful<br>
                    &gt; from a discovery perspective?<br>
                    <br>
                    Given my findings yesterday (no active obscore
                    service right now has<br>
                    measurements rows), I think we shouldn't sweat this
                    measurements<br>
                    thing too much at this point -- in Registry, this
                    term would only be<br>
                    used for (SSAP, for now) services *returning* lists
                    of such<br>
                    measurements (as they would otherwise return
                    spectra), which seems a<br>
                    long shot.<br>
                    <br>
                    Normal discovery of catalogues and similar will of
                    course proceed as<br>
                    usual.<br>
                    <br>
                    However, I largely agree with Laurent:<br>
                    <br>
                    On Tue, 10 Mar 2020 17:42:43 +0100, Laurent Michel
                    wrote:<br>
                    &gt; I would really prefer the list of
                    dataproduct_type to ensur an ascending<br>
                    &gt; compatibility with Obscore. I believe it is
                    worth to keep as fas as possible a<br>
                    &gt; consistance betwenn standards. This might help
                    in many cases to map things<br>
                    <br>
                    So, even though I think we've found that the actual
                    definition of<br>
                    measurement is, well, hard, I'm now rather convinced
                    we shouldn't<br>
                    drop it.  On the other hand, when we follow Marco --<br>
                    <br>
                    On Tue, 10 Mar 2020 13:53:09 +0100, Molinaro, Marco
                    wrote:<br>
                    &gt; Il giorno mar 10 mar 2020 alle ore 13:14 Markus
                    Demleitner &lt;<br>
                    &gt;&gt;   A set of derived measurements obtained
                    from a particular original<br>
                    &gt;&gt;   dataset.  The prototypical example would
                    be a list of sources<br>
                    &gt;&gt;   extracted from an image.<br>
                    &gt;<br>
                    &gt; Does it have to be "original dataset"
                    _singular_?<br>
                    <br>
                    and Laurent's previous proposal, we'd end up with:<br>
                    <br>
                      A set of derived measurements obtained from
                    original datasets or a<br>
                      catalog of sources.<br>
                    <br>
                    I'd say the "obtained from original datasets" in
                    there doesn't really<br>
                    mean much (what else could the measurements be
                    derived from?), so<br>
                    reduced it would work out to<br>
                    <br>
                      A set of derived measurements or a catalog of
                    sources.<br>
                    <br>
                    Which, as Sarah rightly says, is really vague and
                    all-encompassing.<br>
                    As I said, I'd normally throw out such a term on
                    grounds of it<br>
                    colliding with too many other terms without really
                    fitting well into<br>
                    a hierarchy.<br>
                    <br>
                    But again, since it's in obscore, we shouldn't drop
                    it.  But since<br>
                    it, to me, seems ill-defined, perhaps we should be
                    frank and just say:<br>
                    <br>
                      Generic tabular data not fitting any of the other
                    terms.  Because<br>
                      of its lack of specificity, this term should
                    generally be avoided,<br>
                      and new, more precise terms should be introduced
                    instead.<br>
                    <br>
                    Can people live with that?  If we find good use
                    cases for<br>
                    "measurements" later, we still can get more precise
                    in this<br>
                    definition as long as we now say "don't use it,
                    really".<br>
                    <br>
                    Back to Sarah:<br>
                    <br>
                    On Tue, Mar 10, 2020 at 03:58:09PM +0000, Sarah
                    Weissman wrote:<br>
                    &gt; * Do we need a separate category for "Model"?<br>
                    <br>
                    The future Vocabularies 2 standard is designed to
                    make it easy to add<br>
                    new terms exactly when they're actually needed.  So,
                    when you you<br>
                    have an SSA or Obscore service returning such Models
                    instances -- or<br>
                    some future use of this vocabulary needs it --, we
                    can include it.<br>
                    For now, let's see if we can just stick with what
                    Obscore already<br>
                    has.<br>
                    <br>
                    &gt; * Do you expect that more than one label would
                    be applied to a data<br>
                    &gt; product? For example (naively) could a
                    "spectral image cube" be<br>
                    &gt; labeled with "spectrum" and "image" and "cube"?<br>
                    <br>
                    This depends on where the terms are being used.  In
                    obscore, only one<br>
                    label per row is possible, and I don't think that
                    can be changed in a<br>
                    backwards-compatible way; we could, however, in some
                    future version<br>
                    of obscore, introduce a multi-valued
                    "other_dataproduct_types" in the<br>
                    style of EPN-TAP -- but that's a different
                    discussion.<br>
                    <br>
                    In what I'm drafting for SimpleDALRegExt, SSA
                    services can be<br>
                    annotated with zero or more dataproductType elements
                    (current<br>
                    internal draft: <a
                      href="http://docs.g-vo.org/SimpleDALRegExt.pdf"
                      target="_blank" moz-do-not-send="true">http://docs.g-vo.org/SimpleDALRegExt.pdf</a>
                    or on<br>
                    volute).  More on this later on the Registry list.<br>
                    <br>
                    Thanks,<br>
                    <br>
                           Markus</p>
                </div>
              </div>
            </div>
          </div>
        </blockquote>
      </div>
    </blockquote>
  </body>
</html>