<html>
  <head>

    <meta http-equiv="content-type" content="text/html; charset=UTF-8">
  </head>
  <body text="#000000" bgcolor="#FFFFFF">
    <p><br>
    </p>
    <div class="moz-forward-container"><br>
      <meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
      <p>Hi all, </p>
      <p>Two words on this.</p>
      <p>A ) Measurements<br>
      </p>
      <p> I remember the discussion among authors when we added (to the
        initial 1.0 list) the "measurements" term among authors of
        ObsCore 1.1 . Indeed I think the idea to add something like
        "catalog" was given by Daniel DUrand from CADC because they had
        this use case that Pat  is exposing.</p>
      <p>There was however a reason  why we didn't choose to add the
        dataproduct-type "catalog".</p>
      <p>     ObsCore/ObsTAP is for discovery of datasets which have
        time, spatial, spectral and polarisation axes. Selection on the
        ObsCore parameters is not sufficient for catalogs with plenty of
        other parameters which are directly queried via TAP (or even in
        the registry). So we had to find another word for these specific
        tables extracted from the datasets in order to not let beleive
        that ObsCore is a discovery model for any kind of catalog. Hence
        "measurements"</p>
      <p>     HiPS was different because they are really creating HiPS
        for ANY KIND of catalog !!! <br>
      </p>
      <p>     So I think we should keep "measurements" but not with the
        negative definition "Generic tabular data not fitting any of the
        other terms.  Because<br>
          of its lack of specificity, this term should generally be
        avoided,  and new, more precise terms should be introduced
        instead" any of the others will fit I think and yes we have to 
        keep ascendant compatibility with obsCore. This is pretty
        important for interoperability<br>
      </p>
      <p>B ) TimeSeries and :SED. hierarchies<br>
      </p>
      <p>     This is maybe more critical. the definition in ObsCore
        which has been reproduced in Markus' list seems to exclude
        TimeSeries of spectra or Images or whatever thing which is not a
        varying scalar. I don't remember the reason but where do we put
        spectrochronogram then ? Do we create subtypes of spectrum or
        cube ?</p>
      <p>      The question of subtypes and "parent" terms seems to be
        open. In Obscore there was no hierarchy in the terms. But in
        Markus list there is sed as a child from spectrum. I think some
        people complained about using dataproduct_subtype for such
        things and prefer to let this subtype Obscore field available
        for free strings.</p>
      <p>       Imagine in the future (next version of ObscORe) we
        decide to have ObsCore vocabulary for dataproduct_type
        externally defined as the VOcabulary list we are discussing
        here. This could be a way to easilly extend ObsCore vocabulary
        for this specific dataproduct_type attribute.<br>
      </p>
      <p>        We may imagine have "spectrochronogram" and "sed" as
        children elements of spectrum. "timecube" or "spectralcube" as
        children from cubes. <br>
      </p>
      <p>        This will be clear in the vocabulary page but ObsCore
        will manage that at the same level in dataproduct_type (exactly
        like we already have sed and spectrum in parallel today)</p>
      <p>       Thoughts ?</p>
      <p>C ) Miscelaneous.</p>
      <p>      If this vocabulary is to be used in various contexts (and
        indeed it is) why do we link it to SimpleDALRegExt ?
        Vocabularies 2.0 is proposing to manage vocabularies as endorsed
        notes. Why don't we do it this way and refer it from  SimpleDAL,
        ObsCore, DataLink, etc ...</p>
      <p>      The MS discussion indeed shows we need to add new
        refinements in formats conveyed by the "media types". Does one
        exist for Measurement sets already ?</p>
      <p>Cheers</p>
      <p>François<br>
      </p>
      <p>        <br>
      </p>
      <p><br>
      </p>
      <div class="moz-cite-prefix">Le 12/03/2020 à 16:20, Patrick Dowler
        a écrit :<br>
      </div>
      <blockquote type="cite"
cite="mid:CAFK8nrr7JRsj_G3UH9MFWBoT_xnbCAxJra7ipcWor6Y4Bsn8yw@mail.gmail.com">
        <div dir="ltr">
          <div class="gmail_default">note: cross-posting reduced to DAL
            and semantics</div>
          <div class="gmail_default"><br>
          </div>
          <div class="gmail_default">In CAOM, I made an
            ObsCore.dataproduyct_type base vocabulary and then added my
            own custom child of measurements named <a
              href="http://www.opencadc.org/caom/DataProductType#catalog"
              moz-do-not-send="true">http://www.opencadc.org/caom/DataProductType#catalog</a>
            (this is literally ObsCore + extensions so if this idea goes
            ahead we'll have to figure out this vocab refers back to the
            parent vocab). If one queries caom2 in our tap service you
            can see that (fully qualified) value mixed in with ObsCore
            values. In the ObsCore view I am currently limiting the rows
            to exclude custom (fully qualified) terms for compliance
            with the current model (1.1). In the event that:<br>
          </div>
          <div class="gmail_default"><br>
          </div>
          <div class="gmail_default">* ObsCore (1.2) dataproduct_type
            values are defined by a vocabulary, and</div>
          <div class="gmail_default">* providers can extend the
            vocabulary with custom terms (then fully qualified rather
            that just the short form when using base terms)</div>
          <div class="gmail_default"><br>
          </div>
          <div class="gmail_default">I could remove he filtering and
            allow more entries to appear in ObsCore view. If that
            doesn't become possible, we would have to decide to (i)
            leave content as-is or (ii) change those to measurements.</div>
          <div class="gmail_default"><br>
          </div>
          <div class="gmail_default">Currently CADC has:</div>
          <div class="gmail_default"><br>
          </div>
          <div class="gmail_default">select dataProductType,count(*)
            from caom2.Plane group by dataProductType;<br>
                                dataproducttype                    |
             count   <br>
-------------------------------------------------------+----------<br>
                                                                   |
            15538972<br>
             timeseries                                            |  
            866405<br>
             spectrum                                              |
             7911476<br>
             cube                                                  |  
            202980<br>
             <a
              href="http://www.opencadc.org/caom2/DataProductType#catalog"
              moz-do-not-send="true">http://www.opencadc.org/caom2/DataProductType#catalog</a>
            |   160850<br>
             eventlist                                             |  
             88421<br>
             image                                                 |
            16076616</div>
          <div class="gmail_default"><br>
          </div>
          <div class="gmail_default">So, we are using "measurements" but
            it is not surprising that Markus didn't see it. We only have
            one child term in use right now.<br>
          </div>
          <div class="gmail_default"><br>
          </div>
          <div>
            <div dir="ltr" class="gmail_signature"
              data-smartmail="gmail_signature">
              <div dir="ltr">
                <div>
                  <div dir="ltr">
                    <div>
                      <div>--<br>
                      </div>
                      <div>Patrick Dowler<br>
                      </div>
                      Canadian Astronomy Data Centre<br>
                    </div>
                    Victoria, BC, Canada<br>
                  </div>
                </div>
              </div>
            </div>
          </div>
          <br>
        </div>
        <br>
        <div class="gmail_quote">
          <div dir="ltr" class="gmail_attr">On Thu, 12 Mar 2020 at
            00:01, Slava Kitaeff &lt;<a
              href="mailto:slava.kitaeff@uwa.edu.au"
              moz-do-not-send="true">slava.kitaeff@uwa.edu.au</a>&gt;
            wrote:<br>
          </div>
          <blockquote class="gmail_quote">
            <div lang="EN-AU">
              <div class="gmail-m_-3349331814557714389WordSection1">
                <p class="MsoNormal">Hi James et all,</p>
                <p class="MsoNormal"> </p>
                <p class="MsoNormal">Just a quick note on that. It’s a
                  bit of a guess but I’m suspecting that this might be
                  referring to a Measurement Set (MS), which is a
                  specific data format (and a model) used by CASA (and
                  now ASKAPsoft) to store interferometric visibilities.
                  So, the data type is then must be “visibilities”. To
                  confuse things further, CASA can store continuum
                  images and spectral-line image-cubes in the same MS
                  format as CASA tables. In my view, it’d be ideal to
                  have two things describing a) the nature of data and
                  b) its specific format (including version).</p>
                <p class="MsoNormal"> </p>
                <p class="MsoNormal">Cheers,</p>
                <p class="MsoNormal">Slava</p>
                <p class="MsoNormal"><span>________________________________</span><span></span></p>
                <p class="MsoNormal"> </p>
                <p class="MsoNormal"><b><span>Dr Slava Kitaeff</span></b><span></span></p>
                <p class="MsoNormal"><span>Australian SKA Regional
                    Centre Program Manager</span><span></span></p>
                <p class="MsoNormal"><span> </span></p>
                <p class="MsoNormal"><b><span>International Centre for
                      Radio Astronomy Research</span></b><span></span></p>
                <p class="MsoNormal"><span>The University of Western
                    Australia</span><span></span></p>
                <p class="MsoNormal"><span> </span></p>
                <p class="MsoNormal"><b><span>Astronomy and Space
                      Science</span></b><span></span></p>
                <p class="MsoNormal"><span>CSIRO</span></p>
                <p class="MsoNormal"> </p>
                <p class="MsoNormal"><span>Email: </span><span><a
                      href="mailto:slava.kitaeff@icrar.org"
                      target="_blank" moz-do-not-send="true"><span>slava.kitaeff@icrar.org</span></a> </span><span>or <a
                      href="mailto:slava.kitaeff@csiro.au"
                      target="_blank" moz-do-not-send="true"><span>slava.kitaeff@csiro.au</span></a> </span><span></span></p>
                <p class="MsoNormal"><span>Tel.: (+61) (0) 8 6488 7744
                    (ICRAR) or (+61) (0) 8 6436 8865 (CSIRO)</span><span></span></p>
                <p class="MsoNormal"><span>Mob.: +61 404 297 414</span><span></span></p>
                <p class="MsoNormal"><i><span> </span></i><span></span></p>
                <p class="MsoNormal"><b><span>Mailing addresses:</span></b><span> </span><span></span></p>
                <p class="MsoNormal"><span>M468, University of Western
                    Australia, 35 Stirling Highway, Crawley WA 6009,
                    Australia, <b>or</b></span><span></span></p>
                <p class="MsoNormal"><span>CSIRO Astronomy and Space
                    Science, PO Box 1130, Bentley WA 6102, Australia</span><span></span></p>
                <p class="MsoNormal"><span> </span></p>
                <p class="MsoNormal"><span>ICRAR: Discovering the hidden
                    Universe through radio astronomy<b> </b></span><span></span></p>
                <p class="MsoNormal"><span><a
                      href="http://www.icrar.org" target="_blank"
                      moz-do-not-send="true"><span>www.icrar.org</span></a><span>    </span></span><a
                    href="http://www.icrar.org/#subscribe"
                    target="_blank" moz-do-not-send="true"><span>Subscribe
                      to our eNewsletter</span></a><span>    </span><a
                    href="http://twitter.com/icrar" target="_blank"
                    moz-do-not-send="true"><span>ICRAR on Twitter</span></a><span>   </span><a
href="http://www.facebook.com/pages/ICRAR/199692286227" target="_blank"
                    moz-do-not-send="true"><span>ICRAR on Facebook</span></a></p>
                <p class="MsoNormal"> </p>
                <p class="MsoNormal"> </p>
                <div>
                  <p class="MsoNormal"><b><span>From: </span></b><span>&lt;<a
                        href="mailto:dal-bounces@ivoa.net"
                        target="_blank" moz-do-not-send="true">dal-bounces@ivoa.net</a>&gt;
                      on behalf of "Dempsey, James (IM&amp;T, Black
                      Mountain)" <a class="moz-txt-link-rfc2396E"
                        href="mailto:James.Dempsey@csiro.au"
                        moz-do-not-send="true">&lt;James.Dempsey@csiro.au&gt;</a><br>
                      <b>Date: </b>Wednesday, 11 March 2020 at 5:43 pm<br>
                      <b>To: </b>Markus Demleitner &lt;<a
                        href="mailto:msdemlei@ari.uni-heidelberg.de"
                        target="_blank" moz-do-not-send="true">msdemlei@ari.uni-heidelberg.de</a>&gt;,
                      "<a href="mailto:dal@ivoa.net" target="_blank"
                        moz-do-not-send="true">dal@ivoa.net</a>" &lt;<a
                        href="mailto:dal@ivoa.net" target="_blank"
                        moz-do-not-send="true">dal@ivoa.net</a>&gt;, "<a
                        href="mailto:registry@ivoa.net" target="_blank"
                        moz-do-not-send="true">registry@ivoa.net</a>"
                      &lt;<a href="mailto:registry@ivoa.net"
                        target="_blank" moz-do-not-send="true">registry@ivoa.net</a>&gt;,
                      "<a href="mailto:semantics@ivoa.net"
                        target="_blank" moz-do-not-send="true">semantics@ivoa.net</a>"
                      &lt;<a href="mailto:semantics@ivoa.net"
                        target="_blank" moz-do-not-send="true">semantics@ivoa.net</a>&gt;<br>
                      <b>Subject: </b>Re: Vocabularising
                      dataproduct_type</span></p>
                </div>
                <div>
                  <p class="MsoNormal"> </p>
                </div>
                <div>
                  <p class="MsoNormal"><span>Hi Markus,</span></p>
                </div>
                <div>
                  <p class="MsoNormal"><span> </span></p>
                </div>
                <div>
                  <p class="MsoNormal"><span>We are dealing with a lot
                      of catalogues as well as images, cubes etc.
                      Currently these have a blank dataproduct_type in
                      the CASDA obscore instance as there wasn't
                      anything we could use in v1.0. We will probably
                      start using measurements soon as it is better than
                      nothing, but it isn't a term that our community
                      uses. Their expectation is to see catalogue in the
                      type, so I'd be keen to see that as term available
                      for use.</span></p>
                </div>
                <div>
                  <p class="MsoNormal"><span> </span></p>
                </div>
                <div>
                  <p class="MsoNormal"><span>Our catalogues are
                      generally source lists produced from the
                      associated  image/cube.</span></p>
                </div>
                <div>
                  <div>
                    <p class="MsoNormal"><span> </span></p>
                  </div>
                  <div>
                    <p class="MsoNormal"><span>Cheers,</span></p>
                  </div>
                  <div id="gmail-m_-3349331814557714389Signature">
                    <div name="divtagdefaultwrapper">
                      <div>
                        <div>
                          <div>
                            <p class="MsoNormal"><span>James Dempsey</span></p>
                          </div>
                          <div>
                            <p class="MsoNormal"><span>Senior Developer</span><span></span></p>
                          </div>
                          <div>
                            <p class="MsoNormal"><span>Information
                                Services Applications</span><span></span></p>
                          </div>
                          <div>
                            <p class="MsoNormal"><span>CSIRO Information
                                Management &amp; Technology (IM&amp;T)</span></p>
                          </div>
                        </div>
                      </div>
                    </div>
                  </div>
                </div>
                <div class="MsoNormal">
                  <hr width="100%"> </div>
                <div id="gmail-m_-3349331814557714389divRplyFwdMsg">
                  <p class="MsoNormal"><b><span>From:</span></b><span> <a
                        href="mailto:dal-bounces@ivoa.net"
                        target="_blank" moz-do-not-send="true">dal-bounces@ivoa.net</a>
                      &lt;<a href="mailto:dal-bounces@ivoa.net"
                        target="_blank" moz-do-not-send="true">dal-bounces@ivoa.net</a>&gt;
                      on behalf of Markus Demleitner &lt;<a
                        href="mailto:msdemlei@ari.uni-heidelberg.de"
                        target="_blank" moz-do-not-send="true">msdemlei@ari.uni-heidelberg.de</a>&gt;<br>
                      <b>Sent:</b> Wednesday, 11 March 2020 7:48 PM<br>
                      <b>To:</b> <a href="mailto:dal@ivoa.net"
                        target="_blank" moz-do-not-send="true">dal@ivoa.net</a>
                      &lt;<a href="mailto:dal@ivoa.net" target="_blank"
                        moz-do-not-send="true">dal@ivoa.net</a>&gt;; <a
                        href="mailto:registry@ivoa.net" target="_blank"
                        moz-do-not-send="true">registry@ivoa.net</a>
                      &lt;<a href="mailto:registry@ivoa.net"
                        target="_blank" moz-do-not-send="true">registry@ivoa.net</a>&gt;;
                      <a href="mailto:semantics@ivoa.net"
                        target="_blank" moz-do-not-send="true">semantics@ivoa.net</a>
                      &lt;<a href="mailto:semantics@ivoa.net"
                        target="_blank" moz-do-not-send="true">semantics@ivoa.net</a>&gt;<br>
                      <b>Subject:</b> Re: Vocabularising
                      dataproduct_type</span> </p>
                  <div>
                    <p class="MsoNormal"> </p>
                  </div>
                </div>
                <div>
                  <div>
                    <p class="MsoNormal">Hi Sarah,<br>
                      <br>
                      On Tue, Mar 10, 2020 at 03:58:09PM +0000, Sarah
                      Weissman wrote:<br>
                      &gt; * Would too many things be mapped into
                      "Measurement" to be useful<br>
                      &gt; from a discovery perspective?<br>
                      <br>
                      Given my findings yesterday (no active obscore
                      service right now has<br>
                      measurements rows), I think we shouldn't sweat
                      this measurements<br>
                      thing too much at this point -- in Registry, this
                      term would only be<br>
                      used for (SSAP, for now) services *returning*
                      lists of such<br>
                      measurements (as they would otherwise return
                      spectra), which seems a<br>
                      long shot.<br>
                      <br>
                      Normal discovery of catalogues and similar will of
                      course proceed as<br>
                      usual.<br>
                      <br>
                      However, I largely agree with Laurent:<br>
                      <br>
                      On Tue, 10 Mar 2020 17:42:43 +0100, Laurent Michel
                      wrote:<br>
                      &gt; I would really prefer the list of
                      dataproduct_type to ensur an ascending<br>
                      &gt; compatibility with Obscore. I believe it is
                      worth to keep as fas as possible a<br>
                      &gt; consistance betwenn standards. This might
                      help in many cases to map things<br>
                      <br>
                      So, even though I think we've found that the
                      actual definition of<br>
                      measurement is, well, hard, I'm now rather
                      convinced we shouldn't<br>
                      drop it.  On the other hand, when we follow Marco
                      --<br>
                      <br>
                      On Tue, 10 Mar 2020 13:53:09 +0100, Molinaro,
                      Marco wrote:<br>
                      &gt; Il giorno mar 10 mar 2020 alle ore 13:14
                      Markus Demleitner &lt;<br>
                      &gt;&gt;   A set of derived measurements obtained
                      from a particular original<br>
                      &gt;&gt;   dataset.  The prototypical example
                      would be a list of sources<br>
                      &gt;&gt;   extracted from an image.<br>
                      &gt;<br>
                      &gt; Does it have to be "original dataset"
                      _singular_?<br>
                      <br>
                      and Laurent's previous proposal, we'd end up with:<br>
                      <br>
                        A set of derived measurements obtained from
                      original datasets or a<br>
                        catalog of sources.<br>
                      <br>
                      I'd say the "obtained from original datasets" in
                      there doesn't really<br>
                      mean much (what else could the measurements be
                      derived from?), so<br>
                      reduced it would work out to<br>
                      <br>
                        A set of derived measurements or a catalog of
                      sources.<br>
                      <br>
                      Which, as Sarah rightly says, is really vague and
                      all-encompassing.<br>
                      As I said, I'd normally throw out such a term on
                      grounds of it<br>
                      colliding with too many other terms without really
                      fitting well into<br>
                      a hierarchy.<br>
                      <br>
                      But again, since it's in obscore, we shouldn't
                      drop it.  But since<br>
                      it, to me, seems ill-defined, perhaps we should be
                      frank and just say:<br>
                      <br>
                        Generic tabular data not fitting any of the
                      other terms.  Because<br>
                        of its lack of specificity, this term should
                      generally be avoided,<br>
                        and new, more precise terms should be introduced
                      instead.<br>
                      <br>
                      Can people live with that?  If we find good use
                      cases for<br>
                      "measurements" later, we still can get more
                      precise in this<br>
                      definition as long as we now say "don't use it,
                      really".<br>
                      <br>
                      Back to Sarah:<br>
                      <br>
                      On Tue, Mar 10, 2020 at 03:58:09PM +0000, Sarah
                      Weissman wrote:<br>
                      &gt; * Do we need a separate category for "Model"?<br>
                      <br>
                      The future Vocabularies 2 standard is designed to
                      make it easy to add<br>
                      new terms exactly when they're actually needed. 
                      So, when you you<br>
                      have an SSA or Obscore service returning such
                      Models instances -- or<br>
                      some future use of this vocabulary needs it --, we
                      can include it.<br>
                      For now, let's see if we can just stick with what
                      Obscore already<br>
                      has.<br>
                      <br>
                      &gt; * Do you expect that more than one label
                      would be applied to a data<br>
                      &gt; product? For example (naively) could a
                      "spectral image cube" be<br>
                      &gt; labeled with "spectrum" and "image" and
                      "cube"?<br>
                      <br>
                      This depends on where the terms are being used. 
                      In obscore, only one<br>
                      label per row is possible, and I don't think that
                      can be changed in a<br>
                      backwards-compatible way; we could, however, in
                      some future version<br>
                      of obscore, introduce a multi-valued
                      "other_dataproduct_types" in the<br>
                      style of EPN-TAP -- but that's a different
                      discussion.<br>
                      <br>
                      In what I'm drafting for SimpleDALRegExt, SSA
                      services can be<br>
                      annotated with zero or more dataproductType
                      elements (current<br>
                      internal draft: <a
                        href="http://docs.g-vo.org/SimpleDALRegExt.pdf"
                        target="_blank" moz-do-not-send="true">http://docs.g-vo.org/SimpleDALRegExt.pdf</a>
                      or on<br>
                      volute).  More on this later on the Registry list.<br>
                      <br>
                      Thanks,<br>
                      <br>
                             Markus</p>
                  </div>
                </div>
              </div>
            </div>
          </blockquote>
        </div>
      </blockquote>
    </div>
  </body>
</html>