Vocabularising dataproduct_type

[Markus Demleitner]

Hi Laurent,

On Tue, Mar 10, 2020 at 02:07:48PM +0100, Laurent Michel wrote:
> > So... what do you suggest to fix this?  The definition I'm currently
> > proposing is
> 
> I propose :
> measurements: A set of derived measurements obtained from a particular
> original dataset or a catalog of sources compiled from various progenitors.

I've put that in in rev. 5741 (but I've not uploaded it to
ivoa.net/rdf yet), but very frankly I'm not thrilled by "a
catalog of sources compiled from various progenitors" because that's
a *very* wide definition.  In particular, I'm always getting a bit
queasy when things get as close to tautologies as "derived from <one
or more> progenitors" -- I'd argue that isn't a restriction on
"catalog" at all.

If you follow that objection, then what's left of the definition is

  A set of derived measurements obtained from a particular original
  dataset or a catalog of sources.

Hm.  I suspect that's not quite what people wanted, was it?

Perhaps it's time to take a step back and ask ourselves what the
concept actually is, i.e., what is the set of things this term should
address, perhaps as a list of files that is a "measuremen" and,
equally importantly, a list of files that is not?  

Since I'd still like to make this reflect obscore usage: was there a
collection of use cases back when this term was added to the list of
allowed dataproduct_type values?

And talking about obscore usage, I've just run a global obscore query
for

  SELECT TOP 1000
  access_url
  FROM ivoa.obscore
  WHERE dataproduct_type='measurement'

[2] and got back zero rows[1].

This would perhaps indicate that the term isn't all that required and
could be left out for now?

           -- Markus


[1] Disclaimer: I ^C-ed out of CSIRO ASKAP and IAPS TAP because I
lost patience with them; so, if they have measurement-typed rows, I'd
have missed them.

[2] In case you'd like to experiment with global obscore queries,
here's the pyvo program I've used:

import pyvo

QUERY = """
SELECT TOP 1000
  access_url
FROM ivoa.obscore
WHERE dataproduct_type='measurement'
"""


def query_obscore():
    results = []
    for svc_rec in pyvo.registry.search(datamodel="obscore"):
        print("Querying {}".format(svc_rec.res_title))
        try:
            svc = pyvo.dal.TAPService(svc_rec.access_url)
            results.append(
                svc.run_sync(QUERY))
        except KeyboardInterrupt:
            # someone lost their patience with a service.  Query next.
            pass
        except Exception as msg:
            # some service is broken; you *should* complain, but
            # then let's be lazy here
            print("  Broken: {} ({})\n".format(
                svc_rec.access_url, msg))
        
    return table.vstack(results)


def main():
    tab = query_obscore()
    print(tab)
    
	
if __name__=="__main__":
    main()