Global dataset discovery

Markus Demleitner msdemlei at ari.uni-heidelberg.de
Mon Feb 26 11:31:35 CET 2024


Dear Apps, dear Registry,

If you've been in Tucson, you have perhaps heard my talk on global
dataset discovery ("Give me all images of this part of the sky that
are published in the VO";
<https://wiki.ivoa.net/internal/IVOA/InterOpNov2023Apps/twoup.pdf>)
with pyVO.  Well, I have finally written code that is actually doing
something pretty much like this; as promised in Tucson, the exercise
concentrates on images first, because that's probably the hardest
problem we have so far.

This is a long-ish story, a part of which I have written up in this blog
post: <https://blog.g-vo.org/global-dataset-discovery-in-pyvo.html>,
which I'd invite you to read.

As a minimal TL;DR let me state:

* PyVO PR #470 is now worth a look; perhaps we should even consider
  tentatively merging it: <https://github.com/astropy/pyvo/pull/470>.

* The problems of using dataModel to register ObsTAP services hit
  this use case face-on.  This needs to be fixed before global discovery
  including obscore will not suck.  I have hence started a Note to
  finally untangle the question of how to register TAP-published
  tables (which also concerns RegTAP, ObsLocTAP, EPN-TAP, LineTAP,
  and presumably more in the future, but they are generally in better
  shape already): <https://github.com/ivoa/TableReg>.  I'll post more
  on this when it's ready for review

* Declaring IsServedBy relationships and collective services as per
  Discovering Data Collections
  <https://ivoa.net/documents/discovercollections> is another thing
  contributing to making/keeping all-VO dataset discovery viable.  If
  you run more than one SIA or SSAP service and are unsure what this is
  about, please have at least a fleeting look at that EN and feel free
  to ask back if anything looks spooky to you.

Any and all feedback (and, of course, help) is most welcome...

        -- Markus



More information about the registry mailing list