Global dataset discovery
Markus Demleitner
msdemlei at ari.uni-heidelberg.de
Mon Feb 26 11:31:35 CET 2024
Dear Apps, dear Registry,
If you've been in Tucson, you have perhaps heard my talk on global
dataset discovery ("Give me all images of this part of the sky that
are published in the VO";
<https://wiki.ivoa.net/internal/IVOA/InterOpNov2023Apps/twoup.pdf>)
with pyVO. Well, I have finally written code that is actually doing
something pretty much like this; as promised in Tucson, the exercise
concentrates on images first, because that's probably the hardest
problem we have so far.
This is a long-ish story, a part of which I have written up in this blog
post: <https://blog.g-vo.org/global-dataset-discovery-in-pyvo.html>,
which I'd invite you to read.
As a minimal TL;DR let me state:
* PyVO PR #470 is now worth a look; perhaps we should even consider
tentatively merging it: <https://github.com/astropy/pyvo/pull/470>.
* The problems of using dataModel to register ObsTAP services hit
this use case face-on. This needs to be fixed before global discovery
including obscore will not suck. I have hence started a Note to
finally untangle the question of how to register TAP-published
tables (which also concerns RegTAP, ObsLocTAP, EPN-TAP, LineTAP,
and presumably more in the future, but they are generally in better
shape already): <https://github.com/ivoa/TableReg>. I'll post more
on this when it's ready for review
* Declaring IsServedBy relationships and collective services as per
Discovering Data Collections
<https://ivoa.net/documents/discovercollections> is another thing
contributing to making/keeping all-VO dataset discovery viable. If
you run more than one SIA or SSAP service and are unsure what this is
about, please have at least a fleeting look at that EN and feel free
to ask back if anything looks spooky to you.
Any and all feedback (and, of course, help) is most welcome...
-- Markus
More information about the registry
mailing list