[Heig] [EXTERNAL] [BULK] Re: vocabulary update: proposal for dataproduct_type update for high energy data : event-list definition and event-bundle
BONNAREL FRANCOIS gmail
francois.bonnarel at gmail.com
Wed May 7 00:20:34 CEST 2025
Dear all,
I think I agree with Markus that we should avoid multiple #this in
general.
But with one single exception : different formats for the same product
Exemples : DataLink allows to have one single line in ObsCore for an
image which we can deliver in FITS, JPEG, PNG etc...
DataLink allows to one single line for a spectrum
in ObscOre which we can deliver in VOtable, FITS, CSV, etc...
In those cases we will have in DataLInk table record 1 : ID =
ivo://aaa/bbb
semantics : #this
content-type : application/fits
content-qualifier : image
description " XXX survey image Target so and so in FITS format"
record 2 : ID = ivo://aaa/bbb (same as above)
semantics : #this
content-type : image/jpeg
content-qualifier : image
description " XXX survey image Target so and so in Jpeg format"
....
The examples given by Laurent show that a data producer can choose
different strategies according to the mode of usage it expects from users.
strategy 1 : the bundle is exposed in ObsCOre an then in
DataLink #this in semantics row can only be the whole bundle
strategy 2 : the spectrum (or event-list, or whatever science
data) is exposed in ObsCore and then #this is this very spectrum, or
event-list, etc... other material you want to link to the spectrum must
have different semantics (#calibration, #auxiliary ...)
Answer to your last question below
Le 05/05/2025 à 22:45, Jaffe, Tess (GSFC-6601) via heig a écrit :
>
> I can add what our use case was.
>
> There was a local team building a web tool for x-ray spectra viewing
> and quick line identification, so I asked them to use our SSA for
> it. We then added a datalink service descriptor to our SSA results and
> served the response matrices in the datalink table result. (The SSA
> result at the time had the link to the FITS file.) It worked
> internally with their tool for a while but the tool didn't make it to
> production. So that was our use case.
>
> Now we are working more on our ObsTAP and its datalinks. But
> precisely the question of which products get their own row in our
> ObsCore table is what we're working on now. Presumably, whatever we
> decide, this hypothetical tool could work with as long as the service
> is performant. But from the client point of view, being given a
> tarball is more annoying to code for. I would prefer listing the
> spectrum product alone in ObsCore and give each ancillary file its own
> row in the DataLink result table. I.e., list a spectrum in ObsCore
> and return additional rows in the DataLink result for each of the
> files that would constitute the bundle. What would the bundle as a
> tarball facilitate?
We discussed examples in the context of CTA and XMM. I think
"event-bundle" records in ObsCore may be completed by DataLink records
#this with content-type=application/OGIP or
content-type=application/GADF (or later VODF)
In that case a dedicated software can directly process these "standard"
formats of "bundles".
Of course another valid strategy is the one you suggest : product type
in ObsCore : event-list , spectrum and then several records with
#calibration for the different IRF or ARF
An extended vocabulary for IRF/ARF types (TBD) may be used in the
content-qualifier FIELD in each of those rows for better identification.
Probably more adapted for ad hoc processing.
Cheers
François
>
>
>
> ------------------------------------------------------------------------
> *From:* heig <heig-bounces at ivoa.net> on behalf of Mireille Louys via
> heig <heig at ivoa.net>
> *Sent:* Monday, May 5, 2025 1:50 PM
> *To:* heig at ivoa.net <heig at ivoa.net>
> *Subject:* Re: [Heig] [EXTERNAL] [BULK] Re: vocabulary update:
> proposal for dataproduct_type update for high energy data : event-list
> definition and event-bundle
> CAUTION: This email originated from outside of NASA. Please take care
> when clicking links or opening attachments. Use the "Report Message"
> button to report suspicious messages to the NASA SOC.
>
>
>
>
> Hello,
>
> Thanks Tess and Laurent for these examples .
>
> This was a proposal during the HE workshop last week to bring some
> examples
>
> showing an Obscore data discovery ( obscore entry here ) and exploring
> various settings of data link scenarios .
>
> Any volunteer for examples from other archives ?
>
> Best , Mireille
>
>
> Le 05/05/2025 à 18:14, Laurent Michel via heig a écrit :
> > Hello,
> >
> >
> >
> > Le 05/05/2025 à 08:56, Markus Demleitner via semantics a écrit :
> >> Hi Tess,
> >>
> >> On Wed, Apr 30, 2025 at 01:48:29PM +0000, Jaffe, Tess (GSFC-6601) via
> >> semantics wrote:
> >>> Possibly dumb question:
> >>
> >> Not at all; you're touching a topic that has been discussed quite a
> >> few times now without a satisfying result yet: What does it mean if
> >> there are multiple items with the same semantics in Datalink? Is it
> >> "all of them together give the thing" or is it "they are
> >> alternatives"? Or yet something else?
> >>
> >> Previous rounds suggested that the interpretation will probably
> >> depend on the concept, but the details turned out to be fairly messy.
> >>
> >>
> >>> If an ObsCore table lists an event-bundle as a separate row with
> >>> its own product_type, and the access_url follows best practice
> >>> specifying a datalink that will return the bundle, what should the
> >>> DataLink result include as #this? We are actively putting this
> >>> together now at HEASARC. If the product type is simply a spectrum,
> >>
> >> That's excellent news!
> >>
> >>> our datalink result has the spectrum file as #this and the response
> >>> matrices, background, etc. as related products in the same result
> >>> table. If the product itself is a bundle, what is the #this? Do
> >>> we have to provide a tarball or something? Or are there multiple
> >>> #this with different dataproduct_subtypes? The latter doesn't
> >>> sound right to me.
> >>
> >> Given my preamble, I'd avoid multi-#this. The ideal solution would
> >> IMHO be a standard archive if the HEIG can commit to such a thing.
> >> Failing that, I think a tar archive of the individual components
> >> would be the second best thing. CADC does something like this,
> >> although the other way round: They're handing out everything tarred
> >> together as a #package. Offering the components individually,
> >> possibly as #progenitor-s, would help cases when people really only
> >> want to fetch a single part.
> >
> > I agree that having multiple #this is confusing (which one is the good
> > one.??).
> > In my understanding #this must match the product_type as in the
> > Obscore record.
> >
> > If a spectrum bundle is exposed in a separate row, we should have
> > something like this:
> >
> > Obscore row:
> > -----------
> > - product_type=spectrum-bundle (tbd)
> > - access_format=application/x-votable+xml;content=datalink
> >
> > Datalink response:
> > -----------------
> > - link #1
> > - semantics=#this
> > - content_qualifier=spectrum-bundle (TBD)
> > - content_type=application/tar+gzip
> > - description="spectrum file + preview + ARF + RMF + Background
> > spectrum"
> >
> >
> > If the spectrum is exposed in a separate row:
> >
> > Obscore row:
> > -----------
> > - product_type=spectrum
> > - access_format=application/x-votable+xml;content=datalink
> >
> > Datalink response:
> > -----------------
> > - link #1
> > - semantics=#this
> > - content_qualifier=spectrum
> > - content_type=application/fits
> > - description="spectrum file"
> > - link #2
> > - semantics=#package
> > - content_qualifier=spectrum-bundle (TBD)
> > - content_type=application/tar+gzip
> > - description="spectrum file + preview + ARF + RMF + Background
> > spectrum"
> >
> > I do not believe we are able to design a standard HEIG archive because
> > this too much mission/tool specific.
> > Do we really needs the archive content to be machine readable?
> > Anyway, individual files can be exposed with an adapted semantics.
> >
> > Laurent
> >
> >
> >>
> >> But at least for a prototype (and, if that works fine, perhaps also
> >> as a long-term practice), I think nobody would be terribly confused
> >> if #this were just the time series or spectrum, in particular if a
> >> content-qualifier would let machines figure out what it is they'll
> >> get.
> >>
> >> I'm not too happy with #progenitor for the individual components,
> >> though. Perhaps datalink/core should have a concept #component with
> >> the definition "for datasets where #this is composed of multiple
> >> individual artefacts, #component rows offer access to individual
> >> artefacts. Use local-semantics to consistently mark up the roles of
> >> the components." or so.
> >>
> >> In the end, I think we need to see what will help clients consuming
> >> this. Do we have software that we could use to try that out? What
> >> do people use to work this #event-bundle-s?
> >>
> >> -- Markus
> >>
> >
> > --
> > English version: https:
> //https://gcc02.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.deepl.com%2Ftranslator&data=05%7C02%7Ctess.jaffe%40nasa.gov%7C6075ae45df2e4bb5325508dd8bfd5006%7C7005d45845be48ae8140d43da96dd17b%7C0%7C0%7C638820642266465078%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=vYY2ZoY99F8RvAPlu9Ske3PYa0z8E2Ne12cW%2BdF8Kl4%3D&reserved=0
>
> --
> --
> Mireille Louys, MCF (Assistant Professor)
> Centre de données Astronomiques (CDS) Equipe Images, ICube
> Observatoire de Strasbourg Telecom Physique Strasbourg
> 11, rue de l' Université 300, Bd Sebastien Brandt
> CS 10413
> F-67000 Strasbourg F-67412 Illkirch Cedex
>
> --
> heig mailing list
> heig at ivoa.net
> https://gcc02.safelinks.protection.outlook.com/?url=http%3A%2F%2Fmail.ivoa.net%2Fmailman%2Flistinfo%2Fheig&data=05%7C02%7Ctess.jaffe%40nasa.gov%7C6075ae45df2e4bb5325508dd8bfd5006%7C7005d45845be48ae8140d43da96dd17b%7C0%7C0%7C638820642266487767%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=YfJb4jKT78AkfN4F7QVYmofeRGcMreiSO0uI9ObBHWo%3D&reserved=0
> <http://mail.ivoa.net/mailman/listinfo/heig>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.ivoa.net/pipermail/heig/attachments/20250507/d9e880c4/attachment-0001.htm>
More information about the heig
mailing list