[Heig] [EXTERNAL] [BULK] Re: vocabulary update: proposal for dataproduct_type update for high energy data : event-list definition and event-bundle

Mon May 5 19:58:01 CEST 2025

Thanks for these responses, that's exactly what I needed to know.  I just didn't think a tar file was a particularly popular option in the IVOA, but it should be straightforward for us.  But aurent's spec makes sense to me, and we already have tools that tar things up.

(I'm not on the semantics list, just the HEIG, and already overloaded.)

________________________________
From: heig <heig-bounces at ivoa.net> on behalf of Laurent Michel via heig <heig at ivoa.net>
Sent: Monday, May 5, 2025 12:14 PM
To: heig at ivoa.net <heig at ivoa.net>
Subject: Re: [Heig] [EXTERNAL] [BULK] Re: vocabulary update: proposal for dataproduct_type update for high energy data : event-list definition and event-bundle

CAUTION: This email originated from outside of NASA.  Please take care when clicking links or opening attachments.  Use the "Report Message" button to report suspicious messages to the NASA SOC.

Hello,

Le 05/05/2025 à 08:56, Markus Demleitner via semantics a écrit :
> Hi Tess,
>
> On Wed, Apr 30, 2025 at 01:48:29PM +0000, Jaffe, Tess (GSFC-6601) via semantics wrote:
>> Possibly dumb question:
>
> Not at all; you're touching a topic that has been discussed quite a
> few times now without a satisfying result yet: What does it mean if
> there are multiple items with the same semantics in Datalink?  Is it
> "all of them together give the thing" or is it "they are
> alternatives"?  Or yet something else?
>
> Previous rounds suggested that the interpretation will probably
> depend on the concept, but the details turned out to be fairly messy.
>
>
>> If an ObsCore table lists an event-bundle as a separate row with
>> its own product_type, and the access_url follows best practice
>> specifying a datalink that will return the bundle, what should the
>> DataLink result include as #this?  We are actively putting this
>> together now at HEASARC.  If the product type is simply a spectrum,
>
> That's excellent news!
>
>> our datalink result has the spectrum file as #this and the response
>> matrices, background, etc. as related products in the same result
>> table.  If the product itself is a bundle, what is the #this?  Do
>> we have to provide a tarball or something?  Or are there multiple
>> #this with different dataproduct_subtypes?  The latter doesn't
>> sound right to me.
>
> Given my preamble, I'd avoid multi-#this.  The ideal solution would
> IMHO be a standard archive if the HEIG can commit to such a thing.
> Failing that, I think a tar archive of the individual components
> would be the second best thing.  CADC does something like this,
> although the other way round: They're handing out everything tarred
> together as a #package.  Offering the components individually,
> possibly as #progenitor-s, would help cases when people really only
> want to fetch a single part.

I agree that having multiple #this is confusing (which one is the good one.??).
In my understanding #this must match the product_type as in the Obscore record.

If a spectrum bundle is exposed in a separate row, we should have something like this:

Obscore row:
-----------
- product_type=spectrum-bundle (tbd)
- access_format=application/x-votable+xml;content=datalink

Datalink response:
-----------------
- link #1
   - semantics=#this
   - content_qualifier=spectrum-bundle (TBD)
   - content_type=application/tar+gzip
   - description="spectrum file + preview + ARF + RMF + Background spectrum"

If the spectrum is exposed in a separate row:

Obscore row:
-----------
- product_type=spectrum
- access_format=application/x-votable+xml;content=datalink

Datalink response:
-----------------
- link #1
   - semantics=#this
   - content_qualifier=spectrum
   - content_type=application/fits
   - description="spectrum file"
- link #2
   - semantics=#package
   - content_qualifier=spectrum-bundle (TBD)
   - content_type=application/tar+gzip
   - description="spectrum file + preview + ARF + RMF + Background spectrum"

I do not believe we are able to design a standard HEIG archive because this too much mission/tool specific.
Do we really needs the archive content to be machine readable?
Anyway, individual files can be exposed with an adapted semantics.

Laurent

>
> But at least for a prototype (and, if that works fine, perhaps also
> as a long-term practice), I think nobody would be terribly confused
> if #this were just the time series or spectrum, in particular if a
> content-qualifier would let machines figure out what it is they'll
> get.
>
> I'm not too happy with #progenitor for the individual components,
> though.  Perhaps datalink/core should have a concept #component with
> the definition "for datasets where #this is composed of multiple
> individual artefacts, #component rows offer access to individual
> artefacts.  Use local-semantics to consistently mark up the roles of
> the components." or so.
>
> In the end, I think we need to see what will help clients consuming
> this.  Do we have software that we could use to try that out?  What
> do people use to work this #event-bundle-s?
>
>           -- Markus
>

--
English version: https: //https://gcc02.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.deepl.com%2Ftranslator&data=05%7C02%7Ctess.jaffe%40nasa.gov%7Cc7e5b3d1896d4e38f87a08dd8befed7c%7C7005d45845be48ae8140d43da96dd17b%7C0%7C0%7C638820585097929386%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C80000%7C%7C%7C&sdata=zZY4gqz0yDYMgsj0c7x31Tq7Xyp%2B0a0WFBN18ApCiv4%3D&reserved=0
--
jesuischarlie/Tunis/Paris/Bruxelles/Berlin

Laurent Michel
SSC XMM-Newton
Tél : +33 (0)3 68 85 24 37
Fax : +33 (0)3 )3 68 85 24 32
Université de Strasbourg <https://gcc02.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.unistra.fr%2F&data=05%7C02%7Ctess.jaffe%40nasa.gov%7Cc7e5b3d1896d4e38f87a08dd8befed7c%7C7005d45845be48ae8140d43da96dd17b%7C0%7C0%7C638820585097956389%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C80000%7C%7C%7C&sdata=2yLdx6UB4ns9wGn8TndfII43x1tervCgc7Lavqm3xZI%3D&reserved=0<http://www.unistra.fr/>>
Observatoire Astronomique
11 Rue de l'Université
F - 67200 Strasbourg
--
heig mailing list
heig at ivoa.net
https://gcc02.safelinks.protection.outlook.com/?url=http%3A%2F%2Fmail.ivoa.net%2Fmailman%2Flistinfo%2Fheig&data=05%7C02%7Ctess.jaffe%40nasa.gov%7Cc7e5b3d1896d4e38f87a08dd8befed7c%7C7005d45845be48ae8140d43da96dd17b%7C0%7C0%7C638820585097973738%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C80000%7C%7C%7C&sdata=S%2B5RkXR%2F%2FZiLSY9MsetILe%2FJlZB2qYn%2FerCIzkL5dYg%3D&reserved=0<http://mail.ivoa.net/mailman/listinfo/heig>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.ivoa.net/pipermail/heig/attachments/20250505/d8f7aa01/attachment-0001.htm>