Obscore 1.1 Erratum 3

[BONNAREL FRANCOIS]

Dear dm and TCG,

     First of all I have to apologize because I made some mix up in my 
agenda and thought the TCG meeting was this evening and not yesterday

     I was going to send an email about the Obscore 1.1 Erratum 3 today 
in prevision of that TCG meeting.

     That's lost 😭 !!!

      To summarize my point : this discussion cannot be solved in an 
erratum.

*This is not an erratum this is the change in the data model.*

Le 08/12/2022 à 10:56, Markus Demleitner a écrit :
> Dear DM, dear TCG,
>
> at yesterday's TCG we remembered that Obscore 1.1 Erratum 3,
> <https://wiki.ivoa.net/twiki/bin/view/IVOA/ObsCore-1_1-Erratum-3>, is
> still open.  For context: This is about no longer requiring obs_id to
> be non-NULL.  The benefit would be to lift a very noticeable load on
> validators and implementations to ensure this, while no case has been
> identified where this non-NULL requirement actually is necessary to
> write obscore queries or properly interpret them.
>
> There has been a bit of discussion on this in April,
> <http://mail.ivoa.net/pipermail/dm/2022-April/006233.html>  and then
> again in July,
> <http://mail.ivoa.net/pipermail/dm/2022-July/006251.html>.
>
> The July discussion, as far as I can reconstruct it, ended with
> François remarking (I'm not quite sure why I let it peter out back
> then):
>
> On Fri Jul 8 09:45:46 CEST 2022, BONNAREL FRANCOIS wrote:
>>> in all services?  The worst that would happen with this query if we
>>> don't is that all the "single" observations end up in one aggregate
>>> with obs_id NULL, no?
>> Probably yes.
>>>     If you see a major problem in that, could you
>>> elaborate?
>> This is breaking all observation related queries into two very different
>> ones. I find this ugly, and increasing complexity for the users.
> Do you still remember why you felt this way?  I cannot see how NULL
> obs_ids would make a difference to consumers in any way.  Can you
> perhaps discuss a query that you see as adversely affected by the
> change?

Select list of ob_publisher_did related to the same obs_id. The interest 
of this one is so obvious that I cannot understand why you deny it.

observations (or simulations) are not the same concept as datasets and a 
dataset always come either observation or simulation or experiment.

So relaxing the obs_id and allowing to be absent is a change in the model

.We have to decide if we want to move from an Observation dataset Core 
datamodel to a pure dataset datamodel with sometimes (by chance) a 
relationship to an observation id

>
>> By the way , ESO Obstap service has obs_id for all their datasets. For
>> images, Each observation has the image and the mesurements datasets.
>>
>> CADC also has obs_id everywhere. Apparently all of theme has a 1 to 1
>> remaitionship observation/dataset. They use the free syntax obs_id
>> string  to build the ivi identifier obs_publisher_did string
> Of course they have non-NULL everywhere -- it's required by the
> current standard; the same is true for the obscore service(s) I
> operate.
>
> But the background of the erratum is not that having obs_id non-NULL
> is hard to do, it is that it is hard to validate.  Which of course
> would be perfectly ok if the requirement served a purpose, but since
> we still have not identified such a purpose, it would make
> implementors' lives substantially easier at no cost (well: that's
> admittedly my claim) if we dropped it.

Technically as far as I understood the problem occurs only with ObscOre 
tables implemented as "views", because otherwise you can always create 
an index on this obs_id column.

Apparently "material views" can have indexes. So why not use them when 
available in your dbms ?

And in any case you can always replace you view by a procedure updating 
regularly the ObsCore table from the underneath tables.

Changing the data model for validation reasons before looking to other 
technical solutions seems to me a wrong way to solve these issues.

Cheers

François


>
> So...  Are people still concerned about the change?
>
> Thanks,
>
>             Markus

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.ivoa.net/pipermail/dm/attachments/20221208/5214744b/attachment.htm>