Obscore: obs_id not null requirement

[Patrick Dowler]

ObsCore can have multiple result rows from the same "observation", eg with
different calib_level, and iirc the purpose of obs_id is that that set of
rows would have the same obs_id and thus users would know they are
different "products" of the same observation (aka same photons messengers).

In practice, obs_id could be null if that were not the case, eg if every
record was a different observation. I expect it could be OK for some
records to have null and others have a value that occurred 1+ times in the
table even within a single OsbCore as obs_id is a specific kind of "part of
this group of records" and null(s) would mean "not part of a group".

Having said that, this has a clear meaning to me because our ObsCore impl
is a view on CAOM and CAOM is normalised wrt. this obs_id concept (where
ObsCore is denormalised), but I expect it is not so simple to specify use
of nulls here and clearly deal with other variations of underlying models.

--
Patrick Dowler
Canadian Astronomy Data Centre
Victoria, BC, Canada


On Mon, 7 Mar 2022 at 09:56, Markus Demleitner <
msdemlei at ari.uni-heidelberg.de> wrote:

> Dear DAL, dear Ops,
>
> [Ops: I'm cc-ing you because I'm linking to a praise of validation,
> while the thread leading up to this is perhaps not very relevant to
> you; discussion on obs_id should probably happen on DAL exclusively]
>
> On Fri, Mar 04, 2022 at 11:43:29AM +0000, Mark Taylor wrote:
> > I'm not generally in favour of MUST rather than SHOULD constraints
> > for non-NULL column values, on the grounds that data producers
> > who don't have sensible values to put in there may end up giving
> > useless or nonsense values just to comply with the standards,
> > which doesn't provide value to anybody.
> >
> > However, it may be that the best answer to Markus's problem is
> > for me to drop that taplint query, which could be reasonable.
> > The intention is that taplint doesn't place unreasonable demands
> > on the services that it validates, and I'm happy to take advice
> > in cases where that's not happening.
>
> When I tried to compose an answer to this this morning, I quickly
> realised it was turning into a philosophical piece.  Which I then
> turned into a blog post this afternoon:
> <https://blog.g-vo.org/requirements-and-validators.html>
>
> The TL;DR would be: Thanks, but no thanks.
>
> Let's figure out whether we really have to require obs_id to be
> non-NULL.  If we find a reason for that, I'll bite the bullet and
> make the taplint query fast.  If we don't, let's drop the requirement
> (I volunteer for writing an erratum, which should be quick and easy
> after we've done the thinking here).
>
> Thanks,
>
>              Markus
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.ivoa.net/pipermail/dal/attachments/20220307/e5ee4327/attachment.html>