VOEventRegExt

[Markus Demleitner]

Dear VOEvent activists,

On Fri, May 16, 2014 at 11:19:26AM -0700, Matthew Graham wrote:
> A Working Draft of VOEventRegExt has now been posted to the Document Repository: http://www.ivoa.net/documents/VOEventRegExt/20140513/index.html

Here's a first assessment from Registry of your draft of VOEventRegExt.
Much of this is a request to re-use and adapt things we've built within
Registry in the recent years.  This is not (only) about consistency for
consistency's sake, this is more about making it as simple as possible
to allow registries and its users to use existing infrastructure to
fulfill your use cases.  Clearly, this is written from a Registry point
of view -- it's perfectly possible that we don't see problems with our
proposals, so please take all this with as many grains of salt as
necessary.

Talking about use cases: I think it'd be useful to sharpen the use
cases in 1.1 a bit into things the registry actually can do ("Creating a
stream" certainly is not a registry activity).  We'd be delighted to
assist you in coming up with actual recipies (e.g., RegTAP queries) once
the model is there.  This will also let us assess impact on our
infrastructures.

Full disclosure: I've not looked at the underlying VOEvent spec in any
depth.  Please bear with me if that's the reason I couldn't appreciate
some design decisions.


(1) Referring to existing Registry material

This is about the text starting with "The identifier syntax is set up in
a precise way to give control to both supplier and registry
maintainer:", then later on sections 3.1.1, 3.1.2 and some further text
in section 4: These things basically describe and summarise items from
Identifiers, VOResource and VODataservice.

Now, I can see why you might want some sort of self-contained guideline
to creating VOEvent resource record, but frankly, I think authors of
registry records will need to consult other material anyway (or have
some simplified non-normative HOWTO-type document).  

By repeating or condensing material from our other standards, you're
potentially constraining our possibilities for evolution, you increase
the risk for contradictions within the various standards, and you might
well confuse the readers ("what's the relation between this instrument
and the one in VODataService").

So, I'd recommend essentially just referencing the identifiers REC, and
then, instead of 3.1.1 and 3.1.2. say something like (supposing you like
some of our other suggestons):

  As VOEventStream extends the VODataService vs:CatalogService, it has
  all metadata items present there (identifier, curation and content
  information, as well as further information on the provenance of the
  data exposed, coverage in space, time, and spectrum, etc.).  We refer
  the reader to [VOResource] for curation, content, and other basic
  metadata, and to [VODataService] for various advanced metadata.

In a similar vein, I believe "We propose in this document to add three
new kinds of resource (also known as extension schema)" goes a bit too
far in jargon avoidance. "Resource types" shouldn't be less clear (if you
want to be 100% precise, I'd have to be "resource record types").  And
I'd cut the parenthetical remark.  The extension schema is where the
resources are defined *in*, and I don't think the concept of extension
needs to be discussed in this text at all.


(2) Identifiers syntax

This is going to be a bit of a tough one for you, but at the same time
it would be *great* if you could agree to do this in the politically
correct way.

The point is that we need the hash in IVORNs for what the general
fragment identifiers in URIs are for: Referencing things *within* the
thing referenced, in the case of IVORNs the resource record.  Therefore,
we're moving towards using question marks in analogous cases outside of
VOEvent (dataset identifiers).

So, I'd really like to suggest section 2 to be shortened into:

  VOEvents have identifiers associated with them.  Their conventional form
  is

    ivo://authori.ty/service/id?local.event.id

  This is supposed to be resolvable in the registry.  In accordance with
  common IVOA use (reference to upcoming Identifiers REC), clients only
  resolve the identifier up to the question mark in the registry and use
  the record returned to discover a service that allows access to the
  individual VOEvents.  [here, you'd need to say what capability would
  actuall let you do that, and possibly how]

  Historically, the VOEvent community has also been using the hash (#)
  character instead of the question mark.  VOEvent clients are advised
  to accept that as a separator for the forseeable future *when
  resolving VOEvents* and treat it equivalent to the question mark.
  Servers are urged to phase out usage of legacy, hash-separated ids in
  order to free up URI fragments for both referencing items within
  resource records (ivo://authori.ty/service/id#photo could in this way
  refer to the metadata for a special VOEvent Group) or for referencing
  parts of VOEvents (ivo://authori.ty/service/id?my.event#photo could
  reference an actual group within the event).


(3) Re-using vs:tableSet

In 3.1.3 and 3.1.4, you're defining Group and Param.  I appreciate you
want to be as similar to VOEvent itself as possible.  On the other hand,
you're making Registry's lives fairly hard, as this is a relatively
complex structure (which means we'd need special tables to keep it in)
while being farily similar to vs:tableSet -- the tables and columns
defined there are structurally basically the same thing as your groups
and params.

You'd help us a lot -- not having to introduce new tables in RegTAP, not
having to handle new names, not having to introduce additional discovery
options in other interfaces -- if you could just map VOEvent's data
model to vs:tableSet.  This might look weird at first, but it'd really
be win-win: less work for us, immediate discoverability for you.  Oh,
and you'd have a smaller schema to worry about, and the mapping rules
("use the events schema, map Group to table, Param to column, and use a
table with the conventional name _ for Params outside of Groups") would
be shorter than what you currently write.

Incidentally, an alternative mapping would be to use InputParams, where
Group names might just be folded into the parameter names with a dot
(<Group photo><Param magg> -> <inputParam photo.magg>); I don't believe
this is preferable to using tableSet, but then I do not completely
understand your use cases and models.

(4) Less types

I understand that DataStreams are different from DataServices as
the former are non-static.  However, in what discovery scenarios does
that distinction need to be maintained?  Most of the metadata apprears
to be readily mappable.  And what's the motivation for the local type
definitions -- is stopping people from restricting or extending them
actually important for VOEventRegExt?

After that, I'd suggest to drop DataStream and derive VOEventStream
directly from CatalogService.  VOEventStream would probably not
introduce new elements at all, the type would just let people restrict
discovery to event streams (see below); DataDictionary would move to the
tableSet. On accessControl see below.


(5) accessControl -> securityMethod; securityURL in capability

You have several elements connected with access control.  In standard
VOResource, there's interface/securityModel/@standardId for that --
could you make sure your use cases cannot be covered by that?  And
anyway, I'd strongly ask you to move all such information to interface;
this is where people will look for it, and it's also where, e.g., PGP
keys would make sense, as I suspect they'd be bound to an interface
(e.g., a mail address) rather than a capability; think what the identity
information within the PGP key would be if it really were to refer to a
capability rather than in interface.


(6) Completely do away with VOEventServer?

How many streams is a VOEventServer expected to carry?  If it's not
many, discovery queries will be much simpler if you let the capabilities
sit directly on the VOEventStreams and remove VOEventServer.  Without
actual domain knowledge I'd say the break-even line in terms of avoiding
the duplication of capability elements would be if the typical
VOEventServer carried 5 or more streams and the worst one 20 or more.
Looking at the voe:Feed type I'd even suggest that capabilities exposing
interfaces having this should really sit on the individual streams in
the first place.

If you keep the current architecture, I'd urge you to use the standard
VOResource served-by and service-for relations rather than the custom
voeventStream element.  These are really relations, and using VOResource
mechanisms for these increases the chance that common query patterns can
be re-used for VOEvent discovery.  


(7) Add RegTAP support

In RegTAP, typically additional metadata from registry extensions are
managed through the rr.res_detail table.  Defining the keys in there is
up to the extension itself, so once the model has been essentially
hashed out, we should sit together and figure out which keys -- if any
-- you'll need.


(8) Some minor gripes

3.1.3. -- "possible paramteters" -- typo

3.2. "with the kinds of services that might be wanted. These are known
as Capabilities in the VO registry" -- saying that "services are known
as capabilities" is misleading, even more so since Service is an actual
result type.  Couldn't you make this "...including physical access
options and the protocols to use them with.  These are..."?

4.1.3 -- I'm really not be a big fan of accessURL in capability.  That's a
new wrt all other registry extensions, in which access urls are parts of
interfaces, as is access control -- can't that just move there?

4.1.3 -- Is it really a good idea to have a reference URL for a VTP
interface?  I would expect human-readable information to  be pertinent
to all capabilities (and hence in particular interfaces), which is why
VOResource puts the piece of metadata to into the content element of the
resource as a whole.

4.1.3 -- For the signed element I'd propose to check if that couldn't be
somehow integrated info the securityMethod URL.

Cheers,

          Markus