Hot-wiring agenda

[Rob Seaman]

Roy said:

> I was thinking that the VOEvent standard is finished now

The protocol is "finished" (meaning we should continue to expect the  
need for relatively infrequent maintenance updates), but we're just  
getting started with deployment and commissioning.  I wouldn't be  
surprised at our fielding several generations of broker and transport  
prototypes before the distributed system nears maturity.

> All we have to do is make sure they don't change STC under our  
> noses ....

I've completed my first read-through of STC v1.30.  I think that the  
last thing "they" (meaning Arnold) wants to do is change STC :–)

> For the SEAP, the first question is are we ready to standardize a  
> protocol?

Easy.  No.  We're ready to be ready, however.

> Perhaps we should start by asking where are the current VOEvent  
> repositories, why do people want to extract past events, and which  
> events do they want.

Ok, we've started this conversation under a separate thread.

There is a second thread, whether it also falls under the heading of  
"SEAP", or not.  This is the issue of the backbone topology and the  
overall network architecture.  It isn't just the word "packet" that  
makes me think that there are parallels between VOEventNet and  
Internet technologies.  Authors connect directly to publishers.   
Subscribers connect directly to brokers.  How do the packets get from  
the original publisher to the final broker?  Or do subscribers need  
to connect to multiple publishers?  What are the logistics of  
negotiations between brokers?  Do all brokers talk to all other  
brokers?  Do packets need a time-to-live field and other gizmos  
familiar to Cisco?  In short, how are VOEvent packets routed?

> One of the key difficulties here is the <What> section with its  
> free-form parameters. Queries will have predicates like "All events  
> which have a Param called BANANA which have a Float value that is  
> between 2 and 5". If we want to build forms that allow subscription  
> and query, we will need to allow people to insert their own BANANA- 
> type predicates -- because we do not know in advance what Params  
> might be there.

I'm confident that application specific schema will evolve in the  
fullness of time (for some applications, anyway).  Params are an  
option to reduce the cost of VOEvent buy-in to a level that projects  
are willing to pay.  I think they have been completely successful for  
this purpose.

> This has been defined as a Registry extension. Publishers and  
> Repositories of VOEvents should have a valid Registry record.  
> Perhaps we can have a joint Registy/VOEvent session to figure out  
> how to build these schemas.

Ok.  Let's start listing focus/session topics:

SEAP
Broker arbitration
Registry use for VOEvent
Packet authentication

> My take would be to make it work and get people involved in the  
> simplest way first, before generalizing. Subscription to specific  
> Publisher. Later, they can subscribe to an Aggregator if they want  
> that. This provides both of the models you mention.

Yes.  Each publisher may support direct connections indefinitely.   
Assuming we remain as successful as we've been at continuing to build  
the VOEvent community, however, I believe a more scalable  
architecture will be needed soon.

> You're building your own Registry? Why? Surely we can utilize the  
> VO Registry structure? Perhaps we can revive Matthew's Carnivore  
> registry and make sure it is happy with registering VOEvent nodes.

I didn't know his predator was hibernating.  Time for the prey to  
play.  I've appended the diagram from my NVO summer school slides.   
Let's see if I can resurrect my thought process from 1.5 years ago.   
"SEAP" in this slide is some superset of the current transport  
protocol - that is, VOEvent packets being moved from place to place.   
The roles have changed.  Portal here is our current Publisher role,  
while what used to be Publisher is now a Broker (or Aggregator as  
some insist on calling it).

Authors generate content on the left in different formats.  It is  
published and injected into what we now call the backbone.  It  
bounces around the backbone like pachinko, and in some fashion the  
content wends its way to a Subscribing client on the right which  
passes it on to users in application dependent ways.  Just like the  
current backbone, there are three nodes.  The reason for depicting it  
this way is that with three nodes you don't have to worry about the  
topology.  Is it a ring?  Or a star?  We haven't even had to fully  
face fundamental client server/P2P questions yet.  On the other hand,  
we do have a good notion of what publishing means and what  
subscribing means.

So the original breadth of the SEAP notion was something like  
transport (or preset query, if you will) + query (or delayed  
transport, if you won't).  The red arrows, on the other hand, were an  
attempt at depicting whatever service negotiations occur internal to  
the backbone/network of brokers.  OAI is the Open Archives Initiative  
which "promotes interoperability standards that aim to facilitate the  
efficient dissemination of content".  Other people know infinitely  
more about this than I do, but while OAI has been used  
interchangeably with "registry" in the VO, it isn't obvious to this  
naive observer that current registry technology exhausts the notion  
of metadata harvesting that underlies the efficient dissemination of  
content.

In particular, VOEvent packet streams have some pretty interesting  
metadata - not just so-called science metadata, but actual content  
and routing metadata.  I see the meta-conversation between brokers  
that is superimposed on the actual packet transport as an attempt to  
characterize the streaming of events in an adaptive, robust,  
efficient (+ other good adjectives) fashion.  So it would be  
simplistic to call this an internal registry, but I wouldn't be  
surprised to reuse some of that technology.

On the other hand, VOEvent brokers (Publishers, Repositories,  
Filters) should certainly appear in the normal VO registry for all  
the obvious reasons.

Rob
-------------- next part --------------
A non-text attachment was scrubbed...
Name: VOEFlow.pdf
Type: application/applefile
Size: 452 bytes
Desc: not available
URL: <http://www.ivoa.net/pipermail/voevent/attachments/20070312/31d24d80/attachment-0001.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: VOEFlow.pdf
Type: application/pdf
Size: 6140 bytes
Desc: not available
URL: <http://www.ivoa.net/pipermail/voevent/attachments/20070312/31d24d80/attachment-0001.pdf>