VOEvent II summary, part 3

[Rob Seaman]

Hola,

	Sorry for the delay getting to the third installment of notes from  
the VOEvent II workshop - covering the mini-interop session on  
Wednesday.  I'm sure the doughty attendees, Alasdair, Matthew, Phil,  
Ramon, Robert and Scott will correct and extend my comments.  As you  
can see, we had representatives from several projects that are  
implementing VOEvent infrastructure.

	The goal was to ensure that everybody is pulling in the same  
direction.  I think we were entirely successful in this - for the  
near term.  We reached a broad consensus on a number of issues  
relating to bootstrapping the emerging VOEvent network.  That each  
project is also exploring complementary regions of technology space  
for their own internal use is more a feature of the process, than a  
complication.  We simply don't know yet what the long term technology  
suite is likely to be.

	We do, however, have a much clearer vision of the major components  
that will compose a fully functioning VOEvent system:

	1) Author - any entity that creates content suitable for later  
representation as a sky transient alert.  Strictly speaking, the  
author resides outside the network of VOEvent entities.

	2) Publisher - an entity that assigns a unique identifier to a  
conforming VOEvent packet.  Either the author or the publisher may  
format the content into the original VOEvent packet.  Ensuring that  
output packets are conforming XML documents and validating these  
against the schema would be desirable, but there is really no way to  
enforce this.  Validation is ultimately the responsibility of  
subsequent components.  A publisher is a source of VOEvent packets.

	3) Subscriber - any entity that receives VOEvent packets for  
whatever purpose.  A subscriber is a sink of VOEvent packets.

	4) Relay - a component that subscribes to one or more input packet  
streams and emits one or more output streams.  A relay may filter a  
stream based on the contents of the packets, or in arbitrarily  
advanced ways such as coincidence testing two streams against each  
other.  Implicit in the concept of the VOEvent subscriber model is  
the notion of programmable packet filtering.  Relays will accomplish  
this.  Filtering may be rules based.  Filtering may be richly  
adaptive.  Initial relay prototypes will likely be programmed via web  
interfaces (e.g., with checkboxes).

	5) Repository - a component that persists packets either permanently  
or temporarily.  A repository must subscribe to one or more input  
VOEvent streams.  How repositories are queried and retrievals are  
scheduled is not specified, but this might occur either within the  
VOEvent network or external to the network.  In the former case, a  
repository might be considered a relay implementing a filter with  
arbitrary latency, and with rules applied retroactively.

	6) Broker - any heterogeneous combination of two or more of the  
three atomic components:  publisher, repository or relay.   
(Subscription is mixed in as needed.)  We spent a lot of time trying  
out different words such as aggregator or nexus and assigning  
different names to different combinations of the atoms.  Ultimately,  
it seemed more expressive to use a general term to cover all cases.   
A broker, of course, may offer arbitrary applications-level  
functionality in addition to the VOEvent packet handling.

	A typical VOEvent authority will operate a number of such  
components, each of which may be visible through the registries.  It  
will be easiest to refer to the NOAO or TALON VOEvent "broker", for  
instance, rather than tossing around some loosely defined meaning of  
publisher for such instances.  A publisher publishes new packets.   
Period.

	Note that nothing has yet been said about the underlying transport  
protocol(s).  This is likely to remain a free parameter.   
Interoperability requires two parties to agree on a protocol, of  
course, and it is anticipated that a small number of options will  
garner the lion's share of the VOEvent "market".  To bootstrap the  
system, we settled on two transport protocols, one pull and one push:

Pull:
	RSS 2 with enclosures.  We'll undoubtedly have more to say about  
this choice after we gain more experience.

Push:
	A raw socket connection via a specified port.  Negotiation for a  
port address was deemed too machine dependent - for instance, through  
a firewall.  After establishing the connection, each packet will be  
emitted by the publisher, preceded by a 4 byte network ordered signed  
int count of the size of the packet.  An action item is to decide  
between ASCII and UTF-8 encoding.  Once established, the socket  
remains connected indefinitely.  Responsibility for timeouts related  
to dropped communications links are reserved to each end of each  
channel.

	We are under no illusion that this protocol is sufficient for all  
purposes, but it is simple enough to require minimal effort for any  
project to implement, which will aid the goal of growing the network  
until it reaches critical mass.  Since we're about as far from the  
holy grail of guaranteed delivery of messages as you can get, some  
additional features are desirable to add robustness to the packet  
streams.

1) An ack message:
	Upon successful receipt of a packet, the receiving component will  
return an ack packet to the sender.  At its simplest, this will be an  
empty VOEvent element:

		<VOEvent id=<same as original packet> role="ack" version=... />

An ack packet might also contain normal VOEvent content, but this is  
discouraged until we come up with a good reason for doing so.

2) An "I am alive" packet:
	On an arbitrary schedule(s), each broker emits role="iamalive"  
packets to other brokers (meaning other VOEvent participants, perhaps  
including individual subscribers).  Subsequent discussions have  
clarified the format of these packets to include a timestamp and  
explicit publisher ID:

	<?xml version = '1.0' encoding='UTF-8'?>
	<VOEvent role = "iamalive" id = "ivo://talons.lanl/001" version =  
"1.1">
	    <Who>
	        <PublisherID>ivo://talons.lanl</PublisherID>
	        <Date>2005-12-11T01:10:03</Date>
	    </Who>
	</VOEvent>

Upon receipt, the subscribing entities will respond with the same  
packet, but with an updated <PublisherID>.  The timestamp will allow  
calculation of roundtrip network latencies.

Note that the ack and iamalive functionality partially overlaps.   
Possibly we could simplify to a single packet format satisfying both  
roles.

We are continuing to discuss possibilities for ensuring multihop, end- 
to-end route validation.  This might include a time-to-live field  
similar to IP packet routing.  These mechanisms will likely continued  
to be layered upon any future messaging protocols supported by  
VOEvent, at least optionally.  Overtly these apply only to push  
protocols, of course, but the goal of ensuring a reliable packet  
distribution mechanism is critical whether push or pull is used.

Alasdair has a photo of the whiteboard that we scribbled on:

	http://www.flickr.com/photos/aallan/71222080/in/set-1513473

And I've appended a UML version of the same diagram that Phil  
contributed.  Note the presence of separate names for "broker",  
"nexus" and "aggregator".  All of these combination would be called  
"brokers" under the current terminology.

Further results of the Pearl Harbor day interop will likely come out  
as we discuss various issues.

Rob
-----


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.ivoa.net/pipermail/voevent/attachments/20051218/a7ccafc6/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: unknown.png
Type: image/png
Size: 5639 bytes
Desc: not available
URL: <http://www.ivoa.net/pipermail/voevent/attachments/20051218/a7ccafc6/attachment-0001.png>