Apps Messaging: messaging protocol

[John Taylor]

Hi all,
Following on from Mike's message semantics proposal, I thought I'd throw out
a strawman messaging protocol.  Although we are intending to keep messaging
semantics and the protocol separate, they obviously will affect each other
and so it makes sense to discuss them at the same time even if it does mean
keeping track of two threads.

Following Doug's email a while back about keeping the messaging protocol
distinct from whatever wire protocol(s) we choose, I've just described it in
terms of some general operations.  The PLASTIC people will note the
similarities with the PLASTIC protocol, though I've taken this opportunity
to remedy some of its deficiencies.  I begin by listing the operations, and
then I'll go through them in detail.

1) id= register*()
2) declareAppMetaData(id, metadata[])
2b) metadata[] = getAppMetaData(id)
3) declareMessages(id, mtypes[])
4) unregister(id)

5) msgId = sendAsynch(senderId, recipientId, mtype, args[])
6) msgId = broadcastAsynch(senderId, mtype, args[])
7) msgId, response = send(senderId, recipientId, mtype, args[])
8) msgId, responses[] = broadcast(senderId, mtype, args[])

9) hubId = getHubId()
10) ids[] = getApplicationIds()
11) ids[] = getApplicationsUnderstanding(mtype)
12) mtypes[] = getUnderstoodMessagesForApplication(id)

In the above, I've used [] to denote multiple values, rather than
necessarily an array.  I hope my informal notation is clear enough at this
early stage.
===================================================
1) id= register*()
2) declareAppMetaData(id, metadata)
2b) metadata[] = getAppMetaData(id)
3) declareMessages(id, mtypes[])

Once an application has located a hub (by means as yet undecided), its first
step is to register with it - this needs only be done once per session per
application.  The hub provides a session-unique id that is used in
subsequent communications.  This differs slightly from Mike's model where
the application's human-readable-name is used for addressing.  Early on we
canvassed opinion amongst our users as to whether they'd ever want to run
two instances of the same program simultaneously, they said yes, so I prefer
the unique-id mechanism, even though it's less user-friendly.  Note that
asterisk in the register operation.  This denotes that there could be
several different register operations, one for each "wire protocol".  An
application would choose the register operation appropriate to the way it
wanted to communicate.

Operation 2) is for the application to tell the hub about itself: a
human-readable name, and maybe an icon, description etc....   I see this as
being more contentious: we could instead simply use messages between
applications to achieve the same thing.  The advantage of having it as an
operation in the protocol is for those environments that can send messages
but can't easily receive them.  For instance, I could send a message from
the python commandline to Topcat, and all Topcat would have to identify me
would be the hub-generated unique id, which isn't very friendly.  If I'd
also declared some metadata to the hub, then Topcat could interrogate the
hub to find out just who senderId=vom://1234567-891 is. (with operation 2b)

Operation 3) is for the application to tell the hub which message types it
wishes to receive.  The hub won't send it messages it can't understand, and
applications can interrogate the hub (using operation 12) to determine the
appropriate messages to send.  We might need a mechanism to say "send me
messages of any type" [if we do, I hope we make a better job of it than we
did in PLASTIC...which was a mess.]

In PLASTIC, these 3 operations are rolled into 1, but I feel it's better to
separate them.

4) unregister(id)
When an application closes, it should inform the hub.

5) msgId = sendAsynch(senderId, recipientId, mtype, args[])
6) msgId = broadcastAsynch(senderId, mtype, args[])

OK, this is where the fun happens.  5) sends a message from sender to
recipient using the hub-supplied ids for addressing.  6) sends the message
to everyone (if they've elected to receive that mtype).  The hub returns a
session-unique msgId that can be used in further comms, such as to tie reply
messages up with request messages (in Mike's parlance).  In these operations
the message argument list args[] is intended to be quite general.
I'm explicitly writing out the parameters here...if you prefer you can write
it:
5) msgId = sendAsynch(message)
where message wraps the whole lot.  At some point we need to decide which
parameters are part of the message, and which are not.  My initial feeling
is that mtype and args[] are part of the message, while the addressing info
are not.  [We could also add the msgId (if one already exists for that
"conversation") as one of the input params, or we could put it inside args[]
- I'm not sure which is best.]

Plastic devs will note another difference with Plastic - the above
operations reflect the two most common modes of use that we've encountered
in 18 months of experience: one-to-one, and one-to-all.  Plastic has a
one-to-subsetOfAll mode, which is rarely used to send to more than one
application and is extremely annoying.

7) msgId, response = send(senderId, recipientId, mtype, args[])
8) msgId, responses[] = broadcast(senderId, mtype, args[])

These are like 5 and 6, but block and then return a response value.  This
value can be quite general (e.g. some kind of datastructure)

These are going to be contentious. Plastic allows a request-response style
messaging pattern.  This is not exactly radical: SOAP does it, and MPI has
blocking send modes (though they're not quite the same as this).  It _is_
very useful in making the programmer's life easier when dealing with very
quick informational messages such as "tell me what your application's icon
is".  Yes, I know you could do this asynchronously and sit around waiting
for a message to come back with the answer, but I feel that we should be
prepared to sacrifice some purity of style to make the application
programmer's life easier.  Furthermore, my putative python-command-line
hacker cannot receive data back from applications any other way (or at
least, not easily).
However, I'd be interested to know what the community thinks.  Could
including a request-response style seriously restrict our choices when it
comes to picking wire protocols?

9) hubId = getHubId()

We can treat the hub itself as just another application that can be
messaged.  However, you do need to know which application it is, and this
operation does the job.

10) ids[] = getApplicationIds()

Tell me the hub-assigned ids of all the registered applications.

11) ids[] = getApplicationsUnderstanding(mtype)

Tell me the hub-assigned ids of the all the registered applications that
understand messages with this mtype.

12) mtypes[] = getUnderstoodMessagesForApplication(id)

Tell me what mtypes this particular application claims to support.

Security
------------
The question of applications spoofing one another has come up a few times.
While I don't really believe it's a serious risk (those astronomers -
they're such jokers!), I set out below a simple modification to the above
protocol that would make it more secure.  This is  based on proposals by
Mark Taylor in discussions we had a year ago - if they don't make any sense
then blame my faulty memory rather than Mark.

The register operation could be changed as follows:
(id, application-secret) = register*(hub-secret)

The hub-secret is a secret that can only be easily known by applications
running under the user's uid and is intended to defeat other users on the
same machine who might try (e.g.) port scanning.  For instance, it could be
a random string written into the .ivoamsg file.  The application-secret is a
per-application secret that apps must keep track of and use along with their
id to identify themselves.  Thus, operations 2-8 would all include this as
an extra parameter (though it wouldn't be transmitted to any receiving
application by the hub.)  This prevents Topcat pretending to be Aladin and
vice versa (you guys!).

Well, I hope that you have time in between munching Easter Eggs to give me
your thoughts on this.  I'll try to get this up on the wiki so that we can
keep track of modifications and suggestions.

John
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.ivoa.net/pipermail/apps/attachments/20070406/682bf4d3/attachment.html>