Apps Messaging - Semantics of a Message

[Mike Fitzpatrick]

Hi All,

	The thread has died down a bit and while I think we've agreed there
needs to be some sort of "Hub" process (this is not uncommon in other
messaging systems) and made some good progress on  how an app might connect
to a messaging system, there is much left to be discussed about "What is a
Message" and "What does it mean".  Keeping in mind our desire to separate
the message spec from any particular implementation or transport protocol,
I offer the following fodder for discussion.
	I've looked at quite a few messaging systems and some of the ideas
here are drawn from each.  I also claim that these ideas map well onto the
current PLASTIC concepts/usage even if in a final version PLASTIC
implementations will need to be modified a bit.  This is not meant to be
read as a specification and rambles a bit between concept and detail, but
I'm hoping to at least start the discussion about what we hope to
accomplish with application messaging and offer some ideas about how to
meet some requirements.  It incorporates some use cases not in John's
current list on the twiki and tries to be fairly hi-level and free of
implementation details.  Note that the 'Hub' and transport are not
discussed, I'd like to focus on the semantics and purpose of the messages
for the moment.
	So, let the debate begin and thanks for your feedback....


-Mike



Message Concepts
----------------

	A MESSAGE is an abstract container for the information we wish
to send to another application.  Some attributes of a message are required
to ensure proper delivery and handling, but there are also optional
attributes that may only be used in a specific context (e.g. to return a
result/response rather than from an original request for some action).
Attributes such as the type or ID of message will be required to be
delivered to all recipients, however an attribute such as the sender of a
message may be optional and depend on the implementation.

	MESSAGEs can be described generally as being a NOTIFY, a REPLY,
or a REQUEST message.  NOTIFY messages are purely informational and require
no response or confirmation of delivery.  A REQUEST message is one that
asks another application to perform some action; in this case the
application SHOULD reply explicitly with a status message indicating
whether the request was completed, however it is not an error if it does
not.  [Note: Applications may set a property in the Hub to request
confirmation of delivery of every message and so are free to decide for
themselves whether a missing reply is to be considered an error.] A REPLY
message is one sent to return a status code or other result in response to
an originating message; REPLY messages are tagged with the same message ID
as the original and are returned to the sender of the originating message
as well as apps that wish to monitor the message traffic.

	More specifically, a MESSAGE contains an 'mtype' attribute that
defines the semantic meaning of the message.  The concept behind the
'mtype' (described in more detail below) is based loosely on the use of
UCDs in that a (small) controlled vocabulary is sufficient to describe the
majority of concepts needed in applications messaging.  The mtype is made
up of 'atoms' to construct 'words' that are not only meaningful to the
developer, but allow applications to easily match regular expressions by
using wildcard characters to filter messages that may or may not match a
specific capability in the application.  For example,  an mtype expression
of "display.*" might imply some sort of image display capability where
'display.URL' means an app can specifically download/display and image from
a URL.  A task may wish to connect by advertising interest in the more
general display messages, but is free to reject specific messages it cannot
handle.

	Because we wish to loosly couple the capabilities one application
is searching for from the details of what another may provide, we don't
create a rigorous definition of a message and its behavior in an
application.  Instead, the mtype is meant to create a "rough concept" of a
message such as "display an image";  The use of a specific mtype might also
suggest the parameters required (e.g. a filename or URL) to form a valid
request and so sending apps will have some hope that another listening app
can do something sensible without knowing the detailed capabilities of the
receiving application.  As a part of the specification an mtype such as
'display.url' may require that a URL be the only argument and expect no
return value, however by sending a message to a *named* application we
assume the sender wishes to use that app for some specific reason and may
know the details of how a particular message is implemented, and so it is
free to use an argument list peculiar to the target app and perhaps expect
a reference of some kind to the downloaded image for later processing.
Likewise, any two client apps are still free to use a mutually agreed
private set of messages outside of the mtype vocabulary and exchange them
using the same underlying system.  If we were to tag the message as
something like "generic;display.URL' an app MAY choose to provide only
the functionality implied by the spec for that message (e.g. not save a
result reference), otherwise it may always reply with a reference that
a sender might ignore.



Message Attributes
------------------

	The list of message attributes is intentionally small to maintain
simplicity.  We will assume that the attribute names listed here are
retained when a message is serialized or queried.  These include:

    sender 	The <appName> of sender application.  The Hub will supply
		this attribute to the receiving application.  The
		implementation should not require that a "sender ID" be
	 	present in a message request since this allows spoofing
		of the message and the application IDs are more properly
		suited as being an attribute in the HUb than in an app
		knowing what the Hub assigns as some ID.  Just as there
		may be multiple identical instances of a Recipient, a
		Sender should be likewise free from a specific instance
		defined by a Hub ID.

    recipient	The recipient of the message.  This is a String value 
	 	supplied by a sending applicaton that may be one of the 
		reserved words:

		Hub	    Message is intended for the Hub only and
			    will not be forwarded to other apps.  May
			    only be used with SET and GET class messages.
			    (Message classes are discussed below).

		Any	    Sender wishes to broadcast to all clients
			    currently connected.  These are typically
			    used only with STATUS and EVENT class msgs.
			    Applications that cannot, or choose not to,
			    handle the message is free to ignore it without
			    posting a reply notification.

		Additionally, the recipient may be one of:

		<appName>   Indicating the message should be sent to all
			    clients with this <appName>.

		<pattern>   Indicating the message should be sent to all
			    clients that have registered a capability to
			    handle messages matching the specified
			    pattern.  The use of '*' as a <pattern> is
			    a special case of the reserved word 'Any'

		In these last two cases, the Hub will first attempt to
		match based on the <appName> before <pattern>.  Wildcards
		are permitted in the <appName> to allow sending to a subset
		of apps (e.g. sending to "worker*" will deliver the message
		to identical instances of both the 'worker1' and 'worker2'
		apps as well as multiple instances of an app that connected
		simply using the name 'worker').

    msgid	A unique id assigned to each message by the Hub and included 
		as a message attribute for the RECEIVEd message.  This
		value is returned to the sender as a response to the SEND
		method, the msgid will remain the same in a REPLY method 
		message so the sender can identify the originating message.

    mtype 	A UCD-like string indicating the request or message type.
		The semantics and syntax of the mtype are described in more
		detail below.

    refID 	An ID assigned by an app to be used as a reference to a 
		result object in future messages it may receive.  In some
		cases this may be an opaque handle to something like "the
		image you loaded in the display", in other cases it can be
		de-referenced to an actual file/image that was created
		(e.g. the output of a "Save Image" message).  Applications
		may choose to not return a refID if it is a purely
		transient result (e.g. a plot of an image histogram where
		the data for the plot is not saved) or cannot meaningfully
		be referenced by a later message (e.g. a subset of table
		rows that may be highlighted but cannot be addressed as a
		new table).

    arguments 	A whitespace-delimited string specifying the arguments of
		the message.  In the case of a REPLY message this will be
		either a string containing the response value, or in the
		case of a request that does not return a value, one of the
		reserved strings:

		OK          Request was executed w/out error
		ERR         Request encountered an error 
		REJECTED    Recipient refused to process the request
		DELIVERED   Recipient received the message but did not reply


Message Representation
----------------------

	A MESSAGE can be represented in a number of ways suitable to the
needs of the transport protocol and/or what is simplest for the sending or
receiving client.  For example, a simple string of keyword=value pairs
listing the attributes of the message, a serialized XML document, as an RPC
call based on the message type and arguments, or as parameters for an HTTP
GET request.  Attributes not explicitly present in a message are assumed to
be undefined, required attributes may not be blank and an application
should trigger and error if they are.
	Note that no requirement is made that ALL attributes of a MESSAGE
are presented to the recipient.  Just as a sender may not care who gets
the message, a recipient may not care who sent it -- however provisions
should be made for clients to get this information if they choose.  This
can be done e.g. by the recipient querying the Hub for a message attribute
based on its message ID, or by setting a property in the Hub that requests
messages be delivered with this extra information.


Message Delivery
----------------------

	All messages are sent asynchronously from the Sender's perspective.
The method that a client uses to send a message MUST return the msgID
assigned by the Hub for that message.  Because a REPLY message is tagged
with the originating msgID, a client that needs synchronous behavior can
simply block until it receives a reply message with the same id.  This also
allows for multiple recipients to reply individually without the sender
knowing how many potential recipients are available and provides a
"broadcast" behavior even in cases where a recipient app is named (i.e.
because multiple instances of that app could be connected).

	In this model, clients can determine for themselves how many other
apps are available, what their capabilities are, etc.  This differs
slightly from the PLASTIC model where messages are either synchronous
(blocking the client or the Hub waiting for replies), or asynchronous (and
thereby losing all replies) based on the delivery method invoked.  A Hub
may still need to maintain information about attached clients in order to
determine the receipients of a message, but the messages being exchanged
and the number of apps involved in any one exchange is separate from the
underlying transport.


Message Types
----------------------

	As mentioned above, the 'mtype' is a message type similar to the
UCD+ convention of using a controlled vocabulary to build up more complex
meaning.  For example, the mtype 'display.image' is logically understood to
mean "display an image";  However the actual actions taken by applications
can be quite different in response to this message even though they "make
sense" for that particular application.  As an example, if this were sent
to Aladin it would likely cause the app to load the named image as a new
layer in the display;  The same message sent to IRAF however might invoke
the IRAF display task and cause the image to appear in a display server
window (let's assume that server isn't msg-enabled).  In both cases the
action is logical for the given app, the developers are free to choose how
to implement the default action for the message, and as far as the sender
of the message is concerned an image is displayed for the user.
	The list of controlled words and their hierarchy will no doubt
suffer the same debates as with UCDs, but there is promise of consensus.
The initial list should be small and cover current usage but should also be
extensible to apps and use-cases not currently in use.  Below we list a
suggested hieraarchy based on minimal consideration:

    Notify (no response required)
	app.*				Application status msgs
	    connected
	    disconnected
	    error
	status.*			Reply status messages
	    reply
	    delivery
	    progress
		:
	event.*				UI event messages
	    keypress
	    mouseButtonDown
	    mouseButtonUp
		:

    Request (response required)
	set/get				Set/Get property messages
	    info
	    param
	    property
	    capability
	    appName
	    result
	    filename
	load/save			File operations
	    image
	    table
	    URL
	plot.*				Vector Graphics capabilities
	    table
	    rows
	display.*			Image Display capabilities
	    image
	    URL
	select.*			Table selection capabilities
	    rows
	highlight.*			Object highlighting capabilities
	    position
	    rows
	exec.*				Remote task execution capabilities
	    task


Rigorous Message Types
----------------------

	The PLASTIC implementation of messages is based on the idea that a
message type is an 'ivorn'.  The primary argument for this approach is that
the Registry can be used as a central repository for a description of the
message, however there are several problems in a generalized system:

    - The message ivorn is based on an ad-hoc agreement between developers
      of specific apps and does not specify the semantics of what that
      message might mean to later app developers.
    - While an ivorn may be searchable in a registry, the descripion of that
      ivorn isn't generally machine-parsable.  This means an app can't
      effectively resolve an ivorn at runtime to determine a needed
      capability, and even a search for capabilities registered for a 
      specific app would require a network connection for the 'system' to
      operate at all.  Apps should be able to make requests for capabilities
      without an available Registry using only the locally running processes
      (or tasks that can be invoked based on a requested message).
    - IVORNs are more complicated to parse than UCD schemes and so the
      filtering of messages is also more complicated in an app.

That said, the current PLASTIC messages appear to have the same idea of a
message hierarchy (e.g. by including "/fits" or "/table" in the path) even
if these messages are implicitly tied to a specific implementation.