Apps Messaging - Semantics of a Message
Paul Harrison
pharriso at eso.org
Fri Apr 27 03:54:08 PDT 2007
Hi,
I believe that you have decided in this group to split the interface/
transport issues from the message definitions - it is primarily the
message definitions/semantics that I would be interested in trying to
co-ordinate between the groups.
Paul Harrison
ESO Garching
www.eso.org
On 27.04.2007, at 09:34, Mike Fitzpatrick wrote:
> Hi Paul,
>
> Sorry to have apparently ignored your message. You're right that in
> the case of the task invocation being discussed the overlaps make
> the UWS worth considering. Where I think this breaks down however
> is that with a SOAP or REST interface this is limited to a point-to-
> point
> scenario. A use-case in which multiple instances of the same task
> may need to be started (e.g. a pipeline operating on different halves
> of the same image file where the context for each task was established
> earlier) requires a broadcast message, especially if the number of
> worker tasks isn't known a priori.
well in the SOAP interface, which is more complex than the REST one,
this sort co-ordination was done by registering callbacks rather than
using broadcasts, but as I said I am not joining the argument on
transport/interface issues...
>
> I can see where complications could also occur with long-running or
> crashed tasks, e.g. with a message Hub the Hub would likely know
> that a task crashed an disconnected, a hung task might not give the
> same indication with a service call to check its status.
>
> FWIW, I don't see much difference in using messages to invoke some
> function normally accessed from a GUI in one case or implemented as
> a task in another. The devil of course is in how the message is
> encoded
> and your view of what messages should be used for.
>
> In any case, we should discuss this more in Beijing.
gladly.
>
> -Mike
> On 4/16/07, Paul Harrison <pharriso at eso.org> wrote:
>> Hi All,
>>
>> I have come late to this latest long applications thread, and have a
>> couple of general points to make without getting into the fine
>> detail.
>>
>> There does seem to be a dichotomy in this group between the original
>> PLASTIC group and the more expanded "applications messaging" group. I
>> think that this is because PLASTIC was designed as a protocol where
>> essentially you issue a command to an application and it does
>> something, but the original issuer does not care about the results.
>> The proposal for applications messaging does seem to want to create
>> an infrastructure when the issuer does care about the results of an
>> application invocation so that complex "workflows" can be created
>> from the individual applications, which of course adds complication
>> compared with the PLASTIC case.
>>
>> This extended "application messaging" use case seems to overlap
>> hugely with the "Parameterized Application" use case of the Universal
>> Worker Service (UWS-PA) that the Grid WG has developed. The UWS was
>> primarily designed as a client-server protocol, so already has a SOAP
>> profile, and will have a pure REST profile by Bejiing interop.
>> However, it would be conceivable to create an intra-desktop profile
>> to meet the "applications messaging" use case, all using essentially
>> the same registry-storable message description metalanguage.
>>
>> Paul Harrison
>> ESO Garching
>> www.eso.org
>>
>> On 06.04.2007, at 12:20, Mike Fitzpatrick wrote:
>>
>> > Hi All,
>> >
>> > The thread has died down a bit and while I think we've
>> agreed there
>> > needs to be some sort of "Hub" process (this is not uncommon in
>> other
>> > messaging systems) and made some good progress on how an app might
>> > connect
>> > to a messaging system, there is much left to be discussed about
>> > "What is a
>> > Message" and "What does it mean". Keeping in mind our desire to
>> > separate
>> > the message spec from any particular implementation or transport
>> > protocol,
>> > I offer the following fodder for discussion.
>> > I've looked at quite a few messaging systems and some of
>> the ideas
>> > here are drawn from each. I also claim that these ideas map well
>> > onto the
>> > current PLASTIC concepts/usage even if in a final version PLASTIC
>> > implementations will need to be modified a bit. This is not meant
>> > to be
>> > read as a specification and rambles a bit between concept and
>> > detail, but
>> > I'm hoping to at least start the discussion about what we hope to
>> > accomplish with application messaging and offer some ideas about
>> > how to
>> > meet some requirements. It incorporates some use cases not in
>> John's
>> > current list on the twiki and tries to be fairly hi-level and
>> free of
>> > implementation details. Note that the 'Hub' and transport are not
>> > discussed, I'd like to focus on the semantics and purpose of the
>> > messages
>> > for the moment.
>> > So, let the debate begin and thanks for your feedback....
>> >
>> >
>> > -Mike
>> >
>> >
>> >
>> > Message Concepts
>> > ----------------
>> >
>> > A MESSAGE is an abstract container for the information we
>> wish
>> > to send to another application. Some attributes of a message are
>> > required
>> > to ensure proper delivery and handling, but there are also optional
>> > attributes that may only be used in a specific context (e.g. to
>> > return a
>> > result/response rather than from an original request for some
>> action).
>> > Attributes such as the type or ID of message will be required to be
>> > delivered to all recipients, however an attribute such as the
>> > sender of a
>> > message may be optional and depend on the implementation.
>> >
>> > MESSAGEs can be described generally as being a NOTIFY, a
>> REPLY,
>> > or a REQUEST message. NOTIFY messages are purely informational and
>> > require
>> > no response or confirmation of delivery. A REQUEST message is one
>> > that
>> > asks another application to perform some action; in this case the
>> > application SHOULD reply explicitly with a status message
>> indicating
>> > whether the request was completed, however it is not an error if it
>> > does
>> > not. [Note: Applications may set a property in the Hub to request
>> > confirmation of delivery of every message and so are free to decide
>> > for
>> > themselves whether a missing reply is to be considered an error.] A
>> > REPLY
>> > message is one sent to return a status code or other result in
>> > response to
>> > an originating message; REPLY messages are tagged with the same
>> > message ID
>> > as the original and are returned to the sender of the originating
>> > message
>> > as well as apps that wish to monitor the message traffic.
>> >
>> > More specifically, a MESSAGE contains an 'mtype' attribute
>> that
>> > defines the semantic meaning of the message. The concept behind
>> the
>> > 'mtype' (described in more detail below) is based loosely on the
>> > use of
>> > UCDs in that a (small) controlled vocabulary is sufficient to
>> > describe the
>> > majority of concepts needed in applications messaging. The mtype
>> > is made
>> > up of 'atoms' to construct 'words' that are not only meaningful
>> to the
>> > developer, but allow applications to easily match regular
>> > expressions by
>> > using wildcard characters to filter messages that may or may not
>> > match a
>> > specific capability in the application. For example, an mtype
>> > expression
>> > of "display.*" might imply some sort of image display capability
>> where
>> > 'display.URL' means an app can specifically download/display and
>> > image from
>> > a URL. A task may wish to connect by advertising interest in
>> the more
>> > general display messages, but is free to reject specific messages
>> > it cannot
>> > handle.
>> >
>> > Because we wish to loosly couple the capabilities one
>> application
>> > is searching for from the details of what another may provide, we
>> > don't
>> > create a rigorous definition of a message and its behavior in an
>> > application. Instead, the mtype is meant to create a "rough
>> > concept" of a
>> > message such as "display an image"; The use of a specific mtype
>> > might also
>> > suggest the parameters required (e.g. a filename or URL) to form a
>> > valid
>> > request and so sending apps will have some hope that another
>> > listening app
>> > can do something sensible without knowing the detailed capabilities
>> > of the
>> > receiving application. As a part of the specification an mtype
>> > such as
>> > 'display.url' may require that a URL be the only argument and
>> > expect no
>> > return value, however by sending a message to a *named*
>> application we
>> > assume the sender wishes to use that app for some specific reason
>> > and may
>> > know the details of how a particular message is implemented, and so
>> > it is
>> > free to use an argument list peculiar to the target app and perhaps
>> > expect
>> > a reference of some kind to the downloaded image for later
>> processing.
>> > Likewise, any two client apps are still free to use a mutually
>> agreed
>> > private set of messages outside of the mtype vocabulary and
>> > exchange them
>> > using the same underlying system. If we were to tag the message as
>> > something like "generic;display.URL' an app MAY choose to
>> provide only
>> > the functionality implied by the spec for that message (e.g. not
>> > save a
>> > result reference), otherwise it may always reply with a
>> reference that
>> > a sender might ignore.
>> >
>> >
>> >
>> > Message Attributes
>> > ------------------
>> >
>> > The list of message attributes is intentionally small to
>> maintain
>> > simplicity. We will assume that the attribute names listed here
>> are
>> > retained when a message is serialized or queried. These include:
>> >
>> > sender The <appName> of sender application. The Hub will
>> supply
>> > this attribute to the receiving application. The
>> > implementation should not require that a "sender
>> ID" be
>> > present in a message request since this allows
>> spoofing
>> > of the message and the application IDs are more
>> properly
>> > suited as being an attribute in the HUb than in an
>> app
>> > knowing what the Hub assigns as some ID. Just as
>> there
>> > may be multiple identical instances of a Recipient, a
>> > Sender should be likewise free from a specific
>> instance
>> > defined by a Hub ID.
>> >
>> > recipient The recipient of the message. This is a String value
>> > supplied by a sending applicaton that may be one
>> of the
>> > reserved words:
>> >
>> > Hub Message is intended for the Hub only and
>> > will not be forwarded to other apps. May
>> > only be used with SET and GET class
>> messages.
>> > (Message classes are discussed below).
>> >
>> > Any Sender wishes to broadcast to all clients
>> > currently connected. These are typically
>> > used only with STATUS and EVENT class
>> msgs.
>> > Applications that cannot, or choose
>> not to,
>> > handle the message is free to ignore
>> it without
>> > posting a reply notification.
>> >
>> > Additionally, the recipient may be one of:
>> >
>> > <appName> Indicating the message should be sent
>> to all
>> > clients with this <appName>.
>> >
>> > <pattern> Indicating the message should be sent
>> to all
>> > clients that have registered a
>> capability to
>> > handle messages matching the specified
>> > pattern. The use of '*' as a
>> <pattern> is
>> > a special case of the reserved word 'Any'
>> >
>> > In these last two cases, the Hub will first
>> attempt to
>> > match based on the <appName> before <pattern>.
>> Wildcards
>> > are permitted in the <appName> to allow sending to
>> a subset
>> > of apps (e.g. sending to "worker*" will deliver
>> the message
>> > to identical instances of both the 'worker1' and
>> 'worker2'
>> > apps as well as multiple instances of an app that
>> connected
>> > simply using the name 'worker').
>> >
>> > msgid A unique id assigned to each message by the Hub
>> and included
>> > as a message attribute for the RECEIVEd message.
>> This
>> > value is returned to the sender as a response to
>> the SEND
>> > method, the msgid will remain the same in a REPLY
>> method
>> > message so the sender can identify the originating
>> message.
>> >
>> > mtype A UCD-like string indicating the request or
>> message type.
>> > The semantics and syntax of the mtype are
>> described in more
>> > detail below.
>> >
>> > refID An ID assigned by an app to be used as a reference
>> to a
>> > result object in future messages it may receive.
>> In some
>> > cases this may be an opaque handle to something
>> like "the
>> > image you loaded in the display", in other cases
>> it can be
>> > de-referenced to an actual file/image that was
>> created
>> > (e.g. the output of a "Save Image" message).
>> Applications
>> > may choose to not return a refID if it is a purely
>> > transient result (e.g. a plot of an image
>> histogram where
>> > the data for the plot is not saved) or cannot
>> meaningfully
>> > be referenced by a later message (e.g. a subset of
>> table
>> > rows that may be highlighted but cannot be
>> addressed as a
>> > new table).
>> >
>> > arguments A whitespace-delimited string specifying the
>> > arguments of
>> > the message. In the case of a REPLY message this
>> will be
>> > either a string containing the response value, or
>> in the
>> > case of a request that does not return a value,
>> one of the
>> > reserved strings:
>> >
>> > OK Request was executed w/out error
>> > ERR Request encountered an error
>> > REJECTED Recipient refused to process the request
>> > DELIVERED Recipient received the message but did
>> not reply
>> >
>> >
>> > Message Representation
>> > ----------------------
>> >
>> > A MESSAGE can be represented in a number of ways suitable
>> to the
>> > needs of the transport protocol and/or what is simplest for the
>> > sending or
>> > receiving client. For example, a simple string of keyword=value
>> pairs
>> > listing the attributes of the message, a serialized XML document,
>> > as an RPC
>> > call based on the message type and arguments, or as parameters for
>> > an HTTP
>> > GET request. Attributes not explicitly present in a message are
>> > assumed to
>> > be undefined, required attributes may not be blank and an
>> application
>> > should trigger and error if they are.
>> > Note that no requirement is made that ALL attributes of a
>> MESSAGE
>> > are presented to the recipient. Just as a sender may not care who
>> > gets
>> > the message, a recipient may not care who sent it -- however
>> > provisions
>> > should be made for clients to get this information if they choose.
>> > This
>> > can be done e.g. by the recipient querying the Hub for a message
>> > attribute
>> > based on its message ID, or by setting a property in the Hub that
>> > requests
>> > messages be delivered with this extra information.
>> >
>> >
>> > Message Delivery
>> > ----------------------
>> >
>> > All messages are sent asynchronously from the Sender's
>> perspective.
>> > The method that a client uses to send a message MUST return the
>> msgID
>> > assigned by the Hub for that message. Because a REPLY message is
>> > tagged
>> > with the originating msgID, a client that needs synchronous
>> > behavior can
>> > simply block until it receives a reply message with the same id.
>> > This also
>> > allows for multiple recipients to reply individually without the
>> > sender
>> > knowing how many potential recipients are available and provides a
>> > "broadcast" behavior even in cases where a recipient app is named
>> > (i.e.
>> > because multiple instances of that app could be connected).
>> >
>> > In this model, clients can determine for themselves how
>> many other
>> > apps are available, what their capabilities are, etc. This differs
>> > slightly from the PLASTIC model where messages are either
>> synchronous
>> > (blocking the client or the Hub waiting for replies), or
>> > asynchronous (and
>> > thereby losing all replies) based on the delivery method invoked.
>> > A Hub
>> > may still need to maintain information about attached clients in
>> > order to
>> > determine the receipients of a message, but the messages being
>> > exchanged
>> > and the number of apps involved in any one exchange is separate
>> > from the
>> > underlying transport.
>> >
>> >
>> > Message Types
>> > ----------------------
>> >
>> > As mentioned above, the 'mtype' is a message type similar
>> to the
>> > UCD+ convention of using a controlled vocabulary to build up more
>> > complex
>> > meaning. For example, the mtype 'display.image' is logically
>> > understood to
>> > mean "display an image"; However the actual actions taken by
>> > applications
>> > can be quite different in response to this message even though they
>> > "make
>> > sense" for that particular application. As an example, if this
>> > were sent
>> > to Aladin it would likely cause the app to load the named image as
>> > a new
>> > layer in the display; The same message sent to IRAF however might
>> > invoke
>> > the IRAF display task and cause the image to appear in a display
>> > server
>> > window (let's assume that server isn't msg-enabled). In both cases
>> > the
>> > action is logical for the given app, the developers are free to
>> > choose how
>> > to implement the default action for the message, and as far as the
>> > sender
>> > of the message is concerned an image is displayed for the user.
>> > The list of controlled words and their hierarchy will no
>> doubt
>> > suffer the same debates as with UCDs, but there is promise of
>> > consensus.
>> > The initial list should be small and cover current usage but should
>> > also be
>> > extensible to apps and use-cases not currently in use. Below we
>> > list a
>> > suggested hieraarchy based on minimal consideration:
>> >
>> > Notify (no response required)
>> > app.* Application status msgs
>> > connected
>> > disconnected
>> > error
>> > status.* Reply status messages
>> > reply
>> > delivery
>> > progress
>> > :
>> > event.* UI event messages
>> > keypress
>> > mouseButtonDown
>> > mouseButtonUp
>> > :
>> >
>> > Request (response required)
>> > set/get Set/Get property messages
>> > info
>> > param
>> > property
>> > capability
>> > appName
>> > result
>> > filename
>> > load/save File operations
>> > image
>> > table
>> > URL
>> > plot.* Vector Graphics capabilities
>> > table
>> > rows
>> > display.* Image Display capabilities
>> > image
>> > URL
>> > select.* Table selection capabilities
>> > rows
>> > highlight.* Object highlighting
>> capabilities
>> > position
>> > rows
>> > exec.* Remote task execution
>> capabilities
>> > task
>> >
>> >
>> > Rigorous Message Types
>> > ----------------------
>> >
>> > The PLASTIC implementation of messages is based on the
>> idea that a
>> > message type is an 'ivorn'. The primary argument for this approach
>> > is that
>> > the Registry can be used as a central repository for a description
>> > of the
>> > message, however there are several problems in a generalized
>> system:
>> >
>> > - The message ivorn is based on an ad-hoc agreement between
>> > developers
>> > of specific apps and does not specify the semantics of
>> what that
>> > message might mean to later app developers.
>> > - While an ivorn may be searchable in a registry, the
>> > descripion of that
>> > ivorn isn't generally machine-parsable. This means an app
>> can't
>> > effectively resolve an ivorn at runtime to determine a needed
>> > capability, and even a search for capabilities registered
>> for a
>> > specific app would require a network connection for the
>> > 'system' to
>> > operate at all. Apps should be able to make requests for
>> > capabilities
>> > without an available Registry using only the locally running
>> > processes
>> > (or tasks that can be invoked based on a requested message).
>> > - IVORNs are more complicated to parse than UCD schemes and
>> so the
>> > filtering of messages is also more complicated in an app.
>> >
>> > That said, the current PLASTIC messages appear to have the same
>> > idea of a
>> > message hierarchy (e.g. by including "/fits" or "/table" in the
>> > path) even
>> > if these messages are implicitly tied to a specific implementation.
>> >
>>
>>
More information about the apps
mailing list