callAndWait( ) implementation issues?

[Mark Taylor]

On Wed, 7 May 2008, Alasdair Allan wrote:

> I'm currently getting a bit worried by the implementation issues surrounding 
> the callAndWait( ) method in the Hub. The current scheme is that a client can 
> call this and expect a synchronous response from the Hub. However there seems 
> to be no way to synchronously call a client.
>
> Clients only have a recieveCall( ) method, and then asynchronously return a 
> reply( ) call to the Hub in response. In theory the Hub takes this hopefully 
> speedily returned reply( ) and uses it as a return value for the callAndWait( 
> ) which has been hanging with an open socket all the time.
>
> However this reply( ) call will almost certainly be handled by a separate 
> thread (or forked process) or at least a different part of the Hub software 
> than the (hanging open) callAndWait( ) method. I'm having real difficulty 
> imagining how the information in the asynchronous reply( ) call is supposed 
> to make its way over to the synchronous callAndWait( ) except by a 
> complicated system of thread semaphores and shared variables.

Hi Al,

interesting point.  You are correct that in order to implement 
callAndWait() the hub will need to be multithreaded/multiprocess and 
will require some degree of non-trivial IPC (i.e. more than just 
wait()/kill()).  Clearly this makes hub implementations in strictly
single-threaded languages a non-starter, but I will admit that since 
the threading features of the language make this sort of thing 
relatively straightforward in Java, I had not thought about this as 
a serious burden on implementators in other languages which offer 
multi-threading or at least access to fork().

> I can certainly go off an implement this model, however this is going to 
> severely limit the language choice for Hub implementation. For Perl at least, 
> it means that the Perl binary will have to be compiled with ithreads enabled, 
> and while mine is, most peoples aren't. Up till now I've been implementing 
> the Hub using vanilla Perl, this implies that you won't be able to take the 
> Perl implementation out of the box.

Although I haven't written serious Perl for quite some while
(so don't really know what I'm talking about), I am
a bit surprised that this sort of thing would be sufficiently
difficult to require a Perl compiled with non-standard options.
My sketchy research suggests that it ought not to be too much of
a stretch to implement this functionality using SysV style IPC,
if available (i.e. if Perl shmget() is implemented), via a module 
like IPC::Shareable or IPC::ShareLite.  These modules work on the Perl 
which comes with my Linux machine without having to recompile it or 
anything.  Linux and Mac have SysV IPC.  It looks to me like
MS Windows is sufficiently POSIXy to have it too, but I haven't 
tested this.

> However the real problem is that I've got no real feel for how badly this 
> might affect other languages outside the core three of Java, Python and Perl. 
> In my opinion this issue massively complicates an otherwise fairly easily 
> implemented specification.

I'll admit that it does complicate hub implementation, and possibly
limits the languages in which hubs can be implemented.  The benefit
is that it's easier to write clients.  Either of your suggestions
makes it easier to write a hub at the expense of making it harder to
write clients:

> 1) Add a recieveSynchronousCall( ) method in the client.

this would effectively require that clients need to provide two ways
of doing the same thing - fiddly rather than difficult.
I must say I don't like the idea of clients having to work harder to
make life easier for the hub, but it is a possibility.

> 2) Remove callAndWait( ) from the hub.

this would mean that only callable clients could receive return values
from messages.  That would be a shame (writing a callable client is in
general going to be a lot harder than writing a non-callable one, and
will be effectively impossible in single-threaded languages).

> Comments?

My feeling about this is that the burden on hub developers for implementing
callAndWait() is acceptable.  Although we have deliberately moved away
from the PLASTIC model which (pretty much) required hubs to be written
in java because of the Java RMI requirement, in my opinion this does
not mean that we need to bend over backwards to make hub implementation
as easy as possible, or to remove barriers to writing hubs in the widest
possible variety of languages.  Above all I feel that the primary goal
is lowering the burden on client developers, and if this makes hub
developers' lives harder, or restricts the languages in which they can
work, so be it.  It is important that SAMP does not mandate a
particular language or implementation for hubs, and it's great for
robustness to have two or three implementations available, but I don't
believe that there is any need to have a large number of different
hub implementations in all kinds of languages.

So in my opinion we should stick with the existing synchronous facade
built on asynchronous calls model.

Other opinions?

Mark

-- 
Mark Taylor   Astronomical Programmer   Physics, Bristol University, UK
m.b.taylor at bris.ac.uk +44-117-928-8776 http://www.star.bris.ac.uk/~mbt/