web services for remote execution

[Guy Rixon]

Roy,

it's all been done before, somewhere.  What you're describing is a classical
compute-grid.  Loadsa prior art. The particular form you suggest - execution
of a restricted set of pre-installed apps - is CEA's speciality (CEA is a grid
job-manager, in case you were wondering.

CEA does the session-cretaion bit (will send architectural links next week).
It imports and exports files but does not otherwise do the file space. Globus
WS-GRAM also does this.

IVOA async-activities specifies the session-creation interface. OGSA will also
specify this (no details available yet). This also handles cleaning up the
files afterwards. IVOA async activities are based on WS-RF; WSRF.NET is out (U
of  Virginia) and seems rather good - I have a detailed tutorial example for
using it if you wish.

MySpace does file management. GGF's GFS also does so (no implementations yet).
VOSpace and VOStore will also handle this.

Condor and Grid Engine are good systems for sessions on local clusters. DRMAA
is GGF's standard API for talking to these from an application - and it's
implemented for both. CEA may get a link into GridEngine via DRMAA quite soon.

CEA does execution of Unix apps. If I have my way, it will be made
cluster-aware (the DRMAA thing, above) for these same apps.

Let me know if you want more details.  I'm off to Pune, so will be off-line
until Monday.

Cheers,
Guy


On Fri, 24 Sep 2004, Roy Williams wrote:

> Guy
>
> I have been watching some the emails, wondering if any of them fit what I want to do.
> O'Mullane and his RunID seems relevant, The GGF document on "Scenarios for mapping DAIS
> concepts" seems relevant.
>
> I am building an infrastructure for Science Gateways on the grid. This is a set of secure
> web services that allow execution and monitoring of batch jobs. It is intended to be used
> with a set of web pages to expose particular functionality (eg astronomical mosaicking,
> seismic simulations, etc).
>
> The web services would allow:
>
> 1. Creation of a session on the server, including creation of file space, and return of
> sessionID
>
> 2. Put and Get to the file space of a session
>
> 3. Execution of an arbitrary Unix command on the server, which includes:
> -- Queuing jobs on the server as part of session
> -- Monitoring status of remote jobs using sessionID
>
> 4. Removing and deleting a session and its files
>
> I am close to defining my own service interface (WSDL) for these actions. Before I do so,
> can you (or anyone else) tell me if anyone else has already done this (CEA? CondorSOAP?
> GGF?)
>
> Thank You
> Roy
>
>
> --------
> California Institute of Technology
> roy at caltech.edu
> 626 395 3670
>

Guy Rixon 				        gtr at ast.cam.ac.uk
Institute of Astronomy   	                Tel: +44-1223-337542
Madingley Road, Cambridge, UK, CB3 0HA		Fax: +44-1223-337523