VOSpace 2.1 issues

[Dave Morris]

On 2014-09-13 03:06, Mike Fitzpatrick wrote:
> On Fri, Sep 12, 2014 at 6:04 PM, Brian Major <majorb at nrc-cnrc.gc.ca> 
> wrote:
> 
> .... This means a service can offer a
> capability, but it would apply to all nodes or to none at all.  If 
> there
> were "special" pre-defined nodes with a capability then the user would
> still be at the mercy of whatever options the service allowed (e.g. 
> there'd
> be no way to specify a different out name/database/whatever).
> 

There is nothing that mandates all capabilities must be available on all 
the nodes.

A service is free to offer different sets of capabilities on different 
nodes.

Which capabilities are available on which nodes is under the control of 
the server, because only the server really knows where and how the data 
is actually stored, and as a result, which capabilities it is actually 
possible to provide on any given node.

> In the discussion that started all these we had three capabilities in 
> mind:

These use cases are exactly the kind of things we had in mind when we 
designed the VOSpace specification. In fact, the automatic SIA 
capability is explicitly mentioned as an example in the capabilities 
section of the VOSpace specification.

> 1) an SIA service for all images put into the container

Your service can offer a SIA service as a capability on any or all of 
the container nodes within your service.

If you want to be able to control or configure the SIA service from the 
client, you can add an 'EnableSIA' property to enable/disable your SIA 
service on a per node basis.

You can add this now with your own property URI

     'urn:EnableSIA' - true/false to enable the SIA service

You can also add a 'ConfigSIA' capability to enable the client to 
configure your SIA service on a per node bases.

You can add this now with your own capability

     'urn:ConfigSIA' - REST service to get/set the SIA service properties

This would enable a client to switch the SIA capability on and off using 
the 'urn:EnableSIA' property, and use the 'urn:ConfigSIA' capability to 
configure it.

If you want the client to be able to enable/configure the SIA capability 
when the node is created, then map some/all of the SIA configuration 
properties onto node properties, which can be set when the node is 
created.

All of this is possible now.

> 2) a service that takes any table and loads it into a database

Again, this can be supported by defining a set of properties and 
capabilities for enabling and configuring the import process.

A new property to enable your TableImport capability

     urn:EnableTableImport - true/false to enable the table import.

A new capability to configure your TableImport capability

     urn:ConfigTableImport - A REST service to get/set the table import 
properties

All of this is possible now.

> 3) a generic "JobRunner" capability ...

Again, this can be supported by defining a set of properties and 
capabilities for enabling and configuring the executable task.

A new property to enable your JobRunner capability

     urn:EnableJobRunner - true/false to enable the JobRunner capability

A new capability to configure your JobRunner capability

     urn:ConfigJobRunner - A REST service to get/set the job runner 
properties

All of this is possible now.

----

The original VOSpace specification was designed to allow individual 
service providers to extend the service by developing their own 
properties and capabilities.

Over time, new properties and capabilities that proved popular would be 
adopted as official IVOA properties and capabilities.

The adoption process can be as simple as publishing an IVOA note and 
registering the URIs in the registry.

The reason that we did not define any of these extended capabilities in 
the original VOSpace specification was to try to keep the core 
specification as simple as possible while still being able to provide 
support for these types of extensions.

The danger we wanted to avoid was like the Simple Object Access Protocol 
(SOAP), the VOSpace specification would gradually become more and more 
specialized until it was no longer Simple anything.

> .... As it stands there is no way
> for a VOSpace to offer these in a list of supported capabilities, but
> assign only one of those to a specific container, and no way for the 
> user
> to create multiple instances with slightly different parameters.

These are both supported by the current specification.
(see above)

> 
> For something like the SIA service example (one used by the spec as 
> well)
> an attachCapabilities method would need to return the service URL the
> client could then use.  Similarly one can imagine wanting to set 
> parameters
> such as one to force a scan of the current container holdings when 
> building
> the service (e.g. when you attach the capability to an existing 
> container)
> or to use only new images, to pass in DB login params, etc.  You could
> argue these parameters are properties of the node and that setNode 
> would
> suffice, but there should probably be a fixed syntax (e.g.
> "<capName>.<param>") defined for this.
> 
> The idea of a detachCapability is needed in workflows where you might 
> want
> to suspend the capability for some reason.
> 

All of this is supported by the current specification.

All of this can be done now by defining a small set of properties and 
capabilities for each new use case, AutoSIA, TableImport, JobRunner etc.

Adding generic 'attachCapability' and 'detachCapability' methods to the 
core VOSpace service could involve a lot of work to describe all the 
metadata needed to describe which capabilities are applicable which 
nodes.

However .. there is nothing preventing you from adding a new 
'ConfigureCapabilities' capability and making it available on any/all of 
the nodes in your service.

The new 'ConfigureCapabilities' capability could provide the 
'attachCapability' and 'detachCapability' methods.

If the 'ConfigureCapabilities' capability proves to be popular, then it 
can be adopted as an official IVOA capability.

All that would needed for it to become an official IVOA capability would 
be an IVOA note describing the capability, and a URI registered in the 
registry.

All of this is supported by the current specification.

--------

Hope this helps,
Dave

--------
Dave Morris
Software Developer
Wide Field Astronomy Unit
Institute for Astronomy
University of Edinburgh
--------