Proposals for VOSpace (multiple targets)

[Dave Morris]

Hi Sonia,

You raised several good suggestions in your email. To avoid confusion 
I'll reply to each one in a separate email thread.

On 2021-06-11 13:31, Zorba, Sonia wrote:
> 3. Allow to specify multiple target nodes (instead of exactly one 
> target
> node) for transfer operations. This is useful if the user desires to 
> select
> a subset of files inside a folder instead of the whole folder.

This works in your specific case because your staging transfer protocol 
handles multiple files. If we added support for multiple target nodes in 
the standard, we would need to define how this would work for all the 
other transfer protocols.

Specifically, what does this mean?

     <vos:target>vos://..../file-one</vos:target>
     <vos:target>vos://..../file-two</vos:target>
     <vos:direction>pullFromVoSpace</vos:direction>
     <vos:protocol uri="ivo://ivoa.net/vospace/core#httpget"/>

Concatenating the content of the two files into one http stream is 
probably not what was intended.

The standard allows the service to offer multiple protocol endpoints for 
a single transfer. If we added support for multiple targets, then we 
would need to define how to link the different targets to the different 
endpoints.

     <vos:transfer xmlns:vos="http://www.ivoa.net/xml/VOSpace/v2.0" 
version="2.1">
         ....
         <vos:target>vos://..../file-one</vos:target>
         <vos:target>vos://..../file-two</vos:target>
         ....
         <vos:protocol uri="ivo://ivoa.net/vospace/core#httpget">
             <!--+
                 | Would this get one or both targets?
                 | If only one, how do we identify which one?
                 +-->
             <vos:endpoint>http://..../Wahb9Elu</vos:endpoint>
         </vos:protocol>
         <vos:protocol uri="ivo://ivoa.net/vospace/core#httpget">
             <!--+
                 | Would this get one or both targets?
                 | If only one, how do we identify which one?
                 +-->
             <vos:endpoint>http://..../ruphoh4B</vos:endpoint>
         </vos:protocol>
     </vos:transfer>

If we added support for multiple targets, would this include targets 
from different locations? If not, then we would need to define how we 
would prevent them.

     <vos:targets>
         <!--+
             | This is a target in the CADC VOSpace.
             +-->
         <vos:target>vos://CADC/..../file-one</vos:target>
         <!--+
             | This is a target in the INAF VOSpace.
             +-->
         <vos:target>vos://INAF/..../file-two</vos:target>
         ....
         <vos:protocol uri="ivo://ivoa.net/vospace/core#httpget">
             <!--+
                 | Would this get data from both spaces?
                 +-->
             <vos:endpoint>http://..../cie2AiD8</vos:endpoint>
         </vos:protocol>

     </vos:targets>

As it stands I think this raises too many complications.

There is perhaps a simpler solution using the existing standard.

* Create a container node.
* Use links to add the intended targets to the container.
* Apply the staging transfer to the container node.

I think this achieves the same goal, of transferring multiple targets in 
a single transfer, in a way that is understandable to both generic 
VOSpace clients and to human users, without requiring a change to the 
standard.

Cheers,
-- Dave

--------
Dave Morris
Research Software Engineer
Wide Field Astronomy Unit
Institute for Astronomy
University of Edinburgh
--------