Proposals for VOSpace (multiple targets)

[Dave Morris]

Hi Sonia,

My apologies, I didn't listen carefully enough to your talk. I missed 
the part where you said you had already tried the container approach. I 
will endeavour to listen more closely in the future. In my defence, it 
was very early in the morning and I had not had enough coffee ;-)

The example in your email prompted me to try another approach.

On 2021-06-17 10:44, Zorba, Sonia wrote:
> For example, by default a pullFromVoSpace operation could accept only 
> one
> target. However, a pullFromVoSpace operation with a view that returns a
> tar/zip archive would make sense.
> 
>      <vos:target>vos://..../file-one</vos:target>
>      <vos:target>vos://..../file-two</vos:target>
>      <vos:direction>pullFromVoSpace</vos:direction>
>      <vos:protocol uri="ivo://ivoa.net/vospace/core#httpget"/>
>      <vos:view uri="ivo://my-registry/vospace/views#tar" />
> 

Could we add the list of selected files as params on the view ?

     <vos:target>vos://..../parent-container</vos:target>
     <vos:direction>pullFromVoSpace</vos:direction>
     <vos:protocol uri="ivo://ivoa.net/vospace/core#httpget"/>
     <vos:view uri="ivo://my-registry/vospace/views#tar">
         <vos:param 
uri="ivo://my-registry/vospace/views#tar/include">file-one</vos:param>
         <vos:param 
uri="ivo://my-registry/vospace/views#tar/include">file-two</vos:param>
     </vos:view>

This would mean we would only get the individual file selection on views 
that transfer multiple files, like tar and zip. Solving the question 
about what to do with multiple targets on a protocol that only transfers 
a single file.

If we define the 'tar/include' param for the 'tar' view as:

    * The name or path of a child node to include in the result, relative 
to the target container.
    * The name or path MUST start with the name of a direct child of the 
target container.
    * The name or path MAY use '/' to create a relative path within the 
target container.
    * The name or path MUST NOT use '..' to refer to nodes outside the 
target container.
    * The name or path MUST NOT start with '/' to imply an absolute path 
within the VOSpace.

This limits the view to the simple case of only being able to select 
child nodes within the container, which matches the capability of the 
tar file format used for the transfer.

Note - if a client wanted to be able to select files from anywhere in 
the VOSpace, they could treat the top level '/' node as a container, and 
select any files they wanted to include relative to the top of the 
space.

We could also define a 'tar/exclude' param for the 'tar' view as:

    * The name or path of a child node to **exclude** from the result, 
relative to the target container
    * The name or path MUST start with the name of a direct child of the 
target container
    * The name or path MAY use '/' to create a relative path within the 
target container
    * The name or path MUST NOT use '..' to refer to nodes outside the 
target container
    * The name or path MUST NOT start with '/' to imply an absolute path 
within the VOSpace

So a more complex example would be:

     <vos:target>vos://..../parent-container</vos:target>
     <vos:direction>pullFromVoSpace</vos:direction>
     <vos:protocol uri="ivo://ivoa.net/vospace/core#httpget"/>
     <vos:view uri="ivo://my-registry/vospace/views#tar">
         <vos:param 
uri="ivo://my-registry/vospace/views#tar/include">file-one</vos:param>
         <vos:param 
uri="ivo://my-registry/vospace/views#tar/include">path/file-two</vos:param>
         <vos:param 
uri="ivo://my-registry/vospace/views#tar/include">path/child-container</vos:param>
         <vos:param 
uri="ivo://my-registry/vospace/views#tar/exclude">path/child-container/file-three</vos:param>
     </vos:view>

Which translates as a request for a HTTP GET endpoint to download a tar 
file containing 'file-one', 'path/file-two', plus all the contents of 
'path/child-container' apart from 'path/child-container/file-three'.

We could add wild cards to the mix, probably by defining a subset of the 
Unix glob syntax.
https://en.wikipedia.org/wiki/Glob_(programming)

Hope this helps,
-- Dave