Proposals for VOSpace (multiple targets)

[Zorba, Sonia]

Hi Dave,
don't worry about the talk! :-)

I think the solution you are proposing solves the problem in a clean way
without changing the standard, so for me it is ok to proceed in this
direction.
I also agree about not allowing ../ notation in paths.

Currently the recommendation document mentions the possibility of defining
archive views (zip/tar) but it doesn't provide additional details about it.
I think that, even if this will be an optional feature, it would be very
useful to include in the document an example like the one you included in
your previous email, to guide implementers on how to develop this kind of
feature.

I'm also wondering if it could be useful to standardize it, maybe using a
generic archive view with a parameter for defining the format.

Something like this:

     <vos:target>vos://..../parent-container</vos:target>
     <vos:direction>pullFromVoSpace</vos:direction>
     <vos:protocol uri="ivo://ivoa.net/vospace/core#httpget"/>
     <vos:view uri="ivo://ivoa.net/vospace/core#archive">
         <vos:param uri="ivo://ivoa.net/vospace/core#archive/format
">zip</vos:param>
         <vos:param uri="ivo://ivoa.net/vospace/core#archive/include
">file-one</vos:param>
         <vos:param uri="ivo://ivoa.net/vospace/core#archive/include
">file-two</vos:param>
     </vos:view>

Cheers,
Sonia

Hi Sonia,
>
> My apologies, I didn't listen carefully enough to your talk. I missed
> the part where you said you had already tried the container approach. I
> will endeavour to listen more closely in the future. In my defence, it
> was very early in the morning and I had not had enough coffee ;-)
>
> The example in your email prompted me to try another approach.
>
> On 2021-06-17 10:44, Zorba, Sonia wrote:
> > For example, by default a pullFromVoSpace operation could accept only
> > one
> > target. However, a pullFromVoSpace operation with a view that returns a
> > tar/zip archive would make sense.
> >
> >      <vos:target>vos://..../file-one</vos:target>
> >      <vos:target>vos://..../file-two</vos:target>
> >      <vos:direction>pullFromVoSpace</vos:direction>
> >      <vos:protocol uri="ivo://ivoa.net/vospace/core#httpget"/>
> >      <vos:view uri="ivo://my-registry/vospace/views#tar" />
> >
>
> Could we add the list of selected files as params on the view ?
>
>      <vos:target>vos://..../parent-container</vos:target>
>      <vos:direction>pullFromVoSpace</vos:direction>
>      <vos:protocol uri="ivo://ivoa.net/vospace/core#httpget"/>
>      <vos:view uri="ivo://my-registry/vospace/views#tar">
>          <vos:param
> uri="ivo://my-registry/vospace/views#tar/include">file-one</vos:param>
>          <vos:param
> uri="ivo://my-registry/vospace/views#tar/include">file-two</vos:param>
>      </vos:view>
>
> This would mean we would only get the individual file selection on views
> that transfer multiple files, like tar and zip. Solving the question
> about what to do with multiple targets on a protocol that only transfers
> a single file.
>
> If we define the 'tar/include' param for the 'tar' view as:
>
>     * The name or path of a child node to include in the result, relative
> to the target container.
>     * The name or path MUST start with the name of a direct child of the
> target container.
>     * The name or path MAY use '/' to create a relative path within the
> target container.
>     * The name or path MUST NOT use '..' to refer to nodes outside the
> target container.
>     * The name or path MUST NOT start with '/' to imply an absolute path
> within the VOSpace.
>
> This limits the view to the simple case of only being able to select
> child nodes within the container, which matches the capability of the
> tar file format used for the transfer.
>
> Note - if a client wanted to be able to select files from anywhere in
> the VOSpace, they could treat the top level '/' node as a container, and
> select any files they wanted to include relative to the top of the
> space.
>
> We could also define a 'tar/exclude' param for the 'tar' view as:
>
>     * The name or path of a child node to **exclude** from the result,
> relative to the target container
>     * The name or path MUST start with the name of a direct child of the
> target container
>     * The name or path MAY use '/' to create a relative path within the
> target container
>     * The name or path MUST NOT use '..' to refer to nodes outside the
> target container
>     * The name or path MUST NOT start with '/' to imply an absolute path
> within the VOSpace
>
> So a more complex example would be:
>
>      <vos:target>vos://..../parent-container</vos:target>
>      <vos:direction>pullFromVoSpace</vos:direction>
>      <vos:protocol uri="ivo://ivoa.net/vospace/core#httpget"/>
>      <vos:view uri="ivo://my-registry/vospace/views#tar">
>          <vos:param
> uri="ivo://my-registry/vospace/views#tar/include">file-one</vos:param>
>          <vos:param
> uri="ivo://my-registry/vospace/views#tar/include">path/file-two</vos:param>
>          <vos:param
>
> uri="ivo://my-registry/vospace/views#tar/include">path/child-container</vos:param>
>          <vos:param
>
> uri="ivo://my-registry/vospace/views#tar/exclude">path/child-container/file-three</vos:param>
>      </vos:view>
>
> Which translates as a request for a HTTP GET endpoint to download a tar
> file containing 'file-one', 'path/file-two', plus all the contents of
> 'path/child-container' apart from 'path/child-container/file-three'.
>
> We could add wild cards to the mix, probably by defining a subset of the
> Unix glob syntax.
> https://en.wikipedia.org/wiki/Glob_(programming)
>
> Hope this helps,
> -- Dave
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.ivoa.net/pipermail/grid/attachments/20210621/faee7763/attachment.html>