Proposals for VOSpace (multiple targets)

[Zorba, Sonia]

Hi Dave,
thanks for all your replies and for creating a dedicated thread for each
discussion point.

I agree that allowing multiple targets increases complexity, however I
don't think this requirement is so specific to our use case, so I'll try to
insist a bit more on it.

Probably there are cases where having multiple targets doesn't make much
sense, so the service developers should add some checks and return proper
exceptions for those cases.
For example, by default a pullFromVoSpace operation could accept only one
target. However, a pullFromVoSpace operation with a view that returns a
tar/zip archive would make sense.

     <vos:target>vos://..../file-one</vos:target>
     <vos:target>vos://..../file-two</vos:target>
     <vos:direction>pullFromVoSpace</vos:direction>
     <vos:protocol uri="ivo://ivoa.net/vospace/core#httpget"/>
     <vos:view uri="ivo://my-registry/vospace/views#tar" />

Other file storage services (e.g. Google Drive or Owncloud) allow selecting
multiple files in the same folder and downloading them as a zip. I think
users are very used to this feature and expect to be able to do something
like this if they use the service from a UI.

We had already implemented the multiple targets support as you are
suggesting (by creating a new container node containing a reference to all
the selected nodes and then performing the pullFromVoSpace operation on
it), however we are not satisfied by this solution, because it brings
additional complexity too:

- the node exists just to support the transfer operation and has to be
deleted at the end of it;
- during the operations the user would see some unexpected nodes on his/her
folders, so we had to add an extra property to hide them from the UI; these
nodes have no meaning for the user, they just exist as a workaround to
allow multiple targets operations; it seems like an internal implementation
detail that we have to hide;
- there are many different HTTP calls just for starting a single operation
and this visibly slow down the UI, making the user experience worse.

The standard allows the service to offer multiple protocol endpoints for
> a single transfer. If we added support for multiple targets, then we
> would need to define how to link the different targets to the different
> endpoints.
>

If a pullFromVoSpace operation accepts multiple targets only when its
result is a single entity (like in the tar archive case) there is no
problem. Otherwise, I agree with you that handling the link between each
target and its endpoints sounds quite complex. We hadn't considered this
particular case, so thanks for starting a discussion about it.

If we added support for multiple targets, would this include targets
> from different locations? If not, then we would need to define how we
> would prevent them.


>      <vos:targets>
>          <!--+
>              | This is a target in the CADC VOSpace.
>              +-->
>          <vos:target>vos://CADC/..../file-one</vos:target>
>          <!--+
>              | This is a target in the INAF VOSpace.
>              +-->
>          <vos:target>vos://INAF/..../file-two</vos:target>
>

What if the target is single but it is a container node containing nodes
from different locations? I think that this is already allowed by the
standard and implementations should already be able to handle these cases.
I don't think this problem is specific to the multiple targets case.

It seems that CADC VOSpace UI already provides the possibility to select
multiple files and download a zip archive. I'm curious to know how they
handled that use case.

In any case I think that allowing multiple targets could be an optional
feature, so only facilities that really need it have to do the extra work
that this feature requires.

Cheers,
Sonia
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.ivoa.net/pipermail/grid/attachments/20210617/c6cc96f3/attachment-0001.html>