Question about VOSpace and cold storage

[Zorba, Sonia]

Dear all,
I'm working with my colleagues on a VOSpace implementation where part of
the data has to be retrieved from a cold storage (tape library).

The data has to be copied from the tape to a disk and then deleted from the
disk after the user downloaded it. The idea is that the structure of files
and folders is always displayed by the VOSpace, but files can't be
downloaded until they have been copied to the disk. A cleanup job runs
periodically to remove the copied files after some days, in order to free
space on the disk.

We added an extra property into the node for specifying if the file is
already available or it has to be retrieved from the tape library and we
thought to use the pullToVoSpace operation for performing the copy.

VOSpace specification says that the pullToVoSpace operation is used to
"import data into a VOSpace node", and this use case seems to fit well in
this definition.

However the job requires to specify a protocol to use, but in this case it
is an internal operation, so we thought to use a custom protocol with no
endpoints, for example:

<vos:transfer xmlns:vos="http://www.ivoa.net/xml/VOSpace/v2.0"
version="2.1">
  <vos:target>vos://example.com!vospace/mydata1</vos:target>
  <vos:direction>pullToVoSpace</vos:direction>
  <vos:protocol uri="urn:tape-recall"></vos:protocol>
</vos:transfer>

In the beginning we thought to perform this operation as part of the
asynchronous download (pullFromVoSpace), but in that case the job had to
stay in the pending phase until the files were copied to the disk, with no
possibility to check the status of the copy. Rethinking about it, it was
clear that this copy operation is conceptually different from the download,
so it has to be handled as a separate job, that is preparatory for the
download. In that case using the pullToVoSpace operation seemed more
appropriate.

Do you think that this is the correct approach to use? Did someone build
something similar?

Cheers,
Sonia
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.ivoa.net/pipermail/grid/attachments/20201210/7174f05e/attachment.html>