VOSpace vs WebDAV

[Matthew Graham]

Hi,

Here's my tuppence' worth as a GWS greybeard: in terms of core functionality, VOSpace was largely defined in 2007/2008 and has remained unchanged for at least 5 years: VOSpace 2.0 (March 2013) was defined to be the REST equivalent of SOAP-based VOSpace 1.15 (October 2009). Unfortunately, it can be argued that we were too ahead of the game (for example. see the ADASS talk from 2007 about using VOSpace to abstract VMs). It is therefore quite conceivable that there are other external interfaces/services, designed to achieve similar goals, such as distributed storage environments, that have had more development cycles in recent years and are now more suitable for the current data environment. When implementing a storage solution, I would therefore recommend that they should certainly be considered but ultimately it should be the repository use cases which inform any selection criterion.
 
	Cheers,

	Matthew

On May 18, 2014, at 6:44 PM, Walter Landry wrote:

> Brian Major <majorb at nrc-cnrc.gc.ca> wrote:
>> Hi everyone,
>> 
>> Here at the CADC we have had similar talks about VOSpace, but not
>> necessarily about how it compares to WebDAV.  A question we are trying to
>> answer is "What does VOSpace offer for astronomers?"  Indeed, there is very
>> little in the specification that is specific to astronomy.  It is a
>> well-designed, general-purpose virtual storage system specification.
>> However, there are some things to consider:
>> 
>> - There is real value in VOSpace Views.  This feature allows the CADC to
>> deliver to the user only the bytes in which they are interested, not just
>> whole files.
> 
> Thank you.  I had forgotten about this aspect of VOSpace.  I agree
> that a sampling and conversion service is a useful thing to have.  In
> fact, that is what the AccessData standard describes.  So I would
> think it would be best to adopt AccessData for this kind of operation
> separately from the data store.
> 
>> - The specification has been flexible enough to allow the CADC
>> implementation to have customizations and optimizations in our supporting
>> storage systems that may be running on heterogeneous infrastructures.  If a
>> lower-level standard is adopted will there be such flexibility?
> 
> Just looking at WebDAV, it has even fewer requirements on the server
> than VOSpace.  I do not think this is a problem.
> 
>> That said, I agree with Dave here--I think it's time to step back and look
>> at the big picture and the role and look of VOSpace in the future.
>> 
>> Here are some technologies/articles you may find interesting that pertain
>> to this discussion:
>> 
>> - A federated storage system build on WebDAV: Dynamic Federations  (
>> http://www.dynamicfederation.org/DynFed/Welcome.html)
>> - An similar article on that here: https://indico.cern.ch/event/218328/  I
>> think you can substitute "HEP" with "Astronomy".
> 
> For those who have not looked at these HEP slides, they have
> implemented a service with WebDAV plus "extras, WAN transfers, 3rd
> party copy...".  So I looked at the way they support 3rd party copies
> 
>  https://svnweb.cern.ch/trac/lcgdm/wiki/Dpm/WebDAV/Extensions
> 
> I do not see support for asynchronous transfers.  Bummer.  My guess is
> that they decided it was too much work to do it properly.
> 
> Cheers,
> Walter Landry
>