VOSpace vs WebDAV

[Brian Major]

Hi everyone,

Here at the CADC we have had similar talks about VOSpace, but not
necessarily about how it compares to WebDAV.  A question we are trying to
answer is "What does VOSpace offer for astronomers?"  Indeed, there is very
little in the specification that is specific to astronomy.  It is a
well-designed, general-purpose virtual storage system specification.
 However, there are some things to consider:

- There is real value in VOSpace Views.  This feature allows the CADC to
deliver to the user only the bytes in which they are interested, not just
whole files.

- The specification has been flexible enough to allow the CADC
implementation to have customizations and optimizations in our supporting
storage systems that may be running on heterogeneous infrastructures.  If a
lower-level standard is adopted will there be such flexibility?

That said, I agree with Dave here--I think it's time to step back and look
at the big picture and the role and look of VOSpace in the future.

Here are some technologies/articles you may find interesting that pertain
to this discussion:

- A federated storage system build on WebDAV: Dynamic Federations  (
http://www.dynamicfederation.org/DynFed/Welcome.html)
- An similar article on that here: https://indico.cern.ch/event/218328/  I
think you can substitute "HEP" with "Astronomy".
- The object storage system ceph:  (http://ceph.com/)
- A possible alternative to VOSpace Views: The JPEG 2000 Interactive
Protocol (http://www.jpeg.org/jpeg2000/j2kpart9.html) and analysis from
Andreas here: http://arxiv.org/abs/1307.5123

Brian


On Thu, May 15, 2014 at 10:48 AM, Dave Morris <dave.morris at metagrid.co.uk>wrote:

> Hi Walter,
>
> I am one of the authors of the original VOSpace specification, along
> with Paul Harrison, Matthew Graham and Guy Rixon.
>
> Your comparison of the main features of WebDAV and VOSpace are correct.
>
> If what your users want is to be able to drag and drop files from their
> desktop to a remote folder using a standard file browser, then WebDAv is
> indeed probably the best protocol to use. In fact, implementing the full
> VOSpace specification will probably just get in the way. WebDav covers
> everything you need and the many existing 3rd party implementations mean
> it is a lot simpler to deploy.
>
> You are right that asynchronous 3rd party transfers are responsible for
> much of the complexity in the VOSpace specification.
>
> At the time, interoperable asynchronous 3rd party transfers was one of
> the reasons for developing the VOSpace specification. One of the aims of
> our project was to be able to process as much as possible 'in the cloud'
> and only transfer the final processed results to the the client desktop.
> As a result, a lot of the work on the VOSpace specification was designed
> to handle server to server transfers, rather than server to client.
>
>
> Your analysis of vos: URI scheme is correct, part of its function is
> indeed to act as a name resolver service, similar to DNS.
>
> Very similar in fact to the 'Name Mapping Authority Host' in the ARK
> scheme that Norman described. Although the steps involved in resolving
> the service URL via the registry are a bit more complex than they should
> be be due to limitations in the ivo: registry URI scheme.
>
>
> Regarding the suggestion of replacing VOSpace with WebDAV plus
> extensions.
>
> I agree - it probably is time for a fresh look at the problem, a lot has
> happened in the last 10 years (back in 2004 we still thought SOAP was a
> good idea). However, if it really is a re-write from scratch, then to
> avoid confusion and interoperability issues we should probably call it
> by a different name and use a different URI prefix.
>
> Regards,
> Dave
>
> --------
> Dave Morris
> Software Developer
> Wide Field Astronomy Unit
> Institute for Astronomy
> University of Edinburgh
> --------
>
>
>
>
> On 2014-05-15 06:19, Walter Landry wrote:
> > Hello Everyone,
> >
> > At our datacenter, we want to implement some kind of workspace for our
> > users.  So the natural thing to do is to look at VOSpace and see if it
> > could be bent to our needs.  However, I am also familiar with WebDAV
> > [1],
> > and I am having a hard time understanding what advantages VOSpace
> > brings.  Just to run down the basics about WebDAV:
> >
> > 1) WebDAV supports much of the functionality in VOSpace, including
> >    all of these items lifted directly from the VOSpace introduction
> >
> >     * add or delete data objects
> >     * manipulate metadata for the data objects
> >     * obtain URIs through which the content of the data objects can be
> >       accessed
> >
> > 2) WebDAV is a mature, established standard deployed worldwide on a
> >    variety of machines.  Every major desktop OS has WebDAV built in,
> >    and you can get clients and servers for every operating system on
> >    almost any hardware, including phones, tablets, and supercomputers.
> >
> > 3) WebDAV has a number of implementations of the client and server in
> >    every programming language you can think of.
> >
> > 4) WebDAV is based on http, so it is easy to layer any of a number of
> >    authentication schemes on top.
> >
> > 5) WebDAV has an easily understood filesystem-like API.
> >
> > In contrast, the only compelling feature of VOSpace I can think of is
> > the ability to initiate 3rd party transfers.  But it would seem better
> > to add a trivial extension to WebDAV rather than creating a completely
> > new protocol.
> >
> > Moreover, the API for VOSpace is much more complicated, with a lot of
> > indirection.  The vos: URI scheme in particular feels like a
> > reinvention of DNS for a benefit that I do not see.  It almost goes
> > without saying that I have not seen much interest in VOSpace outside
> > of astronomy.
> >
> > Given all this, it feels like the best use of my time would be to
> > install mod_dav on my Apache server and be done in short order.  I
> > would guess that my users would even be happier than if I used
> > VOSpace.  It is pretty addictive to be able to drag and drop from your
> > desktop to a remote WebDAV folder using the standard file browser.
> >
> > Now, I am aware that this has been discussed before.  I have also
> > spoken personally with one of the authors of VOSpace.  Yet I still do
> > not quite see the need for VOSpace.  Could someone enlighten me?
> >
> > Thanks,
> > Walter Landry
> > wlandry at caltech.edu
> >
> > [1] http://webdav.org/
> > [2] http://www.ivoa.net/forum/grid/2007-March/001713.html
> >     http://www.us-vo.org/pipermail/techwg/2005-February/000793.html
>



-- 
Brian Major

Canadian Astronomy Data Centre
Centre canadien de données astronomiques

National Research Council Canada
Conseil national de recherches Canada
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.ivoa.net/pipermail/grid/attachments/20140515/7d4d421f/attachment-0001.html>