The busy attribute on a node

[Patrick Dowler]

What Dave said!

There is also the issue when overwriting existing data. Some systems
let you get the old version of the file until the write finishes (or
even later for eventual consistency systems). I'm not sure if we want
to prescribe the behaviour at this level of detail: does vospace
provide strong consistency or is it allowed to not? If it is allowed
to not, how would we convey that to clients/users who really do have
to know this to use a service safely?

We took the safe route: mark the node busy until the write is
completed, thus blocking all other reads and writes.

PS-Agree on the value of true bug in the spec.

Pat

On 27 April 2016 at 07:20, Dave Morris <dave.morris at metagrid.co.uk> wrote:
> Hiya,
>
> On 2016-04-26 23:16, Matthew Graham wrote:
>>
>>
>> The second issue is that this means the VOSpace node featured in a
>> pushToVoSpace or pullToVoSpace operation should have its busy flag set
>> to true whilst the operation is in progress and this is not explicitly
>> stated under these operations. This would then have the consequence
>> that any pullFromVoSpace/pushFromVoSpace operation using such a node
>> should fail.
>>
>
> Could we change that last sentence to make it a bit more flexible.
>
> Suggestion :
>
> "This would then have the consequence that *some*
> pullFromVoSpace/pushFromVoSpace operations using such a node *may* fail."
>
> "It is up to each specific transfer protocol or view to define how they
> behave when invoked on a busy node."
>
> ----
> Use case #1
>
> Definition:
> "Simple HTTP GET would not be valid for a busy node."
>
> ----
> Use case #2
>
> If we defined a PaginatedProtocol that supported pagination for tabular data
> (one wiki page defining four property URI's would probably cover it), then
> while a long running database query is in progress, the node for the query
> results would be 'busy' and the complete data set would not available, but
> if we know it has already generated 1000 rows then a PaginatedProtocol
> transfer request for the first 10 rows would be valid.
>
> Definition:
> "A PaginatedProtocol transfer request would be valid for a busy node, if the
> requested page is within the range of available data."
>
> ----
> Use case #3
>
> An important VOEvent is broadcast that refers to a tar.gz archive of
> background reference material and suddenly everyone wants a copy of it.
> Using something like BitTorrent might be the best way to distribute the
> tar.gz to multiple destinations.
>
> If we are using something like BitTorrent for the transfer protocol, then it
> would be valid to start a pull request from a node that does not itself have
> all of the data yet and is still busy pulling the data in from somewhere
> else.
>
> Definition:
> "A BitTorrentLike transfer request would be valid for a busy node, if the
> list of blocks and checksums is available."
>
> ----
> Should we add something to the specification to say:
>
> "During a protocol negotiation, a node should only offer transfer protocols
> that are valid at the point when the negotiation is taking place."
>
> In other words, a busy node should not offer a transfer protocol that it
> knows would fail while the node is busy.
>
> In the example of a long running query:
>
> While the query is running the node should not offer simple HTTP GET,
> because it would fail.
>
> The node may offer PaginatedProtocol while the query is running, but it can
> only offer simple HTTP GET once the query has finished and all the data is
> in place.
>
> In the example of a BitTorrent archive:
>
> While the inbound transfer is running the node should not offer simple HTTP
> GET, because it would fail.
>
> The node may offer BitTorrentLike transfer while the inbound transfer is
> running, but it can only offer simple HTTP GET once the inbound transfer has
> finished and the whole of the tar.gz archive is available.
>
>
>
> --------
> Dave Morris
> Software Developer
> Wide Field Astronomy Unit
> Institute for Astronomy
> University of Edinburgh
> --------
>



-- 
Patrick Dowler
Canadian Astronomy Data Centre
Victoria, BC, Canada