The busy attribute on a node

[Dave Morris]

Hiya,

On 2016-04-26 23:16, Matthew Graham wrote:
> 
> The second issue is that this means the VOSpace node featured in a
> pushToVoSpace or pullToVoSpace operation should have its busy flag set
> to true whilst the operation is in progress and this is not explicitly
> stated under these operations. This would then have the consequence
> that any pullFromVoSpace/pushFromVoSpace operation using such a node
> should fail.
> 

Could we change that last sentence to make it a bit more flexible.

Suggestion :

"This would then have the consequence that *some* 
pullFromVoSpace/pushFromVoSpace operations using such a node *may* 
fail."

"It is up to each specific transfer protocol or view to define how they 
behave when invoked on a busy node."

----
Use case #1

Definition:
"Simple HTTP GET would not be valid for a busy node."

----
Use case #2

If we defined a PaginatedProtocol that supported pagination for tabular 
data (one wiki page defining four property URI's would probably cover 
it), then while a long running database query is in progress, the node 
for the query results would be 'busy' and the complete data set would 
not available, but if we know it has already generated 1000 rows then a 
PaginatedProtocol transfer request for the first 10 rows would be valid.

Definition:
"A PaginatedProtocol transfer request would be valid for a busy node, if 
the requested page is within the range of available data."

----
Use case #3

An important VOEvent is broadcast that refers to a tar.gz archive of 
background reference material and suddenly everyone wants a copy of it. 
Using something like BitTorrent might be the best way to distribute the 
tar.gz to multiple destinations.

If we are using something like BitTorrent for the transfer protocol, 
then it would be valid to start a pull request from a node that does not 
itself have all of the data yet and is still busy pulling the data in 
from somewhere else.

Definition:
"A BitTorrentLike transfer request would be valid for a busy node, if 
the list of blocks and checksums is available."

----
Should we add something to the specification to say:

"During a protocol negotiation, a node should only offer transfer 
protocols that are valid at the point when the negotiation is taking 
place."

In other words, a busy node should not offer a transfer protocol that it 
knows would fail while the node is busy.

In the example of a long running query:

While the query is running the node should not offer simple HTTP GET, 
because it would fail.

The node may offer PaginatedProtocol while the query is running, but it 
can only offer simple HTTP GET once the query has finished and all the 
data is in place.

In the example of a BitTorrent archive:

While the inbound transfer is running the node should not offer simple 
HTTP GET, because it would fail.

The node may offer BitTorrentLike transfer while the inbound transfer is 
running, but it can only offer simple HTTP GET once the inbound transfer 
has finished and the whole of the tar.gz archive is available.



--------
Dave Morris
Software Developer
Wide Field Astronomy Unit
Institute for Astronomy
University of Edinburgh
--------