Hotwired 3 TDIG Breakout

[John Swinbank]

Dear Tim, all,

On 19 Nov 2013, at 18:14 , Tim Jenness <tjenness at cornell.edu> wrote:

> I’ve just managed (finally) to get myself subscribed to this list

Welcome!

> On Nov 18, 2013, at 22:17 , Rob Seaman <seaman at noao.edu> wrote:
> 
>> Good comments so far.  Suggest focusing on the use cases up front.  Without speculating on solutions, suggest that they be kept general purpose.  Would not assume that tens of thousands of transients per LSST visit must correspond to tens of thousands of VOEvent alert packets (though might correspond to that number of cutouts or other per-event data structures - on the other hand, one could amalgamate the cutouts into a data cube).
>> 
> 
> One thing that has to be handled is the case of a single event that comes with cutouts and other support information. Is that handled by VOEvent v3 or is that handled as a VOEventContainer with a VOEvent v2 description referring to supporting binaries. 

Right — in fact, perhaps the mention of “VOEvent v3” is the key question here, together with Rob’s emphasis on use cases.

I really like the VOEventContainer concept, as it potentially solves a couple of the big issues that the LSST and others have raised: how to transport large numbers of events, and how to bundle additional data with them.

However, I’m wondering if this is actually a pressing issue for the community now, rather than next decade. Is there anybody who feels like event containers should be an urgent item on the roadmap — perhaps even to the extent that they are willing to take the lead on defining a standard?

If not, then perhaps they should go on the back burner for now. I guess that the ultimate form of an event container will be shaped both by the requirements of LSST and other high-volume event producers and consumers, and by the form that the VOEvent standard itself takes at the time we are rolling out containers. For example, if we introduced an alternative event serialization, or a signature standard, or similar, which arguably might more directly address current use cases, that might later go on to inform the design of an eventual container standard.

I’m obviously reluctant to knock an interesting discussion on its head — I definitely think it’s worth talking through these issues now, and I’d love to hear from anybody who has a need for event containers before LSST comes on the scene. 

[…]

> Yes. Putting the cutout in the thing you ship around the place takes up more network traffic but has many benefits downstream. It also allows the cutout on the original server to be moved around without having to guarantee a URL will work forever.

Can you expand on the “many benefits”? I take the pointing about upstream not needing to guarantee a persistent URL, but what other considerations are important here? Are folks planning to automatically analyse cut-out images to inform their follow-up efforts? If so, I imagine latency is a consideration; obviously, that’s much less of an issue if they are basically indented for human consumption. Anything else?

Cheers,

John