0MQ: The Intelligent Transport Layer

[Dick Shaw]

On Fri, 6 Jan 2012 12:31:37 +0100
  John Swinbank <swinbank at transientskp.org> wrote:
> However, I don't think we can avoid the issue of bulk transport. I assume 
>that the LSST folks (for example) have done their homework, and really will 
>produce a couple of million events which are worthy of distribution per 
>night. They will then face the problem of shifting them down a high-latency, 
>low-bandwidth line. They still need a "super truck" capable of that, even if 
>it's only to carry letters to a sorting office.

We LSST folks have been told by our community that they want access to every 
notification, and that the LSST project should not be the judge of what events 
are "worthy." Tagging events with hints about the nature of the source, etc. 
is ok, but not pre-filtering. Also, building the "truck" is not within the 
current LSST scope. This is not to say that the transport mechanisms are of no 
concern--quite the contrary. I think there is a need for a system of brokers, 
filters, classifiers, and information aggregators to help the community select 
what is most interesting from a science perspective, and to distill the 
information in a way that is both manageable and timely for the end-user. Put 
another way, usage patterns matter a lot when designing the transport 
capabilities of the system.

Cheers,
Dick

#---
Dr. Richard A. Shaw, Scientist
National Optical Astronomy Observatory
950 N. Cherry Avenue
Tucson, AZ  85719
USA
520-318-8398
http://www.noao.edu/noao/staff/shaw/