Hotwired 3 TDIG Breakout

[Frederic V. Hessman]

The fundamental problem isn’t to zip or not to zip (gzipped tar files should be just as good).

The problem is that the “manifest" was supposed to contain basic info normally present in a complete VOEvent document so that the packaged events could reference it and hence be smaller.  This is an understandable approach from the perspective of the LSST but means that the multiple original events issued in such a package cannot be fully viewed outside of the package without the manifest.

This suggests to me that we should include a generic “include” mechanism for VOEvents:  the LSST promises to maintain it’s own list of referenced content so that a stripped down event can be reconstituted without reference to the package in which it was distributed.

However, I took the standard VOEvent example and looked to see how to trim it down to it’s bare-bones size, also with the aid of external references, but frankly, I don’t see how much gain is to be had.  It would be nice to have a brute force complete LSST VOEvent to play with….

Is isn’t the sending of packages of complete VOEvents in a zip file enough data compression?   For this, we don’t need anything new.
Rick


On 16 Nov 2013, at 13:14, Roy Williams <roy at caltech.edu> wrote:

> OK I think I see now. There are two kinds of container here:
> 
> (1) How to put a single VOEvent together with images and other binary content. Zip would be a good way to do this, with images and other files referenced by file:// links. It could have a suffix .xmlx for example.
> 
> (2) How to collect together a lot of (1) into a single big object for data transfer. Here we can have a manifest.xml which is data about the collection itself, common metadata etc, together with all the .xmlx files. Then all these can be zipped up into a .xmlxx file.
> 
> Is that what we agreed?
> Roy
> 
> On 11/16/13 11:30 AM, Matthew Graham wrote:
>> Hi,
>> 
>> I agree with John - people wanted some sort of structure to collect
>> common metadata. Tim Jenness suggested looking at epub which is zip
>> plus a structured inventory.
>> 
>> Cheers,
>> 
>> Matthew
>> 
>> On Nov 16, 2013, at 10:46 AM, Roy Williams wrote:
>> 
>>> 
>>> On 11/15/13 5:29 PM, John Swinbank wrote:
>>>> - Mike introduced his proposal for a “VOEventContainer”:
>>>> essentially a means of bundling multiple VOEvents together with
>>>> supporting data such as images into a single entity. This could
>>>> address both issues surrounding bulk transport as well as the
>>>> stated aim of the LSST folks to include cut-out images with their
>>>> events. The proposal received a positive response from the
>>>> audience, but there was some quibbling over technical details.
>>>> Mike will introduce his proposal and kick off a discussion as to
>>>> its implementation on the mailing list.
>>> 
>>> I heard the opposite of this. My impression was that there was a
>>> rejection of the idea of inventing a special IVOA format for
>>> bundling content. I heard a consensus that *zip* is a perfectly
>>> good way to bundle multiple events, to bundle events with binary
>>> content such as images. That zip is well known, widely implemented,
>>> and trusted. It is the solution used in similar cases, such as epub
>>> and docx format, where an XML document is bundled with images. Try
>>> it yourself: cp document.docx document.zip unzip document.zip What
>>> I heard is that this is a solved problem, and there is no need to
>>> invent anything new for bundling XML files and their images.
>>> 
>>> Roy
>>> 
>>> --- Caltech LIGO roy at caltech.edu 626 395 3670
>>> 
>> 
> 
> -- 
> ---
> Caltech LIGO
> roy at caltech.edu
> 626 395 3670

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.ivoa.net/pipermail/voevent/attachments/20131116/5e5d1b6f/attachment.html>