Draft VOEvent 2.1 => Updated VOEvent 2.0

Sat Nov 4 20:13:06 CET 2017

Hi Rob and others,

I think that we are all pursuing the same goal: using VOEvent in the really world. And that is good ! :-)
As you say Roy: any change of the standard takes way too long for this goal.
As you say Rob: the prose recommendation is normative. 

This said, I think we could consider my draft schema as a better implementation of the current recommendation, as it includes more STC elements than the current schema in use.
My proposal to name it version 2.1 is misleading, I agree. Let's forget this numbering now (see new title).

I just wanted to recall the frame in which we are working in Europlanet. The project started 2 years ago, and we have been investigating VOEvent as a protocol to send observations or predictions of solar and planetary events. Following the recommendation, as Rob said, that should work. Our immediate aim is to be able to use the current VOEvent software, that is currently using an XML schema to validate the messages that are distributed. 

We have neither the expertise, nor the manpower to build from scratch a software like "comet", which is doing the job pretty well, as far as we could check. Comet is using the VOEvent 2.0 XML schema to validate the messages. We have thus built an XML schema that includes a few extra pieces of STC that fit our needs. We have presented our proposal at the last 2 interop meetings, and now we try to come with a proposed solution that should not harm existing project using VOEvents and the comet software. 

I also fully understand that there are other options on the table, like the JSON format. I don't have any strong feeling against this: I don't really care about using XML or JSON, as far as we can use a working infrastructure. 

As said before, we need to be able to use VOEvent now, and we can't really wait for another implementation of a author/broker/subscriber software that may come at best in a year from now. 
We have two options on our side: 
- either we ignore the WhereWhen section of the VOEvent and put everything in the What section, with groups and params, mimicking the WhereWhen section for the STC implementation
- or we propose an updated WhereWhen section that includes the missing STC parts.

The first option was our initial path, and this is how we tested our services with comet up to now, but clearly, it is not consistent with the VOEvent data model. Why bother describing various event concepts into pieces like What, WhereWhen, How, Why, if we can put everything in What?… This is why we are now proposing an intermediate solution that allows us to stay into the VOEvent description model philosophy. 

All that said, and after Rob's comment, I guess that the draft sent a few days ago could qualify for an errata rather than a new version (but I'm thinking loud here…)

Sincerely
Baptiste

> Le 4 nov. 2017 à 16:51, Rob Seaman <seaman at lpl.arizona.edu> a écrit :
> 
> I suspect that anybody who has navigated a standard through the IVOA process would agree with Roy!
> Again, the prose recommendation is normative. Whatever the current XML schema says, VOEvent allows full STC expressions:
> 
> <WhereWhen>
>     <ObsDataLocation>
>         <ObservatoryLocation/>
>         <ObservationLocation/>
>     </ObsDataLocation>
> </WhereWhen>
> Personally, I think the schema should simply have continued to import STC in whole. But discussing this in terms of XML is now pointless. Let's migrate to JSON / AVRO and figure out how to realize the current recommendation in terms of the new paradigm. Review the list of currently important science use cases and make sure the new realization of the format, including additional features of STC if needed, supports them.
> Preserve the current XML VOEvent for backwards compatibility.
> 
> Rob
> --
> 
> On 11/3/17 6:43 AM, Roy Williams wrote:
>> Hi Baptiste
>> I appreciate that you wish to handle events that occur in more complex WhereWhen specifications, such as relative to other planets or satellites, or as a probability distribution (LIGO/Virgo).
>> 
>> In your document "VOEvent 2.0 Schema limitations", there are three problems stated with VOEvent 2.0, and one of those is about gravitational wave probability distributions. For this latter case, a solution already exists within the existing schema, and has been adopted. Perhaps there are similarly simple, flexible ways to handle the other two cases? If LIGO/Virgo can work with typed Groups, can't you too? Please explain why new and complex XML schema is a better solution.
>> 
>> The real problem though, lies with VOEvent itself being (a) tied to XML and (b) in a state of flux.
>> 
>> By extending the schema, you make it much more difficult to express VOEvent in other formats such as JSON or AVRO, formats more suitable for the upcoming ZTF and LSST event streams. However, if we convert the existing 2.0 schema to AVRO, we can have it all ready by the time these new event firehoses are switched on. But if you require the IVOA to approve a 2.1 schema, they will be arguing for the next two years, the standard will be seen as unfinished, and astronomers will not want to work with it. Therefore something else, something ad hoc, will be used for ZTF and LSST, and VOEvent will lose its progress.
>> 
>> Roy
>> 
>> ---
>> Royal Observatory Edinburgh
>> roy at roe.ac.uk <mailto:roy at roe.ac.uk>
>> 07542 869986
>> ________________________________
>> From: Baptiste Cecconi [baptiste.cecconi at obspm.fr <mailto:baptiste.cecconi at obspm.fr>]
>> Sent: 02 November 2017 4:53 PM
>> To: Roy Williams
>> Cc: voevent at ivoa.net <mailto:voevent at ivoa.net>
>> Subject: Re: Draft VOEvent 2.1
>> 
>> Hi Roy,
>> 
>> here are a few links to the IVOA wiki where you will find material for the case we have been raising. I’ll prepare a cleaner document in the coming days.
>> 
>> http://wiki.ivoa.net/internal/IVOA/InterOpMay2017-TD/lesidaner_interop_voevent.pdf <http://wiki.ivoa.net/internal/IVOA/InterOpMay2017-TD/lesidaner_interop_voevent.pdf>
>> 
>> http://wiki.ivoa.net/twiki/bin/view/IVOA/InterOpMay2017-TD-001 <http://wiki.ivoa.net/twiki/bin/view/IVOA/InterOpMay2017-TD-001>
>> 
>> http://wiki.ivoa.net/internal/IVOA/InterOpOct2017TDIG/lesidaner_interop_voevent.pdf <http://wiki.ivoa.net/internal/IVOA/InterOpOct2017TDIG/lesidaner_interop_voevent.pdf>
>> 
>> Sincerely
>> Baptiste
>> 
>> Le 2 nov. 2017 à 16:59, Roy Williams <roy at roe.ac.uk <mailto:roy at roe.ac.uk><mailto:roy at roe.ac.uk> <mailto:roy at roe.ac.uk>> a écrit :
>> 
>> Hi Baptiste
>> I find it difficult to understand what is being proposed from this list of changes, since I was not present at the discussions. Is there a presentation or document that explains what you are trying to achieve, and how your proposed changes achieve your goals? Have you considered using the VOEvent syntax itself without schema changes, with typed Groups? Can you provide an example event using your proposed new schema?
>> Thank you
>> Roy Williams
>> 
>> ---
>> Royal Observatory Edinburgh
>> roy at roe.ac.uk <mailto:roy at roe.ac.uk><mailto:roy at roe.ac.uk> <mailto:roy at roe.ac.uk>
>> 07542 869986
>> ________________________________
>> From: voevent-bounces at ivoa.net <mailto:voevent-bounces at ivoa.net><mailto:voevent-bounces at ivoa.net> <mailto:voevent-bounces at ivoa.net> [voevent-bounces at ivoa.net <mailto:voevent-bounces at ivoa.net><mailto:voevent-bounces at ivoa.net> <mailto:voevent-bounces at ivoa.net>] on behalf of Baptiste Cecconi [baptiste.cecconi at obspm.fr <mailto:baptiste.cecconi at obspm.fr><mailto:baptiste.cecconi at obspm.fr> <mailto:baptiste.cecconi at obspm.fr>]
>> Sent: 02 November 2017 3:20 PM
>> To: voevent at ivoa.net <mailto:voevent at ivoa.net><mailto:voevent at ivoa.net> <mailto:voevent at ivoa.net>
>> Subject: Draft VOEvent 2.1
>> 
>> Dear colleagues,
>> 
>> during the discussions at IVOA last week, Pierre Le Sidaner and I have mentioned our need for updates in the VOEvent 2.0 schema, in order to fit our needs in Solar and Planetary sciences. We have prepared a draft VOEvent 2.1, trying to make sure it stays fully compliant with VOEvent built with version 2.0 of the schema. We would be grateful to have your view on this proposed draft version. The main changes are recalled below (but are available as comments in the XML schema file):
>> 
>> Revision 2.1 2017/11/02 BaptisteCecconi
>> - added "AuthorURI" in "Who"
>> - removed restricted list of "AstroCoordSystem": previously "idValues" type, now "xs:string"
>> - removed "idValues" type (and references)
>> - added annotation in "AstroCoords/Time" (according to STC-1.30)
>> - added "AstroCoords/PositionName" as "xs:string"
>> - added annotation to "AstroCoords/Position2D" (according to STC-1.30)
>> - removed restricted list of attribute "coord_system_id" in "AstroCoords": previous type=idValues (removed), new type=xs:string
>> - added "TimeInterval" of type "TimeInterval" in "Time"
>> - added "TimeOrigin" of type "xs:string" in "TimeInstant"
>> - added "TimeInterval" type with "StartTime" and "StopTime" elements
>> - changed minOccurs attribute in "Position2D/Error2Radius": now set to 0
>> - added "Error2" element in "Position2D" with type "error2" and minOccurs=0
>> - added "Error3" element in "Position3D" with type "error3" and minOccurs=0
>> - added "coord_value" type introducing "ucd" and "unit" attributes for C1, C2 and C3 elements
>> - added "error2" type, with "C1" and "C2" elements
>> - added "error3" type, with "C1", "C2" and "C3" elements
>> - added "TimeFrameType" (taken from to STC-1.30)
>> - added "SpaceFrameType" (taken from to STC-1.30)
>> 
>> The presentations of Pierre Le Sidaner at the last 2 interop meetings contain the rationale for the changes, but we can prepare a more detailed document, if we decide to proceed.
>> 
>> Sincerely
>> Baptiste Cecconi
>> 
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.ivoa.net/pipermail/voevent/attachments/20171104/bb89ff10/attachment-0001.html>