<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body bgcolor="#FFFFFF" text="#000000">
<p>Delighted to read all the replies. Not surprisingly there
continue to be divergent points of view. An IVOA note is precisely
the mechanism to provoke such a response.<br>
</p>
<p><break>make coffee</break></p>
<p>Won't reply to all the messages directly, but most of the
original key design points have been touched upon: diversity of
use cases, schema + code generation, transport, backwards
compatibility, triggering follow-up observations, time series,
tabular data, etc.</p>
<p>The current compromises were taken with full knowledge of future
projects like LSST, Gaia, SKA, and if not ZTF then PTF. The world
has moved on and perhaps some new issues have been revealed, but
most likely there has been a change of emphasis to prefer a binary
format. We always intended to support binary content.</p>
<p>The question with new formats like Avro is whether they are
mature enough. I share somebody's concern with the maturity of
JSON schema, but for that matter many of the problems with XML are
really problems with XML schema.</p>
<p>There are many ways for such a project to fail. But one of the
biggest risks would be starting over from scratch. VOEvent as a
technology project was flawed, as is IVOA in general. As a
community building exercise, VOEvent has been one of IVOA's crown
jewels. VOEvent has built a following among both professional and
amateur projects.</p>
<p>Perhaps some ground rules?</p>
<p>1) VOEvent-2 should remain as a stable standard. Ditto with the
current transport standard.<br>
</p>
<p>2) VOEvent-3 (in JSON and/or Avro) should seek to support
reversible translation from VOEvent-2 as much as possible. If
there are features of VOEvent-2 that cannot be supported, describe
them in a section of the new recommendation. Which is to say,
let's not introduce "simplifications" without overriding
motivation.<br>
</p>
<p>3) If there are new features of VOEvent-3 that cannot easily be
accommodated in native VOEvent-2, one imagines they might be
handled with references to additional formats in the same stream.
Or it may just be that to use embedded images you need to use
tools that understand VOEvent-3.</p>
<p>4) The language and intent of the normative VOEvent-2
specification is:</p>
<blockquote>
<p><i>"[T]he <WhereWhen> element may reference an STC
<ObsDataLocation> element to provide a sky location and
time for the event."</i><i><br>
</i></p>
<p><i>"The STC specification, in particular
<ObsDataLocation> and its contained elements, allows
more exotic coordinate systems (for example, describing
planetary surfaces). Further description of how VOEvent
packets might be constructed to convey such information to
subscribers is outside the scope of this document."</i></p>
</blockquote>
<p>The prototype schema is non-normative and makes simplifications,
but full support for all the richness of <ObsDataLocation>
was explicitly intended. STC is evolving and VOEvent-3 may have
different options. Having survived both VOEvent-1 and VOEvent-2
(and seeing that Arnold still has skin in the game), it is my
humble belief that the best way to avoid the complications of STC
is to surrender to them entirely. We don't have to support all of
STC, but the richness of the astronomical use cases - even for a
single facility like ZTF - requires all of
<ObsDataLocation>. Leave the question of how best to support
this in a JSON schema for somebody else and just incorporate it as
a subschema.</p>
<p>5a) Agree with Rick that VOEvent should continue to focus on
describing astronomical phenomena. Leave controlling
telescope/instrument behavior to another standard(s). Each type of
standard is fairly complex. Combining them would be more than
twice so. If RTML, for example, needs its own JSON realization,
that will surely succeed better alone.</p>
<p>5b) This may also apply to standards like ADES for astrometry.
ZTF needs to handle moving objects as well as it bumper crop of
kilonovae, but it doesn't need to handle them with the same
format. Consider how the various formats will interact in
workflows. Update the broker diagram and build the
bricks-and-mortar infrastructure.<br>
</p>
<p>6) If there is ever a VOEvent-4, suggest these meta-rules
(perhaps there are others) continue in force. Preserve both
VOEvent-2 and VOEvent-3 for backwards compatibility and make the
new standard as interchangeable as possible, but don't fret about
new features.<br>
</p>
<p>Rob</p>
<p>--<br>
</p>
<br>
<div class="moz-cite-prefix">On 10/17/17 3:24 AM, Roy Williams
wrote:<br>
</div>
<blockquote type="cite"
cite="mid:A44BF53AA08AE04889395D4D1A5797B19696B5@MERCURY.roe.ac.uk">
<pre wrap="">I would like to point out that there are two independent ideas in the Note that we circulated:
-- The idea of using JSON as an alternate format
-- The idea that Groups of Params and Tables can be used as data models
These are independent of each other. The data models can be just as well implemented in XML. I have not heard any complaints about the data models. Should I split the Note into two pieces for separate consideration?
Reasons for using JSON. Perhaps (5) is the most important, followed by (6), which is really a question.
(1) JSON and XML can be used together, and pretty much interchanged. Certainly I have had little trouble making an XML-to-JSON converter. However, I will say that we put in a few simplifications that might not convert back easily. For example instead of saying Name1=RA and C1=300 (the XML), we just say RA=300 (the JSON). This means that there is no longer a choice of name, so if you want to call it RiteAscenshun in XML, and it gets converted to JSON and back to XML, then it will come out being called RA. We have also defaulted the units to degrees and meters in the WhereWhen section.
(2) There is no longer the confusion over what is text and what is attributes that XML has. Instead of <tag att=hello>dolly</tag>, we convert to att=hello and tag=dolly.
(3) I like the way Tim refers to XML as Betamax. Many people I have talked to prefer JSON. It is a matter of taste I suppose.
(4) If we are encouraged by the IVOA, it is our intention to build a JSON schema that can be used to validate the content. Once there is a schema, the JSON is much more constrained. JSON can also be "schema-compliant"
(5) The future high-volume transient surveys are proposing to use a format called AVRO, that is very much like JSON, to transport event notices. There are technical efficiencies such as binary encoding, and in particular the ability to put binary values in the notice -- image cutouts. The AVRO schema is written in JSON, and is quite close to the JSON schema alluded in (4). Thus VOEvent is brought in line with the ZTF and LSST projects.
(6) If encouraged by the IVOA, we could shift attention from JSON and work directly with its cousin, AVRO.
Thank you for your attention
Roy (with Eric, Matthew, Rob, and Scott)
</pre>
</blockquote>
<br>
</body>
</html>