[nvo-techwg] Time series data

[Rob Seaman]

Hi Doug, etc.,

> The issue of time series data came up briefly in the NVO telecon  
> today.
> Since all the attention has been focused on spectra little has been
> done about this yet, although the Spectrum/SED data model upon which
> SSA is based was always intended to be general enough to support time
> series data as well.  Both are spectrophotometric sequences, with
> the spectral coordinate varying in one case, and time in the other.

There's a lot of interest in VOEvent in seeing a practical time  
series format emerge soon.  Several other action items (e.g., XML  
signatures for authentication, and the representation of orbits) came  
out of the recent "Hotwired" workshop, confirming that VOEvent will  
continue to have a large cross-section with many other (I and N) VO  
standards and activities.

In this case, I wonder whether "spectrophotometric" only begins to  
cover the pertinence of time series for sky transient - and time  
domain, in general - alerts.  In addition to contribution from gamma  
rays to long wavelength radio waves, we had two talks on neutrino  
telescopes, several on classification schemes including ways to  
recognize signals embedded in sequences of temporally varying  
measurements of all types, coincidence testing, etc.  We take the  
meaning of the word "event" very liberally.

Pragmatically, the piratical individuals engaged in VOEvent and HTN  
related projects just want a time series standard that is  
utilitarian, simple, yet flexible.  They won't have infinite patience  
waiting for it to arrive, however.  To gauge both the completeness  
and elegance of an acceptable standard, the consensus would likely be  
that STC resides somewhere outside our comfort zone.  I doubt we  
could be motivated to accept such a complicated sub-schema again.

> It was mentioned that VizieR is a major resource for light curves
> so I had a look to get a better feel for current practice.  What we
> have now for SSA could probably already be used for light curves such
> as we see in VizieR, but there are two areas where it could stand
> improvement for this data.  The support for photometric systems could
> be better - but we need this for spectra and images as well.  This was
> a major topic of dicussion at the recent Spectroscopy in VO workshop
> in Madrid for example.

If this is an ongoing debate, I'd suggest that a VOEvent-friendly  
prototype would specify a small number of standard systems that could  
be expanded later.

> The second thing is that it would be good to
> be able to have multiple flux values (photometric systems or filters)
> associated with each time value, as time series data often measures
> the object in multiple standard filters/bandpasses/photometric systems
> for each time sample.

Yes.  Something like multiple <params> per time step.

> A final issue is segmentation, as large time
> series taken over a long period of time are often segmented, with
> big time caps between the segments.

Yes, although VOEvent's follow-up citations provide one way of doing  
this.

> All of these issues came up in connection with SSA and spectra as
> well, but they were second order features (for spectra) and lower
> priority for the first version of SSA, hence were deferred.  But with
> the completion of SSA and the Spectrum data model, Characterization,
> etc., we are probably 90% of the way there already.

I guess the question I have is whether that 90% is sufficient for a  
functional prototype.  Alternately, how long would the remaining 10%  
take to converge?  My sense of the debate at last week's workshop is  
that a VOEvent consensus on a 90% solution could be ready in a few  
week's, not months.  This seems much more aggressive than a similar  
consensus for SSA, especially as you are using words like "lower  
priority" and "deferred".

> In terms of the data interface, it appears that a SSA service could be
> trivially modified to support time series data.   The query interface
> would probably work as-is.  For simple light curves, CSV/TSV output
> would probably be popular, and is essentially what existing services
> such as VizieR return now.  HTML or JPEG for a direct graphical view
> is also popular, and VOTable would be ideal for serious analysis;
> again this is much the same as for simple 1-D spectra.

We're a publication format.  Some of the things you mention could be  
included as external references, but really we're just going to want  
to define some XML element to be embedded in our packets.  I would  
imagine these will include VOTable-like <params>.  If alternate  
representations are needed for purposes outside VOEvent, it seems to  
me we just need to ensure that our quasi-human readable packet format  
maps onto an underlying common time series data model.  In that case,  
I would suggest that the VOEvent time series use case should be  
straightforward enough that the broader VO time series DM could and  
should simply support this.

If the SED DM is mature enough to start scribbling possible explicit  
time series representations, VOEvent would be delighted to evaluate  
these and comment.  If not, we will be tendering a simple, packet  
oriented representation of our own for other groups to comment.  In  
either case, we will be seeking a VOEvent related consensus quickly.

Rob