Use case analysis for handling data cubes in VO

[Anita Richards]

> Ian McCrea from RAL brought up an interesting point at an AstroGrid solar / 
> STP meeting yesterday; in some STP time series, the time intervals within a 
> single time series might not all be the same length. (For example, point 0 at 
> 0 seconds, point 1 at 10 seconds, point 2 at 47 seconds, etc). There's 
> potentially a case for the same thing with SDO data in 3 years' time  as one 
> of the instruments can optionally take data at half cadence (but only half 
> the CCD pixels are read), or observers can choose 8 out of 10 channels in any 
> one observation cycle. Either case would lead to weird time series data for a 
> specific channel on this SDO instrument.
>

The sampling on any axis can be irregular -

Spatial - adaptive smoothing

Spectral - an SED or even datacube built up from a series of observations 
in different filters/bands (e.g. quite common to produce a cube of radio 
continuum flux density at 1.4, 5, 8, 15 .. GHz in order to make a spectral 
index map, or of pol. angle at each freq to produce Faraday rotation 
maps...)

Time  - as above, or the maser movie I pointed to and many other such 
observations (probably also Igor's example?) where data are taken whenever 
the telecope is available.  These may be interpolated onto a regular grid 
before data release, or thy may not.

The Characterzation model tries to allow for all such eventualities and it 
would be useful to know if it is adequate for the sorts of Solar as well 
as astronomical 3D data under discussion.

This discussion did start as a more specific and immediately practical 
consideration of access to 3D/ND data, which raises an interesting 
question of how the detail is represented in the data itself.  For 
example, in a radio data cube with spectral points at 1.4, 5, 8, 15 .. 
GHz, the frequency axis in the FITS header will probably be just a counter 
(planes 1, 2, 3, 4...) with the true frequencies hidden in the history, if 
anywhere. Does Doug or anyone know a better way actually in present use? 
Additional metdata is needed to make them accessible to a discovery tool, 
or to most processing tools (apart from custom tools which can dive into 
histories..).  How are the other cases handled?

My feeling is that even if it is too difficult to use the DAL model to 
handle full details of irreguarly sampled data, the simplest solution - 
just to begin with - would be to supply the typical sampling interval on 
each axis, and/or the number of samples, with a flag for whether the 
sampling was regular or not (leaving the fuller levels of detail to Char). 
That would at least allow data discovery and the user could seek extra 
information manually if necessary (just as I often have to follow up 
references from Vizier to find the resolution of tabulated data, for 
example).  That would require having a firm link to the provenance of any 
data.


best wishes

Anita

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Dr. Anita M. S. Richards, AstroGrid Astronomer
MERLIN/VLBI National Facility, University of Manchester, 
Jodrell Bank Observatory, Macclesfield, Cheshire SK11 9DL, U.K. 
tel +44 (0)1477 572683 (direct); 571321 (switchboard); 571618 (fax).