Use case analysis for handling data cubes in VO

[Anita Richards]

Dear Doug, Silvia, Elizabeth, Phil,

Doug Tody recently presented a very interesting draft document describing 
access to 3-D/N-D Image Data in VOs
http://www.aoc.nrao.edu/~dtody/sia/3D-use-cases.pdf

Here are some suggestions for further consideration:

TIME SERIES

The draft appeals for more exampls of data, especially multi-dimensional 
time-series data, which is why this is copied to people who are experts in 
this.  I understand that one problem so far is the lack, in practical use, 
of standardised time formats or tools to convert between formats.  I can't 
believ that it is any more difficult than providing tools for converting 
between the various bizarre celestial coordinates, just that no-one has 
done it yet.

1) Solar data - see the AstroGrid Solar Movie Maker accessible via the 
AstroGrid Workbench. 
http://wiki.astrogrid.org/bin/view/Astrogrid/MovieMaker
is a slightly outdated stand-alone description, or use the current 
workbench from http://software.astrogrid.org
Elizabeth or Silvia can provide information about the technical details.

2) Maser movie - see
http://www.journals.uchicago.edu/ApJ/journal/issues/ApJ/v599n2/58473/58473.html
http://www.journals.uchicago.edu/ApJ/journal/issues/ApJ/v599n2/58473/video2.mpg

Phil can correct/add more details but as I understand it, this is a time 
series of all-channel squashes from monitoring of SiO masers (i.e. axes 
RA, Dec, Time).  The original data are actually a large number of 
observations at intervals of a few weeks, each of which have axes not only 
for RA, Dec, multi-channel Frequency, but also for polarization.  The 
image datacubes can have axes (RA, Dec, Freq) in which the 'observable' is 
flux density in total intensity, linear or circular polarization, Stokes 
parameters etc. or polarization angle. They can also have axes (RA, Dec, 
Velocity). In this example there is a unique mapping from frequency to 
velocity but in similar cubes there can be two or more molecular 
transitions within one band, i.e. more than one possible velocity axis. 
One can also present cubes with axes of RA, Dec, Stokes parameters, which 
could be used either combined to give polarized intensity and angle, or to 
view I, Q, U or especially V separately.

3D VISUALISATION

TopCat (www.starlink.ac.uk/topcat) now offers a number of 3D visualisation 
options for tabular data which recognises celestial coordinates etc. 
My feeling is that as we go to more or less than 2D data the distinction 
between tabular data and 'image' data is more and more blurred. On the one 
hand, many 1D spectra and time series are simply tables of freq (or time 
etc.) v. flux or other observable, possibly plus uncertainties. On the 
other hand, 3+D maser data or other observations of collections of compact 
sources are often presented as tables with headings something like
TIME RA DEC Bmaj BMin BPA VEL Iflux Qflux UFlux Vflux (errors.. ...)


CHARACTERISATION

These data have at least 5 types of axes, some of which are themselves 
multiple (Space, Time, Frequency, Velocity, Polarization). The problem is 
that there is not yet any easy way to visualise or analyse more than a 
selection at any one time.  Thus where the 3D Acess Draft says 'in 
practice data with more than 3 dimensions is rarely seen...' that is true 
literally, because tools don;t exist to visualise it, but not true in 
spirit, in that we do need to describe multi-dimensional data in order to 
extract any desired combination of axes.  The Characterisation model 
(draft at 
http://www.ivoa.net/internal/IVOA/IvoaDataModel/Characterisation_Note_20051129.pdf) 
should allow this, so we need to make sure that it meets the needs of the 
3D Access model - comments welcome to dm at ivoa.net

OTHER COMMENTS

Other comments on 3D draft:

Para 2
I am not sure why we need _uniformly_ sampled data, since in some cases 
(e.g. frequency/wavelength) this is only the case if you pick the right 
units.  In other cases - such as some of the time series examples given 
above - the original data may be taken irregularly in time and the 
extraction and visualisation tools may or may not interpolate onto a 
regular grid.  There is also the case of adaptively smoothed data.
As such tools do exist (or it may not always be necessary) I 
suggest that we leave out 'uniformly'.  The Characterisation model 
provides (possibly too much!) detailed methods for coping with various 
irregularities in sampling.

Access modes

Spectrum extraction - add - or an analogous extraction parallel to any 
other axis, e.g. to produce a light curve or variability with time.

We also need to consider extracting  3 dimensions from >3D data sets, 
including collapsing higher number axes - e.g. the collapse of each 
epoch's frequency axis in the maser movie example above.

The other common requirement (especially for ALMA!) is to convert velocity 
conventions and to generate a velocity access in response to a rest 
frequency and velocity convention.

Generic data set discovery

I will try and compare the proposals against some specific use cases and 
come up with words for this section and elswhere in the light of comments 
on what of the above is relevant/suitable at this stage.

best wishes

Anita

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Dr. Anita M. S. Richards, AstroGrid Astronomer
MERLIN/VLBI National Facility, University of Manchester, 
Jodrell Bank Observatory, Macclesfield, Cheshire SK11 9DL, U.K. 
tel +44 (0)1477 572683 (direct); 571321 (switchboard); 571618 (fax).