3D data model: plan

[Doug Tody]

Igor, Tim -

This looks like exactly what is needed at this point.  We need to
understand better the data we are dealing with, from IFU data to radio
spectral image cubes, and how we would like to deal with it for analysis.
Representative use-cases are a good way to address this.  We want to end
up with interfaces and tools which are multiwavelength and can be used
for all such data.

Some specific comments:

On Thu, 17 Feb 2005, Igor Chilingarian wrote:
> Today I discussed with Francois some questions concerning 3D data model

Remember, it is not just the abstract data model, we also need to consider
what is required for actual data access and analysis.  In fact I think
we should start with the latter, plus list the data instances we want to
support, and only at the end of this analysis try to define the required
data models or extensions to data models.

> Your idea about science case driven approach is a perfect proposition.
>
> I have a kind of plan for what to do for a moment:
>  1) Define clearly the science case for IFU data I'm working on: studies of
>      stellar populations in nearby galaxies using absorption-line spectra
>  2) During the next week hopefully we'll have two science cases formulated
>      for Fabry-Perot data: studies of the gaseous kinematics in disc
>      galaxies, and studies of gaseous shells (SN remnants) in nearby
>      galaxies. I asked scientits working with this type of data to describe
>      these cases.
>
> Each of these science cases will be presented as a short part of text (15-20
> lines) including scientific objectives and brief description of the scientific
> data processing with the requiremets about the descripition of the data.
> It should help to figure out the requirements for the 3D data model, and
> for 3D data access layer. We'll probably need the same to be done for
> other 3D data types, but I don't know personally people working with radio
> and X-ray cubes.

Yes.  Also Tim's examples of JCMT radio and IFU data, or anyone else who
wants to volunteer to contribute use-cases.

>  3) As Francois proposes me, I'll try to add the necessary features to the
>      Characterisation data model to be able to deal with the IFU data.
>      I'll do my best to finish this (with a help of Francois) before the
>      end of March

>  4) At this point it will be clear (hopefully) what we need to have in the
>      3D data model, and whether we need it separated from more general DMs
>      (Characterization and Observation), and it would be possible to start
>      the real work on the document draft.

Yes, but remember that Characterization is a general model and is not
specific to any particular kind of data.  Hence we don't want to be adding
"observational" metadata into this to describe specific data types.  We
may need to do that, but such specialized information should go elsewhere.

At the level of data characterization a 3D dataset merely has a certain
bandpass, resolution, sample size, etc. in each physical axis (spatial,
spectral, time).  The difference between a 2D image and a 3D spectral data
cube is merely that there are multiple samples along the spectral axis.
The difference between IFU data and a regularly sampled spectral data
cube might not be visible at this level, except perhaps for subtleties
like the filling factor in the spatial domain.

For actual 3D datasets, some like spectral data cubes might be best handled
by generalizing the image model to 3D.  Others like IFU data might be
best handled by a collection of extracted 1D spectra (e.g. SSA using a
FITS binary table), or maybe a 3D image with a spatial mask and other
metadata, or maybe via a new model or subclass specific to this data if
it is complex enough to require it.  To fully address this problem might
require some combination of all of these.  I doubt if we want to try to
generate one model with covers everything as it will be too complex.

 	- Doug