RFC initiated for Simple Spectral Access protocol

[Jesus Salgado]

Dear Doug,

I fully agree with the fact that SSA can/should not impose any output
format. I (with my VOSpec developer hat) would prefer to have a single
format so the implementation is easier in the client but We (with my
data provider hat) cannot accept I am forced to change to a format that,
for my particular project/archive, it is not the better one to
characterize my data.

In fact, all the data producers consider their format is the optimal for
their data and, even when some could consider the SED data model (both
XML or FITS serialization) better, I do not know any data producer who
share this opinion. On the other hand, the resources necessary to
transform the legacy/original format to the SED DM one could result in
many data providers never wanting to be part of the VO effort.

So, in my opinion, we should live with legacy/native data and we should
add the necessary info in the SSA VOTable output to enable a VO
application to deal with this format (if possible)

This is why, in case we have a native format referenced in the SSA
output, some fields should be set as compulsory for foreign data, i.e.

Dataset.SSA.SpectralAxis
Dataset.SSA.FluxAxis
(WARNING! The previous two fields which define the columns where the
spectral coordinate and the flux are stored have disappeared from the
SSAP spec 0.97 (they are in version 0.91)! We assume this is by mistake)

(Note: In case it is assumed the VO client is forced to get the project
datamodel from Dataset.DataModel and then, in some way, understand the
structure of the file to allow the data parsing, etc., that would open
another debate)

Dataset.SpectralSI
Dataset.FluxSI

Using this very basic information we have been able to parse native data
from ~18 different data providers without major effort.

Best Regards,
Jesus

On Fri, 2007-06-22 at 07:19 -0600, Doug Tody wrote:
> Two points here:
> 
> 1) SSA does not impose any format; FITS is fully supported as an output
> format.
> 
> 2) We are confusing the standard VO interchange format with the
> native project format as produced by the project and stored in the
> remote archive.  Pass-through of native project spectra is already
> supported (this is format=native) and could make things easier for the
> minimally-compliant data provider - although metadata generation is
> probably the harder problem.
> 
> In the ideal case, a service will read native spectra into an in-memory
> implementation of the Spectrum data model, and dynamically output whatever
> format the client application prefers (we already have software which can
> can do this).  A good service implementation will support pass-through
> of native data as well; the client may or may not be able to do anything
> with this, but native data may be preferable for some analysis.
> 
> The issue we are discussing here, is if there should be a recommended
> format for the VO interchange (wire) format, where the data has already
> been converted to be compliant with the Spectrum model.
> 
> 	- Doug
> 
> 
> On Fri, 22 Jun 2007, Anita M. S. Richards wrote:
> 
> > >> - Section 1.4.1 states "VOTable is preferred if only one format is
> > >> supported".  I would suggest not preferring one data format over another
> > >> and leaving it to the data providers to decide. For an "archival" SSA
> > >> service, which is the more likely type of service to offer only one
> > >> format, FITS may be a better choice.
> > >
> > > This may be a controversial point, but my opinion is that VOTable may
> > > prove to be a better choice as an interchange format (not necessarily for
> > > archive storage) for metadata-rich VO spectra, especially since an SSA
> > > client already has to support VOTable for queries.  Astronomy software is
> > > already adapting to support VOTable.  I suspect that most new spectral
> > > services and client software will prefer to support VOTable over FITS.
> > > I think it is useful to suggest a format for those who will only implement
> > > one format.  If there is sufficient disagreement we can reconsider the
> > > point though.
> > >
> >
> > I'm afraid that I am with Randy on this.  Yes, VOTable is very convenient
> > for some things, but the danger is that if we emphasise it too much for
> > spectra, service providers may think that they don't have to support
> > anything else.  In fact the majority of spectra at present are probably
> > FITS or ascii or goodness knowns what, and in future large data volumes
> > may be xml/binary.  Why should we impose a format on spectra when we don;t
> > for images? VO tools can handle FITS perfectly well. If I had a large
> > collection of FITS spectra with numerous
> > extension tables giving the processing history etc., I would not want to
> > convert the whole lot to VOTable.  Before someone says that it is
> > possible, yes, maybe, but that isn't the point - we should impose the
> > minimum requirements on data providers. It is much better to make sure
> > that there are tools which can handle at least the few commonest formats
> > and convert them if necessary - which may often involve converting a
> > sub-set or cutout. More importantly, the metadata must be preserved.  But
> > that is not really the province of this discussion.
> >
> >
> > best wishes
> >
> > Anita
> >
> > - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
> > Dr. Anita M. S. Richards, AstroGrid Astronomer
> > MERLIN/VLBI National Facility, University of Manchester,
> > Jodrell Bank Observatory, Macclesfield, Cheshire SK11 9DL, U.K.
> > tel +44 (0)1477 572683 (direct); 571321 (switchboard); 571618 (fax).
> >
-- 
Jesus J. Salgado


ESAC Science Archive Team
e-mail: Jesus.Salgado at sciops.esa.int
Tel + 34 91 8131271

European Space Agency/European Space Astronomy Centre
VILLAFRANCA Satellites Tracking Station
P.O. Box 78
E-28691 Villanueva de la Cañada
MADRID - SPAIN

================================================================================================
This message and any attachments are intended for the use of the addressee or addressees only. The
unauthorised disclosure, use, dissemination or copying (either in whole or in part) of its content
is prohibited. If you received this message in error, please delete it from your system and notify
the sender. E-mails can be altered and their integrity cannot be guaranteed. ESA shall not be liable
for any e-mail if modified.
=================================================================================================