general comments to SSAP and SDM from outside
Petr Skoda
skoda at sunstel.asu.cas.cz
Mon Sep 17 05:36:23 PDT 2007
After some waiting for general comments of DAL community, I decided to
make some summary - sorry for long post...
D. Toddy:
> The suggestion to add "normalized" as a form of flux calibration is
> something we should consider.
Thanks for support of my idea - Jonathan has put it in the new revision od
DM - will it be updated in the SSAP as well ?
>> Regarding support for IRAF-Onedspec format for spectra (a linearized
>> spectrum represented as a 1-D image), I feel this is too limited a
>> format to be pushed as a new general standard,
OK - I agree its limited (esp. problems with merged echelle spectra
are quite often met) but I do not want to define it as a
recommended standard but as a tolerated option.
>> We also need
>> client software which can form and issue a query to an SSA service,
>> select spectra, and download them over the Internet;
I think we have it already - SPLAT can do it now, VOSPEC seems to be
capable of it (but does not understand WCS info)and SpecView
could be if 1D image FITS will be accepted by IVOA as accepted format,
>> which is not yet VO-enabled probably cannot do all these things.
>> Hence some new software will be required no matter what we do.
>> One simple solution the might be do do the conversion on the client
>> side, and have an option to write out spectra locally in this format
>> for consumption by legacy software.
I wanted to point out the problems on publishing side - a lot of data is
aleady reduced (e.g. IRAF, MIDAS) and people can publish it as it is or
not at all - they might not have enough manpower to do the massive
conversion of all files (moreover I could not find any convertor allowing
simple conversion of output from IRAF (apall, doslit, dispcor ...) tasks
to binary tables (the package TABLES although being IRAF extension seems
not understand IRAF WCS).
>> It is certainly also possible
>> for an SSA service to have the capability to return spectra in this
>> format, but as I say, it is not general enough to be a core format.
Are the other formats no 4-6 (text, text/html, graphics)listed in sec
3.5 "Packaging model" of spectral
data model document general enough to be core format ? Is especially GIF
or JPG better than image fits for representing spectra ????
I know that "detailed serialization for these formats are not specified"
but why not to allow the FITS 1d image at the same level as JPG ??
I feel some inconsistency in sec. 3.5 ......
(it states the usage of getCapabilities, but AFAIK its postponed, so the
DM should not say - "it uses"
>> If a service supports 1-D images as an output format, this would be
>> an optional capability indicated in the list of supported formats,
>> and a query with FORMAT set to the appropriate MIME type (as in your
>> email) would specify that this is the format the client wants back.
>
Yes it is exactly what I wanted to have.
>>
>> In the case of IRAF itself we are working already to interface IRAF
>> directly to SSA, at which time it will be able to read Spectrum-DM
>> compliant spectra directly, and all the IRAF tasks will be usable
>> directly on VO-compliant spectra.
This will be fantastic !! I already tried to test the vo-cl as received at
NVO Summer School, but there was no support of spectra.
If the tasks like splot or spectool will support SSAP and will be able to
start the analysis (like line EW measurement, fitting ..) on the spectra
downloaded from SSAP servers in the same manner as local ones
(understanding the IRAF WCS) then stellar astronomers would be happy
enough.... and the FORMAT=NATIVE is the solution...
>> For other apps or environments, we have middleware such
>> as VOClient in development which will make this easier in the future.
>> Ultimately I think this is the best solution.
Most of the other spectrum tools are quite obsolete (written in FORTRAn or
IDL) and I an afraid that despite technical difficulties there will not be
enough manpower to enhance them.
>>> Most of the spectra from ground
>>> are served by a single server tool (Pleinpot by P. Prugniel)
>> This sounds interesting - maybe we can get this software updated
>> to support an SSA service interface.
The system already supports SSAP 0.9 (using the recommended VOX_ keywords
and the spectra from ELODIE and SOPHIE as well as HYPERLEDA archives are
in fact preprocessed on-the-fly to be accepted by current SSAP clients
(e.g. certain FITS keywords are not present in spectra, but are generated
from database tables ). The author with whom I colaborate, has plans to
support current SSAP once accepted. The system PLEINPOT has ben quite
hidden behind this archives and it was not stressed enough (and advertised
;-) that it is a another SSAP compatible spectra server (like e.g. SAADA).
But in addition it is a powerful set of astronomical tasks allowing the
flexible on-the-fly pipeline processing of spectra (including wavelength
rebinning, flux normalization , zooming ...) - its extensible wih
dynamically loaded tasks and many more. In addition to serving SSAP it has
a features of normal cgi-bin based web archives with simple generation of
graph of spectrum, simple query form generator etc..
I am currently adapting it as server for small spectra collection and I
will have a poster at ADASS comparing it to other SSAP servers - who is
interested..
-----
> I think the main thing is that the document cover how to represent
> a normalized spectrum, which it does now, so I can agree with this.
> The only thing is that in many cases (e.g., a continuum normalized
> spectrum) I suspect the normalization function was computed from the
> data,
This is a little bit limited in IRAF splot - you can only fit curves using
the data points present in data (either all or in selected ranges).
But experience shows that sometimes it is better to draw some control
points and do it interactively until the fit has the required shape and
placement (eg above existing points in case of blended molecular bands) -
it is the art than a rigorous math - but such is a life ...
Experienced astronomer already knows the tricks.
> as there is for including the background model (indeed, in some cases
> they may be the same).
No - the background is always subtracted , not divided into !
But something as the "continuum model" would be surely needed.
There is a number of techniques of continuum fitting and the proper
description of them may be quite complicated in simple keywords..
On the other hand, even now there is rather uncommon to publish original
and normalized spectra in a consistent way - no 1to1 association of two
separate files - only few packages store the continuum shape together with
original data
>
> What we have is probably sufficient for now, especially since I
> suspect people working with normalized spectra often don't look at
> the normalization function in any case.
Unfortunately it is true - but they are often surprized that the star is
not variable but its a problem of different normalization ;-)
So in general, the VO tools could help to improve the reseach as well
allowing easy checking of original and normalized spectra quickly
(its a responsibility of publishers to collect the corresponding data and
identify their correspondence properly)
--------
Jonathan:
> The question of continuum normalized spectra is an interesting and
> important one. I accept the need for Calibration = NORMALIZED.
I am glad it was accepted !
> Really what you have is two spectra:
>
> - firstly, the normalized spectrum, which I believe we correctly support
> with
>
> Spectrum.Char.FluxAxis.Calibration = NORMALIZED
> Spectrum.Char.FluxAxis.unit = ' ' (blank, i.e. dimensionless)
> Spectrum.Char.FluxAxis.ucd = arith.ratio;phot.flux.density
>
> I do insist that the units are dimensionless.
OK - I give up - dimensionless is better than "n/a"
but the ucd is rater 'phot.count' - see below
> - secondly, the continuum spectrum, which often may be defined
> using a function. We don't have a way of encoding such a function in the
> VO,
> but your service can just instantiate the continuum spectrum, i.e.
> calculate
> the flux values in each bin and make a VO Spectrum object for the
> continuum.
Yes, it seems to be reasonable solution for now - as I said there are only
few tools producing data with separate continuum description (e.g. our
SPEFO - but is has serious limitations anyway)...
> Then for this spectrum you have
>
> Spectrum.Char.FluxAxis.Calibration = ABSOLUTE
in fact most of spectra I have in mind are not absolute calibrated - they
are in something like summed counts (just after 1D extraction - e.g.
summed CCD lines with ADU on y-axis)
so it is rather UNCALIBRATED
> Spectrum.Char.FluxAxis.unit = 'erg cm**-2 s**-1 Angstrom**-1' or
> something
here rather *.unit='count'
> Spectrum.Char.FluxAxis.ucd = phot.flux.density;em.wl
rather *.ucd= phot.count
>
> Given these two spectra, the user can reconstruct the full spectrum by
> multiplying them together. (I don't think we are ready to add metadata to
> explicitly connect them yet.)
You say "the second spectrum is... fit not
> data, so it is probably not correct.." - but a spectrum generated from
> a fit is perfectly acceptable, we make no restriction that the VO Spectrum
> has to be directly observed data.
OK - to be honest even the absolute calibration is done by dividing by
some fit (in IRAF you use standard star with known flux - but you make
several bins on it - then you make fits trough them and this fit (not
data points) is used to get absloute calibration on ground spectra !!
> tied to the format, but we must standardize on formats that are rich
> enough to store the full model. I agree with Doug that the use of FITS
> images for storing spectra, although widespread, is too broken to
> support in the future (nowhere good to put the error metadata etc.).
So what about rest of sec . 3.5 ?? How do you put metadata to JPG ?
Again I think the 3.5. should be better formulated not to confuse
people...
> Nevertheless implementations may map between the old formats and the new
> ones at the cost of losing the extra metadata. We'll have to see what
> formats are most used in the future - that shouldn't affect the model itself.
> I posted a revised version of the document to
>
> http://hea-www.harvard.edu/~jcm/vo/docs/spec101r3/SpectrumDM-20070827.pdf
I will post separately more comments to it ...
-----------------
>
> Doug, I am against reusing the background model element in this
> way - the semantics are different as well as the (subtract vs divide) issue.
I agree but continuum model should be prepared in the future anyway..
--------
Lazslo:
> in mind further issues, like automatic unit conversion. We already have a
> problem with Angstroms (though using only SI units can help overcome this
> problem but makes the life of vo client application developers more
> difficult), because angstrom and amper are noted with the same letter using
>the ascii charset.
- it is a problem in physics in general
The CGS system does not need Amper (I had to modifiy the Mathematica units
package and after substituting the Amper I was able to check the
calculation of Einstein coefficient and transition rates easily ...
But in SI I do not know ....
> On the otherhand, I would not introduce any unit describing normalization or
> whatever else. First of all, characterization has a field for calibration,
> that's perfectly enough
OK - I agree the "unitless" unit is strange ;-)
> and appropriate to describe if the flux axis
> represents normalized values (for example setting the field to NORMALIZED
> implies the fluxes are indeed
> spectrophotometrically calibrated relatively to each other).
Probably there is a misuderstanding - I do not mean ratio of two fluxes
but a unknown (e.g in counts) flux divided by artificial function - see
more in Jonathan's analysis..
> However, it still would be useful to describe different normalization
> methods. One can normalize spectra to a given flux at a given wavelength,
It is sometimes used - there are estimated regions for certain spectral
types where the continuum is seen directly (but it does not work for late
type stars) and the unknown flux is really normalized in these regions -
but than the fit is forced through them and only now is the fit divided
into the spectrum..
> but in stellar population synthesis fluxes are normalized to represent the
> emission of a star population with one solar mass.
probably you are speaking about different kind of normalization -
I mean normalization=rectification (streightening of stellar continuum to
be set at level 1.0)
------------------
Anyway I am glad the discussion about the specfic needs of (especially)
stellar astronomers begun and hope they will be able to benefit from VO
infrastructure soon. I am available for more arguing at ADASS or Interop
soon.
Best regards,
Petr Skoda
*************************************************************************
* Petr Skoda Phone : +420-323-649201, ext. 361 *
* Stellar Department +420-323-620361 *
* Astronomical Institute AS CR Fax : +420-323-620250 *
* 251 65 Ondrejov e-mail: skoda at sunstel.asu.cas.cz *
* Czech Republic *
*************************************************************************
More information about the dal
mailing list