ImageDM and ObsCore
Arnold Rots
arots at cfa.harvard.edu
Tue Nov 19 11:57:07 PST 2013
Sorry, this has been on my plate for a while, now.
Would it help if I referred you to the examples on my STC page?
The G1Arecibo example on http://hea-www.cfa.harvard.edu/~arots/nvometa/STC/
provides a description of a hypothetical cube in all 4 coordinate types.
Cheers,
- Arnold
-------------------------------------------------------------------------------------------------------------
Arnold H. Rots Chandra X-ray
Science Center
Smithsonian Astrophysical Observatory tel: +1 617 496
7701
60 Garden Street, MS 67 fax: +1 617
495 7356
Cambridge, MA 02138
arots at cfa.harvard.edu
USA
http://hea-www.harvard.edu/~arots/
--------------------------------------------------------------------------------------------------------------
On Thu, Nov 7, 2013 at 7:33 AM, Louys Mireille <mireille.louys at unistra.fr>wrote:
> Dear Arnold, dear all ,
> Could we try to work out an example for the Redshift axis description ,
> let say for a CALIFA Data cube , as available at :
> http://califa.caha.es/?q=content/califa-dr1
>
> Any input from people involved in data cubes for another Use case , where
> REDSHIFT or VELOCITY axis is necessary ?
>
> Cheers , Mireille.
>
> Le 30/10/2013 14:52, Arnold Rots a écrit :
>
> I notice that a redshift coordinate is defined but that there are no
> parameters that allow specifying anything along that axis.
>
> That will need to be fixed.
>
> - Arnold
>
>
> -------------------------------------------------------------------------------------------------------------
> Arnold H. Rots Chandra X-ray
> Science Center
> Smithsonian Astrophysical Observatory tel: +1 617 496
> 7701
> 60 Garden Street, MS 67 fax: +1 617
> 495 7356
> Cambridge, MA 02138
> arots at cfa.harvard.edu
> USA
> http://hea-www.harvard.edu/~arots/
>
> --------------------------------------------------------------------------------------------------------------
>
>
>
> On Fri, Oct 25, 2013 at 12:40 PM, François Bonnarel <
> francois.bonnarel at astro.unistra.fr> wrote:
>
>> Hi all,
>> Here are my comments
>>
>> The basic thing to do to understand most of the discripencies between
>> ObsCore, ImageDM (or SpecDM) is to go back to Char2 utype list.
>> Char2 utype list is an evolution of Char1 utype list which has been
>> reworked mainly by Igor, Mireille and me during the past years and which is
>> not yet public. (it will be posted rather soon on IVOA now) I am
>> currently documenting it. It's basically cleaning up Char1 and has inspired
>> most of the ObscOre innovations with respect to Spectrum (Spectrum 1 at
>> that time)
>>
>> The main thing to understand is that Characterisation is reusing STC
>> class instances at the low level. For example
>> Char.(AnyCharAxis).coverage.location.coord designates an AstroCoord
>> instance. AstroCoord because this had to come with a reference to a
>> coordinate system. AstroCoord (or subclasses of it) and AstroCoordArea (or
>> subclasses of it) are the basic classes reused in coverage location, bounds
>> and support.
>> This was first intended to have fully complet STC xml elements in case of
>> xml serialisation of char metadata.
>>
>> But there was another reason to do this : benefit of the semantic
>> power of STC classes and attributes names. You may have a look to the list
>> of char1 utypes if you want. This was used not only for coverage but also
>> for accurate definitions of Errors, PixelSize and Resolution elements. For
>> example STC allows various definition of error bins in 2-d or 3-d spaces:
>> circular, rectangular or ellipsoidal, tilted or not, etc ...
>>
>> The joint file is an upgrade of a work I allready sent Doug and Mark
>> at the end of June. Column 1 is the ImageDM utype list taken from Doug's
>> file. Column 2 is the corresponding ObsCore utype. Column 3 is the specific
>> path of char2 DM, before using terminal stc utypes . Column 4 gives the
>> STC structure reused. Column 5 explains the discripencies and proposes
>> various solutions.
>>
>> Cheers
>> François
>> Le 25/10/2013 18:01, Douglas Tody a écrit :
>>
>> Mireille - Thanks for the quick feedback. I will wait to discuss
>>> until we have comments submitted (due today!). - Doug
>>>
>>>
>>> On Fri, 25 Oct 2013, Louys Mireille wrote:
>>>
>>> Dear Doug , dear all,
>>>>
>>>> Thanks for the convergence effort .
>>>> I inserted my comments below .
>>>>
>>>> Le 19/10/2013 19:44, Douglas Tody a écrit :
>>>> Hi All -
>>>> ....
>>>> The main change to ImageDM coming out of the interop
>>>> discussions is to
>>>> make the ImageDM and ObsCore consistent - ideally ImageDM
>>>> extends
>>>> ObsCore, so that if we already implement ObsCore/ObsTAP we
>>>> can add more
>>>> metadata to support the ImageDM, and ObsTAP and SIAV2 (and a
>>>> future
>>>> generic AccessData also being developed) can play well
>>>> together. Our
>>>> main focus at this point are the data model Utypes; the image
>>>> data
>>>> access model (AccessData etc.) will become the focus once we
>>>> have the
>>>> basic model defined.
>>>>
>>>> To get this started I have gone through the most recent
>>>> ImageDM and
>>>> compared this to ObsCore to see what differences remain. The
>>>> attached
>>>> ImageDM spreadsheet details the differences.
>>>>
>>>> In the spreadsheet, column B contains the ObsCore field IDs.
>>>> Those in
>>>> bold are the mandatory ObsCore fields. Utypes which differ
>>>> between the
>>>> two models are highlighted with a yellow background. The
>>>> Utypes shown
>>>> are the ImageDM/SpectralDM versions. Refer to Appendix B of
>>>> the ObsCore
>>>> doc (pg 35) to see the corresponding Utypes defined by
>>>> ObsCore.
>>>>
>>>> A summary of the differences follows below. The edited
>>>> ImageDM
>>>> spreadsheet highlighting the differences with ObsCore is also
>>>> attached.
>>>>
>>>> We need to decide what to do to bring ImageDM/SpectralDM in
>>>> line with
>>>> ObsCore - do we merely adopt ObsCore as the core model and
>>>> change
>>>> ImageDM/SpectralDM and possibly Char/Char2 etc., or do we
>>>> produce an
>>>> updated version of ObsCore to minimize changes to the other
>>>> data models?
>>>> Please think about it and suggest what you think is the best
>>>> approach.
>>>> We have an opportunity here to possibly make all the major
>>>> data models
>>>> consistent. But we will need to move fast for this next
>>>> update.
>>>>
>>>> - Doug
>>>>
>>>> I think most discrepencies can be treated by:
>>>> - changing Obscore Utypes when possible.
>>>> - include minimal and maximal range values specified in ObsCore for
>>>> resolution , sampling , etc into the ImageDM.
>>>> For the moment ImageDM usually defines only a 'refval' but we could
>>>> anticipate extensions for new use-cases with the metadata already
>>>> defined
>>>> in ObsCore .
>>>>
>>>> I am ok to change "Observation." or "Obs." to "Dataset." and align
>>>> ObsCore for this .
>>>>
>>>> are there some problems in Obstap implementations already running to
>>>> have
>>>> these following changes in the Utypes:
>>>>
>>>> Obs.dataProductType --> DataSet.dataProductType and same for
>>>> Obs.dataProductSubtype
>>>> Obs.calibLevel
>>>>
>>>>
>>>>
>>>> ImageDM vs ObsCore Summary of Differences
>>>> ------------------------------------------
>>>>
>>>> ObsTAP ID vs ImageDM shortname / field ID
>>>>
>>>> These serve much the same purpose, e.g., a column
>>>> name in a
>>>> table, or a shortname for get/put calls. The
>>>> simplest solution
>>>> is to just adopt the ObsCore IDs and extend the list
>>>> to the full
>>>> set of attributes in ImageDM.
>>>>
>>>> Dataset vs Observation
>>>>
>>>> ImageDM and SpectralDM use Dataset.X for attributes
>>>> describing
>>>> the overall dataset, e.g., Dataset.Type,
>>>> Dataset.DataModel, etc.
>>>> ObsCore uses "Obs" for the equivalent Utypes, and is
>>>> oriented
>>>> more toward arbitrary (VO and non-VO) data products,
>>>> e.g.,
>>>> Obs.dataProductType. ObsCore has no equivalent to
>>>> Dataset.DataModel. Convergence requires that we
>>>> choose one or
>>>> the other of Dataset or Obs.
>>>>
>>>> So Ok to homogeneize to Dataset.
>>>> Consequently we can add also Dataset.DataModel for completion.
>>>> However, I think we worked out a precise definition of dataproductType
>>>> in
>>>> ObsCore , that we should re-use in ImageDM.
>>>>
>>>>
>>>> ObsCore Utypes Missing in ImageDM
>>>>
>>>> obs_id DataID.ObservationID
>>>> facility_name
>>>> Provenance.ObsConfig.facility.name
>>>> s_ra, s_dec ImageDM uses the field
>>>> Location.Coord instead
>>>> s_resolution_min ImageDM currently defines only
>>>> Resolution.RefVal
>>>> s_resolution_max
>>>> em_respower_min ImageDM currently defines only
>>>> ResolPower.RefVal
>>>> em_respower_max
>>>>
>>>> ResolPower accessed through Resolution in ObsCore. It belongs to this
>>>> Property as in Charv2.0.
>>>> Char.FluxAxis
>>>>
>>>> ImageDM and SpectralDM use Char.FluxAxis, whereas
>>>> ObsCore uses
>>>> Char.ObservableAxis. ObsCore is more correct in this
>>>> case,
>>>> since the observable is not always flux. But we
>>>> refer to a
>>>> generic Flux quantity in many of our models.
>>>>
>>>> What about adding ObservableAxis in ImageDM for the cases where Flux is
>>>> not appropriate?
>>>> Utype Inconsistencies
>>>>
>>>> I identified about 20 of these (marked with a yellow
>>>> background
>>>> in the spreadsheet; we just need to decide on the
>>>> final Utypes.
>>>> Do we keep the ObsCore Utypes and change all our
>>>> other DMs, or
>>>> have a new version of ObsCore?
>>>>
>>>> ObsCore has calibStatus instead of calibrationStatus in ImageDM and
>>>> Char2.0, Spectrum2.0 .
>>>> I suppose we can update ObsCore too for this.
>>>>
>>>>
>>>> More to come in details , namely by François for the utypes comparison .
>>>>
>>>> Best wishes , Mireille.
>>>>
>>>> Le 23/10/2013 23:13, CresitelloDittmar, Mark a écrit :
>>>> All,
>>>>
>>>> I've been tied up with other work.. so haven't had a chance to
>>>> review this well.
>>>> I'll be giving it good look tomorrow. some quick comments below..
>>>>
>>>> If I recall, ObsCore makes use of Characterisation primarily
>>>> through reference.
>>>> It leaves the details to that model and only talks about it's usage
>>>> of certain
>>>> components. The Reference listed is for Char v2.0, so I'd say that
>>>> as far as
>>>> Utypes go, ObsCore is definitive for the Observation level stuff,
>>>> while Char 2.0
>>>> is definitive for the Characterisation stuff. We need to agree
>>>> which to put in
>>>> Char 2.0, then ObsCore should adjust to that if needed.
>>>>
>>>> On Sat, Oct 19, 2013 at 1:44 PM, Douglas Tody <dtody at nrao.edu>
>>>> wrote:
>>>>
>>>>
>>>> ImageDM vs ObsCore Summary of Differences
>>>> ------------------------------------------
>>>>
>>>>
>>>> Dataset vs Observation
>>>>
>>>> ImageDM and SpectralDM use Dataset.X for
>>>> attributes describing
>>>> the overall dataset, e.g., Dataset.Type,
>>>> Dataset.DataModel, etc.
>>>> ObsCore uses "Obs" for the equivalent Utypes,
>>>> and is oriented
>>>> more toward arbitrary (VO and non-VO) data
>>>> products, e.g.,
>>>> Obs.dataProductType. ObsCore has no equivalent
>>>> to
>>>> Dataset.DataModel. Convergence requires that
>>>> we choose one or
>>>> the other of Dataset or Obs.
>>>>
>>>>
>>>>
>>>> During the adass meetings, I began migrating my UML diagrams to
>>>> reflect
>>>> the ObsCore extention. From that perspective, I recall there is
>>>> some
>>>> difficulty with the Obs vs DataSet objects. They have similar info
>>>> at different
>>>> levels. I was working toward getting "Spectral" to be an extension
>>>> of "Obs"
>>>> where the extension is the data portions, etc.
>>>>
>>>>
>>>> Char.FluxAxis
>>>>
>>>> ImageDM and SpectralDM use Char.FluxAxis,
>>>> whereas ObsCore uses
>>>> Char.ObservableAxis. ObsCore is more correct
>>>> in this case,
>>>> since the observable is not always flux. But
>>>> we refer to a
>>>> generic Flux quantity in many of our models.
>>>>
>>>>
>>>> ImageDM probably shouldn't use FluxAxis.
>>>> SpectralDM can include FluxAxis as an extension to Char.
>>>>
>>>>
>>>> Utype Inconsistencies
>>>>
>>>> I identified about 20 of these (marked with a
>>>> yellow background
>>>> in the spreadsheet; we just need to decide on
>>>> the final Utypes.
>>>> Do we keep the ObsCore Utypes and change all
>>>> our other DMs, or
>>>> have a new version of ObsCore?
>>>>
>>>>
>>>> more on this tomorrow.
>>>> Mark
>>>>
>>>>
>>>>
>>>>
>>>>
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.ivoa.net/pipermail/dm/attachments/20131119/c0ab15db/attachment-0001.html>
More information about the dm
mailing list