ImageDM and ObsCore
Louys Mireille
mireille.louys at unistra.fr
Thu Nov 7 04:33:56 PST 2013
Dear Arnold, dear all ,
Could we try to work out an example for the Redshift axis description ,
let say for a CALIFA Data cube , as available at :
http://califa.caha.es/?q=content/califa-dr1
Any input from people involved in data cubes for another Use case ,
where REDSHIFT or VELOCITY axis is necessary ?
Cheers , Mireille.
Le 30/10/2013 14:52, Arnold Rots a écrit :
> I notice that a redshift coordinate is defined but that there are no
> parameters that allow specifying anything along that axis.
>
> That will need to be fixed.
>
> - Arnold
>
> -------------------------------------------------------------------------------------------------------------
> Arnold H. Rots Chandra X-ray Science Center
> Smithsonian Astrophysical Observatory tel: +1 617 496 7701
> 60 Garden Street, MS 67 fax: +1 617 495 7356
> Cambridge, MA 02138 arots at cfa.harvard.edu <mailto:arots at cfa.harvard.edu>
> USA http://hea-www.harvard.edu/~arots/
> <http://hea-www.harvard.edu/%7Earots/>
> --------------------------------------------------------------------------------------------------------------
>
>
>
> On Fri, Oct 25, 2013 at 12:40 PM, François Bonnarel
> <francois.bonnarel at astro.unistra.fr
> <mailto:francois.bonnarel at astro.unistra.fr>> wrote:
>
> Hi all,
> Here are my comments
>
> The basic thing to do to understand most of the discripencies
> between ObsCore, ImageDM (or SpecDM) is to go back to Char2 utype
> list.
> Char2 utype list is an evolution of Char1 utype list which has
> been reworked mainly by Igor, Mireille and me during the past
> years and which is not yet public. (it will be posted rather soon
> on IVOA now) I am currently documenting it. It's basically
> cleaning up Char1 and has inspired most of the ObscOre innovations
> with respect to Spectrum (Spectrum 1 at that time)
>
> The main thing to understand is that Characterisation is
> reusing STC class instances at the low level. For example
> Char.(AnyCharAxis).coverage.location.coord designates an
> AstroCoord instance. AstroCoord because this had to come with a
> reference to a coordinate system. AstroCoord (or subclasses of it)
> and AstroCoordArea (or subclasses of it) are the basic classes
> reused in coverage location, bounds and support.
> This was first intended to have fully complet STC xml elements in
> case of xml serialisation of char metadata.
>
> But there was another reason to do this : benefit of the
> semantic power of STC classes and attributes names. You may have a
> look to the list of char1 utypes if you want. This was used not
> only for coverage but also for accurate definitions of Errors,
> PixelSize and Resolution elements. For example STC allows various
> definition of error bins in 2-d or 3-d spaces: circular,
> rectangular or ellipsoidal, tilted or not, etc ...
>
> The joint file is an upgrade of a work I allready sent Doug and
> Mark at the end of June. Column 1 is the ImageDM utype list taken
> from Doug's file. Column 2 is the corresponding ObsCore utype.
> Column 3 is the specific path of char2 DM, before using terminal
> stc utypes . Column 4 gives the STC structure reused. Column 5
> explains the discripencies and proposes various solutions.
>
> Cheers
> François
> Le 25/10/2013 18:01, Douglas Tody a écrit :
>
> Mireille - Thanks for the quick feedback. I will wait to discuss
> until we have comments submitted (due today!). - Doug
>
>
> On Fri, 25 Oct 2013, Louys Mireille wrote:
>
> Dear Doug , dear all,
>
> Thanks for the convergence effort .
> I inserted my comments below .
>
> Le 19/10/2013 19:44, Douglas Tody a écrit :
> Hi All -
> ....
> The main change to ImageDM coming out of the interop
> discussions is to
> make the ImageDM and ObsCore consistent - ideally
> ImageDM
> extends
> ObsCore, so that if we already implement
> ObsCore/ObsTAP we
> can add more
> metadata to support the ImageDM, and ObsTAP and
> SIAV2 (and a
> future
> generic AccessData also being developed) can play well
> together. Our
> main focus at this point are the data model Utypes;
> the image
> data
> access model (AccessData etc.) will become the focus
> once we
> have the
> basic model defined.
>
> To get this started I have gone through the most recent
> ImageDM and
> compared this to ObsCore to see what differences
> remain. The
> attached
> ImageDM spreadsheet details the differences.
>
> In the spreadsheet, column B contains the ObsCore
> field IDs.
> Those in
> bold are the mandatory ObsCore fields. Utypes which
> differ
> between the
> two models are highlighted with a yellow background.
> The
> Utypes shown
> are the ImageDM/SpectralDM versions. Refer to
> Appendix B of
> the ObsCore
> doc (pg 35) to see the corresponding Utypes defined by
> ObsCore.
>
> A summary of the differences follows below. The edited
> ImageDM
> spreadsheet highlighting the differences with
> ObsCore is also
> attached.
>
> We need to decide what to do to bring
> ImageDM/SpectralDM in
> line with
> ObsCore - do we merely adopt ObsCore as the core
> model and
> change
> ImageDM/SpectralDM and possibly Char/Char2 etc., or
> do we
> produce an
> updated version of ObsCore to minimize changes to
> the other
> data models?
> Please think about it and suggest what you think is
> the best
> approach.
> We have an opportunity here to possibly make all the
> major
> data models
> consistent. But we will need to move fast for this next
> update.
>
> - Doug
>
> I think most discrepencies can be treated by:
> - changing Obscore Utypes when possible.
> - include minimal and maximal range values specified in
> ObsCore for
> resolution , sampling , etc into the ImageDM.
> For the moment ImageDM usually defines only a 'refval' but
> we could
> anticipate extensions for new use-cases with the metadata
> already defined
> in ObsCore .
>
> I am ok to change "Observation." or "Obs." to "Dataset."
> and align
> ObsCore for this .
>
> are there some problems in Obstap implementations already
> running to have
> these following changes in the Utypes:
>
> Obs.dataProductType --> DataSet.dataProductType and
> same for
> Obs.dataProductSubtype
> Obs.calibLevel
>
>
>
> ImageDM vs ObsCore Summary of Differences
> ------------------------------------------
>
> ObsTAP ID vs ImageDM shortname / field ID
>
> These serve much the same purpose, e.g., a
> column
> name in a
> table, or a shortname for get/put calls. The
> simplest solution
> is to just adopt the ObsCore IDs and extend
> the list
> to the full
> set of attributes in ImageDM.
>
> Dataset vs Observation
>
> ImageDM and SpectralDM use Dataset.X for
> attributes
> describing
> the overall dataset, e.g., Dataset.Type,
> Dataset.DataModel, etc.
> ObsCore uses "Obs" for the equivalent
> Utypes, and is
> oriented
> more toward arbitrary (VO and non-VO) data
> products,
> e.g.,
> Obs.dataProductType. ObsCore has no
> equivalent to
> Dataset.DataModel. Convergence requires that we
> choose one or
> the other of Dataset or Obs.
>
> So Ok to homogeneize to Dataset.
> Consequently we can add also Dataset.DataModel for completion.
> However, I think we worked out a precise definition of
> dataproductType in
> ObsCore , that we should re-use in ImageDM.
>
>
> ObsCore Utypes Missing in ImageDM
>
> obs_id DataID.ObservationID
> facility_name
> Provenance.ObsConfig.facility.name
> <http://Provenance.ObsConfig.facility.name>
> s_ra, s_dec ImageDM uses the field
> Location.Coord instead
> s_resolution_min ImageDM currently
> defines only
> Resolution.RefVal
> s_resolution_max
> em_respower_min ImageDM currently
> defines only
> ResolPower.RefVal
> em_respower_max
>
> ResolPower accessed through Resolution in ObsCore. It
> belongs to this
> Property as in Charv2.0.
> Char.FluxAxis
>
> ImageDM and SpectralDM use Char.FluxAxis,
> whereas
> ObsCore uses
> Char.ObservableAxis. ObsCore is more
> correct in this
> case,
> since the observable is not always flux. But we
> refer to a
> generic Flux quantity in many of our models.
>
> What about adding ObservableAxis in ImageDM for the cases
> where Flux is
> not appropriate?
> Utype Inconsistencies
>
> I identified about 20 of these (marked with
> a yellow
> background
> in the spreadsheet; we just need to decide
> on the
> final Utypes.
> Do we keep the ObsCore Utypes and change all our
> other DMs, or
> have a new version of ObsCore?
>
> ObsCore has calibStatus instead of calibrationStatus in
> ImageDM and
> Char2.0, Spectrum2.0 .
> I suppose we can update ObsCore too for this.
>
>
> More to come in details , namely by François for the
> utypes comparison .
>
> Best wishes , Mireille.
>
> Le 23/10/2013 23:13, CresitelloDittmar, Mark a écrit :
> All,
>
> I've been tied up with other work.. so haven't had a chance to
> review this well.
> I'll be giving it good look tomorrow. some quick
> comments below..
>
> If I recall, ObsCore makes use of Characterisation primarily
> through reference.
> It leaves the details to that model and only talks about
> it's usage
> of certain
> components. The Reference listed is for Char v2.0, so I'd
> say that
> as far as
> Utypes go, ObsCore is definitive for the Observation level
> stuff,
> while Char 2.0
> is definitive for the Characterisation stuff. We need to
> agree
> which to put in
> Char 2.0, then ObsCore should adjust to that if needed.
>
> On Sat, Oct 19, 2013 at 1:44 PM, Douglas Tody
> <dtody at nrao.edu <mailto:dtody at nrao.edu>>
> wrote:
>
>
> ImageDM vs ObsCore Summary of Differences
> ------------------------------------------
>
>
> Dataset vs Observation
>
> ImageDM and SpectralDM use Dataset.X for
> attributes describing
> the overall dataset, e.g., Dataset.Type,
> Dataset.DataModel, etc.
> ObsCore uses "Obs" for the equivalent Utypes,
> and is oriented
> more toward arbitrary (VO and non-VO) data
> products, e.g.,
> Obs.dataProductType. ObsCore has no equivalent
> to
> Dataset.DataModel. Convergence requires that
> we choose one or
> the other of Dataset or Obs.
>
>
>
> During the adass meetings, I began migrating my UML
> diagrams to
> reflect
> the ObsCore extention. From that perspective, I recall
> there is
> some
> difficulty with the Obs vs DataSet objects. They have
> similar info
> at different
> levels. I was working toward getting "Spectral" to be an
> extension
> of "Obs"
> where the extension is the data portions, etc.
>
>
> Char.FluxAxis
>
> ImageDM and SpectralDM use Char.FluxAxis,
> whereas ObsCore uses
> Char.ObservableAxis. ObsCore is more correct
> in this case,
> since the observable is not always flux. But
> we refer to a
> generic Flux quantity in many of our models.
>
>
> ImageDM probably shouldn't use FluxAxis.
> SpectralDM can include FluxAxis as an extension to Char.
>
>
> Utype Inconsistencies
>
> I identified about 20 of these (marked with a
> yellow background
> in the spreadsheet; we just need to decide on
> the final Utypes.
> Do we keep the ObsCore Utypes and change all
> our other DMs, or
> have a new version of ObsCore?
>
>
> more on this tomorrow.
> Mark
>
>
>
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.ivoa.net/pipermail/dm/attachments/20131107/b4f898ff/attachment-0001.html>
More information about the dm
mailing list