ObsCore for velocity cubes

Fri Feb 24 01:18:43 CET 2023

Thanks Françoios. I think the query constraints should not involve the velocities (is there a use case that calls for this?), but only the em_min/max, or equivalently  f_min/max, and on v_resolution (or equivalently on em_resolv_power = c/v_resolution).

The fact that a query could lead to a cube that covers or not a redshifted Halpha or a NII, is not an issue as this is exactly the same behaviour we accept when querying spectra.

em_unit is also not of interest for a ObsCore discovery query, plus as said, v_min and v_max should be express always in m/s, as well as v_resolution.

For me is important that the v_min and v_max be present in the ObsCore result, because they can be used to express the SODA max value of the “velocity” filtering parameter. With that, a user can provide the cuts in velocity, after having discovered a cube using ranges in wavelengths or frequencies.

It also useful to have the rest frequency/wavelength and the velocity flavour in ObsCore, so that they can be passed in input to SODA: in this way cuts could be provided in wavelengths or frequencies, as all the parameters to perform vel vs freq vs wavel transformations are then available (and this allows to treat frequency cubes as if they were velocity cubes).

I’d like to know if anybody as a use case to place a constraint on the v_min/max instead of using em_min/max (or f_min/max); anybody?

Thanks,
Alberto

From: BONNAREL FRANCOIS <francois.bonnarel at astro.unistra.fr>
Date: Thursday, 23 February 2023 at 15:48
To: Alberto Micol <amicol at eso.org>, alberto micol <amicol.ivoa at googlemail.com>
Cc: "dal at ivoa.net" <dal at ivoa.net>, Data Models mailing list <dm at ivoa.net>
Subject: Re: ObsCore for velocity cubes

This email was sent from outside of ESO from the address [francois.bonnarel at astro.unistra.fr]. If it looks suspicious, please report it to phishing at eso.org.

Dear Alberto,

    For sure it's possible to have the parameters of Greisen et al in an extension.
    But I'm still wondering how useful and handy it is.

     Again, definitely I see the interest of delivering these metadata (which are actually fitw wcs metadata) at the SODA level in order to create a velocity cutout  query.

      But I'm still filling chilly for data discovery.
       Let's look at a couple of examples

       select * from obscore-with-vel-extension where dataproduct_type= cube and  v_min = 800  and v_max= 1200 and em_unit = km/s and em_ucd = vel

        then you get for example
                  optical cubes with h alpha line redshifted by  ~1.7 to ~2.6 nm
                  and radio cubes with 21 cm line redshifted by ~0.55 to ~0.8 mm
                  and probably many others

       So, will you say, we have to give the reference line to get  h alpha redshifted line ONLY

       select * from obscore-with-vel-extension where dataproduct_type= cube and v_min = 800  and v_max= 1200 and em_unit = km/s and em_ucd = vel and em_ref = 656.3 10**-9

        But then what you get is something which may contain the redshifted halpha line but maybe also nothing about halpha (because the velocity axis is not a guarantee for redshifted line to be there in the dataset)  but instead the NII line at 658.4.

        Or even worse = (highly) redshifhted halpha and (nearly) non redshifted NII !!! (and there we are close to the 4D use case)

        So I find this misleading and I think we can find also the same kind of mess in the radio domain (imagine very high z pushing 21 cm in low frequency radio or mm lines in the metric domain for example )

       So if you want velocity cubes in the halpha redshift rang eas above just query
          select * from obscore-with-vel-extension where dataproduct_type= cube and em_min = 658.0 10**-9 and em_max = 658.9 10**-9 and em_unit = km/s and em_ucd = vel and em_ref = 656.3 10**-9

      This will be exactly the same results with the same limitations (Halpha redshift and there or not and NII there or not) but at least less confusion. And you will exclude all cubes which are not organized with velocity axes anyway.
       I still wonder if the em_unit is needed there, because it is supposed to describe the unit used in the dataset and I don't think if you look for velocity cubes you want to select those in m/s from those in km/s. I think what you need is to know what is the unit.

       My conclusion adding em_ref for the reference line an a couple of new values for em_ucd should be enough for ObsCore extension and all the other things for SODA.

Cheers
François

Le 22/02/2023 à 17:33, Alberto Micol a écrit :
Sorry let me rephrase more correctly:

Can we aim to describe the Greisen (2006) cubes, limiting to the 3d case (2 spatial 1 velocity), so to cover most of the existing velocity cubes?

Thanks,
Alberto

From: Alberto Micol <amicol at eso.org><mailto:amicol at eso.org>
Date: Wednesday, 22 February 2023 at 17:26
To: BONNAREL FRANCOIS <francois.bonnarel at astro.unistra.fr><mailto:francois.bonnarel at astro.unistra.fr>, alberto micol <amicol.ivoa at googlemail.com><mailto:amicol.ivoa at googlemail.com>
Cc: "dal at ivoa.net"<mailto:dal at ivoa.net> <dal at ivoa.net><mailto:dal at ivoa.net>, Data Models mailing list <dm at ivoa.net><mailto:dm at ivoa.net>
Subject: Re: ObsCore for velocity cubes

Thanks for the explanation!
As said, my aim is to cover the cubes described in Greisen et al, 2006, and not “any kind of imaginable cube”.
This 4D cube you described is something of a different nature. And indeed is no longer a regular 3D cube.
Can’t we aim to describe just only the Greisen et al, 3d cubes?
That would cover a huge fraction of the existing cubes.

Thanks,
Alberto

From: BONNAREL FRANCOIS <francois.bonnarel at astro.unistra.fr><mailto:francois.bonnarel at astro.unistra.fr>
Date: Wednesday, 22 February 2023 at 17:20
To: Alberto Micol <amicol at eso.org><mailto:amicol at eso.org>, alberto micol <amicol.ivoa at googlemail.com><mailto:amicol.ivoa at googlemail.com>
Cc: "dal at ivoa.net"<mailto:dal at ivoa.net> <dal at ivoa.net><mailto:dal at ivoa.net>, Data Models mailing list <dm at ivoa.net><mailto:dm at ivoa.net>
Subject: Re: ObsCore for velocity cubes

This email was sent from outside of ESO from the address [francois.bonnarel at astro.unistra.fr<mailto:francois.bonnarel at astro.unistra.fr>]. If it looks suspicious, please report it to phishing at eso.org<mailto:phishing at eso.org>.

Dear Alberto,
I quote Arnold (2015 Apr 30)

I strongly urge to keep the two separate.

Yes, they have much in common, but they serve very different purposes.

And it is possible to create a 4-D cube where the 4th axis is actually a

spectral axis and the pixels are different lines (rest frequencies).

STC2 will treat the spectral and redshift/Doppler as distinct axes.
Now imagine you have Halpha and Hbeta lines in the same dataset (MUSE cubes?). The may come from different regions and the velocities will be different for both.
You could have a different velocity map for the two lines in the same dataset.
Cheers
François

Le 22/02/2023 à 17:06, Alberto Micol a écrit :
Many thanks François,

I will think thouroughly through your message, but first I would like to ask you to clarify the meaning of your sentence: “As Arnold pointed at that time you have datasets where you provide different doppler shift information at different spectral coordinates. What are the radial velocity bounds in that case ?”
Could you explain that with more words, maybe giving an example?

To explain why I do not understand that sentence, I give you an example of a cube I have at hand here.

In my case a velocity cube is a FITS cube with the third axis defined as:
CTYPE3 = ‘VRAD’
CUNIT3 = ‘m/s’
CD3_3_ = 250
CRVAL3 = 0
CRPIX3 = 141
NAXIS3 = 281

That is enough to state that:
V_min = -35000 m/s
V_max = +35000 m/s
Many thanks,
Alberto

From: BONNAREL FRANCOIS <francois.bonnarel at astro.unistra.fr><mailto:francois.bonnarel at astro.unistra.fr>
Date: Wednesday, 22 February 2023 at 16:46
To: alberto micol <amicol.ivoa at googlemail.com><mailto:amicol.ivoa at googlemail.com>, Alberto Micol <amicol at eso.org><mailto:amicol at eso.org>
Cc: "dal at ivoa.net"<mailto:dal at ivoa.net> <dal at ivoa.net><mailto:dal at ivoa.net>, Data Models mailing list <dm at ivoa.net><mailto:dm at ivoa.net>
Subject: Re: ObsCore for velocity cubes

This email was sent from outside of ESO from the address [francois.bonnarel at astro.unistra.fr<mailto:francois.bonnarel at astro.unistra.fr>]. If it looks suspicious, please report it to phishing at eso.org<mailto:phishing at eso.org>.

Dear Alberto, all,

    I come to this copying also to the dm list (ObsCoire and extensions are dm stuff). As you may remember this point was EXTENSIVELY discussed when we upgraded ObsCore  from 1.0 to 1.1.
    This discussion was mainly in 2014/2015. See for example these threads starting by http://mail.ivoa.net/pipermail/dm/2015-April/005147.html and http://mail.ivoa.net/pipermail/dm/2015-April/005168.html as well as http://mail.ivoa.net/pipermail/dm/2015-May/005174.html

      The conclusion at that time was that we didn't have to extend ObsCore this way eventually.
      For several reasons.
       - radial velocity or redshifts  derivation from spectral frequencies or wl are complex things. querying between vel = 10 km/s and 60 km/s may give different results according to the radial velocity (or dopplershift) flavor. And if we add the constraint em_ucd =  vrad (or vopt, or ) then the query become really specialised.
      - if the velocity cube is simply a transform of the frequency/wavelength axis  we still have to select on wavelength because velocity cubes in radio domain and velocity cube in optical domain are really different things
        -  But generally speaking the velocity axis is not ONLY a transform of the wavelength (or frequency) axis, it provides different information. It is a different axis (although related). As Arnold pointed at that time you have datasets where you provide different doppler shift information at different spectral coordinates. What are the radial velocity bounds in that case ?
        ---> ObsCore will become really complex to manage and fill in these situations

        Considering your use case Alberto, I think we have two distinguish two steps
            - data  discovery (With ObsTAP/ SIA) = it's probably enough to let the user know that you have velocity cubes by providing the appropriate values in em_ucd and em_unit (which are there to describe the dataset content and not the spectral characterization). We may think to add em_ref to give the wl of the reference   line. velocity limits will have to be transformed in wavelength to fill the mandatory em_min/max anyway. With this you already know that the data you discover by a standard ObsCore (or radio extension) query are "'velocity cubes"

             - SODA cutouts : there , the parameters you propose are useful. A discussion started already some time ago to extend SODA . For example adding pixel cutouts, or changing the projection or the spatial sampling of the cutout. In this context we could also choose to extract by forcing velocity constraints, when appropro$iate.
               To do that we need additional metadata (eg : the content of WCS,  and this also for the spectral axis). And for that, there is the old idea to have a SODA mode providing full metadata, CADC already implemented something like that. The velocity '"axis" detailed flavor can be described this way if it exists. This also has to be integrated in the SODA-next discussion. Some more on this will come soon.

Cheers
François

 1Le 22/02/2023 à 11:33, alberto micol a écrit :
Very well, thanks both Markus and Pat.

So, yes, we need a v_resolution, and it must be in fixed units.
Taking the Greisen et al. default, this is [m/s].

We though need the fravergy UCD to be able to discriminate the different velocity flavours.
Is em_ucd the correct field to use?
(Yes, I’m hit strongly by the confusion of throwing untested features from a data model into a relational mapping,
that’s why I need your confirmation on this)

Thanks!
Alberto

On 21 Feb 2023, at 21:11, Alberto Micol <amicol at eso.org<mailto:amicol at eso.org>> wrote:

Hi James,

Thanks for the answer. I have a question/doubt regarding the v_resolution and v_unit.

ObsCore already foresees the em_resolution, so I originally thought we do not need the v_resolution.

But now I am reconsidering this, because I think (in the ObsCore standard we were not super-clear regarding this) the units of em_resolution are the ones provided by the em_unit attribute. Different cubes could have different em_units, and that means that em_resolution could not be used to seamlessly query across all possible cubes.

If the above is correct, then, yes, we need to have a new field,
v_resolution, with units fixed to e.g. m/s,
similarly to the case of em_min/max always provided in [m], whatever the value of the em_unit, to support searches.

In which case, we do not need a v_unit, as v_min/max are fixed in [m/s] and we already have em_unit for the units encoded in the cube.

Would that make sense?

Regarding treating frequency cubes as velocity cubes (and viceversa),
it is indeed useful to provide characterisations both in frequency and in velocity
(and why not energy and wavelength), distinguishing the different cases according to
the value of the em_ucd, e.g., em.freq, em.wl, em.energy (see ObsCore B.6.2), or
spect.dopplerVeloc.radio, spect.dopplerVeloc.opt, spect.dopplerVeloc.rel (*) (see ObsCore B.6.2.6).

So we would have:

em_ucd required
v_resolution required, fixed in m/s
em_unit optional, expressing the unit encoded in the cube for the spectral axis
em_resolution optional, expressing the resolution in the units used by the data cube

If we converge on this, I could put together a little document with a detailed proposal.
Would that considered useful?

Cheers,
Alberto

(*) cross-standard issue: the spect.dopplerVeloc.rel is mentioned in the ObsCore standard,
but it is not a recognised value in the current UCD1+ standard.
We will need to act on this (Mireille?).

From: "Dempsey, James (IM&T, Black Mountain)" <James.Dempsey at csiro.au<mailto:James.Dempsey at csiro.au>>
Date: Thursday, 16 February 2023 at 00:21
To: Alberto Micol <amicol at eso.org<mailto:amicol at eso.org>>, "dal at ivoa.net<mailto:dal at ivoa.net>" <dal at ivoa.net<mailto:dal at ivoa.net>>
Subject: Re: ObsCore for velocity cubes

This email was sent from outside of ESO from the address[prvs=40330730a=James.Dempsey at csiro.au<mailto:prvs=40330730a=James.Dempsey at csiro.au>]. If it looks suspicious, please report it to phishing at eso.org<mailto:phishing at eso.org>.

Hi Alberto,

At CASDA we support cutouts of velocity spectral line cubes.

We currently require the user to provide a BAND or CHANNEL parameter to the SODA service which is translated by the service to the velocity frame of the target cube. This works, but requiring conversions on both the client and server side isn’t ideal. Likewise, in our ObsCore implementation only the em_ucd indicates the cubes are in velocity but it would certainly be useful to have full velocity information for data discovery!

The extension you outline looks like it would work nicely for the same use case in CASDA. However I think we should add the following to fully describe the velocity axis (as opposed to its translation into wavelength for the base obscore).

  1.  v_resolution – value of CDELT3 or CDELT4
  2.  v_unit – although this should always be m/s

Another consideration is that a lot of our users tend to use velocity to interact with frequency cubes. So even for these frequency cubes we might end up filling in velocity metadata if the fields were defined. Tools such as CARTA tend to automatically show the spectral axis as velocity where it can, such that many users would assume the cubes are in velocity not frequency.

Cheers,
James Dempsey
Senior Developer  |  CSIRO
james.dempsey at csiro.au<mailto:james.dempsey at csiro.au>  |  02 6214 2912

From: dal <dal-bounces at ivoa.net<mailto:dal-bounces at ivoa.net>> on behalf of alberto micol <amicol.ivoa at googlemail.com<mailto:amicol.ivoa at googlemail.com>>
Date: Thursday, 16 February 2023 at 12:02 am
To: dal at ivoa.net<mailto:dal at ivoa.net> <dal at ivoa.net<mailto:dal at ivoa.net>>
Subject: ObsCore for velocity cubes
Dear ObsCorers,

I have received with much interest the ObsCore extension for RADIO data, thanks for that very clear document.

Would it be possible to have a similar extension for cubes with velocity as third axis (the other two being the usual spatial ones)?
Let me call them “velocity cubes”...

Here my use case…

At ESO we need to support cutouts of velocity cubes.

Up to now our SODA interface supported spectra, images, and wavelength cubes;
to provide the user with the SODA input parameters and their possible MIN/MAX values
we query ivoa.ObsCore.

Now with the introduction of velocity cubes, we would like to get the necessary SODA input params also from ObsCore.
We can certainly do so by extending our own ObsCore table (ObsCore allows to extend the table), but I was wondering
if others have similar needs, and see if we can agree with a common ObsCore extension.

Velocity cubes should come in a FITS format as described by Greisen et al, 2006, so-called Paper 3.
Shortly summarised, the velocity can be either:

     *   a radio (VRAD),
     *   an optical (VOPT),
     *   a relativistic velocity (ZOPT),
     *   a generic apparent velocity (VELO).
The RESTFRQa (RESTWAVa) keyword conveys the necessary reference frequency (wavelength) to transform velocity into frequency or wavelength.
The velocity unit is m/s.

SODA cutouts on the velocity axis are usually formulated by passing cuts already expressed in velocity
though it must be possible to perform the conversions to and from frequency/wavelength.

What would then be needed in ObsCore is:

v_min
v_max

but also, in case of radio velocity cube:
f_min
f_max (getting these from the RADIO extension)
f_rest = RESTFRQ

and, in case of a optical or relativistic velocity cube
em_rest = RESTWAV
(we already have em_min, em_max)

The em_resolv_power should then be set to:
 lambda                    freq                     c
 ———————  =  ————— = —————
 delta lambda           delta freq       2 * delta_v

where delta_v is the CD3_3 in the cube (in the example of a cube with the velocity on the third axis);
but I would also likely accept a v_pixel_scale (= delta_v).

The UCD for the velocity should then be:

spect.dopplerVeloc.opt

spect.dopplerVeloc.radio
unsure what it should be for the relativistic case.

This would allow us at ESO to implement SODA on top of ObsCore (as already done for all other parameters)
and would allow astronomer to search the velocity cubes with certain characteristics using ObsCore.

What do you think?
Is that doable? I think at ESO we will simply start implementing the above extra ObsCore parameters (unless you suggest different names for that),
while waiting to see if there is more interest to build a proper standard.

Eager to know what you think,
Cheers,
Alberto

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.ivoa.net/pipermail/dal/attachments/20230224/e488bf1c/attachment-0001.htm>