Obsc ore 1.1 WD
CresitelloDittmar, Mark
mdittmar at cfa.harvard.edu
Fri Oct 9 21:49:38 CEST 2015
All,
I do hope we get to spend some quality time on this topic soon (in Sydney?).
I find myself agreeing with different parts of different people's
statements!
+ I see Arnold's point about the event list being pixelated.
I've been thinking along the same lines in the past couple days, so my
earlier classification of the event list in the 'non-pixelated' group
may have
been in error.
The events are measured by a detector with finite pixel size and
integration time,
and these are, at least for Chandra, recorded in the event list (ccd_id,
chipx, chipy)
+ But I not convinced that #events is the proper value for any <domain>_dim
field in ObsCore1.1
+ The element/descriptions of the ObsCore1.1 *_dim fields are directed at
the
regularly sampled pixel array (Simple Image array) use case. My
earlier comments was
simply that the document should be specific about the values to place
provide for this case
AND what values to put for every other case (which I grouped as
non-pixelated).
So, the language may need to be tweaked, but this statement should be
true.
+ We will need to have a review of how these fields should be populated for
each flavor
of image/cube covered by the NDCube model.
Mark
On Fri, Oct 9, 2015 at 2:14 PM, Arnold Rots <arots at cfa.harvard.edu> wrote:
> I beg to differ.
> Pixels (or voxels) are coordinate bins for which we provide the value of
> some dependent variable.
> That is precisely what we do for photon events, too, except that we
> enumerate all the pixels for
> which we have a measurement, rather than provide the dependent variable
> value in an array
> where the position in the array implies what the coordinate bins are. And
> that is another way
> of describing what I called a sparse cube.
>
> For instance, for Chandra data we provide the time stamp of the time bin
> (which does have a
> definite finite size), the CCD pixel coordinates, and the quantized energy
> of the photons.
> To me, that is a perfect example of a list of enumerated pixels.
> Whenever we measure things and express them digitally, we are effectively
> binning, either
> because we truncate, or because we round the value. And that means that we
> are projecting
> our information on a pixel grid.
>
> Let me be clear: I am not trying to make things more complicated; rather,
> I prefer to define
> our concepts in such a way that in a generalization as much as possible is
> covered.
> That does not mean that all possible special cases need to be implemented,
> but that we
> keep the top level definition in mind when we implement, so that we won't
> find ourselves
> later in a situation where we either can't accommodate an important use
> case, or have to
> go to extreme lengths to fit a round peg into a square hole.
>
> Cheers,
>
> - Arnold
>
>
> -------------------------------------------------------------------------------------------------------------
> Arnold H. Rots Chandra X-ray
> Science Center
> Smithsonian Astrophysical Observatory tel: +1 617 496
> 7701
> 60 Garden Street, MS 67 fax: +1 617
> 495 7356
> Cambridge, MA 02138
> arots at cfa.harvard.edu
> USA
> http://hea-www.harvard.edu/~arots/
>
> --------------------------------------------------------------------------------------------------------------
>
>
> On Fri, Oct 9, 2015 at 6:40 AM, François Bonnarel <
> francois.bonnarel at astro.unistra.fr> wrote:
>
>> Hi all,
>> The discussion to decide if an "event list" is a sparse cube because
>> it is actually also a set of measurements or something radically different
>> because it is made of photons is subtle.
>> In any case, i think at the discovery purpose it is not really easy
>> and usefull to code the number of events in one of the "dim" columns we
>> have.
>> Adding a new one to do so ? Or waiting for a model to describe this. I
>> clearly prefer the second solution
>> So "event list" in the dataproduct-type and NULL (or some other
>> undetermined value) in the dim fields should be enough at this stage
>> Cheers
>> François
>>
>> Le 08/10/2015 20:53, Louys Mireille a écrit :
>>
>>> Dear Tom, dear list members,
>>>
>>> I think I also support your point of view.
>>>
>>> Pixels or let say numerical elements along one characterized axis
>>> of a science ready data product are the result of a sampling step in
>>> the detectors, followed eventually by a projection in a
>>> specified coordinate system.
>>> An event , as far as is the physical phenomenon that is
>>> converted/projected into a grid element.
>>>
>>> There are scientific features needed for a safe and rich interpretation
>>> of the content of data set, that are not covered in Obscore.
>>> for instance :
>>> - the signal to noise ratio in flux,
>>> - the number on non null channels in a spectra
>>> - the detection limit for a source in an image
>>> - correlation factors between spaxels in a cube ,
>>> - the proportion of good time intervals in a CTA observation,
>>> - the spatial density of detected sources in an image or cube ( crowding
>>> factor- probably there should be a better english name for that, sorry )
>>> etc
>>> ...
>>> all this information is closer to the astronomers concerns for advanced
>>> dataset selection, but is context dependent.
>>> I think the number of events in an event list belongs to this category.
>>>
>>> This could be interesting to explore further use-cases for these
>>> specialized physical features about the data content.
>>>
>>> ObsCore should keep its general properties for global discovery , and
>>> CubeDM could tackle the 'interpretation profile' for various kinds of
>>> observation.
>>>
>>> Best regards , Mireille
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> Le 08/10/2015 19:45, Tom McGlynn (NASA/GSFC Code 660.1) a écrit :
>>>
>>>> I confess that I find this a bit weird. E.g., consider a telescope with
>>>> several coaligned photon-counting detectors (so multiple
>>>> detectors are looking at the same or overlapping fields of view).
>>>>
>>>> After I combine detectors I can have sets of multiple photons with
>>>> identical energy, time, and position. The photons may have been detected
>>>> in different detectors originally, but the scientist has thrown that
>>>> information away as irrelevant. The whole idea of a grid to me is that we
>>>> only see each grid element once, but now we have the same grid element
>>>> showing up multiple times. To me a grid value intrinsically
>>>> represents some kind of integration over the grid area/volume. Events
>>>> typically represent discrete photons. While there may be errors associated
>>>> with the measurement of the characteristics of the photon (and Heisenberg
>>>> suggests that there will be!) a photon is a different class of beast than a
>>>> grid element. It's sort of like saying that when we are taking a census
>>>> of people we should treat them as a grid of times and places of birth. We
>>>> could do so but seeing the grid as the underlying structure seems wrong...
>>>>
>>>> Regards,
>>>> Tom McGlyn
>>>>
>>>> Laurent Michel wrote:
>>>>
>>>>> Arnold
>>>>>
>>>>>
>>>>> I agree with your remark. Event lists can be classified sparse cube.
>>>>> However, we do not have in Obscore a column giving a the product class
>>>>> (pixelated or not, sparse cube). We just have a product_type column with a
>>>>> limited vocabulary. There is no field giving explicitly the product class
>>>>> which could help to interpret correctly the content of the S-T-E axes.
>>>>>
>>>>> For now the point is to decide whether I) it's better to complete
>>>>> Obscore 1.1 with just the *dim and the extended list of product_type or
>>>>> II) whether we have to target a better description of these new categories
>>>>> of data products (new in term of Obscore) .
>>>>> In my opinion, the option I) is safer until the event lists are better
>>>>> taken into account and modelled in the VO.
>>>>>
>>>>> Bye
>>>>> Laurent
>>>>>
>>>>> Le 05/10/2015 18:48, Arnold Rots a écrit :
>>>>>
>>>>>> I don't think event lists should be classified as /non-pixelated/.
>>>>>> If we want to model data consistently, they should be classified as
>>>>>> /pixel lists/:
>>>>>> each event is a pixel; we just end up with a sparse hypercube.
>>>>>> If we were to add one more parameter: the total number of pixels in
>>>>>> the data product, that would solve the problem. For solid cubes it
>>>>>> would be the product of the *dims, for event lists the number of
>>>>>> events.
>>>>>>
>>>>>> - Arnold
>>>>>>
>>>>>> -------------------------------------------------------------------------------------------------------------
>>>>>>
>>>>>> Arnold H. Rots Chandra X-ray Science Center
>>>>>> Smithsonian Astrophysical Observatory tel: +1 617
>>>>>> 496 7701
>>>>>> 60 Garden Street, MS 67 fax: +1 617 495 7356
>>>>>> Cambridge, MA 02138 arots at cfa.harvard.edu <mailto:
>>>>>> arots at cfa.harvard.edu>
>>>>>> USA http://hea-www.harvard.edu/~arots/
>>>>>> --------------------------------------------------------------------------------------------------------------
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Mon, Oct 5, 2015 at 11:00 AM, Laurent Michel <
>>>>>> laurent.michel at astro.unistra.fr <mailto:
>>>>>> laurent.michel at astro.unistra.fr>> wrote:
>>>>>>
>>>>>> Hello,
>>>>>>
>>>>>> A consensus seems to be taking shape about the model evolution
>>>>>> for pixelated data:
>>>>>> + Adding several fields to provide the dimensionality in each
>>>>>> domain. (s_dim, em_dim, etc) as described in the draft
>>>>>> + These are added as 'required' fields.. so must be populated
>>>>>> for all records
>>>>>> + These fields provide for instance the size of each image axis.
>>>>>>
>>>>>> These new fields can however not be used to provide a robust
>>>>>> representation the dimension of non pixelated data (e.g. number
>>>>>> of event in an event list)
>>>>>>
>>>>>> The proposal to complete the version would be:
>>>>>> 1) To keep the new *dim* fields
>>>>>> 2) To support the EventList and other non pixelated as
>>>>>> product_type
>>>>>> 3) To clearly specify that the *dim* columns must be set with
>>>>>> NaN for non pixelated data.
>>>>>> 4) To provide some use cases about discovering non pixelated
>>>>>> data with ObsTap
>>>>>>
>>>>>> That would quiclky allow us to start the RFC period for ObsCore
>>>>>> without having to wait on a agreement about how to model non
>>>>>> pixelated data
>>>>>>
>>>>>>
>>>>>> Laurent
>>>>>>
>>>>>> --
>>>>>> jesuischarlie
>>>>>>
>>>>>> Laurent Michel
>>>>>> SSC XMM-Newton
>>>>>> Tél : +33 (0)3 68 85 24 37
>>>>>> <tel:%2B33%20%280%293%2068%2085%2024%2037>
>>>>>> Fax : +33 (0)3 )3 68 85 24 32
>>>>>> laurent.michel at astro.unistra.fr <mailto:
>>>>>> laurent.michel at astro.unistra.fr> <mailto:
>>>>>> laurent.michel at astro.unistra.fr
>>>>>> <mailto:laurent.michel at astro.unistra.fr>>
>>>>>> Université de Strasbourg <http://www.unistra.fr>
>>>>>> Observatoire Astronomique
>>>>>> 11 Rue de l'Université
>>>>>> F - 67200 Strasbourg
>>>>>> http://amwdb.u-strasbg.fr/HighEnergy/spip.php?rubrique34
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.ivoa.net/pipermail/dm/attachments/20151009/8aef6ec6/attachment.html>
More information about the dm
mailing list