Obsc ore 1.1 WD
Arnold Rots
arots at cfa.harvard.edu
Fri Oct 9 20:14:25 CEST 2015
I beg to differ.
Pixels (or voxels) are coordinate bins for which we provide the value of
some dependent variable.
That is precisely what we do for photon events, too, except that we
enumerate all the pixels for
which we have a measurement, rather than provide the dependent variable
value in an array
where the position in the array implies what the coordinate bins are. And
that is another way
of describing what I called a sparse cube.
For instance, for Chandra data we provide the time stamp of the time bin
(which does have a
definite finite size), the CCD pixel coordinates, and the quantized energy
of the photons.
To me, that is a perfect example of a list of enumerated pixels.
Whenever we measure things and express them digitally, we are effectively
binning, either
because we truncate, or because we round the value. And that means that we
are projecting
our information on a pixel grid.
Let me be clear: I am not trying to make things more complicated; rather, I
prefer to define
our concepts in such a way that in a generalization as much as possible is
covered.
That does not mean that all possible special cases need to be implemented,
but that we
keep the top level definition in mind when we implement, so that we won't
find ourselves
later in a situation where we either can't accommodate an important use
case, or have to
go to extreme lengths to fit a round peg into a square hole.
Cheers,
- Arnold
-------------------------------------------------------------------------------------------------------------
Arnold H. Rots Chandra X-ray
Science Center
Smithsonian Astrophysical Observatory tel: +1 617 496
7701
60 Garden Street, MS 67 fax: +1 617
495 7356
Cambridge, MA 02138
arots at cfa.harvard.edu
USA
http://hea-www.harvard.edu/~arots/
--------------------------------------------------------------------------------------------------------------
On Fri, Oct 9, 2015 at 6:40 AM, François Bonnarel <
francois.bonnarel at astro.unistra.fr> wrote:
> Hi all,
> The discussion to decide if an "event list" is a sparse cube because
> it is actually also a set of measurements or something radically different
> because it is made of photons is subtle.
> In any case, i think at the discovery purpose it is not really easy
> and usefull to code the number of events in one of the "dim" columns we
> have.
> Adding a new one to do so ? Or waiting for a model to describe this. I
> clearly prefer the second solution
> So "event list" in the dataproduct-type and NULL (or some other
> undetermined value) in the dim fields should be enough at this stage
> Cheers
> François
>
> Le 08/10/2015 20:53, Louys Mireille a écrit :
>
>> Dear Tom, dear list members,
>>
>> I think I also support your point of view.
>>
>> Pixels or let say numerical elements along one characterized axis
>> of a science ready data product are the result of a sampling step in the
>> detectors, followed eventually by a projection in a
>> specified coordinate system.
>> An event , as far as is the physical phenomenon that is
>> converted/projected into a grid element.
>>
>> There are scientific features needed for a safe and rich interpretation
>> of the content of data set, that are not covered in Obscore.
>> for instance :
>> - the signal to noise ratio in flux,
>> - the number on non null channels in a spectra
>> - the detection limit for a source in an image
>> - correlation factors between spaxels in a cube ,
>> - the proportion of good time intervals in a CTA observation,
>> - the spatial density of detected sources in an image or cube ( crowding
>> factor- probably there should be a better english name for that, sorry )
>> etc
>> ...
>> all this information is closer to the astronomers concerns for advanced
>> dataset selection, but is context dependent.
>> I think the number of events in an event list belongs to this category.
>>
>> This could be interesting to explore further use-cases for these
>> specialized physical features about the data content.
>>
>> ObsCore should keep its general properties for global discovery , and
>> CubeDM could tackle the 'interpretation profile' for various kinds of
>> observation.
>>
>> Best regards , Mireille
>>
>>
>>
>>
>>
>>
>>
>>
>> Le 08/10/2015 19:45, Tom McGlynn (NASA/GSFC Code 660.1) a écrit :
>>
>>> I confess that I find this a bit weird. E.g., consider a telescope with
>>> several coaligned photon-counting detectors (so multiple
>>> detectors are looking at the same or overlapping fields of view).
>>>
>>> After I combine detectors I can have sets of multiple photons with
>>> identical energy, time, and position. The photons may have been detected
>>> in different detectors originally, but the scientist has thrown that
>>> information away as irrelevant. The whole idea of a grid to me is that we
>>> only see each grid element once, but now we have the same grid element
>>> showing up multiple times. To me a grid value intrinsically
>>> represents some kind of integration over the grid area/volume. Events
>>> typically represent discrete photons. While there may be errors associated
>>> with the measurement of the characteristics of the photon (and Heisenberg
>>> suggests that there will be!) a photon is a different class of beast than a
>>> grid element. It's sort of like saying that when we are taking a census
>>> of people we should treat them as a grid of times and places of birth. We
>>> could do so but seeing the grid as the underlying structure seems wrong...
>>>
>>> Regards,
>>> Tom McGlyn
>>>
>>> Laurent Michel wrote:
>>>
>>>> Arnold
>>>>
>>>>
>>>> I agree with your remark. Event lists can be classified sparse cube.
>>>> However, we do not have in Obscore a column giving a the product class
>>>> (pixelated or not, sparse cube). We just have a product_type column with a
>>>> limited vocabulary. There is no field giving explicitly the product class
>>>> which could help to interpret correctly the content of the S-T-E axes.
>>>>
>>>> For now the point is to decide whether I) it's better to complete
>>>> Obscore 1.1 with just the *dim and the extended list of product_type or
>>>> II) whether we have to target a better description of these new categories
>>>> of data products (new in term of Obscore) .
>>>> In my opinion, the option I) is safer until the event lists are better
>>>> taken into account and modelled in the VO.
>>>>
>>>> Bye
>>>> Laurent
>>>>
>>>> Le 05/10/2015 18:48, Arnold Rots a écrit :
>>>>
>>>>> I don't think event lists should be classified as /non-pixelated/.
>>>>> If we want to model data consistently, they should be classified as
>>>>> /pixel lists/:
>>>>> each event is a pixel; we just end up with a sparse hypercube.
>>>>> If we were to add one more parameter: the total number of pixels in
>>>>> the data product, that would solve the problem. For solid cubes it
>>>>> would be the product of the *dims, for event lists the number of
>>>>> events.
>>>>>
>>>>> - Arnold
>>>>>
>>>>> -------------------------------------------------------------------------------------------------------------
>>>>>
>>>>> Arnold H. Rots Chandra X-ray Science Center
>>>>> Smithsonian Astrophysical Observatory tel: +1 617
>>>>> 496 7701
>>>>> 60 Garden Street, MS 67 fax: +1 617 495 7356
>>>>> Cambridge, MA 02138 arots at cfa.harvard.edu <mailto:
>>>>> arots at cfa.harvard.edu>
>>>>> USA http://hea-www.harvard.edu/~arots/
>>>>> --------------------------------------------------------------------------------------------------------------
>>>>>
>>>>>
>>>>>
>>>>> On Mon, Oct 5, 2015 at 11:00 AM, Laurent Michel <
>>>>> laurent.michel at astro.unistra.fr <mailto:
>>>>> laurent.michel at astro.unistra.fr>> wrote:
>>>>>
>>>>> Hello,
>>>>>
>>>>> A consensus seems to be taking shape about the model evolution for
>>>>> pixelated data:
>>>>> + Adding several fields to provide the dimensionality in each
>>>>> domain. (s_dim, em_dim, etc) as described in the draft
>>>>> + These are added as 'required' fields.. so must be populated
>>>>> for all records
>>>>> + These fields provide for instance the size of each image axis.
>>>>>
>>>>> These new fields can however not be used to provide a robust
>>>>> representation the dimension of non pixelated data (e.g. number
>>>>> of event in an event list)
>>>>>
>>>>> The proposal to complete the version would be:
>>>>> 1) To keep the new *dim* fields
>>>>> 2) To support the EventList and other non pixelated as
>>>>> product_type
>>>>> 3) To clearly specify that the *dim* columns must be set with
>>>>> NaN for non pixelated data.
>>>>> 4) To provide some use cases about discovering non pixelated
>>>>> data with ObsTap
>>>>>
>>>>> That would quiclky allow us to start the RFC period for ObsCore
>>>>> without having to wait on a agreement about how to model non
>>>>> pixelated data
>>>>>
>>>>>
>>>>> Laurent
>>>>>
>>>>> --
>>>>> jesuischarlie
>>>>>
>>>>> Laurent Michel
>>>>> SSC XMM-Newton
>>>>> Tél : +33 (0)3 68 85 24 37
>>>>> <tel:%2B33%20%280%293%2068%2085%2024%2037>
>>>>> Fax : +33 (0)3 )3 68 85 24 32
>>>>> laurent.michel at astro.unistra.fr <mailto:
>>>>> laurent.michel at astro.unistra.fr> <mailto:
>>>>> laurent.michel at astro.unistra.fr
>>>>> <mailto:laurent.michel at astro.unistra.fr>>
>>>>> Université de Strasbourg <http://www.unistra.fr>
>>>>> Observatoire Astronomique
>>>>> 11 Rue de l'Université
>>>>> F - 67200 Strasbourg
>>>>> http://amwdb.u-strasbg.fr/HighEnergy/spip.php?rubrique34
>>>>>
>>>>>
>>>>>
>>>>
>>>
>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.ivoa.net/pipermail/dm/attachments/20151009/923a983d/attachment-0001.html>
More information about the dm
mailing list