Obsc ore 1.1 WD
Arnold Rots
arots at cfa.harvard.edu
Fri Oct 9 21:59:04 CEST 2015
Mark,
I agree with your second point - event_dim would be the wrong choice.
I suggested a while ago that if we were to have something like pixel_dim,
which provides the total number of pixels (the product of all _dims in the
case of a filled cube), that would be the proper place to put the number
of events: it is the number of pixels provided.
Cheers,
- Arnold
-------------------------------------------------------------------------------------------------------------
Arnold H. Rots Chandra X-ray
Science Center
Smithsonian Astrophysical Observatory tel: +1 617 496
7701
60 Garden Street, MS 67 fax: +1 617
495 7356
Cambridge, MA 02138
arots at cfa.harvard.edu
USA
http://hea-www.harvard.edu/~arots/
--------------------------------------------------------------------------------------------------------------
On Fri, Oct 9, 2015 at 3:49 PM, CresitelloDittmar, Mark <
mdittmar at cfa.harvard.edu> wrote:
> All,
>
> I do hope we get to spend some quality time on this topic soon (in
> Sydney?).
> I find myself agreeing with different parts of different people's
> statements!
>
> + I see Arnold's point about the event list being pixelated.
> I've been thinking along the same lines in the past couple days, so my
> earlier classification of the event list in the 'non-pixelated' group
> may have
> been in error.
> The events are measured by a detector with finite pixel size and
> integration time,
> and these are, at least for Chandra, recorded in the event list
> (ccd_id, chipx, chipy)
>
> + But I not convinced that #events is the proper value for any
> <domain>_dim field in ObsCore1.1
>
> + The element/descriptions of the ObsCore1.1 *_dim fields are directed at
> the
> regularly sampled pixel array (Simple Image array) use case. My
> earlier comments was
> simply that the document should be specific about the values to place
> provide for this case
> AND what values to put for every other case (which I grouped as
> non-pixelated).
> So, the language may need to be tweaked, but this statement should be
> true.
>
> + We will need to have a review of how these fields should be populated
> for each flavor
> of image/cube covered by the NDCube model.
>
> Mark
>
>
>
>
>
> On Fri, Oct 9, 2015 at 2:14 PM, Arnold Rots <arots at cfa.harvard.edu> wrote:
>
>> I beg to differ.
>> Pixels (or voxels) are coordinate bins for which we provide the value of
>> some dependent variable.
>> That is precisely what we do for photon events, too, except that we
>> enumerate all the pixels for
>> which we have a measurement, rather than provide the dependent variable
>> value in an array
>> where the position in the array implies what the coordinate bins are. And
>> that is another way
>> of describing what I called a sparse cube.
>>
>> For instance, for Chandra data we provide the time stamp of the time bin
>> (which does have a
>> definite finite size), the CCD pixel coordinates, and the quantized
>> energy of the photons.
>> To me, that is a perfect example of a list of enumerated pixels.
>> Whenever we measure things and express them digitally, we are effectively
>> binning, either
>> because we truncate, or because we round the value. And that means that
>> we are projecting
>> our information on a pixel grid.
>>
>> Let me be clear: I am not trying to make things more complicated; rather,
>> I prefer to define
>> our concepts in such a way that in a generalization as much as possible
>> is covered.
>> That does not mean that all possible special cases need to be
>> implemented, but that we
>> keep the top level definition in mind when we implement, so that we won't
>> find ourselves
>> later in a situation where we either can't accommodate an important use
>> case, or have to
>> go to extreme lengths to fit a round peg into a square hole.
>>
>> Cheers,
>>
>> - Arnold
>>
>>
>> -------------------------------------------------------------------------------------------------------------
>> Arnold H. Rots Chandra X-ray
>> Science Center
>> Smithsonian Astrophysical Observatory tel: +1 617 496
>> 7701
>> 60 Garden Street, MS 67 fax: +1
>> 617 495 7356
>> Cambridge, MA 02138
>> arots at cfa.harvard.edu
>> USA
>> http://hea-www.harvard.edu/~arots/
>>
>> --------------------------------------------------------------------------------------------------------------
>>
>>
>> On Fri, Oct 9, 2015 at 6:40 AM, François Bonnarel <
>> francois.bonnarel at astro.unistra.fr> wrote:
>>
>>> Hi all,
>>> The discussion to decide if an "event list" is a sparse cube
>>> because it is actually also a set of measurements or something radically
>>> different because it is made of photons is subtle.
>>> In any case, i think at the discovery purpose it is not really easy
>>> and usefull to code the number of events in one of the "dim" columns we
>>> have.
>>> Adding a new one to do so ? Or waiting for a model to describe this. I
>>> clearly prefer the second solution
>>> So "event list" in the dataproduct-type and NULL (or some other
>>> undetermined value) in the dim fields should be enough at this stage
>>> Cheers
>>> François
>>>
>>> Le 08/10/2015 20:53, Louys Mireille a écrit :
>>>
>>>> Dear Tom, dear list members,
>>>>
>>>> I think I also support your point of view.
>>>>
>>>> Pixels or let say numerical elements along one characterized axis
>>>> of a science ready data product are the result of a sampling step in
>>>> the detectors, followed eventually by a projection in a
>>>> specified coordinate system.
>>>> An event , as far as is the physical phenomenon that is
>>>> converted/projected into a grid element.
>>>>
>>>> There are scientific features needed for a safe and rich interpretation
>>>> of the content of data set, that are not covered in Obscore.
>>>> for instance :
>>>> - the signal to noise ratio in flux,
>>>> - the number on non null channels in a spectra
>>>> - the detection limit for a source in an image
>>>> - correlation factors between spaxels in a cube ,
>>>> - the proportion of good time intervals in a CTA observation,
>>>> - the spatial density of detected sources in an image or cube (
>>>> crowding factor- probably there should be a better english name for that,
>>>> sorry )
>>>> etc
>>>> ...
>>>> all this information is closer to the astronomers concerns for advanced
>>>> dataset selection, but is context dependent.
>>>> I think the number of events in an event list belongs to this category.
>>>>
>>>> This could be interesting to explore further use-cases for these
>>>> specialized physical features about the data content.
>>>>
>>>> ObsCore should keep its general properties for global discovery , and
>>>> CubeDM could tackle the 'interpretation profile' for various kinds of
>>>> observation.
>>>>
>>>> Best regards , Mireille
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> Le 08/10/2015 19:45, Tom McGlynn (NASA/GSFC Code 660.1) a écrit :
>>>>
>>>>> I confess that I find this a bit weird. E.g., consider a telescope
>>>>> with several coaligned photon-counting detectors (so multiple
>>>>> detectors are looking at the same or overlapping fields of view).
>>>>>
>>>>> After I combine detectors I can have sets of multiple photons with
>>>>> identical energy, time, and position. The photons may have been detected
>>>>> in different detectors originally, but the scientist has thrown that
>>>>> information away as irrelevant. The whole idea of a grid to me is that we
>>>>> only see each grid element once, but now we have the same grid element
>>>>> showing up multiple times. To me a grid value intrinsically
>>>>> represents some kind of integration over the grid area/volume. Events
>>>>> typically represent discrete photons. While there may be errors associated
>>>>> with the measurement of the characteristics of the photon (and Heisenberg
>>>>> suggests that there will be!) a photon is a different class of beast than a
>>>>> grid element. It's sort of like saying that when we are taking a census
>>>>> of people we should treat them as a grid of times and places of birth. We
>>>>> could do so but seeing the grid as the underlying structure seems wrong...
>>>>>
>>>>> Regards,
>>>>> Tom McGlyn
>>>>>
>>>>> Laurent Michel wrote:
>>>>>
>>>>>> Arnold
>>>>>>
>>>>>>
>>>>>> I agree with your remark. Event lists can be classified sparse cube.
>>>>>> However, we do not have in Obscore a column giving a the product class
>>>>>> (pixelated or not, sparse cube). We just have a product_type column with a
>>>>>> limited vocabulary. There is no field giving explicitly the product class
>>>>>> which could help to interpret correctly the content of the S-T-E axes.
>>>>>>
>>>>>> For now the point is to decide whether I) it's better to complete
>>>>>> Obscore 1.1 with just the *dim and the extended list of product_type or
>>>>>> II) whether we have to target a better description of these new categories
>>>>>> of data products (new in term of Obscore) .
>>>>>> In my opinion, the option I) is safer until the event lists are
>>>>>> better taken into account and modelled in the VO.
>>>>>>
>>>>>> Bye
>>>>>> Laurent
>>>>>>
>>>>>> Le 05/10/2015 18:48, Arnold Rots a écrit :
>>>>>>
>>>>>>> I don't think event lists should be classified as /non-pixelated/.
>>>>>>> If we want to model data consistently, they should be classified as
>>>>>>> /pixel lists/:
>>>>>>> each event is a pixel; we just end up with a sparse hypercube.
>>>>>>> If we were to add one more parameter: the total number of pixels in
>>>>>>> the data product, that would solve the problem. For solid cubes it
>>>>>>> would be the product of the *dims, for event lists the number of
>>>>>>> events.
>>>>>>>
>>>>>>> - Arnold
>>>>>>>
>>>>>>> -------------------------------------------------------------------------------------------------------------
>>>>>>>
>>>>>>> Arnold H. Rots Chandra X-ray Science Center
>>>>>>> Smithsonian Astrophysical Observatory tel: +1 617
>>>>>>> 496 7701
>>>>>>> 60 Garden Street, MS 67 fax: +1 617 495 7356
>>>>>>> Cambridge, MA 02138 arots at cfa.harvard.edu <mailto:
>>>>>>> arots at cfa.harvard.edu>
>>>>>>> USA http://hea-www.harvard.edu/~arots/
>>>>>>> --------------------------------------------------------------------------------------------------------------
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Mon, Oct 5, 2015 at 11:00 AM, Laurent Michel <
>>>>>>> laurent.michel at astro.unistra.fr <mailto:
>>>>>>> laurent.michel at astro.unistra.fr>> wrote:
>>>>>>>
>>>>>>> Hello,
>>>>>>>
>>>>>>> A consensus seems to be taking shape about the model evolution
>>>>>>> for pixelated data:
>>>>>>> + Adding several fields to provide the dimensionality in each
>>>>>>> domain. (s_dim, em_dim, etc) as described in the draft
>>>>>>> + These are added as 'required' fields.. so must be populated
>>>>>>> for all records
>>>>>>> + These fields provide for instance the size of each image
>>>>>>> axis.
>>>>>>>
>>>>>>> These new fields can however not be used to provide a robust
>>>>>>> representation the dimension of non pixelated data (e.g. number
>>>>>>> of event in an event list)
>>>>>>>
>>>>>>> The proposal to complete the version would be:
>>>>>>> 1) To keep the new *dim* fields
>>>>>>> 2) To support the EventList and other non pixelated as
>>>>>>> product_type
>>>>>>> 3) To clearly specify that the *dim* columns must be set with
>>>>>>> NaN for non pixelated data.
>>>>>>> 4) To provide some use cases about discovering non pixelated
>>>>>>> data with ObsTap
>>>>>>>
>>>>>>> That would quiclky allow us to start the RFC period for ObsCore
>>>>>>> without having to wait on a agreement about how to model non
>>>>>>> pixelated data
>>>>>>>
>>>>>>>
>>>>>>> Laurent
>>>>>>>
>>>>>>> --
>>>>>>> jesuischarlie
>>>>>>>
>>>>>>> Laurent Michel
>>>>>>> SSC XMM-Newton
>>>>>>> Tél : +33 (0)3 68 85 24 37
>>>>>>> <tel:%2B33%20%280%293%2068%2085%2024%2037>
>>>>>>> Fax : +33 (0)3 )3 68 85 24 32
>>>>>>> laurent.michel at astro.unistra.fr <mailto:
>>>>>>> laurent.michel at astro.unistra.fr> <mailto:
>>>>>>> laurent.michel at astro.unistra.fr
>>>>>>> <mailto:laurent.michel at astro.unistra.fr>>
>>>>>>> Université de Strasbourg <http://www.unistra.fr>
>>>>>>> Observatoire Astronomique
>>>>>>> 11 Rue de l'Université
>>>>>>> F - 67200 Strasbourg
>>>>>>> http://amwdb.u-strasbg.fr/HighEnergy/spip.php?rubrique34
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.ivoa.net/pipermail/dm/attachments/20151009/a2c13758/attachment-0001.html>
More information about the dm
mailing list