Obsc ore 1.1 WD
Tom McGlynn (NASA/GSFC Code 660.1)
tom.mcglynn at nasa.gov
Fri Oct 9 22:47:07 CEST 2015
Arnold Rots wrote:
> I beg to differ.
> Pixels (or voxels) are coordinate bins for which we provide the value
> of some dependent variable.
> That is precisely what we do for photon events, too, except that we
> enumerate all the pixels for
> which we have a measurement, rather than provide the dependent
> variable value in an array
> where the position in the array implies what the coordinate bins are.
> And that is another way
> of describing what I called a sparse cube.
>
I don't think that's right... Again consider my example with two or
more photons which have identical times, positions
and energies. That's perfectly feasible. To me a grid as a structure
implies several aspects:
- Pixels in the grid are not repeated
- The grid is in some sense regular
- Elements of the grid have boundaries with some reasonably simple
geometry
I'd see these as generally true for even a sparse grid. But none of
these are necessarily true for photon events.
- We can have photon events with identical characteristics so that if
they are treated as grid elements
then the grid elements must be repeated.
- Conversely the characteristics of detected photons can be wildly
different, so that the bounding
ranges of the energy/position/time vary wildly from photon to photon
there is no sense of regularity in the grid
- Although the photons may nominally be assigned some energy or
position, the bounds of the energy
position can be very soft and very complexly shaped so that any attempt
to treat the photon as
bounded by some kind of rectilinear boundaries is wrong. While Chandra
photons may be nicely bounded,
Fermi, EGRET and CompTel photon events exhibit these last two
characteristics in increasingly dramatic ways.
I don't dispute that you can shoehorn events into a sparse grid
structure but I think it conceals rather than reveals their
underlying nature, and I think trying to accommodate these kinds of
events if going to warp the underlying structure
where grids make sense.
Tom
> For instance, for Chandra data we provide the time stamp of the time
> bin (which does have a
> definite finite size), the CCD pixel coordinates, and the quantized
> energy of the photons.
> To me, that is a perfect example of a list of enumerated pixels.
> Whenever we measure things and express them digitally, we are
> effectively binning, either
> because we truncate, or because we round the value. And that means
> that we are projecting
> our information on a pixel grid.
>
> Let me be clear: I am not trying to make things more complicated;
> rather, I prefer to define
> our concepts in such a way that in a generalization as much as
> possible is covered.
> That does not mean that all possible special cases need to be
> implemented, but that we
> keep the top level definition in mind when we implement, so that we
> won't find ourselves
> later in a situation where we either can't accommodate an important
> use case, or have to
> go to extreme lengths to fit a round peg into a square hole.
>
> Cheers,
>
> - Arnold
>
> -------------------------------------------------------------------------------------------------------------
> Arnold H. Rots Chandra X-ray Science Center
> Smithsonian Astrophysical Observatory tel: +1 617 496 7701
> 60 Garden Street, MS 67 fax: +1 617 495 7356
> Cambridge, MA 02138 arots at cfa.harvard.edu <mailto:arots at cfa.harvard.edu>
> USA http://hea-www.harvard.edu/~arots/
> <http://hea-www.harvard.edu/%7Earots/>
> --------------------------------------------------------------------------------------------------------------
>
>
> On Fri, Oct 9, 2015 at 6:40 AM, François Bonnarel
> <francois.bonnarel at astro.unistra.fr
> <mailto:francois.bonnarel at astro.unistra.fr>> wrote:
>
> Hi all,
> The discussion to decide if an "event list" is a sparse cube
> because it is actually also a set of measurements or something
> radically different because it is made of photons is subtle.
> In any case, i think at the discovery purpose it is not
> really easy and usefull to code the number of events in one of the
> "dim" columns we have.
> Adding a new one to do so ? Or waiting for a model to describe
> this. I clearly prefer the second solution
> So "event list" in the dataproduct-type and NULL (or some other
> undetermined value) in the dim fields should be enough at this stage
> Cheers
> François
>
> Le 08/10/2015 20:53, Louys Mireille a écrit :
>
> Dear Tom, dear list members,
>
> I think I also support your point of view.
>
> Pixels or let say numerical elements along one characterized axis
> of a science ready data product are the result of a sampling
> step in the detectors, followed eventually by a projection in a
> specified coordinate system.
> An event , as far as is the physical phenomenon that is
> converted/projected into a grid element.
>
> There are scientific features needed for a safe and rich
> interpretation of the content of data set, that are not
> covered in Obscore.
> for instance :
> - the signal to noise ratio in flux,
> - the number on non null channels in a spectra
> - the detection limit for a source in an image
> - correlation factors between spaxels in a cube ,
> - the proportion of good time intervals in a CTA observation,
> - the spatial density of detected sources in an image or cube
> ( crowding factor- probably there should be a better english
> name for that, sorry )
> etc
> ...
> all this information is closer to the astronomers concerns for
> advanced dataset selection, but is context dependent.
> I think the number of events in an event list belongs to this
> category.
>
> This could be interesting to explore further use-cases for
> these specialized physical features about the data content.
>
> ObsCore should keep its general properties for global
> discovery , and CubeDM could tackle the 'interpretation
> profile' for various kinds of observation.
>
> Best regards , Mireille
>
>
>
>
>
>
>
>
> Le 08/10/2015 19:45, Tom McGlynn (NASA/GSFC Code 660.1) a écrit :
>
> I confess that I find this a bit weird. E.g., consider a
> telescope with several coaligned photon-counting detectors
> (so multiple
> detectors are looking at the same or overlapping fields of
> view).
>
> After I combine detectors I can have sets of multiple
> photons with identical energy, time, and position. The
> photons may have been detected in different detectors
> originally, but the scientist has thrown that information
> away as irrelevant. The whole idea of a grid to me is
> that we only see each grid element once, but now we have
> the same grid element showing up multiple times. To me a
> grid value intrinsically
> represents some kind of integration over the grid
> area/volume. Events typically represent discrete photons.
> While there may be errors associated with the measurement
> of the characteristics of the photon (and Heisenberg
> suggests that there will be!) a photon is a different
> class of beast than a grid element. It's sort of like
> saying that when we are taking a census of people we
> should treat them as a grid of times and places of birth.
> We could do so but seeing the grid as the underlying
> structure seems wrong...
>
> Regards,
> Tom McGlyn
>
> Laurent Michel wrote:
>
> Arnold
>
>
> I agree with your remark. Event lists can be
> classified sparse cube. However, we do not have in
> Obscore a column giving a the product class (pixelated
> or not, sparse cube). We just have a product_type
> column with a limited vocabulary. There is no field
> giving explicitly the product class which could help
> to interpret correctly the content of the S-T-E axes.
>
> For now the point is to decide whether I) it's better
> to complete Obscore 1.1 with just the *dim and the
> extended list of product_type or II) whether we have
> to target a better description of these new categories
> of data products (new in term of Obscore) .
> In my opinion, the option I) is safer until the event
> lists are better taken into account and modelled in
> the VO.
>
> Bye
> Laurent
>
> Le 05/10/2015 18:48, Arnold Rots a écrit :
>
> I don't think event lists should be classified as
> /non-pixelated/.
> If we want to model data consistently, they should
> be classified as /pixel lists/:
> each event is a pixel; we just end up with a
> sparse hypercube.
> If we were to add one more parameter: the total
> number of pixels in
> the data product, that would solve the problem.
> For solid cubes it
> would be the product of the *dims, for event lists
> the number of events.
>
> - Arnold
>
> -------------------------------------------------------------------------------------------------------------
>
> Arnold H. Rots Chandra X-ray Science Center
> Smithsonian Astrophysical Observatory tel:
> +1 617 496 7701 <tel:%2B1%20617%20496%207701>
> 60 Garden Street, MS 67 fax: +1 617 495 7356
> <tel:%2B1%20617%20495%207356>
> Cambridge, MA 02138 arots at cfa.harvard.edu
> <mailto:arots at cfa.harvard.edu>
> <mailto:arots at cfa.harvard.edu
> <mailto:arots at cfa.harvard.edu>>
> USA http://hea-www.harvard.edu/~arots/
> <http://hea-www.harvard.edu/%7Earots/>
> --------------------------------------------------------------------------------------------------------------
>
>
>
> On Mon, Oct 5, 2015 at 11:00 AM, Laurent Michel
> <laurent.michel at astro.unistra.fr
> <mailto:laurent.michel at astro.unistra.fr>
> <mailto:laurent.michel at astro.unistra.fr
> <mailto:laurent.michel at astro.unistra.fr>>> wrote:
>
> Hello,
>
> A consensus seems to be taking shape about the
> model evolution for pixelated data:
> + Adding several fields to provide the
> dimensionality in each domain. (s_dim, em_dim,
> etc) as described in the draft
> + These are added as 'required' fields.. so
> must be populated for all records
> + These fields provide for instance the size
> of each image axis.
>
> These new fields can however not be used to
> provide a robust representation the dimension of
> non pixelated data (e.g. number
> of event in an event list)
>
> The proposal to complete the version would be:
> 1) To keep the new *dim* fields
> 2) To support the EventList and other non
> pixelated as product_type
> 3) To clearly specify that the *dim* columns
> must be set with NaN for non pixelated data.
> 4) To provide some use cases about
> discovering non pixelated data with ObsTap
>
> That would quiclky allow us to start the RFC
> period for ObsCore without having to wait on a
> agreement about how to model non
> pixelated data
>
>
> Laurent
>
> --
> jesuischarlie
>
> Laurent Michel
> SSC XMM-Newton
> Tél : +33 (0)3 68 85 24 37
> <tel:%2B33%20%280%293%2068%2085%2024%2037>
> <tel:%2B33%20%280%293%2068%2085%2024%2037>
> Fax : +33 (0)3 )3 68 85 24 32
> laurent.michel at astro.unistra.fr
> <mailto:laurent.michel at astro.unistra.fr>
> <mailto:laurent.michel at astro.unistra.fr
> <mailto:laurent.michel at astro.unistra.fr>>
> <mailto:laurent.michel at astro.unistra.fr
> <mailto:laurent.michel at astro.unistra.fr>
> <mailto:laurent.michel at astro.unistra.fr
> <mailto:laurent.michel at astro.unistra.fr>>>
> Université de Strasbourg <http://www.unistra.fr>
> Observatoire Astronomique
> 11 Rue de l'Université
> F - 67200 Strasbourg
> http://amwdb.u-strasbg.fr/HighEnergy/spip.php?rubrique34
>
>
>
>
>
>
>
More information about the dm
mailing list