<div dir="ltr"><div><div><div><div><div><div>If identical photons are detected by different detectors, then clearly<br></div>the detector Id should be one of the pixel axes.<br><br></div>I am not quite sure sure what you mean by soft boundaries, but if<br></div>it is a matter of how to translate pulse height into energy, then the<br></div>answer should be that pulse height ought to be the pixel axis.<br><br></div>Cheers,<br><br></div> - Arnold<br></div><div class="gmail_extra"><br clear="all"><div><div class="gmail_signature"><div dir="ltr">-------------------------------------------------------------------------------------------------------------<br>Arnold H. Rots Chandra X-ray Science Center<br>Smithsonian Astrophysical Observatory tel: +1 617 496 7701<br>60 Garden Street, MS 67 fax: +1 617 495 7356<br>Cambridge, MA 02138 <a href="mailto:arots@cfa.harvard.edu" target="_blank">arots@cfa.harvard.edu</a><br>USA <a href="http://hea-www.harvard.edu/~arots/" target="_blank">http://hea-www.harvard.edu/~arots/</a><br>--------------------------------------------------------------------------------------------------------------<br><br></div></div></div>
<br><div class="gmail_quote">On Fri, Oct 9, 2015 at 4:47 PM, Tom McGlynn (NASA/GSFC Code 660.1) <span dir="ltr"><<a href="mailto:tom.mcglynn@nasa.gov" target="_blank">tom.mcglynn@nasa.gov</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><span class="">Arnold Rots wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
I beg to differ.<br>
Pixels (or voxels) are coordinate bins for which we provide the value of some dependent variable.<br>
That is precisely what we do for photon events, too, except that we enumerate all the pixels for<br>
which we have a measurement, rather than provide the dependent variable value in an array<br>
where the position in the array implies what the coordinate bins are. And that is another way<br>
of describing what I called a sparse cube.<br>
<br>
</blockquote></span>
I don't think that's right... Again consider my example with two or more photons which have identical times, positions<br>
and energies. That's perfectly feasible. To me a grid as a structure implies several aspects:<br>
- Pixels in the grid are not repeated<br>
- The grid is in some sense regular<br>
- Elements of the grid have boundaries with some reasonably simple geometry<br>
I'd see these as generally true for even a sparse grid. But none of these are necessarily true for photon events.<br>
<br>
- We can have photon events with identical characteristics so that if they are treated as grid elements<br>
then the grid elements must be repeated.<br>
- Conversely the characteristics of detected photons can be wildly different, so that the bounding<br>
ranges of the energy/position/time vary wildly from photon to photon there is no sense of regularity in the grid<br>
- Although the photons may nominally be assigned some energy or position, the bounds of the energy<br>
position can be very soft and very complexly shaped so that any attempt to treat the photon as<br>
bounded by some kind of rectilinear boundaries is wrong. While Chandra photons may be nicely bounded,<br>
Fermi, EGRET and CompTel photon events exhibit these last two characteristics in increasingly dramatic ways.<br>
<br>
I don't dispute that you can shoehorn events into a sparse grid structure but I think it conceals rather than reveals their<br>
underlying nature, and I think trying to accommodate these kinds of events if going to warp the underlying structure<br>
where grids make sense.<br>
<br>
Tom<br>
<br>
<br>
<br>
<br>
<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><span class="">
For instance, for Chandra data we provide the time stamp of the time bin (which does have a<br>
definite finite size), the CCD pixel coordinates, and the quantized energy of the photons.<br>
To me, that is a perfect example of a list of enumerated pixels.<br>
Whenever we measure things and express them digitally, we are effectively binning, either<br>
because we truncate, or because we round the value. And that means that we are projecting<br>
our information on a pixel grid.<br>
<br>
Let me be clear: I am not trying to make things more complicated; rather, I prefer to define<br>
our concepts in such a way that in a generalization as much as possible is covered.<br>
That does not mean that all possible special cases need to be implemented, but that we<br>
keep the top level definition in mind when we implement, so that we won't find ourselves<br>
later in a situation where we either can't accommodate an important use case, or have to<br>
go to extreme lengths to fit a round peg into a square hole.<br>
<br>
Cheers,<br>
<br>
- Arnold<br>
<br>
-------------------------------------------------------------------------------------------------------------<br>
Arnold H. Rots Chandra X-ray Science Center<br>
Smithsonian Astrophysical Observatory tel: <a href="tel:%2B1%20617%20496%207701" value="+16174967701" target="_blank">+1 617 496 7701</a><br>
60 Garden Street, MS 67 fax: <a href="tel:%2B1%20617%20495%207356" value="+16174957356" target="_blank">+1 617 495 7356</a><br></span><span class="">
Cambridge, MA 02138 <a href="mailto:arots@cfa.harvard.edu" target="_blank">arots@cfa.harvard.edu</a> <mailto:<a href="mailto:arots@cfa.harvard.edu" target="_blank">arots@cfa.harvard.edu</a>><br></span>
USA <a href="http://hea-www.harvard.edu/~arots/" rel="noreferrer" target="_blank">http://hea-www.harvard.edu/~arots/</a> <<a href="http://hea-www.harvard.edu/%7Earots/" rel="noreferrer" target="_blank">http://hea-www.harvard.edu/%7Earots/</a>><br>
--------------------------------------------------------------------------------------------------------------<div><div class="h5"><br>
<br>
<br>
On Fri, Oct 9, 2015 at 6:40 AM, François Bonnarel <<a href="mailto:francois.bonnarel@astro.unistra.fr" target="_blank">francois.bonnarel@astro.unistra.fr</a> <mailto:<a href="mailto:francois.bonnarel@astro.unistra.fr" target="_blank">francois.bonnarel@astro.unistra.fr</a>>> wrote:<br>
<br>
Hi all,<br>
The discussion to decide if an "event list" is a sparse cube<br>
because it is actually also a set of measurements or something<br>
radically different because it is made of photons is subtle.<br>
In any case, i think at the discovery purpose it is not<br>
really easy and usefull to code the number of events in one of the<br>
"dim" columns we have.<br>
Adding a new one to do so ? Or waiting for a model to describe<br>
this. I clearly prefer the second solution<br>
So "event list" in the dataproduct-type and NULL (or some other<br>
undetermined value) in the dim fields should be enough at this stage<br>
Cheers<br>
François<br>
<br>
Le 08/10/2015 20:53, Louys Mireille a écrit :<br>
<br>
Dear Tom, dear list members,<br>
<br>
I think I also support your point of view.<br>
<br>
Pixels or let say numerical elements along one characterized axis<br>
of a science ready data product are the result of a sampling<br>
step in the detectors, followed eventually by a projection in a<br>
specified coordinate system.<br>
An event , as far as is the physical phenomenon that is<br>
converted/projected into a grid element.<br>
<br>
There are scientific features needed for a safe and rich<br>
interpretation of the content of data set, that are not<br>
covered in Obscore.<br>
for instance :<br>
- the signal to noise ratio in flux,<br>
- the number on non null channels in a spectra<br>
- the detection limit for a source in an image<br>
- correlation factors between spaxels in a cube ,<br>
- the proportion of good time intervals in a CTA observation,<br>
- the spatial density of detected sources in an image or cube<br>
( crowding factor- probably there should be a better english<br>
name for that, sorry )<br>
etc<br>
...<br>
all this information is closer to the astronomers concerns for<br>
advanced dataset selection, but is context dependent.<br>
I think the number of events in an event list belongs to this<br>
category.<br>
<br>
This could be interesting to explore further use-cases for<br>
these specialized physical features about the data content.<br>
<br>
ObsCore should keep its general properties for global<br>
discovery , and CubeDM could tackle the 'interpretation<br>
profile' for various kinds of observation.<br>
<br>
Best regards , Mireille<br>
<br>
<br>
<br>
<br>
<br>
<br>
<br>
<br>
Le 08/10/2015 19:45, Tom McGlynn (NASA/GSFC Code 660.1) a écrit :<br>
<br>
I confess that I find this a bit weird. E.g., consider a<br>
telescope with several coaligned photon-counting detectors<br>
(so multiple<br>
detectors are looking at the same or overlapping fields of<br>
view).<br>
<br>
After I combine detectors I can have sets of multiple<br>
photons with identical energy, time, and position. The<br>
photons may have been detected in different detectors<br>
originally, but the scientist has thrown that information<br>
away as irrelevant. The whole idea of a grid to me is<br>
that we only see each grid element once, but now we have<br>
the same grid element showing up multiple times. To me a<br>
grid value intrinsically<br>
represents some kind of integration over the grid<br>
area/volume. Events typically represent discrete photons. While there may be errors associated with the measurement<br>
of the characteristics of the photon (and Heisenberg<br>
suggests that there will be!) a photon is a different<br>
class of beast than a grid element. It's sort of like<br>
saying that when we are taking a census of people we<br>
should treat them as a grid of times and places of birth. We could do so but seeing the grid as the underlying<br>
structure seems wrong...<br>
<br>
Regards,<br>
Tom McGlyn<br>
<br>
Laurent Michel wrote:<br>
<br>
Arnold<br>
<br>
<br>
I agree with your remark. Event lists can be<br>
classified sparse cube. However, we do not have in<br>
Obscore a column giving a the product class (pixelated<br>
or not, sparse cube). We just have a product_type<br>
column with a limited vocabulary. There is no field<br>
giving explicitly the product class which could help<br>
to interpret correctly the content of the S-T-E axes.<br>
<br>
For now the point is to decide whether I) it's better<br>
to complete Obscore 1.1 with just the *dim and the<br>
extended list of product_type or II) whether we have<br>
to target a better description of these new categories<br>
of data products (new in term of Obscore) .<br>
In my opinion, the option I) is safer until the event<br>
lists are better taken into account and modelled in<br>
the VO.<br>
<br>
Bye<br>
Laurent<br>
<br>
Le 05/10/2015 18:48, Arnold Rots a écrit :<br>
<br>
I don't think event lists should be classified as<br>
/non-pixelated/.<br>
If we want to model data consistently, they should<br>
be classified as /pixel lists/:<br>
each event is a pixel; we just end up with a<br>
sparse hypercube.<br>
If we were to add one more parameter: the total<br>
number of pixels in<br>
the data product, that would solve the problem.<br>
For solid cubes it<br>
would be the product of the *dims, for event lists<br>
the number of events.<br>
<br>
- Arnold<br>
<br>
-------------------------------------------------------------------------------------------------------------<br>
<br>
Arnold H. Rots Chandra X-ray Science Center<br>
Smithsonian Astrophysical Observatory tel:<br></div></div>
<a href="tel:%2B1%20617%20496%207701" value="+16174967701" target="_blank">+1 617 496 7701</a> <tel:%2B1%20617%20496%207701><span class=""><br>
60 Garden Street, MS 67 fax: <a href="tel:%2B1%20617%20495%207356" value="+16174957356" target="_blank">+1 617 495 7356</a><br></span>
<tel:%2B1%20617%20495%207356><span class=""><br>
Cambridge, MA 02138 <a href="mailto:arots@cfa.harvard.edu" target="_blank">arots@cfa.harvard.edu</a><br>
<mailto:<a href="mailto:arots@cfa.harvard.edu" target="_blank">arots@cfa.harvard.edu</a>><br></span>
<mailto:<a href="mailto:arots@cfa.harvard.edu" target="_blank">arots@cfa.harvard.edu</a><span class=""><br>
<mailto:<a href="mailto:arots@cfa.harvard.edu" target="_blank">arots@cfa.harvard.edu</a>>><br>
USA <a href="http://hea-www.harvard.edu/~arots/" rel="noreferrer" target="_blank">http://hea-www.harvard.edu/~arots/</a><br></span>
<<a href="http://hea-www.harvard.edu/%7Earots/" rel="noreferrer" target="_blank">http://hea-www.harvard.edu/%7Earots/</a>><span class=""><br>
--------------------------------------------------------------------------------------------------------------<br>
<br>
<br>
<br>
On Mon, Oct 5, 2015 at 11:00 AM, Laurent Michel<br>
<<a href="mailto:laurent.michel@astro.unistra.fr" target="_blank">laurent.michel@astro.unistra.fr</a><br>
<mailto:<a href="mailto:laurent.michel@astro.unistra.fr" target="_blank">laurent.michel@astro.unistra.fr</a>><br></span>
<mailto:<a href="mailto:laurent.michel@astro.unistra.fr" target="_blank">laurent.michel@astro.unistra.fr</a><div><div class="h5"><br>
<mailto:<a href="mailto:laurent.michel@astro.unistra.fr" target="_blank">laurent.michel@astro.unistra.fr</a>>>> wrote:<br>
<br>
Hello,<br>
<br>
A consensus seems to be taking shape about the<br>
model evolution for pixelated data:<br>
+ Adding several fields to provide the<br>
dimensionality in each domain. (s_dim, em_dim,<br>
etc) as described in the draft<br>
+ These are added as 'required' fields.. so<br>
must be populated for all records<br>
+ These fields provide for instance the size<br>
of each image axis.<br>
<br>
These new fields can however not be used to<br>
provide a robust representation the dimension of<br>
non pixelated data (e.g. number<br>
of event in an event list)<br>
<br>
The proposal to complete the version would be:<br>
1) To keep the new *dim* fields<br>
2) To support the EventList and other non<br>
pixelated as product_type<br>
3) To clearly specify that the *dim* columns<br>
must be set with NaN for non pixelated data.<br>
4) To provide some use cases about<br>
discovering non pixelated data with ObsTap<br>
<br>
That would quiclky allow us to start the RFC<br>
period for ObsCore without having to wait on a<br>
agreement about how to model non<br>
pixelated data<br>
<br>
<br>
Laurent<br>
<br>
--<br>
jesuischarlie<br>
<br>
Laurent Michel<br>
SSC XMM-Newton<br>
Tél : <a href="tel:%2B33%20%280%293%2068%2085%2024%2037" value="+33368852437" target="_blank">+33 (0)3 68 85 24 37</a><br>
<tel:%2B33%20%280%293%2068%2085%2024%2037><br>
<tel:%2B33%20%280%293%2068%2085%2024%2037><br>
Fax : +33 (0)3 )3 68 85 24 32<br>
<a href="mailto:laurent.michel@astro.unistra.fr" target="_blank">laurent.michel@astro.unistra.fr</a><br>
<mailto:<a href="mailto:laurent.michel@astro.unistra.fr" target="_blank">laurent.michel@astro.unistra.fr</a>><br>
<mailto:<a href="mailto:laurent.michel@astro.unistra.fr" target="_blank">laurent.michel@astro.unistra.fr</a><br>
<mailto:<a href="mailto:laurent.michel@astro.unistra.fr" target="_blank">laurent.michel@astro.unistra.fr</a>>><br>
<mailto:<a href="mailto:laurent.michel@astro.unistra.fr" target="_blank">laurent.michel@astro.unistra.fr</a><br>
<mailto:<a href="mailto:laurent.michel@astro.unistra.fr" target="_blank">laurent.michel@astro.unistra.fr</a>><br>
<mailto:<a href="mailto:laurent.michel@astro.unistra.fr" target="_blank">laurent.michel@astro.unistra.fr</a><br>
<mailto:<a href="mailto:laurent.michel@astro.unistra.fr" target="_blank">laurent.michel@astro.unistra.fr</a>>>><br>
Université de Strasbourg <<a href="http://www.unistra.fr" rel="noreferrer" target="_blank">http://www.unistra.fr</a>><br>
Observatoire Astronomique<br>
11 Rue de l'Université<br>
F - 67200 Strasbourg<br>
<a href="http://amwdb.u-strasbg.fr/HighEnergy/spip.php?rubrique34" rel="noreferrer" target="_blank">http://amwdb.u-strasbg.fr/HighEnergy/spip.php?rubrique34</a><br>
<br>
<br>
<br>
<br>
<br>
<br>
<br>
</div></div></blockquote>
<br>
</blockquote></div><br></div>