<div dir="ltr"><div><div><div><div><div>Laurent,<br><br></div>I think this distinction is a red herring: instrumentally measured parameters are<br></div>always proxies for world coordinates and this is no different for photon counting<br></div>instruments.<br><br></div>Cheers,<br><br></div> - Arnold<br></div><div class="gmail_extra"><br clear="all"><div><div class="gmail_signature"><div dir="ltr">-------------------------------------------------------------------------------------------------------------<br>Arnold H. Rots Chandra X-ray Science Center<br>Smithsonian Astrophysical Observatory tel: +1 617 496 7701<br>60 Garden Street, MS 67 fax: +1 617 495 7356<br>Cambridge, MA 02138 <a href="mailto:arots@cfa.harvard.edu" target="_blank">arots@cfa.harvard.edu</a><br>USA <a href="http://hea-www.harvard.edu/~arots/" target="_blank">http://hea-www.harvard.edu/~arots/</a><br>--------------------------------------------------------------------------------------------------------------<br><br></div></div></div>
<br><div class="gmail_quote">On Tue, Oct 13, 2015 at 7:32 AM, Laurent Michel <span dir="ltr"><<a href="mailto:laurent.michel@astro.unistra.fr" target="_blank">laurent.michel@astro.unistra.fr</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Hello,<br>
<br>
>From the discussion it seems that event list properties are often related to a instrumental features.<br>
<br>
The initial goal of Obscore was multi-wavelength discovery. It is focused on describing footprint of a data product on the space/time/energy/... axes.<br>
The current Obscore description allows to discover Event lists matching a given coverage along these axes.<br>
The incoming draft will explicit a special use case for that with event lists.<br>
<br>
The detailed features such as the #event/SNR/... of events lists should be modelled in more details in another specification (e.g. NDCube).<br>
<br>
We would like to reach an agreement in Sydney about the scope delimitation of the different models.<br>
<br>
<br>
Cheers,<br>
<br>
Laurent & Mireille<span class=""><br>
<br>
Le 09/10/2015 21:59, Arnold Rots a écrit :<br>
</span><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><span class="">
Mark,<br>
<br>
I agree with your second point - event_dim would be the wrong choice.<br>
I suggested a while ago that if we were to have something like pixel_dim,<br>
which provides the total number of pixels (the product of all _dims in the<br>
case of a filled cube), that would be the proper place to put the number<br>
of events: it is the number of pixels provided.<br>
<br>
Cheers,<br>
<br>
- Arnold<br>
<br>
-------------------------------------------------------------------------------------------------------------<br>
Arnold H. Rots Chandra X-ray Science Center<br>
Smithsonian Astrophysical Observatory tel: <a href="tel:%2B1%20617%20496%207701" value="+16174967701" target="_blank">+1 617 496 7701</a><br>
60 Garden Street, MS 67 fax: <a href="tel:%2B1%20617%20495%207356" value="+16174957356" target="_blank">+1 617 495 7356</a><br></span><span class="">
Cambridge, MA 02138 <a href="mailto:arots@cfa.harvard.edu" target="_blank">arots@cfa.harvard.edu</a> <mailto:<a href="mailto:arots@cfa.harvard.edu" target="_blank">arots@cfa.harvard.edu</a>><br>
USA <a href="http://hea-www.harvard.edu/~arots/" rel="noreferrer" target="_blank">http://hea-www.harvard.edu/~arots/</a><br>
--------------------------------------------------------------------------------------------------------------<br>
<br>
<br></span><span class="">
On Fri, Oct 9, 2015 at 3:49 PM, CresitelloDittmar, Mark <<a href="mailto:mdittmar@cfa.harvard.edu" target="_blank">mdittmar@cfa.harvard.edu</a> <mailto:<a href="mailto:mdittmar@cfa.harvard.edu" target="_blank">mdittmar@cfa.harvard.edu</a>>> wrote:<br>
<br>
All,<br>
<br>
I do hope we get to spend some quality time on this topic soon (in Sydney?).<br>
I find myself agreeing with different parts of different people's statements!<br>
<br>
+ I see Arnold's point about the event list being pixelated.<br>
I've been thinking along the same lines in the past couple days, so my<br>
earlier classification of the event list in the 'non-pixelated' group may have<br>
been in error.<br>
The events are measured by a detector with finite pixel size and integration time,<br>
and these are, at least for Chandra, recorded in the event list (ccd_id, chipx, chipy)<br>
<br>
+ But I not convinced that #events is the proper value for any <domain>_dim field in ObsCore1.1<br>
<br>
+ The element/descriptions of the ObsCore1.1 *_dim fields are directed at the<br>
regularly sampled pixel array (Simple Image array) use case. My earlier comments was<br>
simply that the document should be specific about the values to place provide for this case<br>
AND what values to put for every other case (which I grouped as non-pixelated).<br>
So, the language may need to be tweaked, but this statement should be true.<br>
<br>
+ We will need to have a review of how these fields should be populated for each flavor<br>
of image/cube covered by the NDCube model.<br>
<br>
Mark<br>
<br>
<br>
<br>
<br>
<br></span><span class="">
On Fri, Oct 9, 2015 at 2:14 PM, Arnold Rots <<a href="mailto:arots@cfa.harvard.edu" target="_blank">arots@cfa.harvard.edu</a> <mailto:<a href="mailto:arots@cfa.harvard.edu" target="_blank">arots@cfa.harvard.edu</a>>> wrote:<br>
<br>
I beg to differ.<br>
Pixels (or voxels) are coordinate bins for which we provide the value of some dependent variable.<br>
That is precisely what we do for photon events, too, except that we enumerate all the pixels for<br>
which we have a measurement, rather than provide the dependent variable value in an array<br>
where the position in the array implies what the coordinate bins are. And that is another way<br>
of describing what I called a sparse cube.<br>
<br>
For instance, for Chandra data we provide the time stamp of the time bin (which does have a<br>
definite finite size), the CCD pixel coordinates, and the quantized energy of the photons.<br>
To me, that is a perfect example of a list of enumerated pixels.<br>
Whenever we measure things and express them digitally, we are effectively binning, either<br>
because we truncate, or because we round the value. And that means that we are projecting<br>
our information on a pixel grid.<br>
<br>
Let me be clear: I am not trying to make things more complicated; rather, I prefer to define<br>
our concepts in such a way that in a generalization as much as possible is covered.<br>
That does not mean that all possible special cases need to be implemented, but that we<br>
keep the top level definition in mind when we implement, so that we won't find ourselves<br>
later in a situation where we either can't accommodate an important use case, or have to<br>
go to extreme lengths to fit a round peg into a square hole.<br>
<br>
Cheers,<br>
<br>
- Arnold<br>
<br>
-------------------------------------------------------------------------------------------------------------<br>
Arnold H. Rots Chandra X-ray Science Center<br></span>
Smithsonian Astrophysical Observatory tel: <a href="tel:%2B1%20617%20496%207701" value="+16174967701" target="_blank">+1 617 496 7701</a> <tel:%2B1%20617%20496%207701><br>
60 Garden Street, MS 67 fax: <a href="tel:%2B1%20617%20495%207356" value="+16174957356" target="_blank">+1 617 495 7356</a> <tel:%2B1%20617%20495%207356><span class=""><br>
Cambridge, MA 02138 <a href="mailto:arots@cfa.harvard.edu" target="_blank">arots@cfa.harvard.edu</a> <mailto:<a href="mailto:arots@cfa.harvard.edu" target="_blank">arots@cfa.harvard.edu</a>><br>
USA <a href="http://hea-www.harvard.edu/~arots/" rel="noreferrer" target="_blank">http://hea-www.harvard.edu/~arots/</a><br>
--------------------------------------------------------------------------------------------------------------<br>
<br>
<br></span><span class="">
On Fri, Oct 9, 2015 at 6:40 AM, François Bonnarel <<a href="mailto:francois.bonnarel@astro.unistra.fr" target="_blank">francois.bonnarel@astro.unistra.fr</a><br></span><div><div class="h5">
<mailto:<a href="mailto:francois.bonnarel@astro.unistra.fr" target="_blank">francois.bonnarel@astro.unistra.fr</a>>> wrote:<br>
<br>
Hi all,<br>
The discussion to decide if an "event list" is a sparse cube because it is actually also a set of measurements<br>
or something radically different because it is made of photons is subtle.<br>
In any case, i think at the discovery purpose it is not really easy and usefull to code the number of events<br>
in one of the "dim" columns we have.<br>
Adding a new one to do so ? Or waiting for a model to describe this. I clearly prefer the second solution<br>
So "event list" in the dataproduct-type and NULL (or some other undetermined value) in the dim fields should be<br>
enough at this stage<br>
Cheers<br>
François<br>
<br>
Le 08/10/2015 20:53, Louys Mireille a écrit :<br>
<br>
Dear Tom, dear list members,<br>
<br>
I think I also support your point of view.<br>
<br>
Pixels or let say numerical elements along one characterized axis<br>
of a science ready data product are the result of a sampling step in the detectors, followed eventually by a<br>
projection in a<br>
specified coordinate system.<br>
An event , as far as is the physical phenomenon that is converted/projected into a grid element.<br>
<br>
There are scientific features needed for a safe and rich interpretation of the content of data set, that are not<br>
covered in Obscore.<br>
for instance :<br>
- the signal to noise ratio in flux,<br>
- the number on non null channels in a spectra<br>
- the detection limit for a source in an image<br>
- correlation factors between spaxels in a cube ,<br>
- the proportion of good time intervals in a CTA observation,<br>
- the spatial density of detected sources in an image or cube ( crowding factor- probably there should be a<br>
better english name for that, sorry )<br>
etc<br>
...<br>
all this information is closer to the astronomers concerns for advanced dataset selection, but is context dependent.<br>
I think the number of events in an event list belongs to this category.<br>
<br>
This could be interesting to explore further use-cases for these specialized physical features about the data<br>
content.<br>
<br>
ObsCore should keep its general properties for global discovery , and CubeDM could tackle the 'interpretation<br>
profile' for various kinds of observation.<br>
<br>
Best regards , Mireille<br>
<br>
<br>
<br>
<br>
<br>
<br>
<br>
<br>
Le 08/10/2015 19:45, Tom McGlynn (NASA/GSFC Code 660.1) a écrit :<br>
<br>
I confess that I find this a bit weird. E.g., consider a telescope with several coaligned photon-counting<br>
detectors (so multiple<br>
detectors are looking at the same or overlapping fields of view).<br>
<br>
After I combine detectors I can have sets of multiple photons with identical energy, time, and position.<br>
The photons may have been detected in different detectors originally, but the scientist has thrown that<br>
information away as irrelevant. The whole idea of a grid to me is that we only see each grid element once,<br>
but now we have the same grid element showing up multiple times. To me a grid value intrinsically<br>
represents some kind of integration over the grid area/volume. Events typically represent discrete photons.<br>
While there may be errors associated with the measurement of the characteristics of the photon (and<br>
Heisenberg suggests that there will be!) a photon is a different class of beast than a grid element. It's<br>
sort of like saying that when we are taking a census of people we should treat them as a grid of times and<br>
places of birth. We could do so but seeing the grid as the underlying structure seems wrong...<br>
<br>
Regards,<br>
Tom McGlyn<br>
<br>
Laurent Michel wrote:<br>
<br>
Arnold<br>
<br>
<br>
I agree with your remark. Event lists can be classified sparse cube. However, we do not have in Obscore<br>
a column giving a the product class (pixelated or not, sparse cube). We just have a product_type column<br>
with a limited vocabulary. There is no field giving explicitly the product class which could help to<br>
interpret correctly the content of the S-T-E axes.<br>
<br>
For now the point is to decide whether I) it's better to complete Obscore 1.1 with just the *dim and<br>
the extended list of product_type or II) whether we have to target a better description of these new<br>
categories of data products (new in term of Obscore) .<br>
In my opinion, the option I) is safer until the event lists are better taken into account and modelled<br>
in the VO.<br>
<br>
Bye<br>
Laurent<br>
<br>
Le 05/10/2015 18:48, Arnold Rots a écrit :<br>
<br>
I don't think event lists should be classified as /non-pixelated/.<br>
If we want to model data consistently, they should be classified as /pixel lists/:<br>
each event is a pixel; we just end up with a sparse hypercube.<br>
If we were to add one more parameter: the total number of pixels in<br>
the data product, that would solve the problem. For solid cubes it<br>
would be the product of the *dims, for event lists the number of events.<br>
<br>
- Arnold<br>
<br>
-------------------------------------------------------------------------------------------------------------<br>
<br>
Arnold H. Rots Chandra X-ray Science Center<br>
Smithsonian Astrophysical Observatory tel: <a href="tel:%2B1%20617%20496%207701" value="+16174967701" target="_blank">+1 617 496 7701</a><br></div></div>
<tel:%2B1%20617%20496%207701><br>
60 Garden Street, MS 67 fax: <a href="tel:%2B1%20617%20495%207356" value="+16174957356" target="_blank">+1 617 495 7356</a> <tel:%2B1%20617%20495%207356><span class=""><br>
Cambridge, MA 02138 <a href="mailto:arots@cfa.harvard.edu" target="_blank">arots@cfa.harvard.edu</a> <mailto:<a href="mailto:arots@cfa.harvard.edu" target="_blank">arots@cfa.harvard.edu</a>><br></span>
<mailto:<a href="mailto:arots@cfa.harvard.edu" target="_blank">arots@cfa.harvard.edu</a> <mailto:<a href="mailto:arots@cfa.harvard.edu" target="_blank">arots@cfa.harvard.edu</a>>><span class=""><br>
USA <a href="http://hea-www.harvard.edu/~arots/" rel="noreferrer" target="_blank">http://hea-www.harvard.edu/~arots/</a><br>
--------------------------------------------------------------------------------------------------------------<br>
<br>
<br>
<br>
On Mon, Oct 5, 2015 at 11:00 AM, Laurent Michel <<a href="mailto:laurent.michel@astro.unistra.fr" target="_blank">laurent.michel@astro.unistra.fr</a><br></span>
<mailto:<a href="mailto:laurent.michel@astro.unistra.fr" target="_blank">laurent.michel@astro.unistra.fr</a>> <mailto:<a href="mailto:laurent.michel@astro.unistra.fr" target="_blank">laurent.michel@astro.unistra.fr</a><div><div class="h5"><br>
<mailto:<a href="mailto:laurent.michel@astro.unistra.fr" target="_blank">laurent.michel@astro.unistra.fr</a>>>> wrote:<br>
<br>
Hello,<br>
<br>
A consensus seems to be taking shape about the model evolution for pixelated data:<br>
+ Adding several fields to provide the dimensionality in each domain. (s_dim, em_dim, etc)<br>
as described in the draft<br>
+ These are added as 'required' fields.. so must be populated for all records<br>
+ These fields provide for instance the size of each image axis.<br>
<br>
These new fields can however not be used to provide a robust representation the dimension of<br>
non pixelated data (e.g. number<br>
of event in an event list)<br>
<br>
The proposal to complete the version would be:<br>
1) To keep the new *dim* fields<br>
2) To support the EventList and other non pixelated as product_type<br>
3) To clearly specify that the *dim* columns must be set with NaN for non pixelated data.<br>
4) To provide some use cases about discovering non pixelated data with ObsTap<br>
<br>
That would quiclky allow us to start the RFC period for ObsCore without having to wait on a<br>
agreement about how to model non<br>
pixelated data<br>
<br>
<br>
Laurent<br>
<br>
--<br>
jesuischarlie<br>
<br>
Laurent Michel<br>
SSC XMM-Newton<br>
Tél : <a href="tel:%2B33%20%280%293%2068%2085%2024%2037" value="+33368852437" target="_blank">+33 (0)3 68 85 24 37</a> <tel:%2B33%20%280%293%2068%2085%2024%2037><br>
<tel:%2B33%20%280%293%2068%2085%2024%2037><br>
Fax : +33 (0)3 )3 68 85 24 32<br>
<a href="mailto:laurent.michel@astro.unistra.fr" target="_blank">laurent.michel@astro.unistra.fr</a> <mailto:<a href="mailto:laurent.michel@astro.unistra.fr" target="_blank">laurent.michel@astro.unistra.fr</a>><br>
<mailto:<a href="mailto:laurent.michel@astro.unistra.fr" target="_blank">laurent.michel@astro.unistra.fr</a> <mailto:<a href="mailto:laurent.michel@astro.unistra.fr" target="_blank">laurent.michel@astro.unistra.fr</a>>><br>
<mailto:<a href="mailto:laurent.michel@astro.unistra.fr" target="_blank">laurent.michel@astro.unistra.fr</a> <mailto:<a href="mailto:laurent.michel@astro.unistra.fr" target="_blank">laurent.michel@astro.unistra.fr</a>><br></div></div>
<mailto:<a href="mailto:laurent.michel@astro.unistra.fr" target="_blank">laurent.michel@astro.unistra.fr</a> <mailto:<a href="mailto:laurent.michel@astro.unistra.fr" target="_blank">laurent.michel@astro.unistra.fr</a>>>><span class=""><br>
Université de Strasbourg <<a href="http://www.unistra.fr" rel="noreferrer" target="_blank">http://www.unistra.fr</a>><br>
Observatoire Astronomique<br>
11 Rue de l'Université<br>
F - 67200 Strasbourg<br>
<a href="http://amwdb.u-strasbg.fr/HighEnergy/spip.php?rubrique34" rel="noreferrer" target="_blank">http://amwdb.u-strasbg.fr/HighEnergy/spip.php?rubrique34</a><br>
<br>
<br>
<br>
<br>
<br>
<br>
<br>
<br>
<br>
</span></blockquote><span class="im HOEnZb">
<br>
-- <br>
jesuischarlie<br>
<br>
Laurent Michel<br>
SSC XMM-Newton<br>
Tél : <a href="tel:%2B33%20%280%293%2068%2085%2024%2037" value="+33368852437" target="_blank">+33 (0)3 68 85 24 37</a><br></span><span class="im HOEnZb">
Fax : +33 (0)3 )3 68 85 24 32<br>
<a href="mailto:laurent.michel@astro.unistra.fr" target="_blank">laurent.michel@astro.unistra.fr</a> <mailto:<a href="mailto:laurent.michel@astro.unistra.fr" target="_blank">laurent.michel@astro.unistra.fr</a>><br></span><div class="HOEnZb"><div class="h5">
Université de Strasbourg <<a href="http://www.unistra.fr" rel="noreferrer" target="_blank">http://www.unistra.fr</a>><br>
Observatoire Astronomique<br>
11 Rue de l'Université<br>
F - 67200 Strasbourg<br>
<a href="http://amwdb.u-strasbg.fr/HighEnergy/spip.php?rubrique34" rel="noreferrer" target="_blank">http://amwdb.u-strasbg.fr/HighEnergy/spip.php?rubrique34</a><br>
</div></div></blockquote></div><br></div>