Fwd: Advanced Column Statistics

[ada nebot]

Dear Markus, dear Registry, 

I think it would be great to be able to get information on the distribution 
of a specific column. So I’d like to follow-up on this Note you distributed 
a while ago…  

Has there been any further take-up from providers?   

Some thoughts… 
Adding higher order moments, the skewness and the kurtosis, 
might give information on the underlying distribution. 
That said, any choice can be debated. And percentiles are easy to understand 
and calculate. I just find it a bit odd not to add values so used as mean and 
standard deviation, but given the fact that underlying population might not be 
a normal distribution, if added / used it deserves a bit of caution.

Thanks,
Ada

--
Astronome Adjointe
CDS, Observatoire Astronomique de Strasbourg (ObAS)
UMR 7550 Université de Strasbourg 
11, rue de l'Université, F-67000 Strasbourg
+33 (0) 3 68 85 24 20

> Begin forwarded message:
> 
> From: Markus Demleitner <msdemlei at ari.uni-heidelberg.de>
> Subject: Advanced Column Statistics
> Date: 30 April 2021 at 12:42:21 CEST
> To: interop at ivoa.net
> 
> Dear VO community,
> 
> Since doing Registry searches using spatial, temporal, and spectral
> constraints[1] is slowly becoming a reality with VODataService 1.2 in
> RFC (still open for community comments:
> https://wiki.ivoa.net/twiki/bin/view/IVOA/VODataServiceV12RFC), it is
> now time to think about the next step for blind discovery: Advanced
> column statistics.
> 
> The short version of the use case is: "Give me a catalogue of X
> approximately complete to 15 mag in V".
> 
> Based on Grégory's ideas I have written a note on this, in a format
> somewhere between brainstorming and a roadmap, which Giulia has just
> put up on the document repository:
> 
> https://ivoa.net/documents/Notes/colstatnote/index.html
> 
> It would be great if you could have a look at the text with a view
> at:
> 
> * from the Apps perspective: Does the proposed metadata make sense
>  for you?  Is there additional low-hanging fruit we could pick with
>  sensible extensions or alterations to the plan?
> 
> * from the Registry perspective: How will we make data like this
>  available for discovery purposes?  Is vs:Stats a sane way to
>  represent it?
> 
> * from the DAL perspective: What would extended metadata in
>  VODataService mean for TAP_SCHEMA?
> 
> * for operators: Could you come up the the proposed metadata?  Would
>  you?  I'd be happy to work with you to have a few more
>  prototype implementations (beyond DaCHS, as discussed in the
>  note).
> 
> * for everyone: Are there any elephants in the room that I've
>  overlooked so far?
> 
> I'd suggest followup comments should go to Registry.  And I'm happy
> to have something like a spontaneous breakout on this at the next
> interop (I'm told we'll have gathertown for it...)
> 
> Thanks,
> 
>          Markus
> 
> [1] cf. https://ivoa.net/documents/Notes/Regstc/, which in some way
> is part one of the new note.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.ivoa.net/pipermail/registry/attachments/20211021/ccc42b5e/attachment.html>