[csp] final CSP + TDIG + GWS telecon today in one hour: 5 pm CEST = 11 am EST = 8 am PDT

[Mark Allen]

Dear Bruno,

Following the discussion about the past, current and future priorities, I have put together some notes on the multi-D priority area, and how it reached the milestone of the first set of standards to address the needs, and some of the feedback afterwards. 

Some aspects of the process were very useful, and may serve as a good example on how to manage scientific priority areas. In particular the use of a set of “Minimal Requirements” that were approved by exec, then assessed by TCG in order to guide the work done in WGs. It did of course take much longer than originally hoped, but we really did get the standards we needed. 

There is much more that could be discussed of course (are we really getting the implementations we need, are the tools and services using the standards? etc.) but I thought it would be useful at this point where we are reviewing the current and future priorities.

best regards,
Mark



A short review of the Multi-d IVOA priority area 2012-2017

Multi-dimensional Access minimal requirements (set in 2013):
Data Discovery (Query) – A user queries a data collection with one or more of the following constraints:
RA,Dec
Frequency/wavelength
Polarization states
Spatial size
Angular resolution
Integration time
Time of observation
Data Access – Once a user has the list of observations that satisfy the constraints, they select all or a subset of the observations and:
Download the complete science data for each of the selected observations or;
Download simple cutouts of the science data for each of the selected observations
Simple Cutout
Spatial: a circle (a coordinate and a radius)
Energy: one interval (from energy1 to energy2)
Time: one interval (from time1 to time2)
Polarization: a list

Additional requirements for cut-outs that were considered desirable but out of scope
- Sum along any one or more axes
- Re-bin in one or more axes
- Multiply by a function
- Other action on the data


Timeline:

October 2012
"Time Domain Astronomy, and Multi-dimensional Data were AGREED as priority areas for IVOA.”
FM46S  https://wiki.ivoa.net/internal/IVOA/IvoaRepMin/ivoa-fm46s-20121024.pdf <https://wiki.ivoa.net/internal/IVOA/IvoaRepMin/ivoa-fm46s-20121024.pdf>

May 2013
Focus session on multi-dimensional data at Heidelberg,  Interop Meeting
https://wiki.ivoa.net/twiki/bin/view/IVOA/InterOpMay2013 <https://wiki.ivoa.net/twiki/bin/view/IVOA/InterOpMay2013>

May 2014
Focus session on multi-dimensional data at ESAC/Madrid Interop Meeting
https://wiki.ivoa.net/twiki/bin/view/IVOA/InterOpMay2014MultiD <https://wiki.ivoa.net/twiki/bin/view/IVOA/InterOpMay2014MultiD>

June 2015 
DataLink 1.0 IVOA Recommendation

December 2015
SIA 2.0 IVOA Recommendation 23 December 2015

May 2016
Focus Sessions on “Interoperability of data from major astronomy projects"
https://wiki.ivoa.net/twiki/bin/view/IVOA/InterOpMay2016Focus <https://wiki.ivoa.net/twiki/bin/view/IVOA/InterOpMay2016Focus>


May 2017
- SODA 1.0 IVOA Recommendation 17 May 2017
- Obscore 1.1  IVOA Recommendation 9 May 2017
- Announced the milestone - IVOA multi-d data standards: first set of standards to address Discovery, Access and Simple cut-out of multi-d data

July 2018
IVOA Note on Feedback on the DAL protocols relevant to Multi-d standards (F. Bonnarel) http://www.ivoa.net/documents/Notes/RecentDALProtocolsFeedback/index.html <http://www.ivoa.net/documents/Notes/RecentDALProtocolsFeedback/index.html>


Relevant standards:

- SODA 1.0 IVOA Recommendation 17 May 2017
- Obscore 1.1  IVOA Recommendation 9 May 2017
- SIA 2.0 IVOA Recommendation 23 December 2015
- DataLink 1.0 IVOA Recommendation 17 June 2015


Relevant Implementations:

See : Review by F. Bonnarel at Feb 2019, ASTERICS DADI Technology Forum:

https://www.asterics2020.eu/dokuwiki/lib/exe/fetch.php?media=open:wp4:multi-d_status_at_the_end_of_asterics.pdf <https://www.asterics2020.eu/dokuwiki/lib/exe/fetch.php?media=open:wp4:multi-d_status_at_the_end_of_asterics.pdf>




Mark G. Allen
Director CDS
+33 (0)3 68 85 24 87
+33 (0)7 84 07 87 57
mark.allen at astro.unistra.fr
http://cdsweb.u-strasbg.fr/~allen

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.ivoa.net/pipermail/grid/attachments/20190425/14aeaabc/attachment-0001.html>