Vocabularising dataproduct_type

[alberto micol]

Hi All, I just read this thread, so first thing first: Thanks a lot Markus, this is a very good idea.

4 things:

1.- I’d like to reassure that “measurements” is used in ObsCore, at least at ESO, where currently:

SELECT count(*) qty, dataproduct_type
FROM ivoa.obscore
GROUP BY dataproduct_type

  qty  |dataproduct_type
-------|----------------
12673  |"cube"          
542500 |"image"         
391754 |"measurements"  
1769243|"spectrum"      
2562   |"visibility"    

(maybe, Markus, your query was for “measurement” without the ending “s” ?)

See: 
http://archive.eso.org/programmatic/#TAP?e=4&f=votable&m=200&q=SELECT%20count(*)%20qty%2C%20dataproduct_type%0AFROM%20ivoa.obscore%0AGROUP%20BY%20dataproduct_type&


2.- Our measurements products are FITS binary tables of 3 subtypes: 
- catalog: scientific catalogue (typically all-sky) in single FITS binary table (26 such catalogs)

e.g. the catalog of PESSTO transients, the GAIA-ESO catalog containing radial velocities, metallicities, effective temperatures, etc of about 25000 stars, the Next Generation Transit Survey source catalogue (204,000 records), etc.

- catalogtile: one FITS binary table for each of the tiles an ESO Public Survey (or other observing programmes) is partitioned into 
  (22,502 such catalog tiles)

- srctbl: source tables derived from individual images (~370,000 such srctbl)

The distinction between the three subtypes is conveyed by the dataproduct_subtype field.

Catalogs are exposed both:

- as FITS binary tables individually downloadable, 

- by the record, via a dedicated TAP interface to query the content of those tables,
  currently totalling 40 billion records, mostly from catalog tiles.

Source tables are only exposed as FITS binary tables.
 

3- I want to stress that we make a distinction between “sources” and physical “objects"

sources: are detections on single images (single bands). It is not given that a detection is for a real object, it could be just only a spurious detection. In this sense, sources are not yet objects, unless they get confirmed into “objects" by the analysis process (see physical objects)

physical objects, e.g.:
- objects in catalog tiles: sources in different images (e.g. in multiple wavelength bands) recognised to be detections of the same object (cross-correlation implied)
- objects in all-sky catalogs: whereby typically the measurements are derived from 1 or multiple spectra of the same object

Objects end up in “catalogs”, while sources remain in “source tables”. 

From the above you can immediately understand that I fully second Laurent: 
A catalog can be derived from many source tables (e.g. via cross-correlation of source tables in different bands).


4- multi-typed catalogs: some all-sky catalogs (e.g. the PESSTO multi-epoch and multi-band photometry) are actually time series of SEDs, (I should probably change the subtype to SED to make it discoverable), while others are simpler (e.g. NGTS) light curves, ie time series of photometric points in one single band (that’s where the bulk of the 40 billion records (31E9) come from).

The PESSTO example shows that indeed we need to combine various types, as Baptiste was already indicating.

Thanks and cheers,
Alberto




 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.ivoa.net/pipermail/dal/attachments/20200325/fae687a2/attachment-0001.html>