Time Domain DAL/DM/TDIG

Tue Jul 11 08:17:11 CEST 2017

Hi Matthew,

first, sorry, but my intention was not to speak in "IETF RFC2119"
in my mail (but maybe is just my English that is flawed).
I tried simply to put the constraints on the use cases I'm
working on.

Thus, a metadata characterization that works for me does not
mean it has to be made mandatory for all other cases (also
ObsCore mandates fields that can be set NULL for every record).
I only want to make sure that science cases are not dropped
and alert that some difficult-to-tackle discovery queries may
lay around.

As for query by object type, referring also to your later post,
I can live quite comfortably with positional search, because
external name resolving and services exist to cover most
of the needs, even if this means having multiple discovery
and matching steps.
Where you're probably right is that this specific
topic may be more KDD than TD related, but it plays a role
nonetheless here.

Cheers,
    Marco

2017-07-10 17:07 GMT+02:00 Matthew Graham <mjg at caltech.edu>:
> Hi,
>
> Object classification is the endpoint of a significant scientific workflow and will not necessarily be available for most time series. Even object characterization represents a lot of work and we don’t have a standard set of characterizing statistics - everyone uses their own favorites - and we cannot assume that every data set available will have these (I’m cc’ing Kai as KDDIG chair because the required metadata to describe these sorts of things is the purview of his group). So these have to be optional and not really in a minimal list.
>
> I think we need to be quite careful here about feature creep from specific scientific subdomains because that makes things easier for them. If I am putting up a collection of 100 million time series but it really has zero utility for exoplanet research or AGN variability, I should have to add metadata to each time series that supports that type of work.
>
>         Cheers,
>
>         Matthew
>
>
>> On Jul 10, 2017, at 3:22 AM, Marco Molinaro <molinaro at oats.inaf.it> wrote:
>>
>> Dear Ada,
>> dear TD/DM/DAL-ers,
>>
>> you're right, the query by object type (target name, position, class,
>> subclass, ...) is definitely a must, but I got the impression that the
>> stellar activity is critical for exoplanets (at least).
>>
>> So, this is a first requirement from my use case: we will need the
>> activity value associated with the activity index type. That is, we
>> have to leave room for queries that say
>>
>> stellar activity within these boundaries measured with this index
>>
>> I think (my case) I could live, e.g. having a source_activity column
>> annotated by one UCD among a set that describes the activity index.
>> I don't think there exist such UCDs, but I haven't checked either.
>> And maybe there exist other solutions.
>> Taking the worst case: how can I deal with multiple activity indexes
>> describing my host star?
>>
>> As for the planets and their characteristics, constraining the search
>> on those can be a back end problem, but:
>> - planet (confirmed or supposed) existence
>> - (stats on?) mass, period, eccentricity
>> can be a minimal metadata starting set.
>>
>> And about discovery methods, that's definitely something they need
>> to cross match planets (maybe more for catalogues than time series
>> themselves), and it would require a single element from a controlled
>> vocabulary. I didn't mean going into the single details of the methods,
>> for sure.
>>
>> Finally, checking on François's metadata blocks, I'd say that
>> - object position
>> - time sampling characteristics and stats
>> - object class (activity augmented)
>> have to be in the minimal list.
>>
>> Cheers,
>>    Marco
>>
>> 2017-07-07 13:51 GMT+02:00 AdaNebot <ada.nebot at astro.unistra.fr>:
>>> Dear Marco,
>>>
>>> Thanks for your input.
>>>
>>> On 6 Jul 2017, at 14:47, Marco Molinaro <molinaro at oats.inaf.it> wrote:
>>>
>>> Dear Ada,
>>> dear all,
>>>
>>> regarding the "discovery" item, related to RV Time Series
>>> in the exoplanets domain, I had a meeting with people
>>> from the GAPS project.
>>> (if interested a quick summary can be found here:
>>> https://www.asterics2020.eu/dokuwiki/doku.php?id=open:wp4:wp4gapsinaf)
>>>
>>> The main point I'd like to alert about here (other parts I think
>>> are nicely covered by your summary and review) is that a
>>> discovery for time series when doing research on exoplanets
>>> requires:
>>> - host star characterisation, especially stellar activity indexes
>>>
>>>
>>> Regarding the host star characterisation, I think this is in the lines
>>> "query by object type”.
>>> This is very important not only for exoplanets but for many other scientific
>>> cases.
>>>
>>> - planet(s) ~existence~ and characteristics
>>> [and this may be though because 1 stellar system potentially
>>> means multiple planets]
>>>
>>> I agree with you
>>>
>>>
>>> And, vice-versa, searching exoplanets having time series attached
>>> requires as discovery metadata the search method used (this can be
>>> a small vocabulary).
>>>
>>> I'll try to work on the star/activity/planets part related to the GAPS
>>> project, the vice-versa requires other projects with different than
>>> spectroscopic RV method to join.
>>>
>>>
>>> There is a list of metadata needed to describe and discover TimeSeries that
>>> we came up with during the ASTERICS hackathon meeting,
>>> summarised by F. Bonnarel in Shanghai. I have rewritten that list under:
>>> http://wiki.ivoa.net/twiki/bin/view/IVOA/DAL_DM_TDIG_ShanghaiSession2017#Asterics_hackathon_report_F_Bonn
>>>
>>> To my opinion some of the things there are too detailed/complicated, e.g.
>>> Fourier coefficients.
>>> And I think that we can build a restricted list of minimum requirements
>>> based on those.
>>> Please, have a look at the list and let me know among those fields which you
>>> think are a must.
>>>
>>> Cheers,
>>> Ada
>>>
>>> Cheers,
>>>   Marco
>>>
>>> 2017-07-04 13:08 GMT+02:00 AdaNebot <ada.nebot at astro.unistra.fr>:
>>>
>>>
>>> Dear All,
>>>
>>> In Shanghai it was decided that TimeSeries is a scientific priority of the
>>> IVOA. Based on the DAL/DM/TDIG session, here is a list of items to be
>>> discussed. Please feel free to add any other items you consider relevant to
>>> TimeSeries and you would like to discuss:
>>>
>>> What is TimeSeries data? It is a temporal sequence of measurements
>>> containing a time coordinate and a flux (or similar - magnitude) / RV /
>>> position / polarisation / number of spots / spectra / images. Light-curves
>>> and RV curves dominate the landscape of TimeSeries.
>>> Concerning DM: there are models that work for representation of some of the
>>> science cases proposed by the SPC.
>>>
>>> Modified SpectrumDM works fine for light curves with some modifications.
>>> The draft TimeSeriesCube serialisation examples work with SPLAT. But there’s
>>> still some discussion needed on the model.
>>> Feedback from people who have tried both models is appreciated in this
>>> respect.
>>> Although light-curves and RV curves dominate the landscape of TimeSeries,
>>> for which a modified version of SpectrumDM (or a similar for RVs) would
>>> work, we should have an extensible model towards other kind of data, perhaps
>>> built as a simple derivation/specialization of the SparseCube DM.
>>>
>>> Discovery: Obscore/EpnCore + TAP could work if the time axis information
>>> would be extended. What are the minimum mandatory fields for TimeSeries data
>>> to be found by the user? For that collecting science cases is important. A
>>> set of minimum data has already been compiled during an ASTERICS meeting.
>>> Different communities need to be properly represented. Attending to
>>> astronomy meetings is important to hear the astro-community. See the series
>>> of meetings under:
>>> http://wiki.ivoa.net/twiki/bin/view/IVOA/IvoaVOEvent#Future_Events. Please
>>> if you plan to attend any of these meetings or any other related Time Domain
>>> meeting it would be useful to know it and to post it on the wiki.
>>> Format: Time has to be properly formatted to combine data coming from
>>> different surveys. See email from A. Rots (19 May 2017) sent to DM, DAL and
>>> TDIG + reference A&A 574, 36 (2015).
>>>
>>> To keep track these items will be posted on the TDIG wiki, under the
>>> TimeSeries section:
>>> http://wiki.ivoa.net/twiki/bin/view/IVOA/IvoaVOEvent#Time_Series_Data
>>>
>>> In particular under: http://wiki.ivoa.net/twiki/bin/view/IVOA/DAL_DM_TDIG I
>>> have already written these four items.
>>>
>>> You can also find a summary of the different contributions in the
>>> DAL/DM/TDIG Shanghai session under:
>>> http://wiki.ivoa.net/twiki/bin/view/IVOA/DAL_DM_TDIG_ShanghaiSession2017
>>>
>>> The intention of this email is to start productive discussions on each
>>> topic.
>>>
>>> Thanks!
>>> Ada
>>>
>>>
>