ADQL versioning

[Dave Morris]

Hi Markus,

In general, I agree with your points.

On 2018-02-20 09:24, Markus Demleitner wrote:
> That's a reasonable policy, although I might have bargained harder to
> keep a few RDBMSes out of the requried set if I had known that their
> inclusion would lead to the (IMHO harmful) explosion of optional 
> features.
> 

The current list is based on my own unofficial survey of the platforms 
that I knew people were using, and we can discuss adding or removing any 
of the platforms at any point.

It would probably be a good idea to have a more formal list of who is 
running what platform so that we can keep track and update the platform 
list accordingly. I'll post a separate email to that effect on the 
mailing list.

> ...which is a really high bar, especially considering that quite a
> few of the systems are proprietary software not necessarily easily
> accessible -- even as regards their documentation.
> 

Someone needs to do some basic level of checking to make sure we don't 
accidentally exclude a platform without realising it.

I started out doing most of the testing myself, but found that while 
doing the checks for a single change are not that hard, as the number of 
changes increased I was spending all of the time I had available for 
IVOA work on checking the validity of other people's changes.

As a result, I don't think it is unfair to place at least some of the 
responsibility for the validation work on the person or project who are 
requesting the change. I'm happy to help with setting up the tests but 
I'm unable to do all of the testing on my own.

As you know, I have built a set of Docker containers designed to make it 
easier to work with the various platforms 
[https://github.com/ivoa/cosmopterix]

Based on my experience with this I can say that all of the open source 
platforms were easy to use and to package as Docker containers. In 
contrast, all of the proprietary platforms were harder to work with in 
one way or another.

I have not been able to find a way of working with Sybase, and I haven't 
been able to do any testing on this platform.

Oracle is the next worst, requiring manual acceptance of licence terms 
and a 3Gbyte download to install a simple 1000 row test database.

Microsoft SQLServer was the easiest proprietary platform to work with, 
using their ready built Docker container. So far, out of all the 
proprietary platform providers I have worked with, Microsoft are the 
best at providing good documentation and developer access (I don't 
believe I just said that).

Despite these obstacles we still need to be able to test proposed 
features on these platforms because many of the issues I found involved 
one or other of the proprietary platforms.

We also need to keep in mind that the institutes using these proprietary 
platforms have a lot of money invested in licenses and training and so 
it is unlikely they will be able to change without incurring a high 
cost. Which means if we want ADQL/TAP to be adopted as widely as 
possible we need to make sure it is inclusive.

> So -- regardless whether optional or not: I claim the
> two-interoperable-implementations rule applies.

I agree, the two-interoperable-implementations rule should apply to all 
aspects of the ADQL specification, including the optional language 
features and the official ivo_ namespace functions.

I admit that right now I don't have enough paperwork to demonstrate the 
required level of compliance for version 2.1, so we may end up delaying 
completion of the PR review until I can properly document the required 
implementations.

> please speak up if you have
> implemented (or intend to implement before the end of RFC):

We either already have implemented, or are planning to implement, the 
following at Edinburgh

   * CAST
   * LOWER
   * ILIKE
   * OFFSET
   * Common Table Expressions
   * boolean_value_expression and the accompanying literals
   * bitwise operators (rather than functions)


Hope this helps,
Dave

--------
Dave Morris
Research Software Engineer
Wide Field Astronomy Unit
Institute for Astronomy
University of Edinburgh
--------