TAP 1.1, MAXREC and TOP

[Marco Molinaro]

Dear Markus, DAL,
in principle I agree on the choice of giving top priority to the query
language limit.

The MAXREC=0 drawback, IMO, is minimal and affordable.

My doubt is: wherever a "TOP" clause exists it wins on MAXREC, so
MAXREC survives only if it's set to 0 or if no "TOP" clause exists
(are there other cases I'm missing?).

If so, MAXREC goes completely (I know, it's an oversimplification)
server side because the user can take advantage of the QL clause and
try to override it.
How could a server oppose a "too many rows to return" response? (if
you put a MUST on TOP to override MAXREC)

Maybe, if it's decided to go the way you propose, we can use UWS async
timeouts and limits to this (hoping no one will query sync with the
same search). But in any case this will answer back with an error or
similar document, more or less like an overflow response does.
I'm not sure, but maybe MAXREC should really be at higher priority,
modifiable by user in the range/list allowed by the server, and we
should ask apps to highlight it when the user prepares her query. But
this way, I understand, means getting back to your question.

Another solution may be re-describe the soft and hard limits on
MAXREC. The soft can be overridden, the hard cannot and will affect
also TOP.

Cheers,
     Marco

PS - A note on "many query languages transported by TAP" in your proposed text.
I would remove that "many" reference. I.e., change it to something like

However, ADQL and the optional query languages allowed by TAP, may
have their own mechanism ...

2015-08-31 11:15 GMT+02:00 Markus Demleitner <msdemlei at ari.uni-heidelberg.de>:
> Dear DAL,
>
> I believe we should use the opportunity of TAP 1.1. to clarify the
> interaction of TAP's MAXREC and ADQL's TOP.
>
> True, TAP is more general than ADQL, but the large majority of TAP
> queries are ADQL, and differing behaviours between various servers
> are an annoyance.
>
> Essentially, the question is:  What's to happen when MAXREC=200 is
> specified in a query parameter, and there's a TOP 4000 clause in
> QUERY?
>
> There certainly are good arguments to say "use the minimum of both
> numbers", but I'd argue for "use what's in the query", as that
> probably is most directly what the user wrote. You see, at least in
> TOPCAT, and I suspect in other clients, too, MAXREC is a bit out of
> the focus of the user interface but preset.  It's unlikely that users
> having wrote a TOP clause differing from MAXREC actually set
> MAXREC conciously.
>
> A slight drawback might be that, to figure out the metadata-only case
> of MAXREC=0 (or SELECT TOP 0), a service needs to parse the query.
> It would have to do that anyway, though, to produce the metadata
> requested, so I don't think that's a major problem.
>
> If we can reach an agreement on this, I'd say we should change sect
> 2.7.5 of the current 1.1 draft.  Where there's now
>
>   The MAXREC parameter and its effect on the query result is fully
>   described in [std:DALI]. If the result set is truncated in this
>   fashion, it must include an overflow indicator as specified in
>   section XREF.
>
> I propose we should write
>
>   TAP supports the MAXREC parameter as specified in [std:DALI],
>   version 1.0, where what is limited is the number of database rows
>   in the first result table; where TAP jobs produce multiple tables,
>   the row counts of the other tables are not included.
>
>   However, many query languages transported by TAP have
>   their own mechanisms for limiting the number of records to be
>   returned; for instance, in ADQL there is the <set_limit> production
>   ("TOP n").  When a query gives both MAXREC and such a
>   language-specific row limit, the language-specific limit overrides
>   MAXREC (as it was presumably directly entered by the user).  When a
>   query gives no MAXREC but a language-specific set limit, services
>   SHOULD (next major version: MUST) behave as if MAXREC were set to
>   the language-specific set limit.
>
>   If a result is truncated due to MAXREC (or an equivalent
>   mechanism), the query response must include an overflow indicator
>   are specifiec in sectin <ref>
>
> What does everyone think?
>
> Cheers,
>
>         Markus
>