TAP 1.1, MAXREC and TOP

[Markus Demleitner]

On Mon, Aug 31, 2015 at 12:37:06PM -0700, Patrick Dowler wrote:
> On 31/08/15 02:15 AM, Markus Demleitner wrote:
> >Essentially, the question is:  What's to happen when MAXREC=200 is
> >specified in a query parameter, and there's a TOP 4000 clause in
> >QUERY?
[...]
> From the point of view of the query capturing the query in the
> abstract, some of those contain limits and in that case I favour
> using TOP to keep the definition of the result set in one place.
>
> MAXREC - client-server agreement on size of output, with overflow indicator
> when applicable
> TOP - logical limit on membership in the row set

Well -- even though I'd not say this conclusion is inevitable and I
still believe we'd be doing our users a favour if TOP overrode
MAXREC, I'm fine with it as long as all servers behave identially.

So -- can we put this "MAXREC clobbers TOP" into the spec?  This
could be then, in 2.7.5

  TAP supports the MAXREC parameter as specified in [std:DALI],
  version 1.0, where what is limited is the number of database rows
  in the first result table; where TAP jobs produce multiple tables,
  the row counts of the other tables are not included.

  Where query languages transported by TAP have
  their own mechanisms for limiting the number of records to be
  returned (e.g., ADQL's <set_limit> production,"TOP n"), MAXREC
  still gives the maximum number of rows.  In effect, the number of
  rows returned is the minimum of MAXREC and the language-specific
  set limit.

  If a result is truncated due to MAXREC (or an equivalent
  mechanism), the query response must include an overflow indicator
  are specifiec in sectin <ref>



And, since Walter asked: yes, the designers of ADQL should probably
have forseen that there'd be a parameter-level set limit and left out
TOP to avoid having two things for the same purpose.  They didn't,
and I suspect the annoyance of keeping it (to users and server
operators) is quite a bit lower than the annoyance of removing it at
this point, when there's TOP xy queries all over the place.
Hence, I'm afraid we can't do much better than at least clearly
document the interaction between the two.


And, given that, I'd wish clients would not pass in MAXREC by
default.  Mark... Do you think you could have the Max Row combo in
TOPCAT default to empty?

Cheers,

         Markus