TAP 1.1, MAXREC and TOP

[Gregory Mantelet]

Hello DAL,

     I also agree that TOP should not override MAXREC. They should work 
together but whatever is TOP, MAXREC is always the upper limit.

     As said before, TOP is an explicit request from the user to specify 
how many rows he/she wants. Since it is explicitly asked, no overflow 
must be returned.
     On the contrary, MAXREC is a constraint that the TAP service put on 
the size of the result of any query. Since TAP is a friendly protocol 
:-), it publicly declares what is its default server limit (soft limit) 
and even sometimes allows modification of it up to a declared maximum 
value (hard limit).
      Thus, MAXREC must always have the priority on TOP. That's why 
MAXREC=0 has a sense and not "TOP 0": MAXREC=0 specifies to the TAP 
service that the query should not be executed but that only the metadata 
about the selected columns must be returned ; it is a TAP service's 
work, not a database one...so it depends on a TAP parameter instead of a 
query language's special syntax.

     In brief, according to me, the rules should be:

         a- if MAXREC is NOT specified by the user, it is set by default 
to the soft limit. If no soft limit is specified by the service, MAXREC 
is set by default to the hard limit. If none of these limits is 
set...well, there is absolutely no limit on the maximum number of rows 
that can be returned by the TAP service ; no overflow can be indicated.

         a-bis- if MAXREC is specified by the user, it is limited by the 
hard limit (if any).

         b- if TOP is NOT specified by the user, the result is limited 
by MAXREC (whose the value depends of a-) ; if more than MAXREC rows are 
returned by the database, the TAP service limits the output to MAXREC 
and set an overflow flag.

         c- if TOP is set by the user (whatever if MAXREC is set by the 
user or not...cf a-), at most min(MAXREC, TOP) rows may be returned. If 
MAXREC is the minimum value, an overflow is indicated. Otherwise, there 
is no overflow flag.

     Again: TOP is part of a language query and is used by a user to 
specify what he/she wants in his/her result-set ; MAXREC is a service 
constraint designed for server security reasons and which may be (if 
soft limit < hard limit) allowed to modify up to a specified value (hard 
limit).

Cheers,
Grégory


On 01/09/2015 16:51, Tom McGlynn (NASA/GSFC Code 660.1) wrote:
> I also agree with Pat's approach.  My own preference here (and 
> generally through the standards documents) is also that examples of 
> the behavior should be explicitly noted.  I also like the notion that 
> Marco brought up of soft and hard limits that a server may impose.  So 
> the text might include something like:
> -- 
>
> The number of rows returned by a query may be limited by soft and hard 
> upper limits set by the service provider, by an explicitly included 
> MAXREC parameter, and, for ADQL queries, by the TOP field within the 
> query ADQL.  A soft limit is effectively a default value
> for the MAXREC parameter and can be overriden  by explicitly including 
> that parameter.  However a service may set a distinct hard upper limit 
> for the maximum number of rows returned.  If users sets a value for 
> MAXREC that exceeds the hard upper limit, the query should be executed
> as if MAXREC was set to the hard upper limit.  The soft and hard 
> limits should be described in the service metadata.  These implicit 
> limits and an explicitly specified MAXREC limit the query after any 
> TOP value specified in an ADQL query.  The number of rows will be 
> limited to the minimum of the TOP and effective MAXREC values.   The 
> effect should be as if the query was initially limited by any TOP 
> constraint when the database was queried, and then if necessary 
> truncated by the effective MAXREC limit when the results were sent 
> from the server to the querier.
>
> Consider a case where a service has a soft upper/default limit of 300 
> rows, a hard upper limit of 1000 rows and is sent a query which in the 
> absence of any limits would return 2000 rows:
>
> - If no limits are specified by the user, the soft upper limit of 300 
> rows applies.  Only 300 rows are returned and a query overflow 
> indication is set.
> - If no MAXREC is given but the user starts the ADQL query with 
> "select TOP 400 ..." the soft limit takes precedence.  Only 300 rows 
> are returned and a query overflow indication is set.
> - If the user specifies MAXREC=500 and starts the query with "select 
> TOP 400..." then the TOP parameter limits the query.  No query 
> overflow indication is set and 400 rows are returned.
> - If the user specifies MAXREC=500 and "select TOP 600..." then the 
> explicit MAXREC limits the query to 500 rows and an overflow 
> indication is set.
> - If the user specifies MAXREC=9999999 and "select TOP 9999999 ..." 
> then the hard upper limit truncates the query at 1000 rows and an 
> overflow indication is set.
>
> When an overflow indication is set the associated message SHOULD 
> indicate if the overflow was triggered by a default/soft service limit,
> the hard service limit, or an explicit MAXREC limit.   Overflow 
> indicators are never set when the query is limited by the TOP 
> parameter.  If the TOP parameter and effective MAXREC values are the 
> same, then the overflow indication should not be set.  The query 
> should be treated as limited by the TOP parameter so that the result 
> did not overflow.
>
>