TAP MAXREC interpretation

[Dubois-Felsmann, Gregory P.]

>From DALI 3.3.4:

"The service may also enforce a limit on the value of MAXREC that is smaller than the value in the request. If the size of the result exceeds the resulting limit, the service must only return the requested number of rows."

I think the inclusion of the word "resulting" makes clear that the intent is that the provided value be reduced to the service limit, not rejected.

But I'd support making this more explicit in the next version of the standard.  Can we still get this into DALI 1.2?

I'll have a look at what the behavior of the Rubin services is (bear in mind that our TAP is closely derived from CADC's).

Gregory

________________________________________
From: dal <dal-bounces at ivoa.net> on behalf of Mark Taylor <m.b.taylor at bristol.ac.uk>
Sent: Thursday, December 14, 2023 7:04 AM
To: Pierre Fernique
Cc: dal at ivoa.net
Subject: Re: TAP MAXREC interpretation

Pierre,

although as discussed by Thomas and Gregory one can come up with
ways to defend against this behaviour on the client side,
I'm sure that this is not what's intended by TAP/DALI.

DALI sec 3.3.4 is not absolutely 100% explicit about how this works,
but to me it looks clear what's meant.

I would say the correct action is to contact the service providers and
ask them to change the behaviour.  If they really misunderstood
the intent of DALI it might be worth thinking about clarifying the text,
but I think it's most likely just an implementation oversight.

Mark

On Thu, 14 Dec 2023, Pierre Fernique wrote:

>
> Dear TAP experts,
>
> While testing the various TAP services accessible via Aladin Desktop, I
> discovered an interpretation of the MAXREC parameter that I hadn't
> anticipated, and I'm wondering about its validity.
>
> For example, if I query MAST TAP HSC, with a MAXREC of 2M, I don't get the
> table I want, but an error as shown below.
>
> https://mast.stsci.edu/vo-tap/api/v0.1/hsc/sync?REQUEST=doQuery&LANG=ADQL&*MAXREC=20000000*&QUERY=SELECT+table_name+FROM+TAP_SCHEMA.tables
>
> <VOTABLE version="1.4"
> xsi:schemaLocation="http://www.ivoa.net/xml/VOTable/v1.3
> http://www.ivoa.net/xml/VOTable/v1.3">
>    <DESCRIPTION>MAST VO TAP Service</DESCRIPTION>
>    <RESOURCE type="results">
>       <INFO name="QUERY_STATUS" value="ERROR">
> *Error in query maxrec: ensure this value is less than or equal to 100000*
>      </INFO>
>    </RESOURCE>
> </VOTABLE>
>
> But I would have expected to receive the result correctly, possibly truncated
> (OVERFLOW mention) at the internal limit of the service (100000).
>
> In fact, such an approach makes it difficult for the client to use MAXREC,
> except by trial and error appoach. Either it's too high and I get an error, or
> it's too low (but by how much) and I may not get the full result when I could
> have. And finally, if I don't specify a MAXREC, the server will apply the
> default value, even though I could have asked for more.
>
> Any advice on how to deal with this problem?
>
> Thanks
> Pierre Fernique
>

--
Mark Taylor  Astronomical Programmer  Physics, Bristol University, UK
m.b.taylor at bristol.ac.uk          https://www.star.bristol.ac.uk/mbt/