Filtering the UWS JobList by Phase and Last

[Joshua Fraustro]

Hi Paul,

Thanks for the clarification.

I assume then that LAST is intended to be analogous to LIMIT/TOP, that it is applied after all earlier constraints?
That also clears up the similar issue with AFTER. I cannot readily think of a situation where its evaluation would conflict with PHASE at all.

I think a simple clarification to the above effect would suffice. Informally posing the problem to some Institute colleagues, I heard similar interpretations as mine and some grumblings about “rules do not commute” and “requiring operation precedence”… 😊

Thanks,
Joshua

From: Paul Harrison <paul.harrison at manchester.ac.uk>
Date: Thursday, June 27, 2024 at 3:27 AM
To: Joshua Fraustro <jfraustro at stsci.edu>
Cc: GWS IVOA <grid at ivoa.net>
Subject: Re: Filtering the UWS JobList by Phase and Last

Hi,

you are right that it does need some clarification, although as you note most implementations make the intended interpretation that the LAST filter is applied last, which is the crucial point, as without that neither “logical AND” nor “intersection or sets” alone will produce the intended interpretation.

Paul.


On 26 Jun 2024, at 23:04, Joshua Fraustro via grid <grid at ivoa.net> wrote:

For services that implement the UWS pattern, the job list returned at the root /{jobs} path may be filtered using three parameters:


  *   PHASE: return only jobs with the supplied ExecutionPhase
  *   AFTER: return jobs with a creationTime after the supplied datetime
  *   LAST: return the last N jobs, ordered by creationTime, descending

As described by the UWS document, under 2.2.2.1 Job List [ivoa.net]<https://urldefense.com/v3/__https:/www.ivoa.net/documents/UWS/20161024/REC-UWS-1.1-20161024.html*jobList__;Iw!!PDiH4ENfjr2_Jw!E1iP25arWzG2_X0Da9_n1-y8s8hZHgUt-UKH3YbG33aySrH4LqMD1fe1lOhMl0EstrJ1tinTyjxbG_odNPCm$>, “If multiple filters are specified by the client, then the server should return a jobs list which is the result of a logical AND of the various filter conditions”.

Consider the following basic job list of only execution phase. Assume it has been appropriately presorted by creationTime.

job5: phase=PENDING # newest
job4: phase=COMPLETED
job3: phase=PENDING
job2: phase=COMPLETED
job1: phase=COMPLETED # oldest

A client provides the following filters in his query:

/{jobs}?PHASE=COMPLETED&LAST=2

If the service performs these iteratively, which I think a programmer’s gut reaction would be, the orders could be:

  1.  Filter by PHASE, then by LAST, the result would be [job4, job2].
  2.  If the service evaluates by LAST, then by PHASE, the result would be [job4].

I know at MAST we use order 1, and doing some quick checking it appears that Vizier and GAVO do as well.

What I assume is meant by “logical AND” however, is closer to, “the intersection of the sets whose members are the results of the various filter conditions.” In which case the answer becomes:


?PHASE=COMPLETED
[job4, job2, job1]

AND

?LAST=2
[job5, job4]

==

[job4]


Is this the intended meaning? If it is, we should probably clarify the language as such, as that would be the ‘most correct’ interpretation of “logical AND”. If not, we would need to decide what the prescribed evaluation order is.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.ivoa.net/pipermail/grid/attachments/20240627/7be40e9e/attachment.htm>