UWS 1.1 job list, sorting NULL values

[Paul Harrison]

> On 2015-10 -20, at 08:04, Kristin Riebe <kriebe at aip.de> wrote:
> 
> Hi all,
> 
> just one more comment on validation:
> 
>>> Thus, errors like validation/preliminary checks (e.g. ADQL
>>> query validation) are part of a job and are then reported as error
>>> in a job description where startTime is NOT NULL. Since UWS is a
>>> generic protocol, I think that such validation or pre-checks should
>>> not be done externally to a job execution, except if is specified
>>> explicitely in the protocol. Is it right?
>> 
>> I think that this is also the correct approach - in the specification
>> document, UWS does not really know anything about the job description
>> language and as such operations like ADQL validation would most
>> naturally not occur until the job has started execution.
> 
> I think it may be more useful to have a validation check before
> execution of the job. At least for our SQL-service that is the approach
> we chose: we're checking submitted jobs for missing UWS parameters and
> also for basic SQL-errors (typos, basic grammar, valid table and column
> names) *before* the jobs are passed on to the queuing system. Otherwise,
> the user may submit a job, wait possibly for hours until the job is
> started, only to realize then that there is a typo in the SQL-statement.
> This may not be exactly what UWS prescribes, but that's what we find
> more useful. Perhaps validation steps are something to consider further
> in future UWS-versions.


The important point here is that the final validation is not done before the client has
issued a PHASE=RUN command, as the client is allowed to alter parameters
before this point, as the UWS model is that there is a certain amount of negotiation is done about the job execution parameters before the final commit. This has aways implied that that the server has to “understand” the submitted job to be able to do something like return the Quote for instance - however, it does lack an explicit validation step, and perhaps you are right that it would be a good idea to consider it in future versions as it would make the initial PENDING phase more useful.

My comment to Grégory above is not necessarily incompatible with how you have designed your server, as long as you are waiting until after the PHASE=RUN command to set the job into an ERROR status for the basic syntactical errors. This is because from a client point of view a job may be found in either the QUEUED, EXECUTING, or ERROR status after the PHASE=RUN, and it does not matter that the first part of your “queueing” process is to do a validation step which will result in the job entering either the QUEUED or ERROR phase. If you set the startTime on the jobs that enter the ERROR phase as a result of this validation then the end result for the client would be indistinguishable from the approach that Grégory uses in his library. It is always worth remembering that UWS is just a facade and that there is a fair degree of freedom in the underlying implementation as long as the interface behaves in a way that the client expects.

Regards,
	Paul.