TAP, automated site monitoring, and gzip encoding.

[Mark Taylor]

On Thu, 30 Jun 2011, Tom McGlynn wrote:

> One solution that I had hoped might work was to use a GZIP transfer encoding
> (or content encoding) for the query results.  Unfortunately it doesn't look
> like clients currently note the HTTP encoding headers.
> 
> NASA is probably a bit more paranoid about this than some, but I suspect that
> this will become a more common issue as time goes on.
> Support for content or transfer encoding is an HTTP level issue so I don't
> think it requires any change to the TAP standard, just clients that look for
> the appropriate HTTP headers.  Would it be reasonable to request that clients
> support gzip encoding?  In addition to address this security issue I suspect
> this would generally substantially decrease the size of downloaded data and
> make our queries more responsive.
> 
> 	Tom McGlynn

FWIW, although TAP does not address this, the SSA standard 
(PR-SSA-1.1-20110417) does discuss compression in section 7.3:

   7.3 Data Compression

   If the query parameter COMPRESS is present then the service may return
   a compressed dataset, using some standard compression technique such
   as gzip, in place of a normal dataset, without indicating this in the
   query response. Basically the client is indicating that it is prepared
   to receive either compressed or uncompressed datasets and does not
   care which is delivered (the service should pick whichever is more
   efficient). This should be distinguished from protocol-level compression,
   which is transparent to the client, and may occur at the level of the
   HTTP protocol if both client and server support HTTP protocol compression.

   In case of an HTTP GET the keyword Content-Encoding informs the receiver
   about the encoding of the output data, and should have a value such as
   gzip. Note that the encoding is distinct from the MIME-type (Content-Type)
   of the returned data object.

the tone seems to suggest that Content-Encoding is something that
clients might (but not MUST) be expected to do as a matter of course.

Probably DALI ought to say what the general assumption is for DAL
services about content- and/or transfer-encoding.

Mark

--
Mark Taylor   Astronomical Programmer   Physics, Bristol University, UK
m.b.taylor at bris.ac.uk +44-117-928-8776 http://www.star.bris.ac.uk/~mbt/