gzipped images in SIAP 1.0 (fwd)

[Doug Tody]

Hi Pat -

I am aware of Transfer-Encoding but am still trying to understand what
good practice is for actually using this: most of the HTTP compression
stuff I have seen talks only about using Content-Encoding for compression.
In part this may be because Content-Encoding was available in HTTP 1.0
and Transfer-Encoding is a HTTP 1.1 feature.  Transfer encoding addresses
other issues such as chunked transfers, and there can be issues with
things like combining chunking with a compressed stream.

Using Content-Encoding for compression will certainly work and I suspect
it is the most likely approach to work widely and reliably.  It should
work whether an already gzipped file is being returned, or whether the
stream is compressed on the fly.  What the HTTP client does with the
compressed content, e.g., whether it uncompresses it automatically, is
another matter (for display in a browser for example this would always
be done).

> Neither Transport-Encoding nor Content-Encoding modify the Content-Type
> (mimetype) of the resource.

Yes, this makes sense.

> > On Tue, 22 May 2007, Roy Williams wrote:
> > > -- Does anyone remeber the intention of the comma-delimited list of MIME
> > > types?  Should my code look for "application/x-gzip,image/fits"
> > >   Or maybe the other way around?
> >
> > If this refers to the FORMAT query parameter, which takes a list
> > of MIME types, then this tells the service to describe, in the
> > query response, only images with the given MIME types.  If it can't
> > generate the requested MIME type it should return nothing.  Hence if
> > the client asks for image/jpeg and the client cannot return a JPEG,
> > the response will be a null query (REQUEST_STATUS=OK and no data).
>
> This bit appears on the surface to be useful, and I think somewhat in
> conflict with everything else I and others have posted. Will have to
> think about it...  it is basically moving the content-type negotiation
> from where it naturally goes (the download stage) to the query stage,
> which seems much more efficient in the case where all formats are not
> available (probably the typical case).

The first question asked was what SIAP 1.0 intended, and this is what
I have addressed above.  SIAP has always worked this way, and I am
surprised that anyone is confused.

More fundamentally, the purpose of a queryData is to find out what data
is available, and how to get it.  FORMAT allows the client to narrow the
set of possible data formats to those it is interested in: to a client app
there is a big difference between FITS and JPEG for example.  FORMAT is
most definitely a client app level matter.  Once the client finds out
what data of interest is available, it can get it with a simple GET,
using any standard Web tool or code.  No HTTP-level header manipulation,
e.g., to set the "Accept" HTTP header to specify the desired format,
is required, and by the time we try to access the data we already know
that it is available in the desired format.

FORMAT should definitely be controlled directly by the client app.
Generic content compression on the other hand, works well as a HTTP
level feature.  My feeling is that good practice would probably be to
use Accept-Encoding and Content-Encoding for this, and handle it at
the level of the HTTP protocol.

Regarding what a protocol-level COMPRESS parameter should do - I don't
think we understand this well enough yet.  Probably we should just leave
it out for now; most of what we want can be provided already by enabling
HTTP level compression in a smart implementation.  In the future, a
COMPRESS parameter might be a good place to allow a client to enable
advanced features such as FITS-level tiled compression, or to allow
full pass-through of whole file (e.g., GZIP) compressed data.

	- Doug