DALServer V0.1 information (SSAP, DAL)

[Doug Tody]

Hi All -

As I promised earlier, here is some summary information on our new
DALServer toolkit, which is the basis for the two developmental SSAP
services announced yesterday.

I apologize for posting about a specific implementation to the DAL
list, but this is potentially relevant for speedily getting the
first SSAP services going and completing the specification, and I
do not know for sure what initial implementations are in progress.
This information is provided so that anyone working on the initial
SSAP implementations can determine whether they wish to look at
this software in more detail.  If anyone is interested in using
or collaborating on the DALServer package, we can follow up on the
details offline.

 	- Doug


DALServer Toolkit V0.1
----------------------


Provides

     o	A reference implementation of the server-side of the DAL
 	services (the initial 0.1 version has full support only
 	for SSAP).

     o	A stand-alone toolkit for implementing DAL services within
 	a Java servlet environment; handles all the protocol details,
 	leaves the information management to the data provider.

     o	A testbed for working out the remaining details of the SSAP
     	protocol, the basis for the 2nd-generation DAL protocols.


Requirements

     o	Java 1.5 or greater.

     o	An XML parser, e.g., Xerces.

     o	A compliant Java applications server, e.g., Apache Tomcat.
 	Version 5.5.17 was used for development but any recent version
 	should work.

     o	The Savot VOTable packages from CDS (a compatible version
 	is included).  Other VOTable implementations could be used
 	but are not supported at this time.

     DALServer is an explicitly Java-based implementation; other
     environments (dotNet, Ruby, etc.), would require a different
     implementation.


What is involved in implementing a new SSAP data service?
---------------------------------------------------------

DALServer includes a fully functional generic SSAP service which can
be deployed to an application server and run without modification,
other than editing some configuration files.  This consists of a
"Servlet" class, which handles all matters relating to the protocol,
and a "Service" class, which knows how to handle the metadata for a
specific data collection.

To build a new data service, essentially what one does is subclass
the "Service" class, replacing the *queryData* operation and usually
the *getData* operation with custom versions which know how to talk
to the local archive and DBMS, and transform project-specific data
and metadata to what is required for SSAP.  Input consists of the
fully processed request parameters, presented as a parameter set.
Output consists of a "request response" object, which implements the
SSAP data models.  The service computes the SSAP-compliant metadata
and writes this to the request response object with a series of simple
"setValue" calls.

All details of returning data to the client, e.g., serializing
the request response as a VOTable, is handled transparently by the
framework.  (Note that the VOTable output is autogenerated and not
pretty-printed as with most hand-generated VOTables).

The request response object can represent either a data query (the
queryData operation), or an actual Spectrum dataset (the getData
operation).  Spectra can be statically precomputed and returned as
files, or dynamically serialized from the request response object
and returned in a variety of formats.


Features
--------

     o	Protocol-specific input parameter handling (e.g., knows the
 	details of the SSAP parameter set, can parse and reorder
 	range-list parameters, deal with ISO 8601 dates, and so forth).

     o	All request parameters defined by the standard protocol are
     	supported.

     o	Support is provided for additional service-defined parameters,
     	and adhoc client-supplied parameters.

     o	Pass-through of local configuration parameters is included
 	as part of the input parameter set.

     o	Support is provided for multiple operations and versioning.
 	The current version supports the queryData, getData,
 	and getCapabilities operations.  Support for a "health"
 	or ping-type operation is planned as well ("stageData" will
 	eventually be added as well).

     o	A true getData operation is provided based on the Pubisher
 	dataset ID, with runtime data format selection (this remains
 	hidden behind an opaque URL in the protocol however).

     o	Concurrent multi-protocol support is provided (that is,
 	HTTP and host callable now, and SOAP or whatever later with
 	no effect on the service implementation).

     o	Multiple servlet instances may execute simultaneously within
     	the same servlet container.

     o	Full error handling is provided for all service operations,
 	including data retrieval.

     o	Keyword factories are provided for the SSAP and Spectrum data
 	models, based on a form of automatic code generation from a
 	common data model (ssap-keywords.xls/csv).  This frees the
 	service code from having to know all the details of the data
 	model (e.g., the correct UCD and UTYPE tags, GROUP membership,
 	VOTable dataType, FITS keyword, etc.), and provides some
 	isolation from changes in data model versions.

     o	Dataset retrieval supports pregenerated static files (a simple
 	mechanism for this is built-in), dynamically created data,
 	including custom data or data generated from the built-in
 	models, or fully streaming data.

     o	Dynamically computed, model-based data can be serialized and
 	output in a variety of data formats.  Currently, CSV, VOTable,
 	and streaming pass-through of native data are supported.
 	Support for FITS binary table format is planned.

     o	Support for data associations in the query response.

     o	A primitive scoring algorithm is provided, and TOP is
     	implemented.

     o	The query response table may be sorted on any key.


Not Yet Implemented
-------------------

     o	Support for DAL protocols other than SSAP.  This is mainly
 	just a matter of subclassing the general framework for each
 	class of data.

     o	Autogeneration of FITS format data.

     o	Although the operation functions, the content of the SSAP
 	getCapabilities operation is not yet finalized.

     o	The health/ping/heartbeat operation still needs to be added (low
     	priority).

     o	Service-level logging would be very useful.


For Further Information
-----------------------

At the moment the best place to go for further information is my development
server at  http://webtest.aoc.nrao.edu/ivoa-dal/dalserver.html