Logging suggestion

[Ani Thakar]

There are two different logging services that we are talking about here, 
I think:

	- site logging
	- VO logging

The first is the logging service at a given site that may or may not 
provide a query interface for users to view the logged data.  This logs 
all the services at a given site (e.g. STScI).

The second is the VO-wide log harvesting that collects the logs from all 
the VO services in a VO domain (e.g. NVO).  

Which one do you mean by "centralized logging service"?

The logging interface in the support interfaces draft only deals with 
the minimum that any site or service needs to do to get its log data 
harvested to the VO logging VOStore. 

It's fine if each site has a central logging service that has a log 
harvesting interface hanging off it that the VO log harvester can use to 
retrieve log data from it periodically.  Each individual service does 
not need to support a log harvesting interface.

In the current proposal the VO log harvester initiates the xfer of log 
data by giving the site a range of dates and a VOStore identifier.  The 
site then pushes the log data to the VO harvester store.

This way if there are any changes to the VO harvester store location or 
authentication, all the VO sites need not be updated. 

Anyway, more at the tech-WG telecon tomorrow.

	Ani

On Wed, 7 Dec 2005, KevinBenson wrote:

> Yes this is sounding quite good.  I like the idea of a more centralized 
> logging service where by other services can push the data to the central 
> logging service.
> 
> Now for the central logging service I don't think in this first version 
> it would need a "push" model, I would prefer to let clients/apps come 
> and get the data.
> 
> A possible suggestion is "OAI" for obtaining the data from centralized 
> logging services.  Deals with dates such as "from" or "until".  One of 
> the IVOA components "registry" currently uses it.  And also deals with 
> large amounts of data with the notion of a "resumptionToken" allowing a 
> client to page through the results.  And finally there are already 
> several OAI tools to help the implementation.
> 
> cheers,
> Kevin
> 
> Doug Tody wrote:
> 
> > One way this could work:
> >
> >     o    Each site operates a logging service which is distinct from 
> > other
> >         service instances (for data access, computation, etc.).
> >
> >     o    Services send lightweight messages to the local, centralized
> >     logging service whenever anything happens.  This helps keeps the
> >     services simple and stateless.
> >
> >     o    The logging service has the ability to both push logging data to
> >         subscribers, as well as respond to queries for logging data.
> >
> >     o    In push or publish/subscribe mode, it would be relatively 
> > easy to
> >         buffer data and send a message containing logging data only
> >     periodically.  This would avoid the problems of managing very
> >     large amounts of logging data, but would still be efficient due
> >     to the buffering.
> >
> >     o    The logging service could also expose a query interface for
> >         a client to directly access logging data, e.g., by a range of
> >     dates, everything since a certain date, or some other method.
> >     Since it is a separate service this does not complicate other
> >     service interfaces.
> >
> >     o    If security is an issue it can easily be dealt with by the 
> > logging
> >         service.  This is a somewhat different case than for normal
> >     data access/analysis services, as only grid infrastructure needs
> >     to access logging information.
> >
> > It would also be possible for simple services which are not heavily used
> > (small sites) to send small amounts of logging information to a remote
> > logging service, to avoid the need for small sites to support a logging
> > service locally.    - Doug
> 
> 

-- 
Aniruddha R. Thakar, Research Scientist
Center for Astrophysical Sciences, JHU 
3701 San Martin Drive, Baltimore MD 21218-2695
410-516-4850, Fax: 410-516-5096  
thakar at jhu.edu, http://www.sdss.jhu.edu/~thakar
-----------------------------------------------------------------------
Public opinion is a weak tyrant compared with our own private opinion. 
[Thoreau]