Java and setting HTTP user agent

[Mark Taylor]

Thomas,

I'm copying this to the apps list, as it's of general interest -
hope that's OK.

On Wed, 25 Mar 2009, Thomas Boch wrote:

> Hi Mark,
> 
> While trying to set a sensible user agent for Aladin (using
> System.setProperty("http.agent", "Aladin-"+Aladin.VERSION); ), I discovered
> that the JVM was appending a "Java/1.x.y..." string at the end of the value
> actually sent (which is "Aladin-v5.911 Java/1.6.0_03").
> I was just wondering if you found the same behaviour.
> 
> Maybe this is meant to prevent bot developed in Java to pretend to be what
> they are not (?), ie faking a Firefox user agent, or something like that.
> Cheers,

yes this is what I see.  If one wanted to take the extra effort 
it would be possible to manipulate the URLConnection directly 
each a URL was opened and put any chosen string in the User-Agent 
field, but I'm not going to bother.

In fact the effect of doing it the easy way is exactly what RFC 2616 
suggests (sec 14.43):

   The field can contain multiple product tokens (section 3.8) and
   comments identifying the agent and any subproducts which form a
   significant part of the user agent. By convention, the product
   tokens are listed in order of their significance for identifying
   the application.

       User-Agent     = "User-Agent" ":" 1*( product | comment )

   Example:

       User-Agent: CERN-LineMode/2.15 libwww/2.17b3

Looking at section 3.8, I see (which I'd missed before) that the
form of these "product tokens" is name/version, so you should 
alter your code slightly to 

   System.setProperty("http.agent", "Aladin/"+Aladin.VERSION);

instead.

Mark

-- 
Mark Taylor   Astronomical Programmer   Physics, Bristol University, UK
m.b.taylor at bris.ac.uk +44-117-928-8776 http://www.star.bris.ac.uk/~mbt/