Re: infinite loop with http requests


Chris Uppal wrote:
Daniel Pitts wrote:

Oliver Wong wrote:
I recommend against using google as your test server. Google does
some funky stuff when it detects that Java is connecting to it, which
may give you unexpected results.
[...]
Good suggestion except for two things, He isn't using Java's URL API,
which is what's responsible for setting the User-Agent string. Second,
you can override the User-Agent string, and google couldn't possible
know the difference.

I agree with Oliver's advice. Google is perfectly at liberty to treat requests
differently depending on how they /appear/ to have been submitted.

If I were them I would group requests into at least three categories: ones that
appear to be legit (as far as we can tell from the various meta-info in a
request); those that appear to come from frequently abused clients (such as the
Java stuff); and those where we can't tell much. I would be less aggressive
about -- say -- shutting off an over-eager client IP address if the requests
appeared to be from a normal browser than if they appeared to come from
uncontrolled code. And I'd put the "can't tell" ones somewhere in the middle.

But the bottom line is not that Google /can/ treat requests differently
depending on apparently immaterial meta stuff, but that it /does/ do so --
which makes it a very poor example domain for a beginner (to HTTP) to test
against.

-- chris

Okay, while my point was that you can "trick" google into thinking that
it is probably a legit client, your point is well taken.

I suppose a good way to learn HTTP is to set up a webserver in your own
development environment (such as apache, resin, etc...), and use it
instead of a third party website. That way you also have control over
the content being produced.

- Daniel.

.

Relevant Pages

  • Re: infinite loop with http requests
    ... which is what's responsible for setting the User-Agent string. ... Google is perfectly at liberty to treat requests ... Java stuff); and those where we can't tell much. ...
    (comp.lang.java.programmer)
  • Re: HTTP connection
    ... There are no sessions in HTTP. ... There are requests, to which there will ... Check out Ebay or your local bookstore, or Google. ... also described in an RFC) but it adds little that you actually need. ...
    (alt.comp.lang.borland-delphi)
  • Re: Open Office Cocoa
    ... noi attualmente usiamo Flash per il playback dei Google Video ... lato client di Internet" usiamo piu` che altro ... AJAX e mi sembra che ci abbiamo fatto delle belle cosine, ... Quanto a Java client-side, non credo che Google al momento ne abbia ...
    (it.comp.macintosh)
  • Re: 7.0 wishlist?
    ... Publicly accusing a specific person of stupidity, on the other hand, is an insult. ... This is a forum for discussing Java programming, not for assigning blame and arguing over who is at fault. ... domain-specific libraries, and ImageIO plugins and the like. ... I've had to filter JSTOR because that site produced so many Google hits I couldn't use on so many diverse topics, and there's a ton of other candidates for the same treatment, mostly scientific journal sites. ...
    (comp.lang.java.programmer)
  • Re: Just out of curiosity: Which languages are they using at Google and what for?
    ... is it PHP or Java or .NET? ... Which technology is rendering the google main page? ... special-purpose language for that specialized, very-high-volume task, ... at the problem -- but latency does not work the same way: ...
    (comp.lang.python)