Re: page suck attack

robert wrote:
Hi
Every so often my site is "attacked" in which all URLS on my site are retrieved in the span of minutes. What is this called?? I mean what do I google for? I don't know where to begin.

I'm not sure if I am going to implement such at thing but I would like to be able to research it to know my options.

thank you
robert

Hi Robert,

It would appear to me that you've been hit by a scraper bot / content harvester.

If you could give a couple of lines from the log file I'd be able to confirm pretty quickly.

These scrapers are normally easy to block, all you need to do is block there IP address / Subnet.

Many Regards

Nathan
.

Relevant Pages

  • Re: [PHP] page suck attack
    ... robert wrote: ... slurp. ... both of them also check my site over a span of days. ... The others come from regular isps as far as their IP tells me and the hits are within milliseconds. ...
    (php.general)
  • Re: page suck attack
    ... retrieved in the span of minutes. ... Could also be someone using wget to download your site. ...
    (php.general)