Re: Checking if referrer is web crawler

In message <1151495324.852580.282450@xxxxxxxxxxxxxxxxxxxxxxxxxxx>
StevePBurgess@xxxxxxxxx wrote:

Hi. I have a book affiliate website. Whenever a visitor clicks on one
of the books, a script adds one to a field in a mysql database and then
takes the visitor to the shopping basket on the book website.

I have noticed that the book links are getting lots of hit. At first, I
was pleased about the potential income this might mean - but then it
occurred to me that many of these hits are web crawlers (this was
confirmed by webaliser).

Any suggestions of ways of checking if the link is being "clicked" by a
webcrawler so that I can not increment the field in the sql database?

I've checked HTTP_REFERER but it seems to be empty for what I assume
are crawled clicks.

Cheers

Steve


Under user agent they appear to identify themselves as either:

bot
crawler
spider
slurp
crawling

--
Kev Wells http://kevsoft.topcities.com
http://kevsoft.co.uk/
ICQ 238580561
Useless Fact 04 The number of islands around mainland Britain is 6289.
.

Relevant Pages

  • Checking if referrer is web crawler
    ... I have a book affiliate website. ... a script adds one to a field in a mysql database and then ... I have noticed that the book links are getting lots of hit. ... occurred to me that many of these hits are web crawlers (this was ...
    (comp.lang.php)
  • Re: using forms to add records to mysql database - block submit on entering
    ... insert data into a mysql database using html forms. ... selecting 'submit' a new record in my database is created. ... But when I enter data in only one content and hit the enter ... reject the form submission if something is missing. ...
    (comp.lang.php)
  • MySql UTF-8
    ... Anybody managed to get a solution for using UTF-8 in a mysql database (v ... and classic asp? ... I seem to have hit a hurdle with the myodbc driver ...
    (microsoft.public.inetserver.asp.db)
  • fetchall_hashref error: invalid number of parameters?
    ... I am trying to hit a MySQL database using DBI::mysql. ... table information using the SQL - "SHOW TABLE STATUS". ... The statement handle prepares and then executes as it should, ...
    (perl.beginners)
  • using forms to add records to mysql database - block submit on entering
    ... I'm new to php and I'm trying to write some php code so that I can ... insert data into a mysql database using html forms. ... But when I enter data in only one content and hit the enter ...
    (comp.lang.php)