RE: Problem with script requiring valid referrer

From: Chris Devers (cdevers_at_pobox.com)
Date: 02/08/05


Date: Tue, 8 Feb 2005 14:50:59 -0500 (EST)
To: Chris Schults <cschults@grist.org>

On Tue, 8 Feb 2005, Chris Schults wrote:

> > This is really a question for the CGI list, but whatever.
>
> Sorry, didn't realize there was one. Thanks for the heads up!

I'm not trying to be dismissive; just pointing out that there's a
similar list that more specifically covers this topic. But a lot of the
same people are on both lists, so it's not the end of the world.
 
> > Have you looked in the web server logs? Are you recording referer data?
> > If not, you should be, it's useful in cases like this. If you're
> > recording it, you should be able to verify or refute the claims.
>
> Yes, our web logs are recording referrer data, but I wouldn't know
> what to look for. I'll ask the CGI list.

If you're using Apache's combined log format, individual entries in the
log should show both the requested page as well as the referer, along
with data like the requester's IP address. All you need to do is look up
hits for the URL in question and see if the referer field looks right.

A typical entry might look something like this:

  42.68.173.42 - - [08/Feb/2005:13:07:53 -0500] "GET /path/to/page.cgi HTTP/1.1" 200 18818 "http://www.grist.com/" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)"

So if you're digging for data on the command line of a Unixy system, you
might get at the data in question with a statement something like...

  $ grep '/path/to/page.cgi' /apache/log/access_log

And you can whittle it down with something like this

  $ grep '/path/to/page.cgi' /apache/log/access_log | \
> sed 's#^.*HTTP/1.." [0-9]* [0-9]* "\(.*\)".*#\1#'

Which should return a list of the referers recorded. Or you can
manipulate that in various ways, depending on what you want to see.

-- 
Chris Devers


Relevant Pages

  • Re: "Hooked on Hector?" Amazing Berlioz List from Amazon
    ... >> Don't spend too much time looking at those lists. ... > Robert Shaw's recording of the Requiem. ... Mr. Zeidler gives his reasons, ... And a Berlioz performance that is unremarkable simply misses the point. ...
    (rec.music.classical.recordings)
  • Re: "Hooked on Hector?" Amazing Berlioz List from Amazon
    ... >> lister enthusiastically buys into the standard HIP propaganda hook, ... >> that anybody propagates or swallows such stuff): hear Hector as Hector ... > Don't spend too much time looking at those lists. ... Robert Shaw's recording of the Requiem. ...
    (rec.music.classical.recordings)
  • Re: Syslog
    ... > Subject: Syslog ... I have a server running that logs to syslog for ... Top-posting makes list mails a bit difficult to follow ...
    (freebsd-questions)
  • More Directions Home-CD & Film
    ... lists of the BS/7 content appeared and these lists have changed several ... It is a 1959 recording made by Dick Kangas and may be the earliest ... Dylan composition on tape. ... as well as to selected press on whom an embargo has been ...
    (rec.music.dylan)
  • Re: Exchange 2003 Black Lists
    ... These logs are in IIS format. ... How to configure connection filtering to use Realtime Block Lists and how to configure recipient filtering in Exchange 2003 ...
    (microsoft.public.exchange.admin)