Re: Googling
- From: chas.owens@xxxxxxxxx (Chas. Owens)
- Date: Tue, 25 Dec 2007 21:44:30 -0500
On Dec 25, 2007 3:50 PM, yitzle <yitzle@xxxxxxxxxxxxxxxxxxxxx> wrote:
Hi
Summary: Is there an easy method to search Google and get the top
result (title & URL)?
I'm trying to write a script that gets URLs based on a name. I figured
using Google would be the simplest method. I need to search a specific
site, and can construct a query so that the top result is often enough
the correct result.
However, Net::Google requires a SOAP API key, and Google's site says
they are no longer providing new keys. I thought I might be able to
use WWW::Mechanizer, but the HTML that Google returns is fairly ugly
and I haven't tried parsing that just yet.
Is there an easy method to search Google and get the top result (title & URL)?
Using a script to scrape Google's result pages is against their Terms
of Service and their robots.txt.
from http://www.google.com/accounts/TOS
snip
5.3 You agree not to access (or attempt to access) any of the Services
by any means other than through the interface that is provided by
Google, unless you have been specifically allowed to do so in a
separate agreement with Google. You specifically agree not to access
(or attempt to access) any of the Services through any automated means
(including use of scripts or web crawlers) and shall ensure that you
comply with the instructions set out in any robots.txt file present on
the Services.
snip
from http://www.google.com/robots.txt
User-agent: *
Allow: /searchhistory/
Disallow: /news?output=xhtml&
Allow: /news?output=xhtml
Disallow: /search
snip
.
- References:
- Googling
- From: Yitzle
- Googling
- Prev by Date: Re: Googling
- Next by Date: replace chars
- Previous by thread: Re: Googling
- Next by thread: replace chars
- Index(es):
Relevant Pages
|
|
