Re: Automating Searches


John Ersatznom wrote:
nowwho wrote:
Hey,
Thanks for the information so far. I didn't realise there was so much
legal stuff envolved, its for a once off educational project. Didn't
think it would amount to spamming. The pogram would only be run about
50 times in total. There is a set number of queries, and a set number
of results returned. As its an eductional project I never thought of
the legal side!

It's not spamming -- I don't know what the other guy was smoking when he
wrote the post you're replying to. There is NO DIFFERENCE discernible to
Google if you

a) do 10 searches during the day by typing in a Firefox window while
doing research or
b) have your computer do the searches with less/no typing on your part

Google is being "ripped off" iff you do something like:

a) use huge amounts of their bandwidth -- well in excess of a normal
user doing a bit of heavy research say, generating large numbers of
searches or delving very deeply into the result set. Fetching 10
first-pages-of-results one for each of 10 queries, whether done by one
mouse click or ten typed-in queries, has little impact on them, and of
course the one mouse click case makes it actually 10 queries instead of
11 because you mistyped one and had to do it again :)
b) or use google search results to populate your own rival "search
engine" site with revenue-generating ads or what-have-you, either by
scraping google's database or by just putting up a page with a script
that takes peoples' queries and passes them to google, then takes the
result page and replaces google's sponsored links with umpteen flashing
banner ads. Then you're using google's work output to actually compete
against google, rather than simply using google for research. That makes
a crucial difference.

The point of the exercise is to get the URL's returned into an offline
database. It's an excersise purly to pull back the URL's from the
different search engines.

Using code to drive Google lightly and for personal/educational/research
reasons rather than commercial ones doesn't seem to be evil to me,
especially if they cannot in practise distinguish it from "normal" use
anyway, as it isn't producing excessive traffic or being used to compete
against google in some way.

I don't think its a question of good or evil, I think people are
worried that the code could be used for commercial reasons.

In fact, where do you draw the line? Firefox with manually-typed queries
is OK. Then we have Firefox with a MRU for queries; Firefox with query
guessing or autocompletion based on your current activities and
interests; Firefox with a plugin to take the result set too and
transform it e.g. to show 50 rather than 10 hits or to weed out
"supplemental results" that are usually MFA sites that really ARE
ripping off google; Firefox with a plugin to run the query of your
choice and bookmark the results every few days; ... Firefox with a
plugin to gradually build up a database of hits for various queries by
occasionally fetching the nth page of results for one of them, but you
don't publish these anywhere, just use them personally ...

I think the two things that mark a transition to being evil are causing
them excessive traffic and competing with them using their own data in
some way. (Also generating content-free MFA pages to generate revenue
via AdSense ads and SEOing them, but that's more using AdSense than
using the search engine proper, though the SEO will impact the latter
and pollute the results.)

This is an educational project and as computers is not my main interest
of study I don't know what a MFA, SEO are. Can this be explained?


I don't see any way to derive some kind of moral law that makes typing
something morally superior to doing it with one click, and actually
scheduling an automatic (infrequent) job or whatever actually sinful.
There's no inherent virtue in inefficiency, and computers exist to
enable automating tasks. Hyperlinks automate looking up and finding that
dusty reference or whatever; librarians may complain that they rot young
brains but the actual upshot is a gain in productivity, rather than some
kind of evil decadence setting in.

Any help with using the Google API or other suggestions would be a
great help. I also assume that Googe's API won't work with the other
serch engines, so would I have to write a different class for each
search engine?

.

Relevant Pages

  • Re: Automating Searches
    ... its for a once off educational project. ... There is a set number of queries, ... Google is being "ripped off" iff you do something like: ... Firefox with manually-typed queries ...
    (comp.lang.java.programmer)
  • Re: OT: This Administration is starting to worry me.
    ... When you can prduce the legal response to these arguments from Google we'll have more to talk factually about on this issue. ... * The motions requests that Google comply with a subpoena filed by the Attorney General and "produce" for inspection and copying the materials the Government is asking for. ... * First, the subpoena asks Google to produce an electronic file contain, "ll URL's that are available to be located on your companys' search engine as of July 31, 2005. ... Google was asked to "produce an electronic file containing ll queries entered into the Google engine between July 1 and July 31 inclusive. ...
    (rec.outdoors.rv-travel)
  • Re: [Legal] Firefox not truly Free?
    ... > The main disadvantage of the deal with Google is that native language ... "Mozilla Firefox" will have Google as the default search engine. ... Firefox do change the search engine to their local version of Google. ...
    (Debian-User)
  • Re: Microsoft doesnt learn.
    ... making MSN Search the default search engine in the new version of its ... Funny how Google (or Microsoft) didn't complain when Firefox has Google ...
    (comp.sys.mac.advocacy)
  • Re: Microsoft doesnt learn.
    ... Justice Department and the European Commission over Microsoft making MSN Search the default search engine in the new version of its dominant Internet Explorer browser, ... Funny how Google didn't complain when Firefox has Google as it's default search! ...
    (comp.sys.mac.advocacy)
Loading