search engine challenge

From: Frank (frank.sonck_at_pandora.be)
Date: 01/26/04 

Date: Mon, 26 Jan 2004 12:26:40 GMT

Hello,

I'm running a site with +20.000 articles. The articles (html files) are
saved on the server as txt files. Alle other data (author, date, category
and so on) are in a MySQL db. Before we had the articles put in the db also
and then performed SQL queries for the search engine. But this is no longer
feasable since there are too many articles and the db has gotten too big.
The search engine does all of the db and the server cpu goes max.
I'm looking for a php type search engine that automatically indexes the txt
files, produces 1 index file with all indexed words + the id's of articles
having those words. Like that the search script doesn't have to query all
the articles (the whole db) anymore but just this one index file. Would be
nice also if there would be possibility to have a blacklist of words (the,
a,...) and other admin things.

Anyone has experience with this?

Greetz,
Frank.

Relevant Pages

  • Workstation/Server file access and file caching
    ... me to specific MS KB articles that can help me figure this out. ... system and accesses the database located on a shared folder on a Windows 2000 ... The operation in question is the recreation of an index file. ... with a Windows 2000 client and an NT4 server where there could be a problem ...
    (microsoft.public.win32.programmer.networks)
  • Re: search engine challenge
    ... > are saved on the server as txt files. ... Before we had the articles put ... > in the db also and then performed SQL queries for the search engine. ... hard work, and then either using the Google site search, or the Google ...
    (comp.lang.php)
  • Re: search engine problems...
    ... > feasable since there are too many articles and the db has gotten too big. ... > The search engine does all of the db and the server cpu goes max. ... Put the articles back into the database and index the database properly. ...
    (alt.php)
  • search engine problems...
    ... I'm running a site with +20.000 articles. ... The search engine does all of the db and the server cpu goes max. ... the articles anymore but just this one index file. ...
    (alt.php)
  • Re: search engine optimization question
    ... "Bob Bedford" wrote: ... more like search engine suicide than search engine optimization. ... > articles and it doesn't help for referencing ... popular or most recent articles, or a rotating list of links to ...
    (comp.lang.php)