Re: CL Scaling for High Traffic Web Sites
- From: Tim Bradshaw <tfb+google@xxxxxxxx>
- Date: 30 Apr 2007 02:08:23 -0700
On Apr 29, 11:09 pm, bob <papersm...@xxxxxxxxx> wrote:
The idea of memcached is to do make use of existing memory for as
cheaply as possible, retrieving data from multiple machines through a
centralized interface, while offering orders of magnitudes of
performance over disks.
My experience, based on having worked at a significantly large site
was that the best approach by far was to a customised traditional web
cache (a distributed squid in some form or other) to cache everything
that could "easily" be cached. This typically is more than 90% of
your traffic. For the remaining traffic I think that non-stupid
design of the system would mean that a decent database can cope. (in
fact I know this is true).
The front-end cache needs significant thought - we had a very smart
guy who did ours, basically replacing some expensive commercial thing
which killed the site more-or-less every day with a customised squid
which worked very well. Even then there were things it was critical
to get right - for instance we had some imagemap which caused URLs to
come in which had coordinates in which were essentially always a cache
miss and this used to kill the site frequently. Fixing this made a
huge difference.
The non-stupid design thing also matters. At some point I worked out
that we were averaging 1000 requests a second at the front end, well
over 90% of which were satisfied from cache, and the back end database
was sustaining 4000 IOPS. So that's worse than 400 IOPS per uncached
request. I kind of realised we were doomed at that point.
--tim
.
- Follow-Ups:
- Re: CL Scaling for High Traffic Web Sites
- From: Tim Bradshaw
- Re: CL Scaling for High Traffic Web Sites
- From: bob
- Re: CL Scaling for High Traffic Web Sites
- References:
- CL Scaling for High Traffic Web Sites
- From: bob
- Re: CL Scaling for High Traffic Web Sites
- From: Alex Mizrahi
- Re: CL Scaling for High Traffic Web Sites
- From: bob
- Re: CL Scaling for High Traffic Web Sites
- From: Alex Mizrahi
- Re: CL Scaling for High Traffic Web Sites
- From: bob
- CL Scaling for High Traffic Web Sites
- Prev by Date: Re: What are the domains that lisp doesn't fit int?
- Next by Date: Re: Agents service?
- Previous by thread: Re: CL Scaling for High Traffic Web Sites
- Next by thread: Re: CL Scaling for High Traffic Web Sites
- Index(es):
Relevant Pages
|
