[pLog-svn] website

[Jon Daley]

On Wed, 15 Oct 2008, Andy Wright wrote:
> On Wed, 2008-10-15 at 01:59 -0400, Jon Daley wrote:
>> 	These aren't ddos attempts - the worst offender is Microsoft's MSN
>> search bot - I blacklisted him, since he sends hundreds of requests within
>> seconds.
>>  	It also happens when there are lots of search engines all crawling
>> at the same time, so even those that do obey the crawl delay, when they
>> all come in at once, it adds up to a lot of requests.
>>
>>  	Today, I think I finally (hopefully) got the right setting to help
>> stay alive during these overload situations, primarily by turning down the
>> MaxClient setting, but checking out the other settings within the prefork
>> module settings for apache.
>
> Did this method work?  What did your team end up deciding on for
> bringing the site back up?

 	It worked partially.  As long as the server continues to stay up, 
the main problem was a kernel bug:
http://bugzilla.kernel.org/show_bug.cgi?id=11418

 	The rest of it has to do with what happens when it gets 
overloaded, and failing gracefully.