apache Limit Apache and MySQL Processes and RAM Use

[mikeaag]

Hi all,

Hoping someone can help me with a problem im having.

Im currently running a drupal based website, which has been coded by someone else. Its hugely inefficient, doing anywhere from 300-1100 database queries and using around 90mb of ram for each page load.

Needless to say, our server takes a beating when more than 1 person is on it. In fact, it only takes about 15 people on the site at the same time to bring down the server.

I know this is a software issue, but we are currently trying to make the situation better by configuring the server a bit.

Here are some of the things i think would be beneficial, but dont know how to implement:

1. limit the number of apache and mysql processes
   
2. When i look at HTOP, it says 0/0mb for SWP. Does this mean i dont have a swap file set up? how do i check?
   
3. Also, i would like to be able to implement something similar to twitter's fail whale.
   

The site is currently hosted on a VPS, with 3GB guaranteed RAM and 3GB burstable RAM. Processing power doesnt seem to be an issue. The server is running Ubuntu 10.04 LTS.

Any help on this matter would be greatly appreciated. If more info is need, just let me know :D

Thanks in advance

Mike

[SuperKid]

This could be a misconfiguration of PHP if your site is coded in PHP. Sometimes the amount of PHP requests are being limited and therefore once PHP can't take no more your website will appear offline to other users. You have plenty of memory for a website to hold many, many, many users with the right configuration.

If you have lots of users, have you considered compiling your website in Facebooks open source HipHop software? I can tell you right now your cpu usage on the server will be cut more than half. Also another option to look at is to use nginx (www.nginx.com) It uses very very little memory, you could even run your site with 512Mb memory with nginx or even less.

I think limiting the amount of processes mysql and apache can make will be pointless, MySQL service only runs once on your server unless you have multiple mysql instances setup. If you need any help I will need your current PHP, MySQL and Apache configuration files posted.

Matt.

[mikeaag]

This could be a misconfiguration of PHP if your site is coded in PHP. Sometimes the amount of PHP requests are being limited and therefore once PHP can't take no more your website will appear offline to other users. You have plenty of memory for a website to hold many, many, many users with the right configuration.

If you have lots of users, have you considered compiling your website in Facebooks open source HipHop software? I can tell you right now your cpu usage on the server will be cut more than half. Also another option to look at is to use nginx (www.nginx.com) It uses very very little memory, you could even run your site with 512Mb memory with nginx or even less.

I think limiting the amount of processes mysql and apache can make will be pointless, MySQL service only runs once on your server unless you have multiple mysql instances setup. If you need any help I will need your current PHP, MySQL and Apache configuration files posted.

Matt.

Hi Matt,

Thanks for your quick reply.

I have attached a print screen of HTOP from the server so you can see how many processes are running currently with no one on the site.

From the screen shot you can see that there are multiple MySQL processes and multiple apache2 processes as well.

I've seen HipHop before, but not looked too much into it. I'll have a bit more of a read to see if its possible to use. Have you used it before? is it easy to set up? I only have SSH access to the server, and im a complete beginner when it comes to linux commands :D

Finally, if you could direct me to how i can get the those configuration files, i'd be more than happy to provide them for you

Thanks again,

Mike

[AnthonySterling]

Hi Mike,

Those values do sound excessive, the configuration files would be handy. You could get a quick-win by installing APC on the server, this should help by caching some of the code that is running automatically.

Forget Hip-Hop, there's plenty you can do before even considering something so drastic.

[mikeaag]

Hi Mike,

Those values do sound excessive, the configuration files would be handy. You could get a quick-win by installing APC on the server, this should help by caching some of the code that is running automatically.

Forget Hip-Hop, there's plenty you can do before even considering something so drastic.

Hi Anthony,

Thanks for your reply.

I've been looking into APC, however i have absolutely no idea how it works, how to install it and how to get it working with the drupal site running on the server :(

Have you set up APC on a server before? if so, would you mind giving me a quick run down on how to do it (i will also be googling, but some of the resources ive come across confuse/scare me as i dont know what they do)

I've been monitoring the server, and have noticed a that the apache and mysql processes dont seem to close that quickly, and because of that, they spawn at an alarming rate. For example, 2 days ago, at around 5pm, the site went from 1 user (me) to about 5. This caused the number of processes to jump from around 45 all the way up to about 140. Needless to say, this caused the RAM usage to sky rocket and within about 15 seconds, the apache2 service had switched itself off.

I started looking at setting apache's MaxClient settings. but again, i dont really understand what it is im doing, and by the time i had spent 20 minutes battling my way through the directories on the server (through ssh :( ) to find the config file, there were about 3 different settings for the maxclient.

I will attempt to hunt down the config files and get a copy of them uploaded on here.

As a side note - if anyone is familiar with drupal - I managed to decrease the number of queries that some pages ran down from around 1100 to about 400. This however did nothing to the amount of RAM the each page requires to run, they still all use 80-90mb. Would APC reduce this? if not, are there any other techniques i can employ :D

Thanks in advance

Mike