Phusion Passenger Tweaking: Apache stuck in Sending(W)
For some reason I found this information missing from Google searches so I'm going to write one up and wait for Google to index it.
I was having a problem where Apache's process list would slowly begin to fill up with requests stuck in a Sending (or W) state. After my MaxClients was reached, the server would effectively shut down.
Searching for "apache processes stuck in sending W state" yields a whole bunch of people using mod_php and Apache. And it seems that no one ever solved it in a reproducable way. And I'm using Phusion Passenger anyway.
So it seems I had to use my brain. This server:
- Low traffic
- lots of virtual hosts
- No code pushes coincided with the problem.
I couldn't find any other symptoms anywhere. Ruby wasn't getting stuck. The CPU load was low and memory looked normal. All the Apache stats were good.
Then I looked at passenger status.
sudo passenger-status |
And I saw only 6 instances. If I ran passenger status again I could see those instances swapping rapidly between one site and all the others.
I found in the Passenger docs, PassengerMaxPoolSize, which defaults to 6. In my passenger.load (Ubuntu config mod file) I added:
1 2 |
PassengerMaxPoolSize 20 PassengerUseGlobalQueue on |
Voila!
It seems one site had starting to get heavy traffic and was constantly pulling instances. But Passenger didn't handle the context switching very well. By giving more pools, that site is able to keep a few all the time so there's no switching.