I’ve had some hosting problems today. They actually had minimal impact (the biggest impact was to my own site) and I wanted to take some time to explain. This may get a little technical, but don’t hesitate to ask if you don’t understand anything.

The TimThumb Vulnerability

There’s been an issue lately with something called “The TimThumb Vulnerability”. Some WordPress themes and plugins use a tool called TimThumb to resize images. TimThumb had an problem which could let malicious hackers get access to the web server. Not good. The problem is described over on the WPCandy blog.

Now. More by chance than design, I don’t have timthumb installed anywhere in my WordPress installation, so my host was never at risk from anything I had installed.

But it seems that there are some nasty people out there who are scanning WordPress websites for the presence of timthumb. They run scripts which find WordPress installs and then blast them with requests for timthumb in known locations in the hope that they get lucky and can break in.

If you’re technical and interested, here’s some logs showing the scanning in progress:

ID M SS CPU VHost IP Request
14166 W 30 0.08 oikos-host.org.u66.96.128.60 GET /wp-content/themes/snapshot/timthumb.php?src=/g0../0d1.gif
14268 W 30 0.13 oikos-host.org.u66.96.128.60 GET /wp-content/themes/profitstheme_11/timthumb.php?src=/g0../0
14094 W 29 0.55 oikos-host.org.u66.96.128.60 GET /wp-content/themes/irresistible/timthumb.php?src=/g0../0d1.
13798 W 29 1.53 oikos-host.org.u66.96.128.60 GET /wp-content/themes/magazinum/timthumb.php?src=/g0../0d1.gif
14104 W 29 0.50 oikos-host.org.u66.96.128.60 GET /wp-content/themes/max-3.0.0/timthumb.php?src=/g0../0d1.gif
14110 W 29 0.23 oikos-host.org.u66.96.128.60 GET /wp-content/themes/eVid/timthumb.php?src=/g0../0d1.gif HTTP
14112 W 29 0.44 oikos-host.org.u66.96.128.60 GET /wp-content/themes/newsworld/timthumb.php?src=/g0../0d1.gif

This is also evident in the error.log where someone’s trying to access directory contents too:

[Wed Aug 31 16:28:45 2011] [error] [client 74.86.158.106] Directory index forbidden by Options directive: /var/www/vhosts/oikos-host.org.uk/httpdocs/
[Wed Aug 31 16:28:46 2011] [error] [client 74.86.158.106] Directory index forbidden by Options directive: /var/www/vhosts/oikos-host.org.uk/httpdocs/
[Wed Aug 31 16:34:57 2011] [error] [client 74.86.158.106] Directory index forbidden by Options directive: /var/www/vhosts/oikos-host.org.uk/httpdocs/
[Wed Aug 31 16:34:58 2011] [error] [client 74.86.158.106] Directory index forbidden by Options directive: /var/www/vhosts/oikos-host.org.uk/httpdocs/

Taken Down

So my host was being scanned for timthumb in this way, and it was affecting the performance of the server that my host runs on. So, as a preventative measure, my hosting company moved my files out of the way and effectively took one of my shared WordPress platform down. It turns out that other users on the server DID have timthumb installed and this was also causing issues.

This would have been fine but they, for some reason, neglected to inform me. Here’s what the host said:

Normal procedure in the case of malware is to suspend a domain then inform the client but due to the high number of infected sites and the impact it was having on the server we had to take emergency action in this case and just suspend them all.

It’s also worth noting that I do have monitoring of my hosts in place so I normally spot when the servers are down or not responding. However, in this case, the server was up and was responding with a web page – it was just the default web server homepage instead of the real pages it should have been sending out.

Quickly (?) Fixed

To be fair to the hosting company, once the problem was found and reported, sorted it out very quickly and efficiently.  However, from what I can tell my hosting was down for just over a day.

When I queried the hosting company further about whether or not other people had had sites taken down and not been informed they said:

“I think every other domain is back on and has been dealt with. We didnt realise there were any left off…Unfortunately its better to upset 10-20 customers than lose 5-600, its an unpleasant choice we sometimes have to take”

Lessons Learned

Of course, this is not great, and I will be reviewing my hosting and monitoring arrangements as a result. I care greatly about my clients and their uptime is important to me.

Oh, and if you have your own WordPress installed anywhere, be sure to go do a search for timthumb.php and make sure it’s up to date!!

This all happened on my smaller hosting platform and very few of my clients were affected. But if you were affected or want to know more please drop me a line.

Thanks – and sorry I got caught out.