Server meltdown

Our service provider went down today, which resulted in WOT being unavailable for an hour. Service has now been restored, our apologies for the inconvenience.

Here’s a brief summary of what happened:

Sunday, Aug 17th, 16:26 UTC Server temperatures started going through the roof. Air conditioning broken at the co-location facility?

Sunday, Aug 17th, 17:09 UTC Our load balancers are offline, intermittent problems with network connectivity, servers are still up.

Sunday, Aug 17th, 17:16 UTC Load balancers are up again. Everything works, but servers are just about to melt.

Sunday, Aug 17th, 18:11 UTC Hard drives are now 10°C above their maximum operating temperature.

Sunday, Aug 17th, 18:12 UTC Half of our servers have been unplugged? We’ve received no communication from the ISP, except that they’re working on it.

Sunday, Aug 17th, 18:18 UTC And we’re completely down.

Sunday, Aug 17th, 19:10 UTC Some of the servers are back up, load balancers are up, down, up…

Sunday, Aug 17th, 19:13 UTC More servers are up. Temperatures are elevated, but seem to be going down.

Sunday, Aug 17th, 19:15 UTC We’re up again. Thanks for your patience!

Update: The outage was caused by a catastrophic power failure at the data center, which brought down several Finnish web services.

Update 2: There were also some issues with email delivery on our website and confirmation emails were delayed for newly registered users. If you contacted us after Sunday and still haven’t received a response, please resend your message.

Share on facebook
Share on twitter
Share on linkedin

Leave a Reply

Your email address will not be published. Required fields are marked *