(The quickest way to register)

Blog

Server meltdown

Sun 17 Aug 2008 07:42:56 PM UTC — Sami

Our service provider went down today, which resulted in WOT being unavailable for an hour. Service has now been restored, our apologies for the inconvenience.

Here's a brief summary of what happened:

Sunday, Aug 17th, 16:26 UTC Server temperatures started going through the roof. Air conditioning broken at the co-location facility?

Sunday, Aug 17th, 17:09 UTC Our load balancers are offline, intermittent problems with network connectivity, servers are still up.

Sunday, Aug 17th, 17:16 UTC Load balancers are up again. Everything works, but servers are just about to melt.

Sunday, Aug 17th, 18:11 UTC Hard drives are now 10°C above their maximum operating temperature.

Sunday, Aug 17th, 18:12 UTC Half of our servers have been unplugged? We've received no communication from the ISP, except that they're working on it.

Sunday, Aug 17th, 18:18 UTC And we're completely down.

Sunday, Aug 17th, 19:10 UTC Some of the servers are back up, load balancers are up, down, up...

Sunday, Aug 17th, 19:13 UTC More servers are up. Temperatures are elevated, but seem to be going down.

Sunday, Aug 17th, 19:15 UTC We're up again. Thanks for your patience!

Update: The outage was caused by a catastrophic power failure at the data center, which brought down several Finnish web services.

Update 2: There were also some issues with email delivery on our website and confirmation emails were delayed for newly registered users. If you contacted us after Sunday and still haven't received a response, please resend your message.

Trackback URL for this post:

http://www.mywot.com/ru/trackback/1435

Comments

  1. User picture
    • Yatti420 on Mon 18 Aug 2008
    • 12:05:15 AM UTC

    Maximimum Hard Drive Temps?

    What kind of Hard Drives are in use? What is the Max Operating Temperature?? Luckily nothing broke completely..

    Peace
    ---------
    Jared Gray
    • User picture
      • Sami on Mon 18 Aug 2008
      • 02:19:02 AM UTC

      Re: Maximum temperatures

      We're using a few different models, but most hard drives these days have environmental specifications, which tell their operating temperature range. Usually it's from 5 to 55°C.

      Some of the drives in our servers reached 65°C during this incident before being shut down.
  2. User picture
    • Yatti420 on Wed 03 Sep 2008
    • 09:48:36 PM UTC

    Thanks..

    Thanks..

    Peace
    ---------
    Jared Gray
  3. User picture
    • Countryboy on Fri 19 Sep 2008
    • 01:37:47 AM UTC

    WOT shut down

    Good to hear servers are back to normal temp. and running again!
    I don't feel safe without WOT riding shotgun when I am surfing the web.
    Mr. Palmer