Explosion at theplanet knocks out 9000 servers

Neowins previous web servers host, ThePlanet based in Dallas Texas, suffered a major setback yesterday as an electrical fire allegedly caused an explosion, knocking out around 9000 of their customers hosted websites.

Marcus Langenveld writes: "Apparently there was an electrical fire at ThePlanet's H1 data center in Houston Texas. According to some sources there was an actual explosion powerful enough that it knocked down walls in an equipment room. So far there are no reports of damage to servers or networking equipment but power is offline at the facility and approximately 9000 websites are affected. This will surely be all over the tech news websites by morning. <- You got that right! - Ed.

View: Marcus Langenvelds Blog + theplanet customer statement
View: Updated information by the staff @ ThePlanet Forum

Update
Just over an hour ago staff member, Todd Mitchell posted an update on the situation:

Morning,

We are continuing to work through various issues this morning. We will have additional contractors on-site this morning starting at approx. 7 AM. Some will hand-off from contractors who worked overnight and others will start the recovery/installation of new electrical gear to power the data center.

We are still working through the EV1 DNS and ServerCommand items. We are making progress on both items and expect to have both functional within the next 120 minutes.

In addition to the above, the network engineering group worked overnight to prepare the network for the recovery of H1. We expect the reconvergence of the network to go smooth once H1 comes back online.

We do not have an Estimated Time to Repair at present; we should have a better estimate this morning. Our staff and management continue to work through the night and morning-- we will continue to provide hourly updates.

Todd

Report a problem with article
Previous Story

Facebook probed after law students file complaint

Next Story

Dell Eyes Two-Thirds of Sales From Outside U.S. in 5 Years

39 Comments

Commenting is disabled on this article.

A major local forum here in my town got taken down by it. Before it went down, their servers were really lagging anyway, so I figured it was them taking it down. Good to know it wasn't because of that.

yep
Nameservers for www.msgpluslive.net:

* ns1.ev1servers.net returned (SERVFAIL)
* ns2.ev1servers.net returned (SERVFAIL)

Some of the Messenger Plus! services are hosted at EV1 yes. For the main web site, it's just the DNS that was knocked off.

The fire was at The Planet's Houston datacenter aka EV1 (Everones Internet) before TP bought them out. This was not at their Dallas DC.

I was wondering why the web was behaving oddly... At least some sites in particular

Hope it gets sorted out quickly

hehe.. howard from Horwardforums.com is a funny guy:

ZOMG!!! The HowardForums Image Server is down (something at the Data Center caught fire). This will cause pages to load slowly. Turn images off to speed things up or use wap.howardforums.com or if you're just bored.....


I was wondering why that was happening. Now it makes sense.

only if someone caused the problem. It's possible that it was just some hardware they had laying around that had a spac. Will have to wait and see.

luckily my server wasn't affected. they reported that none of the servers or networking equipment were damaged so it looks like the data on the systems is fine.

Wow that blows, every minute a single site is down costs them money, imagine how much is being lost over this issue

(toadeater said @ #8.1)
Not as much as I'm losing filling up my gas tank thanks to another famous product of Texas.

No one else is going to make it any better either, not NY, not IL, not AZ. You should be able to figure out what I mean.

"We have determined that no servers in the data center have been damaged. Nonetheless, they are down because power is out."

update 2:

Hello,

The team here at The Planet continues to work through the various issues that we continue to encounter. We are still making progress on the previous items that I mentioned in my last post. DNS infrastructure has been migrated to another data center and propagation has begun. We are working through some database issues with ServerCommand and fully expect those to be resolved within the next hour.

I’d also like to address the idea of migrating from one data center to another. During the early stages of the H1 data center we opportunistically relocated some customers to another data center. However, due to network and data center (power/cooling) constraints, this option is no longer available and requests for migration cannot be honored. Please rest assured that our teams are working diligently to return service to all affected customers.

At this time we do not have an Estimated Time to Repair at present; we should have a better estimate this morning. Our staff and management continue to work through the night and morning-- we will continue to provide hourly updates.

Todd