Internet chaos as US Amazon EC2 goes offline

By Owen Williams, Hot! 30

Amazon EC2 in US-East experienced a major outage today, with major sites across the web that rely on their services going down. These sites include Foursquare, Reddit (pictured below), 500px, CloudApp and other major outfits.

The Amazon status website currently says there were "network connectivity" issues with North Virginia for up to 30 minutes and ongoing issues with RDS databases and the internet. It appears all hosts within the "US-East availability zone" went down. The purpose of the availability zone is to allow the service to be accessible during an outage such as this, but it failed in this instance.  

During the outage the Amazon EC2 forums erupted with users complaining about the services being down. One user on the forum said; "ALWAYS DOWN, network of ec2 us east is unavaiable at all, why paying?." Another user said he was leaving the service immediately, stating "Awesome. Down again! Even tho it has been 30 minutes, I am moving everything over to Storm on Demand." 

For those who aren't familiar with Amazon EC2 (Amazon Elastic Computer Cloud), is a service that lets customers rent computer resources from "the EC2 cloud" and "provides storage, processing, and Web services to customers." The service is extremely popular and many major websites and services rely on it to operate.

It is not yet known what the cause of the outage was, but it is likely to leave many customers questioning the volatility of the service, considering how many major outages it has experienced this year. The news of this outage comes hot on the heels of Amazon experiencing a major issue in Ireland, where their datacenter was struck by lightning, causing an ongoing outage for many users.

RT @kwestin: Appears Amazon EC2-East just went down bringing Netflix, Foursquare, Turntable, Instagram and more down with it.less than a minute ago via TweetDeck Favorite Retweet Reply


RT @davetufts: Amazon EC2 us-east: also downgraded by S&Pless than a minute ago via Twitter for iPhone Favorite Retweet Reply

@findingnewo my service is hosted on AppHarbor, which uses Amazon and it is down...less than a minute ago via MetroTwit Favorite Retweet Reply

Update 1: It appears parts of EC2 are coming back online partially, but appear to be up ittermittently.

Update 2: Services now appear stable, however some major sites that rely on EC2 are still unavailable.

Update 3: Services have now been fully restored, according to the dashboard. 

Comments (30)

Reply
figgy Reply

Badly effected here. I have an end of the week project deadline and this loss of time is going to cost us big.
Hope the data and my recent work is safe - I loathe to redo it all.

excalpius Reply

figgy said,
Badly effected here. I have an end of the week project deadline and this loss of time is going to cost us big.
Hope the data and my recent work is safe - I loathe to redo it all.

AFFECTED, not effected.

Sorry to hear of your problems though!

Greasy Reply

Ahhhh the cloud...

Isn't that **** awesome!!!!

xiralos Reply

Greasy said,
Ahhhh the cloud...

Isn't that **** awesome!!!!

thunderstorms FTW!

Kevin Flynn Reply

Greasy said,
Ahhhh the cloud...

Isn't that **** awesome!!!!

We'll all still be using Chrome OS by this time next year. Windoze doesn't stand a chance!

FrozenEclipse Reply

Kevin Flynn said,

We'll all still be using Chrome OS by this time next year. Windoze doesn't stand a chance!

Not sure if serious, sarcastic, or stupid.

articuno1au Reply

FrozenEclipse said,

Not sure if serious, sarcastic, or stupid.


Clearly sarcastic

youboreme Reply

FrozenEclipse said,

Not sure if serious, sarcastic, or stupid.

The latter would be you.

MichaelJTKnox Reply

I love the way people complain about this like it's unbelievable that it could happen...
Without electricity, us IT people are all screwed. Cloud = Bad weather ahead lol

Gerowen Reply

I will never understand why anybody trusts their business resources to 3rd party cloud services. If I can run my own fairly secure webserver using a $100 PC and a $35 router with custom firmware, you'd think somebody with the money to hire a dedicated specialist would be able to do the same, and therefore maintain direct control over their online assets instead of entrusting them to someone else.

war Reply

"Disaster Recovery" perhaps. That is to say, most people don't have multiple data centers throughout the world. So when your 1 little router and $100 PC goes out, say due to bad weather, then your site is down. Nice!

Don't you think it's nice to have "multiple copies" of your site "out there" just for situations like that?

remixedcat Reply

CoMMo said,
I will never understand why anybody trusts their business resources to 3rd party cloud services. If I can run my own fairly secure webserver using a $100 PC and a $35 router with custom firmware, you'd think somebody with the money to hire a dedicated specialist would be able to do the same, and therefore maintain direct control over their online assets instead of entrusting them to someone else.

also many ISPs block web servers. You don't have the redundancy or fault tolerance. netflix, on the other hand could have easily built thier own datacenter so they can have more control or used a specialist private cloud company instead of a jack of all trades outfit like ec2.

Nashy Reply

What the hell is EC2?

Owen W Reply

Nashy said,
What the hell is EC2?

Elastic Compute 2. It's "the cloud"

SharpGreen Reply

Owen W said,

Elastic Compute 2. It's "the cloud"

Elastic Compute Cloud***
The "2" means two "c"'s

Owen W Reply

SharpGreen said,

Elastic Compute Cloud***
The "2" means two "c"'s

Wow, I must have been out of it. I totally know what it is, I use it at my day job :-/

frankGawd4Eva Reply

Could be a separate issue but Netflix is currently down for me.. East coast ..

Owen W Reply

jbarcus81 said,
Could be a separate issue but Netflix is currently down for me.. East coast ..

Still? I believe this is related.

frankGawd4Eva Reply

Owen W said,

Still? I believe this is related.

Down for quite some time now.. since about 10pm EST or so.. been watching http://downrightnow.com/netflix

greenwizard88 Reply

jbarcus81 said,
Could be a separate issue but Netflix is currently down for me.. East coast ..
Yeah, Netflix was quite slow for me on the east coast too, this would explain it.

youboreme Reply

and to think, some people still think it wise to use "the cloud" to store personal data instead of conventional hdds...

SharpGreen Reply

LordBattleBeard said,
and to think, some people still think it wise to use "the cloud" to store personal data instead of conventional hdds...

Especially when the only reason the particular cloud service in question exists, is so that Amazon isn't wasting money on excess data center capacity that they aren't using at any one time.

dodgetigger Reply

SharpGreen said,

Especially when the only reason the particular cloud service in question exists, is so that Amazon isn't wasting money on excess data center capacity that they aren't using at any one time.

I still don't understand what happens when it is right before christmas, and Amazon needs all this capacity for themselves...

SharpGreen Reply

dodgetigger said,

I still don't understand what happens when it is right before christmas, and Amazon needs all this capacity for themselves...

I guess they just have several metric **** tons of datacenter capacity then. I don't really know.

remixedcat Reply

don't let amazon sour your cloud perception!!!! They make cloud look bad. ;-(

I say they should take that out of thier name....

bugsbungee Reply

remixedcat said,
don't let amazon sour your cloud perception!!!! They make cloud look bad. ;-(

I say they should take that out of thier name....

Developers make the cloud look bad. If you want resilience, use multiple regions, then your website will only go down if all your regions go down...Its totally a cost thing though, multiple regions are costly and license agreements don't always extend beyond country borders.

remixedcat Reply

Places like netflix should have built thier own datacenters. they would have more control over it.

Neobond Reply

Noticed Foursquare was down yesterday but didn't think much of it lol.

Enron Reply

Microsoft had their stuff fixed without making a big deal out of it.

Mike Frett Reply

Enjoy your cloud, people. And be prepared to expect a lot of this. Better to walk backwards than to step forwards I say.