Full Version: Server problems, 31/03/05
Yes, we know, there were server problems. We're back now.
Yay!!!! thanks guys... i was dying without neowin 
nice
QUOTE(Mr magoo @ Mar 31 2005, 08:27)
Yes, we know, there were server problems. We're back now.
[right][snapback]585702486[/snapback][/right]
[right][snapback]585702486[/snapback][/right]
Yeah, what happend?
database errors. wooohooo. i'm back!
Datacenter issues at ThePlanet (where Neowin is hosted). Lots of sites hosted there were down, including ASO servers (timdorr's hosting), vbulletin.org, etc.
edit: and of course Neowin.net. How could I ever forget Neowin.net...
edit: and of course Neowin.net. How could I ever forget Neowin.net...
i nearly died
so then i had a lunch break
and when i came back
i got neowins msn alert
so i though
hmmmmmmmmmmmmm
i thought itwas down
so checked
and hoooray
so then i had a lunch break
and when i came back
i got neowins msn alert
so i though
hmmmmmmmmmmmmm
i thought itwas down
so checked
and hoooray
goddamn theplanet i was dying without neowin! so glad everything's back to normal [hopefully] now =]
[damn, i'm so sad >.>]
[damn, i'm so sad >.>]
phew... i was tryin to get on for ages. I did notice that a few other sites were down as well, obviously hosted by the planet.
Weirdly enough, MY site, hosted at ASO in The Planet was working. 
Duck Dodgers saved the day?
supposedly you could connect to servers via ip, which i don't think was true...
you can read up about this even at
http://www.webhostingtalk.com/showthread.p...0336&perpage=15
theres currently 61 pages and growing every few minutes
you can read up about this even at
http://www.webhostingtalk.com/showthread.p...0336&perpage=15
theres currently 61 pages and growing every few minutes
Had a funny feeling it was down. No biggie, keep up the great work guys 
QUOTE(Joel @ Mar 31 2005, 13:48)
Weirdly enough, MY site, hosted at ASO in The Planet was working.
[right][snapback]585702551[/snapback][/right]
[right][snapback]585702551[/snapback][/right]
timdorr has quite a few servers in all of the different datacenters there, the first one, which houses two of Neowin's servers was entirely down.
I'll post the official explanation from ThePlanet when they get it up... but looking at the routing graphs in Orbit... this was *bad*
ThePlanet's web site is back up now ... so they must have fixed most of everything that went wrong ...
servermatrix hosted sites/ servers are still down though :'(
Here's the reason peeps
At approximately 4:15AM CST, a pair of redundant Powerware 500KVA UPS units failed creating a power failure in section B of our DLLSTX2 datacenter. Emergency teams were deployed within minutes and power was restored within minutes but intermmittment power outages did occur until 6:45AM CST. Powerware, JT Packard, and electricians are currently onsite with over 100 Planet technicians working to resolve the issue. We do not anticipate any further outages . A formal RFO will be released once the team debriefs. We apologize for all issues that has caused.
In laymens: A couple of UPS's blew up and took a section of the DC with them, Neowin's servers are in that section.
At approximately 4:15AM CST, a pair of redundant Powerware 500KVA UPS units failed creating a power failure in section B of our DLLSTX2 datacenter. Emergency teams were deployed within minutes and power was restored within minutes but intermmittment power outages did occur until 6:45AM CST. Powerware, JT Packard, and electricians are currently onsite with over 100 Planet technicians working to resolve the issue. We do not anticipate any further outages . A formal RFO will be released once the team debriefs. We apologize for all issues that has caused.
In laymens: A couple of UPS's blew up and took a section of the DC with them, Neowin's servers are in that section.
Latest statement from theplanet.com
QUOTE
Location: DLLSTX2
Severity: Level 5
Description: At 4:09AM CST, a power failure occurred in a redundant pair of Powerware UPS units feeding power to section B of the DLLSTX2 datacenter. The power failure was caused by a faulty fuse in UPS unit B-1. As the load transferred to UPS unit B-2, the spike in the load created an overloaded breaker and UPS B2 also lost power. This resulted in a power outage to the main power distribution unit feeding section B of the datacenter floor. Emergency teams were notified and the power was restored within 20 minutes. The power continued to cycle until 6:45AM due to the faulty fuse and the inability of the redundant UPS units to remain in bypass mode. Customers may have noticed several power cycles during this time period. Powerware, JT Packard, and electricians found the problem and replaced the faulty hardware and fuses. At 6:45AM, all electrical service was restored to normal and the NOC team began to bring all servers back online. The technical staff is currently placing a console on all servers to verify server restarts. Customers with operating systems that require a file check may have experienced extended downtime during the file check. Powerware and JT Packard will continue to monitor the UPS systems for the next 24 to 48 hours. The Planet does not anticipate any further outages.
Severity: Level 5
Description: At 4:09AM CST, a power failure occurred in a redundant pair of Powerware UPS units feeding power to section B of the DLLSTX2 datacenter. The power failure was caused by a faulty fuse in UPS unit B-1. As the load transferred to UPS unit B-2, the spike in the load created an overloaded breaker and UPS B2 also lost power. This resulted in a power outage to the main power distribution unit feeding section B of the datacenter floor. Emergency teams were notified and the power was restored within 20 minutes. The power continued to cycle until 6:45AM due to the faulty fuse and the inability of the redundant UPS units to remain in bypass mode. Customers may have noticed several power cycles during this time period. Powerware, JT Packard, and electricians found the problem and replaced the faulty hardware and fuses. At 6:45AM, all electrical service was restored to normal and the NOC team began to bring all servers back online. The technical staff is currently placing a console on all servers to verify server restarts. Customers with operating systems that require a file check may have experienced extended downtime during the file check. Powerware and JT Packard will continue to monitor the UPS systems for the next 24 to 48 hours. The Planet does not anticipate any further outages.
QUOTE(Joel @ Mar 31 2005, 09:48)
Weirdly enough, MY site, hosted at ASO in The Planet was working.
[right][snapback]585702551[/snapback][/right]
[right][snapback]585702551[/snapback][/right]
My site is hosted at ASO too and was working just fine too
so what, random errors few andthen. nobody's perfect 
QUOTE(Hurmoth @ Mar 31 2005, 17:14)
My site is hosted at ASO too and was working just fine too
[right][snapback]585703483[/snapback][/right]
[right][snapback]585703483[/snapback][/right]
The data center only lost 50% power, thats why some servers worked and others didn't.
QUOTE(DigiWire @ Mar 31 2005, 13:28)
The data center only lost 50% power, thats why some servers worked and others didn't.
[right][snapback]585703558[/snapback][/right]
[right][snapback]585703558[/snapback][/right]
Ahh... ok. Sucks Neowin went down. I can live with my site going down
but Neowin, that's just not right
This is a "lo-fi" version of our main content. To view the full version with more information, formatting and images, please click here.