Page 5 of 5

Re: Power outage this morning (the 4th)

Posted: Thu Jan 04, 2007 10:14 pm
by tigrus
vurumai wrote:Having personally lived with the aftermath of a power spike and UPS failure causing a outage to a large data centre - I really feel for guy in charge of Jolt's data center - he will not be getting much sleep atm.

Let me share with you some of the realities of data center facilities management. For simplicity, i will only cover the power related part only.

Data centres come in many "grades", and essentially you get what you pay for.

A top of the line datacentre, will have redundancy for everything.

For example; power facilities for a 1 Mega Watt data center will include;

* 2x phyiscally separate 1MW transformers supplied from different grid power circuits
* 2x physically separated UPS, each with sufficient battery strings to supply power for 90 mins
* 2x 1 MW generators

All the above will be housed in physically separated, fire proof rooms.

This scenario is refered to N+1 redundancy, and allows for the failure any facility without impacting uptime. This type of facility is extremely resilient / expensive and used for TRUELY critical systems eg. share trading, stock market, telco's etc. It is unlikely that any game company would consider hosting in such an expensive environment.

A more common datacentre power environment will have

1x transformer
1x UPS and battery string - good for 30 mins
1x generator

In this scenario, the failure of any component will put everything hosted in the environment at risk.

In the event of a failure from the electicity grid, both AC and DC powered equipment will revert to the UPS (uninteruptable power supply). UPS/battery string failures are suprisingly common and often only come to light in an power failure event.

You get what you pay for !!!

For a system to be totally immune to data center failure, then ALL components of it (in the case of Ryzom, website, klients & game servers) would need to be deployed in a "high availablity" architechture, REPLICATED to a physically separate site. Not only does this double all infrastructure costs, it also make the application significantly more complex to support and manage. These types of deployments are undertaken by corporates who want to protect their revenue generating services.

I would suggest that Ryzom is certainly not even close to being in that category !!!!!

Jolt's data center staff will be having a very difficult time at the moment. GF/Ryzom will be one of a myriad of customers impacted.

Because "hosting" is their core business, I am very sure that they will be taking resolution of these issues very seriously.

http://www.schlockmercenary.com/d/20000813.html