Comparing Centralized vs. Distributed Power Systems

Ultimately, the true cost of implementing a power protection infrastructure is measured over the useful life of the equip­ment and varies based on the scope of the installa­tion. Be­cause many of today’s information technology (IT) systems use distributed architecture, it is often as­sumed that the sup­porting critical power infrastructure should be distributed as well. A real world analysis of Total Cost of Ownership shows that, from a number of different perspectives, a cen­tralized power solution is often the most cost effective ap­proach while offering the added benefits of increased reliability and simplified monitoring.

This article discusses key aspects of the power infrastruc­ture in­volving uninterruptible power supplies (UPS) and asso­ciated hardware. It uses real-world scenarios to demon­strate the advantages and disadvantages of various configurations.

Power System Deployment

Any significant installation of hardware in an IT environ­ment can be disruptive to critical IT operations, especially installa­tions involving the power that makes everything else work. On the surface, it may appear that installing a UPS or group of UPS systems in the bottom of each equipment rack might be the easiest approach. However, in most deployments that involve more than a couple of racks of equipment, tak­ing a centralized approach not only streamlines require­ments for IT personnel, it all but eliminates interruptions to overall operations.

In a centralized configuration, virtually all of the power related work is performed by contractors who install a UPS and power distribution sized for the needs of the organiza­tion. Because they complete these installations independ­ently of the IT staff, there is very little disruption to other IT functions beyond a one-time “transfer of power” when the new power infrastructure is deployed. Once these power contractors complete their work, the IT staff can focus on their jobs with power essentially built-in to the data center.

Typically, a distributed system takes about one day per rack to deploy with most of the work performed by IT per­sonnel who are generally not as familiar with power is­sues as they are with their core systems. Because of the ongoing nature of the work, multiple systems can be down frequently over many days as different components are powered up and down.

Power Density & Space Utilization

As mainframe and minicomputers have given way to dis­trib­uted network architectures, rack designs have been opti­mized to accommodate high-density computing. The most cost effective use of this rack space is to populate it with computers and their core peripherals. Because power equipment does not require the frequency of ac­cess or rear access that other IT equipment needs, it can easily be located in a less expensive area. In fact, power can actually be more secure when located away from more heavily accessed areas, re­moved from accidental or in­tentional intrusions.

A centralized UPS can be sited away from traffic areas, in a separate room or in a corner, and actually reduce total space required for the overall installation by elimi­nating the need for rear access as well as inefficient use of rack space. While centralized UPSs may seem large, the aggre­gate space re­quirement for distributed systems typically ends up being even larger and occupying considerably more footprint. (See Figure) Rear access requirements, common in most distributed systems, can actually triple the space needed for installation. When that space is lo­cated in prime IT areas, the real cost of an installation can increase rapidly.

Another hidden cost involved with distributed installations is the interconnect cabling. In a distributed environment, users often find that they are given only a single-source for pro­prietary interconnect cables. Meanwhile in a com­parably sized centralized installation, contractors gener­ally install the standard wiring according to the National Electric Code (NEC) after a competitive bidding process that assures the most cost effective installation.

Expandability & the Rightsizing Myth

Simply stated, distributed power installations are more ex­pensive on a cost per kilowatt basis than a centralized in­stallation. While a centralized installation may be frac­tionally more expensive to build with excess capacity to allow for future growth, expansion of a distributed system can be extremely costly in terms of equipment purchase as well as downtime and disruption to power during subse­quent installations. In a distributed installation with doz­ens, per­haps hundreds of in­dividual UPS systems, users are paying for what amounts to duplicate or multiple user interfaces as well as packaging, cabling and mechanical features that add nothing to reliability.

Planning ahead is essential for any large installation. In the case of power infrastructure, this includes making accom­modations for proper wiring, distribution and circuit breakers that are all extremely expensive to add after the fact. Whether the installation involves one 200 kVA UPS or one hundred 2kVA systems, the electrical service to the area must be sized for the total current required. Clearly, it is not practical to increase the overall service to the facility in 2kVA increments and, in most cases, nei­ther is it practical or cost effective to add UPS systems in a piecemeal fash­ion. Adding power equipment to racks as power needs in­variably grow can result in downtime as connected equipment must be taken offline. Rightsiz­ing centralized power from the beginning facilitates “plug-and-play” IT in­stallations with virtually no disruption to nearby equipment.

In some ways establishing power requirements is similar to determining cell phone minute usage in an organization. Users sharing a central pool of power (or minutes) tend to use that resource more effectively because the heavier us­ers and lighter users balance each other. If every rack contains a 3kVA UPS with each rack using 1.5 to 2.5kVA of steady state power and no more than 3kVA of peak power during startup, it is easy to see that quite a bit of excess power is being wasted. When a piece of equipment added to a rack causes the total power requirement to exceed 3kVA, the configuration must be completely reengineered to accommodate a small increase in power.

Efficiency

The efficiency of a power system is almost invisible to the user, but the operating cost difference using a high efficiency UPS can equal the cost of the entire power system in three to five years. Not only does lower efficiency boost utility bills, the wasted power shows up in the form of heat that requires added air con­ditioning that increases infrastructure costs and results in still higher utility bills.

Energy efficiency of a UPS is the difference between the amount of energy (as documented by the utility meter) that goes into the UPS versus the amount of useful en­ergy that is available to power the connected equipment. While all UPS systems lose some energy in the form of heat when it passes through the internal UPS components, centralized systems — with their optimized designs and reduced parts count — typically have a 2 to 10% efficiency advantage com­pared to distributed systems. Although this may not sound substantial, the fact that UPSs operate 24-hours per day every day means that even a small improvement in efficiency can translate to tens of thousands of dollars in savings in only a year or two.

In addition, centralized UPSs are designed to maintain efficiency with different types of loads (connected equipment) and varying load levels. Distributed systems often specify high efficiency when fully loaded but typically operate at a fraction of their rated capacity where their efficiency is considerably lower.

Reliability          

While marketing materials may boast of superior reliability, every engineer knows that the statistical reliability of a given piece of equipment can be calculated by multiplying the number of components by the mean time between failures (MTBF) for each component.

In simplest terms, the more components contained in any given system, the lower the reliability. At the same time, increased parts counts generally translate to higher costs. In a simple analogy, it is easy to see that an array of four light bulbs each supplying 25 Watts will cost more while requiring added time and cost to install and maintain compared to one 100 Watt light bulb.

By definition a distributed configuration has more parts — and more parts that can fail. In fact, the basic com­ponent count in a large UPS is not substantially different than that in a small UPS. But kilowatt-for-kilowatt one large UPS has far fewer compo­nents and those components are far more robust than those in multiple small UPSs needed to supply the same power. Even if the smaller devices had a higher individual MTBF (which they don’t), the increased number of devices leads to an inherently less reliable system overall.

Additionally, because the large UPS incorporates more robust components and a system design geared towards heavy-duty operation, centralized systems have greater fault-clearing ability – as much as 200% of rated output – that leads to inherently higher reli­ability. Power disturbances such as short circuits that will shut down lesser systems are barely noticeable in a centralized UPS.

Monitoring & Integration

The ability to monitor the operation of any UPS is an important factor in overall uptime for any installation. A sin­gle 200kVA UPS has one interface on the UPS front panel. In addition, it can easily be monitored remotely by a variety of means from pager or e-mail notification of power alerts to integration with network management system (NMS) con­soles that will display a host of system parameters. The same is true of most high-quality individual distributed UPSs. The problem occurs when an installation that could be sup­ported by a single UPS is instead supported by dozens or hundreds of smaller UPSs. Network managers will still re­ceive notification of problems with the system, but with multiple UPSs, it is not always easy to identify which de­vice needs attention. Oftentimes, IT managers don’t even know where all of the UPSs are located. Moreover, in the event of widespread power disturbances, it is not uncommon to have all of the UPSs in a given area broadcasting cautionary notifications simultaneously. In such a circumstance an IT manager would literally be inundated with alert messages.

In terms of integration, it is important to note that the UPS itself makes up a relatively small part of a total power instal­lation. Whether centralized or distributed, batteries and power distribution make up about 60% of a typical installation. When power requirements expand, these components are actually easier to add on an as needed basis than the UPS hardware in most cases.

Maintenance & Service

Even the most reliable power protection system has components with a finite life — especially batteries. Yet few IT organizations have staff trained to deal with the maintenance, trouble­shooting and service of power systems. Not only does preventive maintenance prolong system life, the availability of factory-trained service engineers can be vital to minimizing downtime. Surprisingly, most resellers of UPS systems used in distributed applications either require depot repair for the UPSs or offer only third-party service by engineers who typically specialize in more lucrative computer repairs or other types of service. For critical installations a 7X24 service con­tract — with factory-trained engineers — adds to uptime far more than it might add to total cost of ownership.

Battery Maintenance & Configuration

Batteries are the weakest link in any UPS system, centralized or distributed. However, for a number of reasons, batteries have a longer life expectancy in centralized systems. It is not uncommon to see batteries in large systems last 5 to 7 years as opposed to 3 to 4 years in distributed systems.

Due to different cost structures and size constraints, batteries in centralized systems are often drop-shipped directly from the battery manufacturer and tend to be of a higher quality than those in smaller UPS systems. In addition, centralized battery banks are generally de­signed to facilitate regular inspection and maintenance without disruption to connected equipment. The combi­nation of higher quality batteries and ease of preventive maintenance is an impor­tant factor in increased battery life. In addition, the double-conversion technology typi­cally used in centralized UPSs minimizes battery cycles compared to the line interactive systems commonly used in distributed installations that actually cycle the batteries to provide power conditioning. In any battery, fewer discharge cycles translate to longer bat­tery life.

Since battery cost is a significant part of the cost of any UPS, maximizing battery life is a key element in reducing total cost of ownership.

Even with the best batteries, careful monitoring and proactive maintenance are critical to assuring uptime. It will always be easier to moni­tor and maintain a centralized system than dozens of separate battery installations. Since every installation contains multiple batteries, monitoring of mul­tiple battery systems can be even more challenging than monitoring multiple UPS systems. In centralized envi­ronments, battery monitoring is simplified because more sophisticated moni­toring options are available for large systems and also be­cause the cells are typically in one place rather than tucked into every equipment rack in the facility.

Conclusion

The fact that a single small rack-mounted UPS seems inexpensive and easy to install can mislead users to a false sense of economy. In reality, the reduced space requirements, robust design, industrial-strength components, greater operating efficiency, optimized battery systems, superior reliability and lower cost per KVA strongly support use of centralized configurations. When the complete installation, operation, maintenance and upgrade costs are considered, the savings of a centralized UPS installation are considerable.