By some estimates about two to three percent of all electricity generated in the USA today is used by or for the Internet [1,2]. Some of that power goes into making chips, but most of it is spent simply moving data. We don’t see the power in use because much of the action takes place behind locked doors in unremarkable, often windowless buildings called Internet data centers (IDCs), also known as telecom exchanges or collocation centers and, less formally, as Internet hotels, server farms and carrier condos. They’re home to all sorts of Internet-related companies, through whose servers, routers and modems flow the bits and bytes of modern e-commerce.

That sensitive electronic equipment, along with the air conditioners needed to keep it cool, is responsible for the IDCs’ large power demands. The “raised-floor,” equipment-filled rooms of an IDC can use three to five times more electric power per unit area than conventional office space [3]. It has been estimated that a fully occupied 500,000-sq ft (46,500-sq m) Internet hotel can consume as much power as an equivalently sized factory — or even a small airport [4].

But the sheer size of an IDC’s power demand is only one of its striking features; the other is its demand for high power quality.

 

• First, the power must be ultra-reliable. IDC tenants now ask for “six nines” (99.9999%) reliability and there’s talk of nine-nines (99.9999999%) being needed in the future. To achieve those levels, IDCs are equipped with redundant utility feeds, uninterruptible power supplies (UPS), and diesel-driven or battery back-up.
  
• Second, the power has to be squeaky clean, free from noise, voltage fluctuations, harmonic distortions and stray currents.

Clean power. That’s the copper connection.

The Importance of Good Grounding

IDC tenants normally ensure that all of their sensitive electronic equipment operates at a uniform and constant ground potential by installing equipotential grounding (EG) systems, also known as performance grounding or signal reference grounding systems near the equipment in question. An EG system is not electrically separate from the building’s conventional equipment grounding and lightning protection systems; it is simply another part of the overall system, physically located at the user end of the service. In fact, all three elements of the grounding system — safety, lightning protection, EG — are electrically bonded, as required by the National Electrical Code®. EG systems do differ somewhat from conventional grounding systems in that they are designed to present minimum impedance at frequencies ranging up to tens of megahertz. The IDC owner’s responsibility is to furnish a common, high-quality (low-resistance, low-impedance) grounding system for the entire building.

But, many IDCs are established in older buildings in central business districts near potential customers and high-capacity communications trunks. Such buildings, attractive as they might be to an IDC developer, unfortunately don’t ordinarily have the type and quality of electrical systems that sensitive electronic equipment requires. An electrical retrofit is usually necessary.

From Newsprint to New Economy: The Downtown Phoenix Telecom Exchange

Such is the case with the Downtown Phoenix Telecom Exchange (DPTE) (Top Image). This five-story, 380,000-sq ft building was built in 1947 as the printing plant for the Arizona Republic, the state’s leading daily newspaper. In 1999, the building was bought by Sterling Networks, Inc., a subsidiary of Sterling Capital, a Northbrook, Illinois-based venture capital organization that currently owns or controls more than five million square feet of commercial real estate across the country. Sterling transformed the building into an Internet hotel.

Doug Newcomb is vice president, Sterling Network Services, at the Downtown Phoenix Telecom Exchange. When he took on the job, he brought with him 30 years of experience in the communications industry, much of it gained at AT&T, Bell Laboratories and Lucent Technologies.

“The reason we’re successful at the DPTE is that Sterling was 10 months ahead of the market,” says Newcomb. “We’re in an ideal location in terms of power and network availability, as well as security. But, we found that Arizonans would rather keep their solid old buildings than tear them down and build new ones, and a lot of the older office space isn’t equipped for the quality of power that our tenants need.”

Sterling fixed that by entirely gutting the old printing plant, removing every wire and cable downstream of the utility disconnects. In its place, Sterling installed an electrical system specifically tailored to the needs of an IDC. The Smith Group, a Phoenix-based architect-engineering firm, designed the new system.

Smith took advantage of the building’s four existing 4000-A/480-V services (supplied by Arizona Public Service) but added three motor-generator sets (rated at 135-, 750- and 1500-kW) as backups. The two larger units back up utility power; the small one is reserved for life-safety systems. Tenants can utilize this backup system or install their own, as several tenants have elected to do. Generator sets are located in the subbasement.

Because the DPTE was built for a new and growing market, Smith’s electrical system plan called for maximum flexibility. Doug Newcomb sees that as a smart idea, considering the varied mix of tenants who have signed on. “Our tenants can design their own electrical systems,” he explains. “We offer a mixture of circuits, and all services are hot swappable. Some of our large tenants designed their systems for power densities as high as 150-300 W/sq ft (Figure 1). We can accommodate that, although I don’t think those levels will actually be realized.

“A few tenants initially told us they also wanted to install their own grounding systems, complete with individual grounding electrodes. The City of Phoenix was willing to go along with the idea, but I wouldn’t permit it — ground loop currents would have degraded our system’s power quality. We furnish the building’s only central grounding system, with access points at each tenant’s facility. It took us awhile to make our point, and it turned out to be a learning experience for the city inspectors, but they eventually approved our plan.

“Incidentally, I have the ground resistance of our system checked twice each year by a competing utility, not by Arizona Public Service, which supplies our power. That way I know the company doing the testing has no vested interest in the results.”

Copper Tree…

The grounding system Mr. Newcomb describes was based on recommendations made by Lucent Technologies, which had conducted a survey of the building before reconstruction began. Mr. Newcomb accepted most of Lucent’s recommendations, but then he added some features based on his own experience.

The grounding system resembles a tree, with a low resistance electrode as its root. The 20-ft deep, chemical type electrode is located near the center of the subbasement (Figure 2). The electrode is surrounded by bentonite, which retains moisture and helps ensure good earth contact. The electrode’s resistance is checked twice yearly and has remained between 3.0 ohms and 3.5 ohms. Electrodes could not be installed outside the building because it extends to the property line on all sides.

A single length of insulated 750-kcmil tinned copper cable connects the grounding electrode to the office principal ground bus (OPGB), a 1/4-in x 4-in x 48-in (6 .3-mm x 100- mm x 1,219-mm) copper plate mounted on a nearby wall, midway between the building’s north and east service entrances (Figure 3). The OPGB can be thought of as the main trunk of the grounding tree. All grounding circuits in the building terminate at this plate.

Connections to the OPGB are arranged in the PANI pattern (see Sidebar). Two insulated 750-kcmil tinned copper conductors extend from the top center of the OPGB as the building’s two vertical grounding risers. These conductors, which are continuous (not spliced) along their entire length, are the tree’s main “branches.” They extend along the inside of the building’s east and west exterior walls and terminate at master ground bars (MGBs) at the roof level.

A 750-kcmil horizontal riser connects the two vertical risers at the third-floor level. Two MGBs, one from each vertical riser, are installed on each of the building’s three floors and basement level. The grounding bars are mounted on insulated standoffs and are positioned either vertically, horizontally or overhead for easy access (Figure 4).

Other all-copper connections to the OPGB (left to right in Figure 3) include bare AWG #6 to the emergency power system transformer, AWG #4 to the lightning protection system, bare 350 kcmil to building steel, AWG 4/0 conductors to the sprinkler standpipe and water line, (Figure 5), another bare AWG 4/0 to building steel, and a pair of bare 750-kcmil conductors to the building’s north and east electrical vaults. All conductors are tagged for identity at termination points.

Connections between the MGBs and tenants’ EG grounding systems are made via conductors routed along overhead-cable trays to equipment cages, where they terminate at overhead- or wall-mounted grounding bars. Grounding conductors for large tenant areas are also routed along suspended trays. Thus, the DPTE’s “raised floor” is actually overhead (Figure 6).

Lightning Protection

Franklin-type air terminals protect HVAC equipment and an antenna tower located on the roof. A continuous ring of braided copper lightning cable connects the terminals, and an extension of this cable is bonded to the roof level’s MGB. A riser bonded to the MGB connects to the “P” quadrant of the OPGB.

Down-conductors are connected to the rooftop ring at approximately 100-ft (30-m) intervals. These conductors run along the inside of the building’s exterior walls and are bonded to copper-clad ground rods driven near the periphery of the subbasement. In addition, one down-conductor is bonded to the MGB on each floor. A ring of copper lightning cable connects the lightning-protection electrodes, and an AWG 4/0 lead from this ring is bonded to the OPGB. Braided copper lightning cable is preferred over solid conductors for connections between air terminals and earth because the cable’s relatively large surface area facilitates the conduction of high-frequency lightning discharges.

Conductors Sized for Specified Resistance

All of the DPTE’s grounding conductors are sized to maintain resistance below fixed levels, either 0.005 ohm or 0.010 ohm for the length of the conductor run, as specified in the Lucent standard upon which the system is based. For example, there is a maximum resistance (for each leg) of 0.005 ohm between the OPGB and the grounding electrode, between the OPGB and MGBs on individual floors, between MGBs and subsidiary grounding bars at tenants’ equipment and between MGBs and branch-circuit panel boards. Resistance is held to less than 0.010 ohm between the OPGB and building steel, the main incoming water line, backup generator sets, cable entrance ground bars, and lightning protection grounds.

Table 1 lists maximum conductor lengths for common wire gages at the two resistance levels in question. Using the table, the grounding conductor size for each leg of the various circuits can be selected based on the length of the leg and the resistance limit for type of service being protected.

The table refers to resistance only and does not consider high-frequency surge currents, for which appropriate impedance limits should be established.

Good Installation Practice Ensures High Power Quality

Several other features of the DPTE’s grounding system are worth noting:

Grounding conductor floor penetrations are sealed with polyvinyl chloride (PVC) (Figure 7). Metallic materials such as steel conduit should not be used in such locations because of the possibility that a high-magnitude surge current (as from lightning or a large ground fault) might occur along the enclosed grounding conductor inducing large currents, resistive heating and a possible fire hazard if metallic penetration sleeves were used.

 

• When grounding conductors are intentionally routed inside metallic conduit (for shielding, for example), the conduits are bonded to the conductor at both ends. Figure 8 shows an example as installed by a DPTE tenant for its grounding system. Encasing grounding conductors in this manner is an IEEE recommended practice. It avoids increased voltage drop along the conduit during a surge and takes advantage of the lower voltage drop offered by the large-diameter conductor [5].

 

• Bend radii in runs of grounding conductors are no smaller than 1 ft (305 mm) to maintain low inductive reactance along the full length of the conductor (Figure 4).

 

• Grounding conductors are continuous between terminations. While there are bonds to the conductor, there are no splices along its length. Bonds are made exothermically or by means of approved copper- alloy clamps (Figure 4).

 

• Connections to ground bars are made with tinned copper lugs fitted with tamper-resistant (pentagonal headed) bolts. All bars are clearly labeled, and all conductors are tagged with routing and termination information (Figure 9).

Sterling installed a Reliable Power Meters (RPM) Multipoint Power Quality monitoring system to keep track of power quality on a real-time basis (Figure 10). Timm Davis, of Phoenix-based Davis Technological Solutions (dts@cox.net), whose company oversaw the installation, explains: “We have meters on the building’s incoming power service and on each tenant’s service, allowing us to continuously monitor all aspects of power and power quality, especially the harmonics, which can adversely affect other tenants. That way, if a tenant has a problem, we can trace down the source and recommend a solution.”

Inexpensive Insurance

Doug Newcomb believes that the grounding system installed at the DPTE, large as it is, was relatively inexpensive. “The cost of the building and its reconstruction came to around $25 million,” says Newcomb. “About one quarter of that went to the electrical infrastructure, including $260,000 for new transformers.

“New breakers for the four service sections cost between $23,000 and $33,000. The grounding system itself costs about $300,000. That sounds like a lot of money for grounding, but it really isn’t much when you compare it with the total cost of the reconstruction.

“And remember,” says Newcomb, “this is a pretty large building; it covers an entire block and we’re protecting five floors. We used only top-quality, full-sized tinned copper conductors as recommended in the Lucent Technologies specifications, but even so, the copper was a small cost item compared with labor and building prep costs. Incidentally, the Lucent Technologies specs we used at the DPTE are still the standard for the telecommunications industry. Everyone uses them.

“Finally, you have to compare the cost of the system with the cost of the equipment and data it protects. Our tenants come to us expecting reliability, security and flawless power quality. Our grounding system guarantees that they’ll get it.”

Think about that reliability the next time you send an email. Copper makes it happen.

 

References

1. Kawamoto, Kaoru, J.C. Koomey, B. Nordman, R.E. Brown, M.A. Piette, A. Meier, "Electricity Used by Office Equipment and Network Equipment in the U.S.," LBNL 45917, U.S. Dept. of Energy, August 2000.

2. Stahlkoph, Karl E. and J.H. Douglas, "Energy Infrastructure for a Digital Economy," Electric Power Research Institute, Palo Alto, California, 2001.

3. The term, "raised floor" refers to installations in which the signal reference grid is located under sensitive electronic equipment in a subfloor plenum or beneath floor covering. Raised floors are commonly used, but the reference grid or ground plane can just as well be located above the equipment.

4. Dresher, W., "Copper in the New Economy," Proc. Cobre/Copper '03, Santiago, Chile, December 2003.

5. IEEE Std 142-1992 (The Green Book©), §2.2.3 -6. Institute of Electrical and Electronic Engineers, Inc., Piscataway, N.J.

_______________________________________________________

David Brender is National Program Manager for the Copper Development Association (CDA) in New York City. His duties involve directing and managing the electrical programs at CDA, including their Energy Efficiency and Power Quality Initiatives, research activities, National Electrical Code, and IEEE work, among others. He represents the copper industry before such bodies as the Department of Energy and EPA, where he has given testimony, as well as technical and public forums.

Dave has authored several magazine articles on energy efficiency and power quality subjects, and has produced video and CD publications as well. He is a principal member of Panel 5 of the National Electrical Code, and directs the industry participation on three additional panels.

He is a graduate of New York University with a bachelor’s degree in Electrical Engineering, and holds an MBA degree from Fairleigh Dickinson University. He is a member of the IEEE, the International Association of Electrical Inspectors, and the Association of Energy Engineers.

Dave is a licensed Professional Engineer in New York and Pennsylvania, as well as New Jersey, where he resides. He has a certification as a Power Quality Professional from the Association of Energy Engineers.

Doug Newcomb is vice president, Sterling Network Services, at the Downtown Phoenix Telecom Exchange. He was responsible for a major portion of the reconstruction undertaken by Sterling Networks when it converted the 50-year-old Arizona Republic printing plant to a modern Internet hotel. Mr. Newcomb can be reached at dnewcomb@worldnet.att.net.

_______________________________________________________________


PANI CONNECTIONS

Below is a generalized view of the PANI convention for arranging connections to a main grounding board. PANI segregates surge Producers, surge Arresters, and Non-isolated and Isolated networks. Its objective is to minimize impedance to earth in the event of a lightning or fault-induced surge, thus giving maximum protection to safety-grounded, networked equipment. The MGB is constructed from 1/4-in (6.3-mm) copper plate, 6 in (156 mm) wide. Length may vary. A similar sequence of connections is used at all MGBs in the building. (Source: United States Department of Agriculture Rural Utilities Service Bulletin 1751F-810, Electrical Protection of Digital and Lightwave Telecommunications Equipment.)