Blowing hot and cold in the data centre

Many diabolical factors are conspiring to turn data centres into energy-burning sinkholes. Packing power-sucking, heat-blowing servers to maximize efficiency means even more energy has to be expended for cooling. Only about a third of a typical server's power draw is needed to run the machine - the other two-thirds is for cooling. And energy prices continue their relentless rise as data centres scramble to add yet more gear to keep up with computing demand.Managed data centre services providers in particular are feeling the heat of this fiendish equation. Contractually bound for one to three-year terms, they must absorb those unpredictable jolts in energy costs since they can't pass them on to customers. Energy management isn't a fluffy green marketing tactic for them - it's a major factor in business survival. How are they faring in today's erratic energy climate?Between a hot rack and hard spaceCo-location providers of "ping and pipe" - managed space and network connections for customer-supplied computer equipment - are in a quandary. Energy costs have increased about 10% over the past three years, says Frank DiPanfilo, operations manager at MCI Canada, a Toronto-based provider. Since most of their contracts are annual, any energy spikes in the course of the year are borne by MCI, he says.Although there are economic incentives to reduce energy costs, many variables are beyond providers' control. "There's not much we can do," says DiPanfilo, explaining that MCI's guarantee of redundancy is a major selling point with customers. "We always have to have 50% of our equipment on standby in case something goes wrong. If you run an efficient data centre and try to cut back on the cost of hydro, then you must shut something down." He says reducing lighting during off-peak times is mildly helpful, but cutting back on air conditioning to cool the ambient air isn't an option.Major vendors are touting liquid cooling systems that tackle cooling at the rack level as a solution, but DiPanfilo says he's leery. "All the engineers and other people I've talked to in the industry are skeptical," he says. Persuading customers is an even harder sell than industry insiders. "It's technology that might be very effective in the long run - but customers don't want to see water anywhere near their equipment," he says. And this technology isn't cheap, says Nauman Haque, senior research analyst at the London-based Info-Tech Research Group. "It's about $10,000 to $20,000 per rack. There's no payback in the first year, although there will be savings over three to five years," he says.Since customers supply equipment in the co-location scenario, there are few incentives at their end to tackle energy consumption at the source by upgrading to energy-efficient equipment or consolidating their servers. DiPanfilo says customers continue to pack their racks to the brim with ever-increasing numbers of power-hogging servers. "What we receive at our end are servers that need power." Nor are customers asking about MCI's energy management practices, he says. "It's not really their problem - the pain is on our side."He says if he had a choice, he would select a bigger site in an outlying suburban area. "You could spread out the heat more and it's cheaper," he says - but this option has a catch. "Most of our customers are in the downtown core, so they like to be within driving distance from their equipment."Pain managementFully managed services providers are in a different boat compared to co-location providers, says David Woelfle, CTO at Toronto-based EDS Canada. The company supplies most of the equipment and customers pay to host a set of servers or applications, typically in three- to five-year contracts, he explains. Energy costs are also EDS' problem, but the company controls more of the variables in that fiendish equation. "It affects everyone's thinking," he says.EDS has several initiatives underway to tackle the problem. The company is focusing on implementing more energy-efficient gear and improving its standardized virtualization offerings to improve utilization on mid-tier hardware. "We often see 20% utilization in many server farms and it's getting worse," says Woelfle. "That's unsustainable - every couple of years the horsepower doubles, so if we don't change how we use them that will go down further to 10% or 5%." Base demand is growing faster than improvements in computing power per unit of energy consumption, he says. "Physically more servers are required to do clients' work, and that's where virtualization comes in. You can easily push server utilization from 20% to about 60%."EDS is also evaluating liquid cooling systems in the US, but these haven't been introduced in Canada yet. "Water and data centres are not the best combo," Woelfle says, echoing DiPanfilo's sentiments. "The consensus at the moment is that liquid cooling is a last resort. People would rather avoid it because it introduces a lot of complexity they'd rather not have, like worrying about pressure monitoring, pinhole leaks and so on."While customers sometimes ask what EDS is doing about rising energy prices, Woelfle says this isn't as big a concern in Canada as it is in the US. "The bigger worry here is a repeat of the 2003 blackout," he says. "They're more worried about how good your standby power is. They want to see the diesels and pat them on the side to ensure they're working. They're less worried that you've figured out how to squeeze a couple of kWs off energy consumption."Coming attractionsMany data centres are running out of power as they hit their infrastructure limits, says Woelfle. "Ten years ago, we built them at 1.5 kWs per square foot - today it's 15. With new technology we can get around 30 kWs but nobody's built out for that except very new facilities." However, bandwidth is no longer a constraint, so building new facilities in cheaper faraway areas is becoming a viable option, he says.In the US, many companies are moving their data centres to distant Northwestern states where both space and power are cheap, says Bill Courtright, executive director of the Parallel Data Lab at Carnegie-Mellon University in Pittsburgh, PA. The Lab recently set up a data centre observatory to study rising operational costs. One of its findings is that people aren't realistic about capacity planning and forecasting at the design phase, he says. "A common mistake is to design for current energy densities. It's surprising to see how often people say they're not going to need that much power but later on, they do." Although it's easier to lay bigger pipes at the construction phase, it can be hard to justify the costs to build extra capacity that won't be needed for several years, he explains.Another finding is that most machines draw 80% of their peak power even when they're idle, says Courtright. "So you're burning almost the same amount of power during quiet times." But turning sleepy servers off isn't an option in 24/7-type commercial operations, says Woelfle. "When you see servers lightly used, they're doing backup or low-level work but they still don't go to zero." However, he says automation software is being commercialized that stops disks spinning when they're not being read, although this is only appropriate for low-level applications. "It's getting a lot of interest for certain uses, and the attraction is energy savings."Another approach is to look at ways to generate more electricity. At its sunny Sacramento CA and Plano TX sites, Woelfle says EDS is exploring solar cogeneration. "Data centres typically have big flat roofs, so putting up solar panels is easy." In warmer climates, energy consumption for cooling is typically highest when the sun shines, so this is an elegant solution, he says. "It won't get you off the electricity grid, but even if you're generating 20% or 30% of your energy with solar, it becomes interesting. And the economics are becoming more attractive as solar photovoltaics are getting cheaper." In colder climates, geothermal cogeneration is also being explored at sites with the right geological features, he says. "That works if you've got facilities with the right real estate to do the set-up and drilling, and we're looking at a couple of those."Since many things must be done in concert to reduce energy consumption, there's interest from many quarters in assembling a set of best practices and standards for a green data centre certification, says Courtright. "I've talked to a lot of people who want to work on that standard." Woelfle says EDS is eyeing developments in this area, which is still in the very early stages.Energy management overall is an emerging trend, says Info-Tech's Haque. "I'd say all energy efficiency practices are under-utilized as this whole area is very new," he says. "People are just starting to realize how much it's costing them. A 30-kW rack in a large data centre probably costs around $30,000 to $40,000 annually in energy costs today, but three years ago, the average energy consumption was 5 to 10 kWs so it wasn't much of a concern."Haque points out that energy efficiency is being driven from the top down in the US by the government, which is concerned to tackle the $33 billion spent by organizations annually on energy. "The US Senate and House of Representatives have passed bills promoting data centre efficiency. The Environmental Protection Agency is on the hook to develop standards for equipment, and a working group is looking at data centre certification." In typical fashion, these initiatives and standards will eventually trickle into Canada, he says.

Blowing hot and cold in the data centre

News | April 17, 2007

Unlock all the Canadian telecom, broadcasting and digital media news you need.

FREE TRIAL

SUBSCRIBE

Inside The Wire Report

Information

Hill Times Publishing

Get in Touch With Us