There are plenty of ways to start measuring your data center’s Power Usage Effectiveness, or PUE, which is a comparison of the total facility power to the IT load. But in recent months Microsoft has opened the curtains on its own operations, thanks largely in part to two data center pros there, Michael Manos and Christian Belady.

Back in April, Manos spoke to us about Microsoft’s data center strategy. It was one of the first times the Redmond, Wash. The next month, Manos showed us a demo of Scry (video here), the tool the company uses to measure everything in their data centers.

Now Manos and Belady have written a series of three lengthy blog posts on PUE, why it’s important, and how to go about measuring it. For those of you out there still a little daunted by the task of measuring key analytics in your data center facilities, this could give you a good start.

With the upcoming holiday sure to brighten the skies across the U.S. with colorful flashes of light, I thought it would be a good time to share some of the advancements in Flash-memory based storage devices. Solid-state drives (SSD) can help bridge the server and storage performance gap and decrease data center power requirements. A few weeks ago, at Hewlett-Packard Co.’s Technology Forum, a relatively new entrant into the field, Fusion io announced that is adapting Fusion-io’s ioMemory architecture to HP’s enterprise-class servers, including the HP BladeSystem c-Class system.

“Adapting this technology specifically for HP servers offers radical increases in associated performance for a broad range of applications and workloads and can dramatically improve the effectiveness of data center architectures,” said Fusion-io CEO Don Basile. “With our ioMemory architecture, we’re getting more than 200,000 IOPS [I/O operations per second] within HP BladeSystem c-Class server blades today.”

When I met with Basile for a mere 30 minutes, he shared an intense amount of information, including a variety of statistics that illustrate the power of the technology. The ioMemory technology is still costly enough that it’s not ideal for long-term archival storage, but the speed that it provides makes it valuable in the active data requirement area. Basile explained that the technology is capable of 3.2 GB/s of sustained bandwidth, with extremely low latency (50 microseconds). In terms of I/O, a traditional enterprise application server that is I/O-intensive may cost $15 to $20 per I/O versus $0.15 per I/O using the Fusion-io system.

In addition, Basile explained that all this speed is possible with less energy expenditure and less heat than a typical storage area network (SAN). Heat and power are reduced because of the lack of mechanical heat that would otherwise be generated by spinning disks. In addition to using less than 1% of the power required by a typical SAN, the footprint is minimal: 16 ioMemory cards can fit into 10U without any oversubscription. Basile noted that consumer electronics such as the iPhone have helped lower the cost of silicone chips.

HP is not alone in integrating SSD technology. Earlier this year EMC announced that it had added SSD in its enterprise Symmetrix system, and more recently rumors have circulated that the company will add it to the Clariion storage array. Sun has also announced that it will release a new version of the Solaris operating system designed to integrate flash and traditional disk-drive storage. And Sun’s CEO Jonathan Schwartz says that it’s not just a flash in the pan. Sun’s Adam Leventhal has also produced an informative technical article outlining the technology and its optimized uses.

In the age of increasing energy costs, efficiencies in data centers are welcome, and the Flash technology is likely to play a key role in the data center of the future. If you’re interested in the topic, check out the Flash Memory Summit in August.

When it comes to data center metrics the one most often talked about is square footage. Nobody ever announces that they’ve built a facility with Y-tons of cooling, or Z-Megawatts. The first metric quoted is X-square feet. Talk to any data center manager however and they’ll tell you that floor space is completely irrelevant these days. It only matters to the real estate people. All that matters to the rest of us is power and cooling - Watts per square foot. How much space you have available is nowhere near as important as what you can actually do with it.

If you look at your data center with a fresh eye, where is the waste really happening?

Since liquid-cooled servers are at the far right-hand side of the bell curve, achieving electrical density for the majority of us is usually a matter of effectively moving air. So what is REALLY preventing the air from moving in your data center? I won’t rehash the raised floor vs. solid floor debate (since we all know that solid floors are better) but even I know that the perforated tiles, or the overhead duct work is not the REAL constraint. A lot of folks have focused a lot of energy on containment; hot aisle containment systems, cold aisle containment systems, and even in-row supplemental cooling systems.

In reality however, all of these solutions are addressing the environment around the servers, not the servers themselves which are after all, the source of all the heat. Why attack symptoms? Let’s go after the problem directly: The server.

First of all, the whole concept of a “rack unit” needs to be discarded. I’ve ranted before on the absurdity of 1U servers, and how they actually decrease data center density when deployed as they are currently built. I’d like to take this a step further and just get rid of the whole idea of a server case. Wrapping a computer in a steel and plastic box, a constrained space, a bottleneck for efficient airflow is a patently absurd thing. It was a good idea in the day of 66 Mhz CPUs and hard drives that were bigger than your head, but in today’s reality of multi-core power hogs burning like magnesium flares it is just asking for trouble. Trouble is what we’ve got right now. Trouble in the form of hot little boxes, be they 1U or blade servers. They are just too much heat in too constrained spaces.

Virtualization won’t solve this problem. If anything it will just make it worse by increasing the efficiency of the individual CPUs making them run hotter more of the time. Virtualization might lower the power bills of the users inside the server, but it won’t really change anything for the facility that surrounds the servers in question. The watts per square foot impact won’t be as big as we hoped and we’ll still be faced with cooling a hot box within a constrained space.

So here is my challenge to the server manufactures: Think outside of the case.

This isn’t a new idea really, nor is it mine. We’ve all seen how Google has abandoned cases for their servers. Conventional wisdom says that only a monolithic deployment such as a Google data center can really make use of this innovation. Baloney. How often does anyone deploy single servers anymore? Hardly ever. If server manufacturers would think outside of the case, they could design and sell servers in 10 or 20 rack unit scale enclosures. They could even sell entire racks. By shedding cases altogether, both server cases and blade chassis, they could create dense, electrically simple, easy to maintain, and most importantly easy to cool servers. The front could be made of I/O ports, fans, and drives. Big fans for quiet efficiency. The backs could be left open, with electrical down one side and network connections down the other. Minimize the case itself to as little as possible… think of Colin Chapman’s famous directive about building a better race car: “Just add lightness.” The case of a server should serve one purpose only: To anchor it to the rack. Everything else is a superfluous obstruction of airflow. No need for steel, as plenty of lighter weight materials exist that can do the job with less mass.

Go look in your data center with this new eye and envision all those server cases and chassis removed. No more artificial restriction of airflow. Your racks also weigh less than half of what they do today. You could pack twice the computing horsepower into the same amount of space and cool it more effectively than what you have installed.

Ten years from now we’ll look back at servers of this era and ask ourselves “what were we thinking??” The case as we know it will vanish from the data center, much like the horse and buggy a century before. We’ll be so much better without them.

Last week I attended a data center tour with Sun Microsystems’ Dean Nelson, Sr. Director of Global Data Center Design Services.

Nelson’s team consolidated four large Sun Microsystems campuses in California – consolidating over 200,000 square feet of data center down to 80,000 square feet, while still allowing capacity to grow.

This new data center, built into Sun’s existing office space in Santa Clara, is designed for modular growth and is very energy efficient (1.28 PUE in one of the rooms, according to Nelson)

In this first video, Nelson demonstrates the Starline Busway, a power component Nelson Describes as track lighting on steroids. These products allow Sun data center staffers to plug in anything from 120 single phase power to 100 amp 3 phase without an electrician, allowing for modular growth and flexibility. The busways also have IP connections that track power usage in real time.

In this second video, Nelson shows us the hot-aisle cold-aisle containment strategy Sun is using. It’s essentially a ceiling on the hot aisle that prevents hot return air from mixing with the cool intake air.

Data center power and cooling services companyAPC is offering a number of free online tools, called APC TradeOff Tools, that give IT pros a way to view how infrastructure changes effect costs and performance in the data center.

“The tools answer questions like, ‘what will the ROI be if I increase the water chiller temperatures?’ or ‘what do I need to do to hit my energy efficiency or carbon footprint target’?,” said Neil Rasmussen, senior vice president of innovation for APC.

The West Kingston, RI-based company’s new tools include the power efficiency calculator that allows IT managers to generate “what if” scenarios regarding virtualization, power sizing, efficiency, power density, and cooling decisions.

Another new tool is the Data Center Carbon Calculator, which allows users to input data about their infrastructure and see the impact any changes would have on data center efficiency, energy costs and carbon footprint.

“If a company makes a carbon efficiency pledge, they can use this tool to drill down into different ways to achieve that goal,” Rasmussen said.

The Data Center Capital Cost Calculator details the impact of physical infrastructure design changes on capital costs; the Virtualization Energy Cost Calculator shows the impact of server virtualization and data center design choices on energy and space savings; and the Data Center Power Sizing Calculator gives details about the impact of server and storage configurations on IT load capacity and required utility input power.

There is also the Data Center AC vs. DC Calculator, which compares the efficiency of each, and the Data Center InRow Containment Selector, which recommends cooling options based on the data center infrastructure.

All of the tools can be accessed here on APC’s website.

The Environmental Protection Agency program has been trying to get data centers to commit to submitting it data so that it can develop an Energy Star rating for data centers. Its original deadline for data centers to submit a form of interest was June 1, but the agency has now extended it for one more month in the hope of getting more companies to bite.

Andrew Fanara of the EPA’s Energy Star program spoke last month about the National Data Center Energy Efficiency Information Program.

Late last month, I spoke to Fanara at a different event in Chicago about how the data collection was going. He said at the time that it had been slow, although there were a few users at the event who expressed interest in submitting data.

The basic idea is that data centers commit to monitoring, measuring and submitting information about their data center’s energy use to the EPA on a regular basis. All of the information is kept confidential, although the EPA will let each individual company know how they did. The EPA will then collect that data and use it to help create an Energy Star rating for data centers.

Earlier this week I got in touch with Fanara to see how the data collection was going, especially considering the deadline extension. He seemed more encouraged, saying that about 200 data centers had signed on thus far.

That’s much better than at the Chicago event, where Fanara said that “response from the industry has been extremely slow.”

Hopefully in another month they’ll get even more users to commit to submitting data. If you’re interested, visit the Energy Star’s data center site.

Folsom, Calif.-based data center monitoring company Synapsense is looking to targeting computational fluid dynamics (CFD) software with its wireless data center monitoring system. The startup is two years old and has been shipping commercial products for six months. The company currently has 15 proof of concept projects at unnamed “Fortune 50” companies, plus support from the data center energy efficiency engineers at Lawrence Berkley National Labs (LBNL), according to Ray Pfeifer, vice president of business development at Synapsense.

Synapsense’s battery powered monitors track data center environmental conditions and use low power wireless [2.4 gigahertz] to communicate that data to a server. Synapsense’s software synthesizes that information and displays it as a live image [example below], which allows data center managers to look at real time maps of their data center and view air pressure distribution, humidity and temperature.

The following is an excerpt of a Q&A with Pfeifer:

Why wireless sensors? Why not wired?
Ray Pfeifer: The majority of the data centers are not new. The IT equipment refreshes every 3-5 years. The facility is there 15-20 years. Low power wireless becomes the first practical way to get in to monitor legacy data centers unobtrusively.

We can deploy in a 10,000 sq ft data center in a day to two days. It takes you weeks or months to do that with a wired solution. Because of the flexibility of wireless, as your data center changes, racks come and go, you can very easily reconfigure the wireless to match the environment. Some data center operators are moving 10-20% of their IT equipment in and out every month.

What specifically do the sensors monitor?
Pfeifer: Our standard installation measures the temperature on inlet and discharge of the racks, the temperature at the inlet and discharge of the Computer Room Air Coniditoner (CRAC) units, humidity at the CRAC units, and sub-floor air pressure.

But what we do with all this data is the interesting thing. A typical data center supplies anywhere two to three times the air that it needs because the majority of the air is being wasted. The strategic placement of the sensors allows you to adjust your airflow methodology. When you raise the return temps on the CRAH units, they become more efficient, you can shut units off.

The LiveImaging gives you a visual map: you can see where your hot spots are, you can see where you’re over-cooling. It enables a data center operator to reconfigure the data center and understand what they need to do to reconfigure it.

How does real-time imaging stack up against CFD analysis?
Pfeifer: There is some great CFD software out there, but the expertise to build a good model is pretty significant, so the people using CFD models are generally professional services folks. A full data center assessment is a $50,000 event for a 25,000 sq ft data center. The problem is, that’s a one time snapshot. They take your readings and they leave. In 90% of those instances, the facility guy looks at it, and it goes on a shelf. A year later, they hire someone else to do another one. If you put sensors cost effectively, do the analysis, collect data real time, you can present it visually to non-eningeering staff. Look at color. If it’s red, it’s hot. This is a dynamic tool that allows continuous commissioning the data center.

What’s the next step for Synapsense?
Pfeifer: The next piece that will complete the full solution will be energy metering, for both infrastructure and IT equipment, down to branch circuit level monitoring. We will also provide real-time DCIE/PUE and overlay it on your existing data center. We’re also working with the LBNL team as they develop DC Pro, the Department of Energy’s data center assessment tool. We’re putting those assessment tools into our software solution.

Beltway data center managers, listen up! Data Center Decisions is hosting a data center energy efficiency seminar at the Capital Hilton, Washington, DC.

According to Public Law 109-431, the Environmental Protection Agency has been tasked with studying data center energy efficiency, and as part of that law, governmental agencies are required to show leadership through implementation of best practices for their own facilities.

This one-day event is a free opportunity for data center pros in the public sector to get in front of this issue before it becomes a regulatory requirement. The speakers for this event include Ken Brill, Founder and Executive Director of The Uptime Institute, and Joe Clabby, President of Clabby Analytics. The speakers will address best practices for energy efficiency across both the IT and Facilities aspects of the data center. Click here for online registration.

Yesterday I went to an Aperture-sponsored event in downtown Chicago that Andrew Fanara from the federal Environmental Protection Agency spoke at. Much of it was information that he has spoken about before and that we’ve reported, all of it around the data center energy efficiency issue that the EPA has gotten more involved with in the past couple years.

A major focus of the event was measurement. Leaders in the industry say that data centers must learn to measure how much power they’re consuming in order to reduce it. Then they can have before-and-after accounts of their Power Usage Effectiveness number, which is an efficiency metric dividing your total facility load by the IT load.

Your PUE number is like golf — the closer to 1, the better. At least that has always been the common wisdom. The goal, says experts, is to reduce your PUE. But sometimes an IT energy efficiency project can play games with that number.

Steve Yellen from Aperture said a virtualization project can temporarily hurt your PUE number. Take this example: You have 10 megawatts coming into a facility, and 5 of them are taken up by the IT load. You virtualize and consolidate servers, thereby reducing your server footprint, and thereby reducing your IT load. So now your IT load is only 4 megawatts even though your facility load is still 10 megawatts. So your PUE would go from 2 to 2.5.

Presumably there would be an adjustment. You would see that the IT load had decreased, and so you would adjust your facility load accordingly. According to Yellen, everything would be hunky dory again, right? Wrong. Your PUE would still take a hit. Let’s take the same example:

• Your facility load is 10 megawatts and your IT load is 5 megawatts, so your PUE is 2.
• You virtualize and consolidate so that your IT load becomes 4 megawatts, a one-megawatt reduction. Your PUE is now 2.5. Uh-oh.
• So you adjust, reducing your facility load by one megawatt to match with the IT load reduction. So now your facility load is 9 megawatts while your IT load is 4 megawatts. Your PUE is now 2.25, which is still worse than the PUE of 2 you had before you virtualized and consolidated. Still uh-oh.

In fact, the more energy you save with your virtualization/consolidation project, the worse it could be for your PUE. Say your project reduced your IT load by two megawatts instead of one. So you reduce your facility load by two megawatts as well. That means the facility load is 8 megawatts and the IT load is 3 megawatts, yielding a PUE of 2.67. Uh-oh.

Taking it a step further, any project that improves your IT load alone will yield a worse PUE. If you buy those new super-duper efficient servers, that could make your PUE worse. If you install blanking panels and move perf tiles around the right way, that will improve your PUE.

Let’s not panic here, because there is a good side to this. If I consolidate servers, I have fewer servers to cool. That presumably means that I’ll be able to reduce my facility load further because I might be able to shut down one of the cooling units. And maybe fewer servers means I can get rid of one of my uninterruptible power systems (UPS) units. In the end, it might all even out, but it may just leave you with a zero-sum game instead of an improved PUE number, which is what you think it would do.

In the end, what’s most important is reducing your overall power load, and if you can document how it all happened, all the better.

 Trends in data center power was the topic at AFCOM New England Chapter’s meeting this week, and apparently it’s a subject that resonates with members—at least judging by the nearly 100 attendees who showed up. (As the New England chapter enters its third year, President Rocko Graziano, whose real job is manager of infrastructure operations and services at L.L. Bean, said this was the largest meeting yet). Two speakers gave the audience their take on the some emerging trends they see taking shape.

Rudy Kraus, CEO of Validus DC Systems, a provider of direct current (DC) power infrastructure for data centers and telecommunications facilities, naturally sees a bright future for data centers powered by DC rather than AC-based electricity. Kraus cited a number of statistics from the likes of the Uptime Institute and McKinsey outlining just how much power data centers can save by switching to more efficient DC power. If data centers in the United States converted only 10% of their capacity to DC power, that would eliminate $1 billion in electric bills. The co2 emissions for a 10 megawatt data center with 17,500 servers would drop from 99,776,400 pounds to 59,865,840 pounds. Kraus invited members of the audience to do their own comparison by visiting an online calculator offered by Intel that analyzes facility-level efficiency of AC and DC servers.

The other speaker, Brian Ouellette, of J.S. Fleming Associates, a provider of power and cooling systems, spoke about the five power trends heading to a data center near you. The top trend, that energy efficiency is gaining importance, is pretty self-evident. The other four trends centered on ways to make data centers more efficient: New ways to scale UPS architectures into adaptive models that can adjust to changing power requirements; two-stage power distribution that reduces restrictions to cooling air flow, among other benefits; increasing use of monitoring with tools such as smart power strips (that monitor in-rack power) and branch circuit monitoring (that monitor each PDU output circuit). Ouellette also pointed out that data centers don’t have to go high-tech in order to become more efficient. When Ouellette asked the audience whether they use blanking panels in their data centers, only five people raised their hands. “Blanking panels are a great way to get the air where you need it,” he said. “Otherwise, you’ll get cross-contamination of air from your hot aisle and cold aisle.”