Thinking outside the case: Running naked servers
When it comes to data center metrics the one most often talked about is square footage. Nobody ever announces that they’ve built a facility with Y-tons of cooling, or Z-Megawatts. The first metric quoted is X-square feet. Talk to any data center manager however and they’ll tell you that floor space is completely irrelevant these days. It only matters to the real estate people. All that matters to the rest of us is power and cooling - Watts per square foot. How much space you have available is nowhere near as important as what you can actually do with it.
If you look at your data center with a fresh eye, where is the waste really happening?
Since liquid-cooled servers are at the far right-hand side of the bell curve, achieving electrical density for the majority of us is usually a matter of effectively moving air. So what is REALLY preventing the air from moving in your data center? I won’t rehash the raised floor vs. solid floor debate (since we all know that solid floors are better) but even I know that the perforated tiles, or the overhead duct work is not the REAL constraint. A lot of folks have focused a lot of energy on containment; hot aisle containment systems, cold aisle containment systems, and even in-row supplemental cooling systems.
In reality however, all of these solutions are addressing the environment around the servers, not the servers themselves which are after all, the source of all the heat. Why attack symptoms? Let’s go after the problem directly: The server.
First of all, the whole concept of a “rack unit” needs to be discarded. I’ve ranted before on the absurdity of 1U servers, and how they actually decrease data center density when deployed as they are currently built. I’d like to take this a step further and just get rid of the whole idea of a server case. Wrapping a computer in a steel and plastic box, a constrained space, a bottleneck for efficient airflow is a patently absurd thing. It was a good idea in the day of 66 Mhz CPUs and hard drives that were bigger than your head, but in today’s reality of multi-core power hogs burning like magnesium flares it is just asking for trouble. Trouble is what we’ve got right now. Trouble in the form of hot little boxes, be they 1U or blade servers. They are just too much heat in too constrained spaces.
Virtualization won’t solve this problem. If anything it will just make it worse by increasing the efficiency of the individual CPUs making them run hotter more of the time. Virtualization might lower the power bills of the users inside the server, but it won’t really change anything for the facility that surrounds the servers in question. The watts per square foot impact won’t be as big as we hoped and we’ll still be faced with cooling a hot box within a constrained space.
So here is my challenge to the server manufactures: Think outside of the case.
This isn’t a new idea really, nor is it mine. We’ve all seen how Google has abandoned cases for their servers. Conventional wisdom says that only a monolithic deployment such as a Google data center can really make use of this innovation. Baloney. How often does anyone deploy single servers anymore? Hardly ever. If server manufacturers would think outside of the case, they could design and sell servers in 10 or 20 rack unit scale enclosures. They could even sell entire racks. By shedding cases altogether, both server cases and blade chassis, they could create dense, electrically simple, easy to maintain, and most importantly easy to cool servers. The front could be made of I/O ports, fans, and drives. Big fans for quiet efficiency. The backs could be left open, with electrical down one side and network connections down the other. Minimize the case itself to as little as possible… think of Colin Chapman’s famous directive about building a better race car: “Just add lightness.” The case of a server should serve one purpose only: To anchor it to the rack. Everything else is a superfluous obstruction of airflow. No need for steel, as plenty of lighter weight materials exist that can do the job with less mass.
Go look in your data center with this new eye and envision all those server cases and chassis removed. No more artificial restriction of airflow. Your racks also weigh less than half of what they do today. You could pack twice the computing horsepower into the same amount of space and cool it more effectively than what you have installed.
Ten years from now we’ll look back at servers of this era and ask ourselves “what were we thinking??” The case as we know it will vanish from the data center, much like the horse and buggy a century before. We’ll be so much better without them.
Posted: July 1st, 2008 under Data center physical infrastructure, Data center power efficiency, Data center cooling, Blade servers, Server virtualization in the data center, Green data center.
Yeah, and maybe one day in the very near future and say WOW, I could (and should) have an IBM z series mainframe!!!
Comment by LJ — July 3, 2008 @ 10:11 am
Very thought provoking idea!
A few things to consider would be shipping these naked beasts (components bolted in a case may resist UPS treatment than a collection of parts, mabybe not), and air quality: with all the hardware exposed, you’d want to beef up air purification systems.
Nice article!
LKG
Comment by Laurent Gharda — July 3, 2008 @ 10:30 am
Spoken as a true “orange”!!
My worry would be a need to increase the chill factor in the data centre. I get cold enough already working on servers….
Comment by charms — July 3, 2008 @ 11:33 am
Just a couple of questions:
- How is EMI/EMC shielding accomplished?
- How do you protect exposed PCBs from damage (physical or ESD)?
- How is coooling air effectively channeled past the first obstruction it hits?
Comment by Jim — July 3, 2008 @ 3:48 pm
You’re CONFUSED. Lack of information has left a hole in your “head”. Egenera and F7 have been doing this for years. I am assuming you want a CHEAP solution. This is where innovative resellers can offer some value-add to the market.
Comment by ADK — July 4, 2008 @ 1:29 am
I dont’t understand. You talk of taking a fresh eye but you are being blind to the fact that in the third paragraph, you nixed the most efficient form of cooling servers: “WATER”.
Water is not out on the bell curve. It is here and about to come on strong. It is a proven solution, so why can’t everyone get their heads out of the sand and start demanding it from the Server vendors.
As an old mainfrane guy I remember sending water to the mainframes and I remember that even when we lost the chillers the mainframe kept chugging along with the 1/4hp (UPS driven) Taco pumps under the raised floor just circulating the water in the piping system. Point in fact we had enough water in the pipes to cool the mainframes for 15-20 minutes to coincide with our battery runtime. What data center has that much cool air?
As soon as the Server vendors realize that they can save money by shipping Servers naked, you’ll be paying extra for the enclosures.
As a cooling solution it may help but it is not enough to get us where we need to be with cooling efficiencies.
We need water, with reliable distribution and connectors. All the components are out there, now it is up to the user community to get on-board and make it mainstream (no pun intended).
Comment by Dennis Cronin — July 6, 2008 @ 6:29 pm
So how do you ensure proper airflow through the heatsink’s of the CPU’s if the air is not directed/forced through it the chassis is now an open archtecture?
Are we to apply fans to the top of the CPU’s taking additional space, or place water cooled heat sinks on the CPU’s?
Since the point is to cool the CPU, please explain how this is to be done in an architecture that does not direct or control air flow without giving up spcae or adding liquid elements into your servers.
Comment by JJ — July 9, 2008 @ 8:17 am
The first thought that came to my mind as i was reading this was where did I put that article about someone running a few THOUSAND Linux partition on an IBM z/Series? I’m with LJ run your apps on a mainframe. I was at a client site with one of our Linux guys who had never seen a mainframe and blindly walked him past a server rack then behind 2 z890’s. After we passed the z box I said a mainframe only puts out as much heat as 2 or 3 servers. After he said I was all wet I took him to the front of the z890 then his jaw dropped.
Comment by MarkC — July 10, 2008 @ 4:10 pm