Water-Side Economizers for Data Centers

When the National Center for Atmospheric Research (NCAR) decided to build a new supercomputing center, it wanted to take advantage of the free cooling available by the high, dry Wyoming environment.

The NCAR Wyoming Supercomputing Center (NWSC) opened in 2021 — a 170,000-square-foot data center with two 12,000-square-foot raised-floor spaces. The site houses the IBM Yellowstone 1.6 Petaflop supercomputer in one of those modules, with the other set aside for future expansion.

“We have pursued ‘right-sized’ construction in order to limit capital construction costs while ensuring that we have the needed infrastructure to meet scientific community computing needs now and for the next 20 to 30 years,” said Krista Laursen, NWSC project director. “We have built infrastructure that will support computing and data archival needs for at least the next decade, and, should facility expansion be necessary down the road, it can be accomplished in a straightforward fashion.”

In addition to the computing space, the NWSC has 6,000 square feet of data storage and archival space, a 2,500-square-foot network operations center, administrative offices, a visitor center, and 100,000 square feet of space for the mechanical and electrical equipment.

Installation Challenges/Specifications

NCAR chose the Wyoming site, in part, because the climate would allow it to maximize the use of outside air and minimize the use of chillers.

“The high and dry climate in Wyoming is ideally suited for the free cooling design used in the NWSC,” Laursen said. “Such a design option might not make sense in other geographic regions.”

Average summer afternoon temperatures in Cheyenne only reach the lower 80s, dropping into the 50s at night. From October to April, nighttime lows drop below freezing. With such a wide temperature range, all of the air handlers are designed to use 100% outside air economization when appropriate. Because NCAR doesn’t want to take the chance of temperature excursions on its high-end equipment, the data center also includes two evaporative towers sending chilled water to heat exchangers at the fan wall. To ensure the inlet temperature stays at the desired 75°F, the NWSC also has one 1,000-ton chiller and two 100-ton chillers to further cool the water as needed, but it is not anticipated that these will be utilized more than a few hundred hours per year.

Using outside air in a dry climate, however, risks electrostatic discharge damage to the equipment. In Cheyenne, the relative humidity drops below 40% on summer afternoons, outside the ASHRAE recommended operating range. The bigger issue is on the winter days. Although the relative humidity tends to stay above 50% at the outside temperatures, it plummets when heating the air up in the data center: That missing moisture needs to be added to the incoming air to prevent electrostatic damage to the equipment.

The Installation

Mark Labac, president of Edge Mechanical Systems, was the Mee representative responsible for the NCAR humidification system. The MeeFog system consisted of two FM-200-B271P ceramic plunger pumps producing 2 gpm at 1,000 psi. The pumps send the water through stainless steel tubing to arrays of impaction pin nozzles in two of the air-handling units (AHUs) and in the air supply for one of the data rooms.

“The MeeFog system has some very thorough water treatment systems upstream so that the water is very clean by the time we are discharging it,” said Terry Autry, mechanical engineer and senior vice president for RMH Group, the engineering and design firm that oversaw the construction and commissioning of the NWSC’s mechanical and electrical systems. “The network of distribution nozzles discharge that fog into the return air path up high, then, as that air continues down into the fan walls on the lower level, that fog  is absorbed into the airstream.”

The MeeFog system is tied into the Johnson Metasys control system that runs the AHUs, ensuring both the temperature and humidity stay in the appropriate ranges no matter the outside temperature and humidity.

“Every AHU is set up for full outside air economization,” Autry said. “The offices, the UPS and battery rooms, the computing spaces, everything has that capability.”

Cost/Benefit Analysis

Assuming a cost of 0.065 cents per kWh for electricity, 3,500 hours of operation per year, and 1,000 pounds of moisture output per hour, a MeeFog system costs $700 annually. In addition, the MeeFog systems cool the air as they humidify it.

“We do get some beneficial cooling from the MeeFog humidification so the other cooling systems have to work that much less,” Autry said.

But, he doesn’t include the energy savings from the cooling in any efficiency calculations. Instead, he said the real value just comes from preventing damage to the equipment. With millions of dollars of equipment at stake, that value overshadows any other.

“Wyoming is very dry, so you need to avoid any static discharge situations,” Autry said. “The MeeFog system is there to avoid reaching too-low humidity in the data center.”