WestGrid: Why it's important and what's unique


Architect's design: Ray Wan Architects

posted: 3 February 2010      

What is WestGrid?

Why is HPC important?

Computing is considered by many to be a new "pillar" of science, and is now used in virtually every scientific discipline.  High performance computing (HPC) can increase computing capacity by a factor of 1000 or more, and allows scientists to build virtual, experimental models that might otherwise be: too expensive, unsafe or physically too large to consider.

What do we mean by too expensive, unsafe or physically too large?

Too expensive:  The Canadian government has recently invested heavily in three research facilities: the uVic-led NEPTUNE, uSask's Canadian Light Source and the Sudbury-based SNOLAB.  Current estimates suggest it will cost $40 million annually to keep these three facilities running, and project leaders and universities can be left scrambling to secure funding.  In cases where experiments can be run in a virtual environment, special facilities don't have to be constructed - saving on research costs.  There will always be a need for highly-specialized, technical, experimental facilities, but in some cases, HPC can be a much less expensive option. 

Unsafe:  Experiments that involve radioactive material and biological systems may be unsafe to conduct in real life, but sometimes, they can be simulated virtually.  The US government no longer carries out nuclear bomb tests; instead, they simulate them using an HPC virtual environment.

Too large:  Scientists, like astrophysicists, study the universe, but we can't build a universe to study it.  HPC provides for a means to address all three of these research challenges, thereby enabling research that might not otherwise be possible.  However, a universe can be modeled in a virtual environment.

 

THE WESTGRID BUILDING

Bringing HPC equipment to the University of Manitoba is not a simple, straight-forward process.  These computers require special facilities, and at other sites, the retrofits alone to existing structures have cost over $4 million. This puts the $2.6 million for the new UM facility into perspective. 

What are the requirements?

Load, Power and Cooling:  The HPC equipment is very heavy; (a 19" rack holding equipment may weigh as much as 2,800 lbs), and it needs a large power source. Additionally, as computer equipment consumes power, it produces heat, so significant cooling capability is also needed.  One option, locating the computers in the EITC would have cost $2.8 million to retrofit for the power alone.  A second option, the basement of Machray Hall, was ruled out because the Red River can pose flood threats in the spring; consequently, housing $4 million dollars of computing equipment in a basement was not an option!  In the end, the University was able to secure funding (provided by CFI, MRIF and WED) to build a new facility for $2.6 million, in a location with an available power source and infrastructure that will permit a unique "green" solution to cooling.

Why is the building special?

The building is unique in several ways. First, it has been built to handle a floor load of 500 lb/sq ft., and it is anticipated to have the necessary load-bearing capacity to handle the University's HPC needs for 25-30 years.  There is enough space in the new facility to bring in new equipment as well as to facilitate the convenient replacement of old equipment as it becomes obsolete. One of the building's most unique features is enabled by its close proximity to the hot and cold water supplies used for building heating and cooling.  In a novel "green" cooling design, the heat generated by the HPC equipment will be captured during the winter months and used to heat buildings downstream from the new facility.  It is estimated that the HPC equipment itself will generate heat equivalent to 200 high-end barbeques running constantly on full power. In Winnipeg's climate, there will be ample opportunity to utilize this heating source.

Construction Challenges

Nothing new is ever simple, and the construction of facility was a case in point. To support the weight of equipment in the building, many piles were needed, and the installation of  the pilings presented a unique challenge.  Normally, pilings are brought in and pounded into place using cranes.  However, the adjacent building, Buller, houses a new Protein X-ray diffractometer, and the vibration that would have been caused by the pounding of pilings threatened this brand new, very expensive, piece of equipment.  As a result, holes were drilled, sonotubes were sunk, and all the pilings were hand-poured. 

When completed, the building will be one of the best-outfitted facilities in the WestGrid system - complete with room for future growth.  As of January 2010, construction is essentially complete, the new equipment is expected to arrive in late May, and the target date for WestGrid to power up at the U of M is the end of June.