#129 HPC OnDemand
Subscribe to get the latest
on 2023-03-21 00:00:00 +0000
with Darren W Pulsipher, Alan Chalker,
In this episode Darren interviews Alan Chalker, director of strategic program at Ohio Super Computer Center about Open OnDemand for HPC clusters worldwide.
Keywords
#datamanagement #hpc #data #technology
In this podcast episode, Darren Pulsipher, the chief solution architect of the public sector at Intel, interviews Alan Chalker from the Ohio Supercomputer Center about breaking down barriers to high-performance computing (HPC). Alan is the director of strategic programs at Ohio Supercomputer and has been working on an NSF-funded project called Open OnDemand for over a decade. The project aims to make HPC more accessible to general consumers who are used to doing things online, like online banking and shopping. Open OnDemand simplifies the process of using HPC by eliminating the need for command-line inputs. Alan’s background includes getting his undergraduate degree in electrical and computer engineering at Ohio State University and then earning his doctorate in biomedical engineering from USC Chapel Hill.
History of Open OnDemand
In 2006 & 2007, a web interface was developed with Edison Welding Institute in collaboration with some techies, who later named it Open OnDemand. It started as an online welding simulation and expanded to include a polymer and general-purpose one. Upon showcasing it at various conferences, other research computing institutions expressed interest in deploying it on their systems. To make it open source, the National Science Foundation awarded them a three-year, $300,000 program that made the prototype more robust. The success of Open OnDemand led to another five-year program worth $3 million. Today, it is deployed on every continent except Antarctica, serving over 400 research computing institutions.
Expanding Influence of HPC
Supercomputers have expanded beyond traditional fields like computer science and engineering. At OSC, anthropology and political science students are using the supercomputer for their research, as well as students in horticulture and crop science courses. The demand for the supercomputer is increasing, with over 8,500 individuals using OSC’s supercomputers from all over the world last fiscal year. Additionally, during the pandemic, many universities could continue teaching and researching remotely through virtual desktops provided by the supercomputer.
Comparing CSP and HPC Pricing Models
Pricing models for the supercomputer are based on core hours and terabyte months, and the government mandate allows for subsidized pricing for Ohio-based academic entities. Cloud service providers charge by the wall clock time, not core hours, and order by data storage and egress network costs. Commercial industry clients are starting to utilize supercomputers for traditional HPC simulation workloads saving money from running them in the retail public clouds.
Examples of Commercial use
The day before Darren and Alan sat down and talked, there were tornadoes in the Columbus, Ohio, area. Weather prediction is significant to many industries, and supercomputers are well-suited. The center generates weather forecasts every 4 to 6 hours for clients such as shipping companies and airlines. While traditional high-performance computing workloads are still typical, emerging ones include analyzing tweets of congressional members relative to COVID-19, anthropology, horticulture, and crop sciences. Anything that is time-limited or involves too much data can benefit from high-performance computing capabilities. The demand for these capabilities is expected to grow due to increased accessibility. Making HPC easier to consume is similar to what the cloud did for grid computing back in the day.
OSC Capacity
OSC’s massive capacity for its high-performance computing system is constantly expanding to meet demand. At the time of the recording, they had 55,000 cores, mainly from Intel, with 400 accelerators spread across 1600 nodes. They anticipate a new acquisition that would bring them up to 75,000-80,000 seats due to the growing demand in biomedical fields. The system can handle large amounts of data, with 20 petabytes of actual disk storage and network connectivity at 350 gigabits per second of read/write speed. One of the significant benefits of OSC is the lack of egress costs for their clients due to the founding of the organization through a National Science Foundation grant.
Open OnDemand
Many universities and HPC centers are leveraging Open OnDemand as a simple web interface to make HPC more available for researchers who need to learn or understand the complexities of scheduling jobs, decomposing problem sets, and managing data across a cluster. Even the cloud service providers have Open OnDemand interfaces to their HPC offering.