Supercomputers

Содержание и основные понятия

Back to the table of contents
Alterna · Chapter 7 · page 7.2
Supercomputers
Cars the size of a room that consume electricity from a small city, solve problems that are not available to a regular PC, sort out how it's done and why.
Supercomputer
A computational system that far exceeds the productivity of conventional computers.
FLOPS
- floating point operations per second.
Today's supercomputers are over.
Exaphlops barrier
— 10
18
That's billions of times more powerful than your laptop.
GEF · Basic
EGE No. 22
01
Productivity units
From megaflops to exaphlops: capacity scale
FLOPS — Floating Point Operations Per Second
Expensive
Degree
Example
MFLOPS
10⁶
Calculator
GFLOPS
10⁹
Smartphone, laptop
TFLOPS
10¹²
Game Video Card (RTX 4090 ~ 83 TFLOPS)
PFLOPS
10¹⁵
Major cluster, Lomonos II
EFLOPS
10¹⁸
Frontier (USA), Fugaku (Japan)
Scale
1 exaphlops = 1 million million million,000,000 operations per second. If every person on Earth (8 billion) made 125,000 calculations per second, they would work together as one exaufflops supercomputer.
02
Supercomputer architecture
Cluster: thousands of knots connected by a super-rapid network
Cluster architecture
The modern supercomputer is
Cluster
: multiple computational nodes (servers) connected to the high-speed network (InfiniBand, 200+Gbit/s).
Node Manager
High Speed Network (InfiniBand)
Host 1
CPU + GPU
Host 2
CPU + GPU
Host 3
CPU + GPU
Hinge N
CPU + GPU
... Thousands of knots...
Warehouse (Lustre)
Refrigeration
Figure 2.
Typical supercomputer architecture: node manager + computational nodes + network + repository
Components
Component
Appointment
Node Manager
Task planning (SLURM, PBS), monitoring
Computation nodes
CPU (AMD EPYC, Intel Xeon) + GPU (NVIDIA H100)
Network
InfiniBand NDR (400 Gbit/s) or Slingshot
Storage
Parallel FS: Lustre, GPFS (data petabytes)
Refrigeration
Liquid (45 °C) is more efficient than air
03
GPU calculations
Video cards are not just for games: thousands of kernels for science
CPU vs GPU
CPU
GPU
Nucleus
4 - 96 (high)
10,000+ (simple)
Tactical frequency
3 - 5 GHz
1-2 GHz
Architecture
MIMD (various tasks)
SIMD (one operation, lots of data)
Better for
Complex logic, branches
Matrixs, graphs, neuronets
Memory
DDR5, up to 2 TB
HBM3, up to 80 GB (rapid)
CUDA and OpenCL
CUDA
The NVIDIA is a GPU programming platform.
OpenCL
- Open standard for any GPU.
Examples of tasks for GPU: neuronet training, molecular dynamics, branding, cryptography.
Relationship to Chapter 2
In chapter 2 (PC), we considered the video maps as components, and in supercomputers, GPUs are the main computing devices.
04
TOP500 and world leaders
Rating of the world's most powerful computers, updated twice a year
ToP500 Rating
Since 1993, every six months (June and November) a list of the world ' s 500 most powerful supercomputers has been published.
Linpack
The solution to the linear equation system.
#
Name
Country
Productivity
Specialities
1
Frontier
♪ USA
1.2 EFLOPS
First exaphlops (2022) AMD EPYC + MI250X
2
Aurora
♪ USA
1.0 EFLOPS
Intel Xeon + Ponte Vecchio GPU (2024)
3
Eagle
♪ USA
561 PFLOPS
Microsoft Azure, cloud supercomputer
4
Fugaku
♪ Japan
442 PFLOPS
ARM processors A64FX (formerly #1)
5
LUMI
Finland
380 PFLOPS
AMD EPYC + MI250X, European flagship
Russia at TOP500
Russian supercomputers:
Lomonos 2
(MOG, ~2.5 PFLOPS)
Chervonenko
(Rosatom) The rating is a unit of systems due to supply restrictions.
Trent.
Expected by 2030
Zetaflops
It's 1,000 times bigger than the exaphlops. The main directions are quantum computing, neuromorphic chips, optical processors.
05
Application of supercomputers
Where Quintillions of Operations are needed per second.
🌤️
Weather forecast
meteorology
Modeling the Earth's atmosphere with a resolution of 1 km. The more powerful the computer is, the more accurate the forecast goes.
💊
Development of drugs
Bioinformatics
Molecular dynamics: Modelling protein-drug interactions. Folding@home - 2.4 EFLOPS (volunteers).
🧠
Artificial intelligence
Neuronets
Learning big language models (GPT, LLAMA) requires thousands of GPUs and weeks of computation.
🔬
Physics and chemistry
Modelling
Quantum chemistry, thermonuclear synthesis, modelling nuclear reactions, astrophysics.
🎬
CGI and Rendering
Film industry
Pixar brands, Weta Digital, are thousands of kernels for special effects.
🔐
Cryptography
security
Selection of encryption keys, vulnerability analysis, connection to Chapter 5 (TLS, RSA).
06
Parallel programming
How to write a program for thousands of kernels: MPI and OpenMP
Two approaches
Technology
Type of memory
How it works
OpenMP
Total
The instructions are in the code (#pagma mp parallel). Flows within a single process. Just add parallelism to the existing code.
MPI
Distribution
Messages exchanged between cluster nodes. Each node is a separate process. It is spread by thousands of nodes.
CUDA / OpenCL
GPU
Parallel computations on video cards, thousands of streams, SIMD architecture.
Gustafson Law
Unlike Amdal's law,
Gustafson
(1988) states that if the task is increased in proportion to the number of kernels, the acceleration increases almost linearly.
S(N) = N − (1 − p) × (N − 1)
Practical sense: supercomputers decide
large
It's a problem, it's not an acceleration.
Amdal vs Gustafson
Amdal.
:: The task is fixed, and the acceleration is limited.
Gustafson
: kernels are fixed, increasing the task of acceleration almost linear.
Both are just looking at the problem from different angles.