HPC-By es: Basics o Pa alleliza ion wi h
OpenMP
Compe ence Cen e o High Pe o mance Compu ing in Hessen (HKHLR)
D . Tim Jamme
HKHLR is unded by he Hessian Minis y o Sciences and A s
HKHLR S a HPC-By es: Basics o Pa alleliza ion wi h OpenMP 2025-10-16
3/17 Wha is he HKHLR?
Hessisches Kompe enzzen um ü Hochleis ungs echnen (HKHLR)
B ain Powe Supe cha ges Compu ing Powe
▶Compe ence Cen e o High Pe o mance Compu ing in Hesse
▶T aining
▶Consul ing
▶In o ma ion
▶Moni o ing
Mo e in o on ou webpage:
h ps://www.hkhl .de/en
Pa icipa ing uni e si ies:
HKHLR S a HPC-By es: Basics o Pa alleliza ion wi h OpenMP 2025-10-16
4/17 In oduc ion
Hessisches Kompe enzzen um ü Hochleis ungs echnen (HKHLR)
▶Mode n CPUs ha e mul iple co es
▶E en mo e in an HPC clus e
▶HPC ha dwa e is no as e pe se bu o e s many CPU co es
▶Di e en CPU co es can compu e in pa allel
▶Dis ibu e he wo k among se e al CPU co es
HKHLR S a HPC-By es: Basics o Pa alleliza ion wi h OpenMP 2025-10-16
5/17 Sha ed-memo y s. dis ibu ed-memo y
Hessisches Kompe enzzen um ü Hochleis ungs echnen (HKHLR)
19.09.2025 | In oduc ion o he Lich enbe g High Pe o mance Compu e | 8
Cache
Pa allelisa ion (2):
Sha ed and dis ibu ed memo y
Sha ed Memo y (e.g. MT o OpenMP)
Dis ibu ed Memo y (e.g. MPI)
In e connec / Ne wo k …
MT = Mul i-Th eading (only „inside“ a node)
OpenMP = Open Mul i P ocessing (only „inside“ a node)
MPI = Message Passing In e ace („inside“and „ac oss“nodes, oo)
Compu e Node
Cache
…
Cache
Cache
MPI
HKHLR S a HPC-By es: Basics o Pa alleliza ion wi h OpenMP 2025-10-16
6/17 OpenMP
Hessisches Kompe enzzen um ü Hochleis ungs echnen (HKHLR)
▶Se o compile di ec i es, lib a y ou ines, en i onmen a iables
▶Speci y high-le el pa allelism in C, C++, and Fo an
▶Rela i ely easy o pa allelize exis ing code
▶Add p agma di ec i e o con ol pa allelism:
▶#p agma omp pa allel in C/C++
▶!$omp pa allel in Fo an
▶Compile wi h - openmp compile op ion
HKHLR S a HPC-By es: Basics o Pa alleliza ion wi h OpenMP 2025-10-16
7/17 Basic Pa allelism
Hessisches Kompe enzzen um ü Hochleis ungs echnen (HKHLR)
C/C++
1// Assume: in N;
2// A ays A[N], B[N], C[N];
3
4#p agma omp pa allel o
5 o (in i=0; i < N; i++) {
6A[i] = B[i] + C[i];
7}
Fo an
1! Assume: in ege :: N
2! eal :: A(N), B(N), C(N)
3
4!$omp pa allel do
5do i=1, N
6A(i) =B(i) +C(i)
7end do
8!$omp end pa allel do
▶Each h ead wo ks on a di e en pa o he a ay
▶A ay pa s a e au oma ically assigned by OpenMP
▶Fine- une wi h he schedule clause
HKHLR S a HPC-By es: Basics o Pa alleliza ion wi h OpenMP 2025-10-16
8/17 Synch oniza ion and Sa e y
Hessisches Kompe enzzen um ü Hochleis ungs echnen (HKHLR)
▶Va iables a e sha ed by de aul
▶I mul iple h eads use he same a iable →da a ace
▶Example on nex slide
▶P e en by con olling access:
▶p i a e: each h ead has i s own copy
▶ educ ion: all h eads sum he esul a he end
▶Example: #p agma omp pa allel o de aul (none) sha ed(A) p i a e(B,C)
▶Synch oniza ion ensu es co ec o de :
▶#p agma omp ba ie : wai un il all h eads a e eady
▶#p agma omp c i ical: only one h ead can execu e his block
▶#p agma omp a omic: make ope a ion a omic
HKHLR S a HPC-By es: Basics o Pa alleliza ion wi h OpenMP 2025-10-16
9/17 Da a Race Example
Hessisches Kompe enzzen um ü Hochleis ungs echnen (HKHLR)
C/C++
1// Assume: in N;
2// double sum = 0.0;
3// double A[N];
4
5#p agma omp pa allel o
6 o (in i=0; i < N; i++) {
7sum += A[i]; // <-- Da a ace!
8}
Fix by using a educ ion:
1#p agma omp pa allel o educ ion(+:sum)
2 o (in i=0; i < N; i++) {
3sum += A[i];//Sa e: h eads ha e a local sum
4}// OpenMP will Sum he Sums o all h eads
Fo an
!$omp pa allel do
do i=1, N
sum =sum +A(i) ! <-- Da a ace!
end do
!$omp end pa allel do
Fix by using a educ ion:
!$omp pa allel do educ ion(+:sum)
do i=1, N
sum =sum +A(i) ! Sa e now
end do
!$omp end pa allel do
HKHLR S a HPC-By es: Basics o Pa alleliza ion wi h OpenMP 2025-10-16
16/17 Addi ional Resou ces
Hessisches Kompe enzzen um ü Hochleis ungs echnen (HKHLR)
▶h ps://www.hkhl .de/en/e en s/cou ses-and- u o ials
▶h ps:// u-da ms ad .cloud.panop o.eu/Panop o/Pages/Viewe .aspx?id=a 359b3a-
a07-4a28-a695-ae8b00 e96d2&s a =10.273728
▶h ps://www.openmp.o g/ esou ces/
HKHLR S a HPC-By es: Basics o Pa alleliza ion wi h OpenMP 2025-10-16
17/17 Image Sou ces
Hessisches Kompe enzzen um ü Hochleis ungs echnen (HKHLR)
▶Slide 4: CC0 ia wikimedia commons
(h ps://commons.wikimedia.o g/wiki/File:CPU_clock_speed_and_Co e_coun _G aph.png)
▶Slide 5: CC BY 4.0 HRZ/HPC TU Da ms ad
HKHLR S a HPC-By es: Basics o Pa alleliza ion wi h OpenMP 2025-10-16
