High-performance computing: Difference between revisions
No edit summary |
No edit summary |
||
| Line 4: | Line 4: | ||
= Hello World = | = Hello World = | ||
[[Amdahl's law]] | *[[Amdahl's law]] | ||
[[Gustafson's Law]] | *[[Gustafson's Law]] | ||
[[GFLOPS]] | *[[GFLOPS]] | ||
[[Memory Coalescing]] | *[[Memory Coalescing]] | ||
[[N-Body Problems]] | *[[N-Body Problems]] | ||
[[SoA vs AoS]] | *[[SoA vs AoS]] | ||
[[Cache line]] | *[[Cache line]] | ||
[[NUMA]] | *[[NUMA]] | ||
=== MPI === | === MPI === | ||
[[MPI: Setup & Hello World]] | *[[MPI: Setup & Hello World]] | ||
[[MPI: Send & Recv]] | *[[MPI: Send & Recv]] | ||
[[MPI: Scatter]] | *[[MPI: Scatter]] | ||
[[MPI: Bcast]] | *[[MPI: Bcast]] | ||
[[MPI: gather]] | *[[MPI: gather]] | ||
[[MPI: Reduce]] | *[[MPI: Reduce]] | ||
[[MPI: Probe]] | *[[MPI: Probe]] | ||
=== OpenCL === | === OpenCL === | ||
[[OpenCL: Setup & Hello World]] | *[[OpenCL: Setup & Hello World]] | ||
[[OpenCL: OpenCL Specification]] | *[[OpenCL: OpenCL Specification]] | ||
[[OpenCL: Hello World (lengkap)]] | *[[OpenCL: Hello World (lengkap)]] | ||
[[OpenCL: Memory Objects]] | *[[OpenCL: Memory Objects]] | ||
[[OpenCL: Data Transfers]] | *[[OpenCL: Data Transfers]] | ||
[[OpenCL: Memory Region]] | *[[OpenCL: Memory Region]] | ||
[[OpenCL: 2D Array]] | *[[OpenCL: 2D Array]] | ||
= Tier 1: Fondasi Wajib = | = Tier 1: Fondasi Wajib = | ||
Revision as of 07:49, 3 July 2026
Halaman induk roadmap belajar HPC. Link merah = artikel yang belum ditulis. Tulis satu per satu; kalau semua sudah biru, kuliah master tinggal sambil tidur.
Fondasi yang sudah ada: Amdahl's law, Gustafson's Law, GFLOPS, Memory Coalescing, N-Body Problems, SoA vs AoS, Cache line, NUMA, seri MPI dan OpenCL.
Hello World
MPI
- MPI: Setup & Hello World
- MPI: Send & Recv
- MPI: Scatter
- MPI: Bcast
- MPI: gather
- MPI: Reduce
- MPI: Probe
OpenCL
- OpenCL: Setup & Hello World
- OpenCL: OpenCL Specification
- OpenCL: Hello World (lengkap)
- OpenCL: Memory Objects
- OpenCL: Data Transfers
- OpenCL: Memory Region
- OpenCL: 2D Array
Tier 1: Fondasi Wajib
OpenMP
- OpenMP: Setup & Hello World
- OpenMP: Parallel For & Scheduling
- OpenMP: Reduction & Sinkronisasi
- OpenMP: Tasking
- OpenMP: Affinity & First-Touch
- OpenMP: False Sharing
- OpenMP: Hybrid MPI+OpenMP
CUDA
- CUDA: Setup & Hello World
- CUDA: Thread Hierarchy (Grid, Block, Warp)
- CUDA: Memory Model & Shared Memory
- CUDA: Matmul dengan Tiling
- CUDA: Reduction & Scan
- CUDA: Streams & Overlap
- CUDA: Occupancy & Warp Divergence
- CUDA: Perbandingan dengan OpenCL
Teori Algoritma Paralel
- Work-Span Model & Brent's Theorem
- Parallel Prefix Sum (Scan)
- Parallel Sorting
- Isoefficiency & Scalability
- Roofline model
Metode Numerik
- IEEE 754 & Floating Point Error
- BLAS: Level 1, 2, 3
- Dekomposisi LU & Cholesky
- Sparse Matrix: Format CSR & CSC
- SpMV
- Iterative Solvers: Jacobi & Gauss-Seidel
- Conjugate Gradient
- Stencil Computation & Heat Equation
- FFT: Konsep Dasar
MPI Lanjutan
- MPI: Isend & Irecv (Non-Blocking)
- MPI: Overlap Komunikasi-Komputasi
- MPI: Derived Datatypes
- MPI: Communicators & Topology
- MPI: One-Sided (RMA)
- MPI: Algoritma di Balik Collectives
Tier 2: Kehidupan di Cluster
Slurm & Environment
Profiling & Benchmark
- Profiling: perf
- Profiling: likwid
- Profiling: Nsight Compute & Nsight Systems
- Profiling: Score-P & TAU
- Benchmark: STREAM
- Benchmark: HPL & HPCG
- Benchmark: OSU Microbenchmarks
SIMD
Interconnect
Tier 3: Pembeda
Distributed Deep Learning
- Distributed Training: Data, Model, Pipeline Parallelism
- NCCL & Ring Allreduce
- Mixed Precision Training