High-performance computing: Difference between revisions
No edit summary |
No edit summary |
||
| (3 intermediate revisions by the same user not shown) | |||
| Line 1: | Line 1: | ||
= Hello World = | |||
*[[Amdahl's law]] | |||
*[[Gustafson's Law]] | |||
*[[GFLOPS]] | |||
*[[Memory Coalescing]] | |||
*[[N-Body Problems]] | |||
*[[SoA vs AoS]] | |||
*[[Cache line]] | |||
*[[NUMA]] | |||
=== MPI === | |||
*[[MPI: Setup & Hello World]] | |||
*[[MPI: Send & Recv]] | |||
*[[MPI: Scatter]] | |||
*[[MPI: Bcast]] | |||
*[[MPI: gather]] | |||
*[[MPI: Reduce]] | |||
*[[MPI: Probe]] | |||
=== OpenCL === | |||
*[[OpenCL: Setup & Hello World]] | |||
*[[OpenCL: OpenCL Specification]] | |||
*[[OpenCL: Hello World (lengkap)]] | |||
*[[OpenCL: Memory Objects]] | |||
*[[OpenCL: Data Transfers]] | |||
*[[OpenCL: Memory Region]] | |||
*[[OpenCL: 2D Array]] | |||
= Tier 1: Fondasi Wajib = | = Tier 1: Fondasi Wajib = | ||
Latest revision as of 06:15, 8 August 2026
Hello World
MPI
- MPI: Setup & Hello World
- MPI: Send & Recv
- MPI: Scatter
- MPI: Bcast
- MPI: gather
- MPI: Reduce
- MPI: Probe
OpenCL
- OpenCL: Setup & Hello World
- OpenCL: OpenCL Specification
- OpenCL: Hello World (lengkap)
- OpenCL: Memory Objects
- OpenCL: Data Transfers
- OpenCL: Memory Region
- OpenCL: 2D Array
Tier 1: Fondasi Wajib
OpenMP
- OpenMP: Setup & Hello World
- OpenMP: Parallel For & Scheduling
- OpenMP: Reduction & Sinkronisasi
- OpenMP: Tasking
- OpenMP: Affinity & First-Touch
- OpenMP: False Sharing
- OpenMP: Hybrid MPI+OpenMP
CUDA
- CUDA: Setup & Hello World
- CUDA: Thread Hierarchy (Grid, Block, Warp)
- CUDA: Memory Model & Shared Memory
- CUDA: Matmul dengan Tiling
- CUDA: Reduction & Scan
- CUDA: Streams & Overlap
- CUDA: Occupancy & Warp Divergence
- CUDA: Perbandingan dengan OpenCL
Teori Algoritma Paralel
- Work-Span Model & Brent's Theorem
- Parallel Prefix Sum (Scan)
- Parallel Sorting
- Isoefficiency & Scalability
- Roofline model
Metode Numerik
- IEEE 754 & Floating Point Error
- BLAS: Level 1, 2, 3
- Dekomposisi LU & Cholesky
- Sparse Matrix: Format CSR & CSC
- SpMV
- Iterative Solvers: Jacobi & Gauss-Seidel
- Conjugate Gradient
- Stencil Computation & Heat Equation
- FFT: Konsep Dasar
MPI Lanjutan
- MPI: Isend & Irecv (Non-Blocking)
- MPI: Overlap Komunikasi-Komputasi
- MPI: Derived Datatypes
- MPI: Communicators & Topology
- MPI: One-Sided (RMA)
- MPI: Algoritma di Balik Collectives
Tier 2: Kehidupan di Cluster
Slurm & Environment
Profiling & Benchmark
- Profiling: perf
- Profiling: likwid
- Profiling: Nsight Compute & Nsight Systems
- Profiling: Score-P & TAU
- Benchmark: STREAM
- Benchmark: HPL & HPCG
- Benchmark: OSU Microbenchmarks
SIMD
Interconnect
Tier 3: Pembeda
Distributed Deep Learning
- Distributed Training: Data, Model, Pipeline Parallelism
- NCCL & Ring Allreduce
- Mixed Precision Training