High-performance computing
Hello World
MPI
- MPI: Setup & Hello World
- MPI: Send & Recv
- MPI: Scatter
- MPI: Bcast
- MPI: gather
- MPI: Reduce
- MPI: Probe
OpenCL
- OpenCL: Setup & Hello World
- OpenCL: OpenCL Specification
- OpenCL: Hello World (lengkap)
- OpenCL: Memory Objects
- OpenCL: Data Transfers
- OpenCL: Memory Region
- OpenCL: 2D Array
Tier 1: Fondasi Wajib
OpenMP
- OpenMP: Setup & Hello World
- OpenMP: Parallel For & Scheduling
- OpenMP: Reduction & Sinkronisasi
- OpenMP: Tasking
- OpenMP: Affinity & First-Touch
- OpenMP: False Sharing
- OpenMP: Hybrid MPI+OpenMP
CUDA
- CUDA: Setup & Hello World
- CUDA: Thread Hierarchy (Grid, Block, Warp)
- CUDA: Memory Model & Shared Memory
- CUDA: Matmul dengan Tiling
- CUDA: Reduction & Scan
- CUDA: Streams & Overlap
- CUDA: Occupancy & Warp Divergence
- CUDA: Perbandingan dengan OpenCL
Teori Algoritma Paralel
- Work-Span Model & Brent's Theorem
- Parallel Prefix Sum (Scan)
- Parallel Sorting
- Isoefficiency & Scalability
- Roofline model
Metode Numerik
- IEEE 754 & Floating Point Error
- BLAS: Level 1, 2, 3
- Dekomposisi LU & Cholesky
- Sparse Matrix: Format CSR & CSC
- SpMV
- Iterative Solvers: Jacobi & Gauss-Seidel
- Conjugate Gradient
- Stencil Computation & Heat Equation
- FFT: Konsep Dasar
MPI Lanjutan
- MPI: Isend & Irecv (Non-Blocking)
- MPI: Overlap Komunikasi-Komputasi
- MPI: Derived Datatypes
- MPI: Communicators & Topology
- MPI: One-Sided (RMA)
- MPI: Algoritma di Balik Collectives
Tier 2: Kehidupan di Cluster
Slurm & Environment
Profiling & Benchmark
- Profiling: perf
- Profiling: likwid
- Profiling: Nsight Compute & Nsight Systems
- Profiling: Score-P & TAU
- Benchmark: STREAM
- Benchmark: HPL & HPCG
- Benchmark: OSU Microbenchmarks
SIMD
Interconnect
Tier 3: Pembeda
Distributed Deep Learning
- Distributed Training: Data, Model, Pipeline Parallelism
- NCCL & Ring Allreduce
- Mixed Precision Training