High-performance computing: Difference between revisions
Appearance
No edit summary |
No edit summary |
||
| (25 intermediate revisions by the same user not shown) | |||
| Line 1: | Line 1: | ||
= | = Hello World = | ||
* [[ | *[[Amdahl's law]] | ||
*[[Gustafson's Law]] | |||
*[[GFLOPS]] | |||
*[[Memory Coalescing]] | |||
*[[N-Body Problems]] | |||
*[[SoA vs AoS]] | |||
*[[Cache line]] | |||
*[[NUMA]] | |||
= OpenCL = | === MPI === | ||
* [[OpenCL: Setup & Hello World]] | *[[MPI: Setup & Hello World]] | ||
*[[MPI: Send & Recv]] | |||
*[[MPI: Scatter]] | |||
*[[MPI: Bcast]] | |||
*[[MPI: gather]] | |||
*[[MPI: Reduce]] | |||
*[[MPI: Probe]] | |||
=== OpenCL === | |||
*[[OpenCL: Setup & Hello World]] | |||
*[[OpenCL: OpenCL Specification]] | |||
*[[OpenCL: Hello World (lengkap)]] | |||
*[[OpenCL: Memory Objects]] | |||
*[[OpenCL: Data Transfers]] | |||
*[[OpenCL: Memory Region]] | |||
*[[OpenCL: 2D Array]] | |||
= Tier 1: Fondasi Wajib = | |||
== OpenMP == | |||
* [[OpenMP: Setup & Hello World]] | |||
* [[OpenMP: Parallel For & Scheduling]] | |||
* [[OpenMP: Reduction & Sinkronisasi]] | |||
* [[OpenMP: Tasking]] | |||
* [[OpenMP: Affinity & First-Touch]] | |||
* [[OpenMP: False Sharing]] | |||
* [[OpenMP: Hybrid MPI+OpenMP]] | |||
== CUDA == | |||
* [[CUDA: Setup & Hello World]] | |||
* [[CUDA: Thread Hierarchy (Grid, Block, Warp)]] | |||
* [[CUDA: Memory Model & Shared Memory]] | |||
* [[CUDA: Matmul dengan Tiling]] | |||
* [[CUDA: Reduction & Scan]] | |||
* [[CUDA: Streams & Overlap]] | |||
* [[CUDA: Occupancy & Warp Divergence]] | |||
* [[CUDA: Perbandingan dengan OpenCL]] | |||
== Teori Algoritma Paralel == | |||
* [[Work-Span Model & Brent's Theorem]] | |||
* [[Parallel Prefix Sum (Scan)]] | |||
* [[Parallel Sorting]] | |||
* [[Isoefficiency & Scalability]] | |||
* [[Roofline model]] | |||
== Metode Numerik == | |||
* [[IEEE 754 & Floating Point Error]] | |||
* [[BLAS: Level 1, 2, 3]] | |||
* [[Dekomposisi LU & Cholesky]] | |||
* [[Sparse Matrix: Format CSR & CSC]] | |||
* [[SpMV]] | |||
* [[Iterative Solvers: Jacobi & Gauss-Seidel]] | |||
* [[Conjugate Gradient]] | |||
* [[Stencil Computation & Heat Equation]] | |||
* [[FFT: Konsep Dasar]] | |||
== MPI Lanjutan == | |||
* [[MPI: Isend & Irecv (Non-Blocking)]] | |||
* [[MPI: Overlap Komunikasi-Komputasi]] | |||
* [[MPI: Derived Datatypes]] | |||
* [[MPI: Communicators & Topology]] | |||
* [[MPI: One-Sided (RMA)]] | |||
* [[MPI: Algoritma di Balik Collectives]] | |||
= Tier 2: Kehidupan di Cluster = | |||
== Slurm & Environment == | |||
* [[Slurm: sbatch, srun, squeue]] | |||
* [[Slurm: Job Array & Resource Request]] | |||
* [[Module Environment (Lmod)]] | |||
== Profiling & Benchmark == | |||
* [[Profiling: perf]] | |||
* [[Profiling: likwid]] | |||
* [[Profiling: Nsight Compute & Nsight Systems]] | |||
* [[Profiling: Score-P & TAU]] | |||
* [[Benchmark: STREAM]] | |||
* [[Benchmark: HPL & HPCG]] | |||
* [[Benchmark: OSU Microbenchmarks]] | |||
== SIMD == | |||
* [[SIMD]] | |||
* [[SIMD: AVX2 Intrinsics]] | |||
* [[SIMD: Auto-Vectorization & Laporan Compiler]] | |||
== Interconnect == | |||
* [[Topologi Jaringan HPC: Fat Tree, Dragonfly, Torus]] | |||
* [[InfiniBand & RDMA]] | |||
= Tier 3: Pembeda = | |||
== Distributed Deep Learning == | |||
* [[Distributed Training: Data, Model, Pipeline Parallelism]] | |||
* [[NCCL & Ring Allreduce]] | |||
* [[Mixed Precision Training]] | |||
== Ekosistem & Riset == | |||
* [[Kokkos: Konsep Dasar]] | |||
* [[SYCL: Konsep Dasar]] | |||
* [[Membaca Paper HPC: SC, PPoPP, IPDPS]] | |||
= Proyek Latihan = | |||
* [[Proyek: Heat Equation 2D dengan MPI Halo Exchange]] | |||
* [[Proyek: Hybrid MPI+OpenMP Heat Equation]] | |||
* [[Proyek: Port Matmul OpenCL ke CUDA]] | |||
* [[Proyek: Benchmark Cluster Heterogen (STREAM, HPL, HPCG, OSU)]] | |||
* [[Proyek: Analisis Roofline Matmul]] | |||
[[Kategori:HPC]] | |||
[[Kategori:Roadmap]] | |||
Latest revision as of 06:15, 8 August 2026
Hello World
MPI
- MPI: Setup & Hello World
- MPI: Send & Recv
- MPI: Scatter
- MPI: Bcast
- MPI: gather
- MPI: Reduce
- MPI: Probe
OpenCL
- OpenCL: Setup & Hello World
- OpenCL: OpenCL Specification
- OpenCL: Hello World (lengkap)
- OpenCL: Memory Objects
- OpenCL: Data Transfers
- OpenCL: Memory Region
- OpenCL: 2D Array
Tier 1: Fondasi Wajib
OpenMP
- OpenMP: Setup & Hello World
- OpenMP: Parallel For & Scheduling
- OpenMP: Reduction & Sinkronisasi
- OpenMP: Tasking
- OpenMP: Affinity & First-Touch
- OpenMP: False Sharing
- OpenMP: Hybrid MPI+OpenMP
CUDA
- CUDA: Setup & Hello World
- CUDA: Thread Hierarchy (Grid, Block, Warp)
- CUDA: Memory Model & Shared Memory
- CUDA: Matmul dengan Tiling
- CUDA: Reduction & Scan
- CUDA: Streams & Overlap
- CUDA: Occupancy & Warp Divergence
- CUDA: Perbandingan dengan OpenCL
Teori Algoritma Paralel
- Work-Span Model & Brent's Theorem
- Parallel Prefix Sum (Scan)
- Parallel Sorting
- Isoefficiency & Scalability
- Roofline model
Metode Numerik
- IEEE 754 & Floating Point Error
- BLAS: Level 1, 2, 3
- Dekomposisi LU & Cholesky
- Sparse Matrix: Format CSR & CSC
- SpMV
- Iterative Solvers: Jacobi & Gauss-Seidel
- Conjugate Gradient
- Stencil Computation & Heat Equation
- FFT: Konsep Dasar
MPI Lanjutan
- MPI: Isend & Irecv (Non-Blocking)
- MPI: Overlap Komunikasi-Komputasi
- MPI: Derived Datatypes
- MPI: Communicators & Topology
- MPI: One-Sided (RMA)
- MPI: Algoritma di Balik Collectives
Tier 2: Kehidupan di Cluster
Slurm & Environment
Profiling & Benchmark
- Profiling: perf
- Profiling: likwid
- Profiling: Nsight Compute & Nsight Systems
- Profiling: Score-P & TAU
- Benchmark: STREAM
- Benchmark: HPL & HPCG
- Benchmark: OSU Microbenchmarks
SIMD
Interconnect
Tier 3: Pembeda
Distributed Deep Learning
- Distributed Training: Data, Model, Pipeline Parallelism
- NCCL & Ring Allreduce
- Mixed Precision Training