Jump to content

High-performance computing: Difference between revisions

From Haidhi Wiki
No edit summary
No edit summary
 
(15 intermediate revisions by the same user not shown)
Line 1: Line 1:
* [[Amdahl's law]]
= Hello World =
*[[Amdahl's law]]
*[[Gustafson's Law]]
*[[GFLOPS]]
*[[Memory Coalescing]]
*[[N-Body Problems]]
*[[SoA vs AoS]]
*[[Cache line]]
*[[NUMA]]  


= MPI =
=== MPI ===
* [[MPI: Setup & Hello World]]
*[[MPI: Setup & Hello World]]
* [[MPI: Send & Recv]]
*[[MPI: Send & Recv]]
* [[MPI: Scatter]]
*[[MPI: Scatter]]
*[[MPI: Bcast]]
*[[MPI: gather]]
*[[MPI: Reduce]]
*[[MPI: Probe]]  


= OpenCL =
=== OpenCL ===
* [[OpenCL: Setup & Hello World]]
*[[OpenCL: Setup & Hello World]]
* [[OpenCL: OpenCL Specification]]
*[[OpenCL: OpenCL Specification]]
* [[OpenCL: Hello World (lengkap)]]
*[[OpenCL: Hello World (lengkap)]]
* [[OpenCL: Memory Objects]]
*[[OpenCL: Memory Objects]]
* [[OpenCL: Data Transfers]]
*[[OpenCL: Data Transfers]]
* [[OpenCL: Memory Region]]
*[[OpenCL: Memory Region]]
* [[OpenCL: 2D Array]]
*[[OpenCL: 2D Array]]
 
= Tier 1: Fondasi Wajib =
 
== OpenMP ==
* [[OpenMP: Setup & Hello World]]
* [[OpenMP: Parallel For & Scheduling]]
* [[OpenMP: Reduction & Sinkronisasi]]
* [[OpenMP: Tasking]]
* [[OpenMP: Affinity & First-Touch]]
* [[OpenMP: False Sharing]]
* [[OpenMP: Hybrid MPI+OpenMP]]
 
== CUDA ==
* [[CUDA: Setup & Hello World]]
* [[CUDA: Thread Hierarchy (Grid, Block, Warp)]]
* [[CUDA: Memory Model & Shared Memory]]
* [[CUDA: Matmul dengan Tiling]]
* [[CUDA: Reduction & Scan]]
* [[CUDA: Streams & Overlap]]
* [[CUDA: Occupancy & Warp Divergence]]
* [[CUDA: Perbandingan dengan OpenCL]]
 
== Teori Algoritma Paralel ==
* [[Work-Span Model & Brent's Theorem]]
* [[Parallel Prefix Sum (Scan)]]
* [[Parallel Sorting]]
* [[Isoefficiency & Scalability]]
* [[Roofline model]]
 
== Metode Numerik ==
* [[IEEE 754 & Floating Point Error]]
* [[BLAS: Level 1, 2, 3]]
* [[Dekomposisi LU & Cholesky]]
* [[Sparse Matrix: Format CSR & CSC]]
* [[SpMV]]
* [[Iterative Solvers: Jacobi & Gauss-Seidel]]
* [[Conjugate Gradient]]
* [[Stencil Computation & Heat Equation]]
* [[FFT: Konsep Dasar]]
 
== MPI Lanjutan ==
* [[MPI: Isend & Irecv (Non-Blocking)]]
* [[MPI: Overlap Komunikasi-Komputasi]]
* [[MPI: Derived Datatypes]]
* [[MPI: Communicators & Topology]]
* [[MPI: One-Sided (RMA)]]
* [[MPI: Algoritma di Balik Collectives]]
 
= Tier 2: Kehidupan di Cluster =
 
== Slurm & Environment ==
* [[Slurm: sbatch, srun, squeue]]
* [[Slurm: Job Array & Resource Request]]
* [[Module Environment (Lmod)]]
 
== Profiling & Benchmark ==
* [[Profiling: perf]]
* [[Profiling: likwid]]
* [[Profiling: Nsight Compute & Nsight Systems]]
* [[Profiling: Score-P & TAU]]
* [[Benchmark: STREAM]]
* [[Benchmark: HPL & HPCG]]
* [[Benchmark: OSU Microbenchmarks]]
 
== SIMD ==
* [[SIMD]]
* [[SIMD: AVX2 Intrinsics]]
* [[SIMD: Auto-Vectorization & Laporan Compiler]]
 
== Interconnect ==
* [[Topologi Jaringan HPC: Fat Tree, Dragonfly, Torus]]
* [[InfiniBand & RDMA]]
 
= Tier 3: Pembeda =
 
== Distributed Deep Learning ==
* [[Distributed Training: Data, Model, Pipeline Parallelism]]
* [[NCCL & Ring Allreduce]]
* [[Mixed Precision Training]]
 
== Ekosistem & Riset ==
* [[Kokkos: Konsep Dasar]]
* [[SYCL: Konsep Dasar]]
* [[Membaca Paper HPC: SC, PPoPP, IPDPS]]
 
= Proyek Latihan =
* [[Proyek: Heat Equation 2D dengan MPI Halo Exchange]]
* [[Proyek: Hybrid MPI+OpenMP Heat Equation]]
* [[Proyek: Port Matmul OpenCL ke CUDA]]
* [[Proyek: Benchmark Cluster Heterogen (STREAM, HPL, HPCG, OSU)]]
* [[Proyek: Analisis Roofline Matmul]]
 
[[Kategori:HPC]]
[[Kategori:Roadmap]]

Latest revision as of 06:15, 8 August 2026

Hello World

MPI

OpenCL

Tier 1: Fondasi Wajib

OpenMP

CUDA

Teori Algoritma Paralel

Metode Numerik

MPI Lanjutan

Tier 2: Kehidupan di Cluster

Slurm & Environment

Profiling & Benchmark

SIMD

Interconnect

Tier 3: Pembeda

Distributed Deep Learning

Ekosistem & Riset

Proyek Latihan

Kategori:HPC Kategori:Roadmap