These articles describe the fundamental concepts of CUDA Programming. Note: the documents below assume understanding of the content in documents that precede them.

  1. What is a GPU? What is CUDA? Why would I care?

  2. Writing our first CUDA kernel

  3. Thread Hierarchy (WIP)

  4. Function Annotations

  5. Thread Divergence (WIP)

  6. SAXPY (WIP)

  7. Separable Compilation

  8. The Memory Hierarchy (WIP)

  9. Arithmetic Intensity

  10. Shared Memory (WIP)

  11. Parallel Reduction, Part 1: Data Races and Atomics (WIP)

  12. Parallel Reduction, Part 2: Shared Memory (WIP)

  13. Parallel Reduction, Part 3: Determinism and the Two-Pass Approach (WIP)

  14. CUDA Streams (WIP)