TUTOR

M20 · Level III — AI Systems Engineering

GPU Computing & Performance Engineering

Understand and engineer parallel computation on modern accelerator systems, from hardware abstractions and memory movement to kernels, profiling, optimisation and distributed execution.

Topics

  • Algorithm
  • Instruction
  • Kernel
  • Memory
  • Parallelism
  • Scheduling
  • Measurement
  • Optimisation

Technical concepts

  • 01Computing Abstractions: From Arithmetic to Parallel Hardware
  • 02A First Performance Model
  • 03Arithmetic Intensity and the Roofline Idea
  • 04Memory Hierarchy
  • 05Locality and Data Reuse
  • 06Vectorisation and SIMD Reasoning
  • 07GPU Execution Model
  • 08Kernel Launches and Granularity
  • 09Coalesced Memory Access
  • 10Shared/Local Memory and Tiling
  • 11Occupancy and Resource Pressure
  • 12Control Flow and Divergence
  • 13Parallel Reduction
  • 14Atomic Operations and Contention
  • 15Synchronisation
  • 16Overlap: Compute, Transfer and Communication
  • 17Profiling: Measure Before You Optimise
  • 18Benchmark Design
  • 19Amdahl's Law
  • 20Strong and Weak Scaling
  • 21Load Balancing
  • 22Kernel Fusion
  • 23Mixed and Reduced Precision
  • 24Compiler and Runtime Lowering
  • 25Memory Planning and Reuse
  • 26Multi-GPU Communication
  • 27Communication-Compute Trade-offs
  • 28Data, Model and Pipeline Parallelism
  • 29Hardware Topology and Placement
  • 30Performance Limiters: Compute, Memory, Latency, Synchronisation

Practical implementation

  • 01Laboratory I: Reference vs Optimised Kernel
  • 02Laboratory II: Memory Access Pattern Study
  • 03Laboratory III: Tiled Matrix Multiplication
  • 04Laboratory IV: Parallel Reduction
  • 05Laboratory V: Profiling and Occupancy Study
  • 06Laboratory VI: Kernel Fusion
  • 07Laboratory VII: Precision vs Performance
  • 08Laboratory VIII: Multi-Device Scaling
  • 09Laboratory IX: End-to-End Inference Performance
  • 10Laboratory X: Reproducible Performance Engineering Case

Tools and technologies

  • Linux VM/container
  • CPU/GPU simulators or accessible accelerators
  • profiling
  • kernel experiments
  • memory hierarchy studies
  • multi-device communication labs

Project

Laboratory X: Reproducible Performance Engineering Case

Expected learning outcomes

  • 01Can derive work and approximate data movement.
  • 02Can explain memory hierarchy and locality.
  • 03Can reason about accelerator execution hierarchy.
  • 04Can write and verify basic kernels.
  • 05Can diagnose memory, compute, divergence and synchronisation bottlenecks.
  • 06Can measure occupancy and resource pressure without treating them as isolated goals.
  • 07Can optimise via tiling, fusion, batching or precision with evidence.
  • 08Can reason about multi-device communication and scaling.
  • 09Can produce reproducible profiler-backed performance reports.
  • 10Can defend a hardware-aware optimisation in a technical viva.

Assessment

  • 01Daily practice
  • 02Laboratories
  • 03Research reproduction
  • 04Integrated project
  • 05Technical viva
  • 06Engineering record

Source document: TUTOR Frontier Module 20 GPU Computing Performance Engineering