TUTOR

M14 · Level III — AI Systems Engineering

Generative Models

Understand and engineer models that learn distributions or generative procedures, from probabilistic foundations through autoregressive, latent-variable, adversarial and diffusion/score-based systems.

Topics

  • Distribution
  • Latent State
  • Likelihood
  • Sampling
  • Denoising
  • Adversarial Learning
  • Conditioning
  • Evaluation
  • Generative Systems

Technical concepts

  • 01What Is a Generative Model?
  • 02Maximum Likelihood as Distribution Fitting
  • 03Autoregressive Generation and the Chain Rule
  • 04Latent Variables: Generating Through Hidden State
  • 05Variational Autoencoders and the ELBO
  • 06The Reparameterisation Trick
  • 07VAE Diagnostics: Posterior Collapse and Latent Usage
  • 08Normalising Flows: Exact Density Through Invertible Maps
  • 09Coupling Layers and Tractable Jacobians
  • 10Generative Adversarial Learning
  • 11GAN Training Dynamics and Failure
  • 12Energy-Based Generative Models
  • 13Score Functions and Score-Based Generation
  • 14Diffusion Models: The Forward Noising Process
  • 15Denoising Objectives and Reverse Generation
  • 16Sampling as Numerical Integration or Iterative Transformation
  • 17Latent-Space Generation
  • 18Conditional Generation
  • 19Guidance and Conditional Control
  • 20Multimodal Generative Systems
  • 21Sampling Temperature, Truncation and Diversity
  • 22Evaluating Generative Models
  • 23Memorisation, Novelty and Training-Data Leakage
  • 24Generative Robustness and Distribution Shift
  • 25Generative Models for Scientific Simulation
  • 26Generative Models as World-Model Components
  • 27Quantum Technology Bridge: Generative Modelling and Hybrid Sampling

Practical implementation

  • 01Laboratory Theory: Toy Autoregressive Generator
  • 02Laboratory I: Autoregressive Generator from First Principles
  • 03Laboratory II: Variational Autoencoder
  • 04Laboratory III: Normalising Flow
  • 05Laboratory IV: Adversarial Training Dynamics
  • 06Laboratory V: Diffusion on a Low-Dimensional Distribution
  • 07Laboratory VI: Conditional Generation and Guidance
  • 08Laboratory VII: Multimodal Generation
  • 09Laboratory VIII: Generative Evaluation Harness

Tools and technologies

  • Linux VM/container
  • toy distributions first
  • scratch implementations
  • reproducible sampling/evaluation
  • framework implementation
  • technical viva

Project

End-to-end generative system + evaluation harness

Expected learning outcomes

  • 01Can formulate a generative distribution or conditional distribution.
  • 02Can derive autoregressive likelihood and sample from the resulting model.
  • 03Can implement a VAE or latent-variable system and interpret the ELBO.
  • 04Can implement an invertible transform and verify its density calculation.
  • 05Can diagnose adversarial instability and mode coverage.
  • 06Can derive and implement a simple diffusion/denoising system.
  • 07Can control generation with conditioning/guidance and measure trade-offs.
  • 08Can evaluate generated distributions using multiple complementary criteria.
  • 09Can reproduce a generative research claim with an auditable deviation log.

Assessment

  • 01Daily practice
  • 02Laboratories
  • 03Research reproduction
  • 04Integrated project
  • 05Technical viva
  • 06Engineering record

Source document: TUTOR Frontier Module 14 Generative Models