Lectures

You can download the lectures here. We will try to upload lectures prior to their corresponding classes.

• 

  Lecture 1: Introduction to diffusion models
  [slides] [video] [panopto] [feedback]
  

  • Applications of diffusion models
  • What are diffusion models?
  • How to train and sample from diffusion models from scratch

  Suggested Readings:

  • Diffusion models from scratch
  • smalldiffusion codebase
  • Elucidating the Design Space of Diffusion-Based Generative Models
  • Analyzing and Improving the Training Dynamics of Diffusion Models
• 

  Lecture 2: Perspectives on diffusion
  [slides] [video] [panopto] [feedback]
  

  • Introduce different interpretations and perspectives on diffusion
  • Derivation of DDPM and DDIM, SDE reversal and probability flow ODE
  • Autoregressive interpretation

  Suggested Readings:

  • What are diffusion models?
  • SDE interpretation blog post
  • Step-by-step diffusion: an elementary tutorial
• 

  Lecture 3: Conditioning and guidance
  [slides] [video] [panopto] [feedback]
  

  • Conditional diffusion training and inference
  • Classifier and classifier-free guidance

  Suggested Readings:

  • Guidance: a cheat code for diffusion models
  • Classifier-free guidance paper
  • Latent diffusion models
  • Guiding a diffusion model with a bad version of itself
• 

  Lecture 4: Generalization in diffusion models
  [slides] [video] [panopto] [feedback]
  

  • What is generalization, how and when do diffusion models generalize?
  • Different inductive biases during training affect generalization

  Suggested Readings:

  • Generalization in diffusion models arises from geometry-adaptive harmonic representations
  • Closed-Form Diffusion Models
  • Nuclear Norm Regularization for Deep Learning
  • An analytic theory of creativity in convolutional diffusion models
• 

  Lecture 5: Score distillation and advanced applications
  [slides] [video] [panopto] [feedback]
  

  • What is score distillation sampling (SDS)?
  • Derivation through score matching and KL divergence
  • Improved 3D shape generation with DDIM inversion and SDS

  Suggested Readings:

  • DreamFusion: Text-to-3D using 2D Diffusion
  • ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillation
  • Score Distillation via Reparametrized DDIM
  • Differentiable 3D representations
• 

  Lecture 6: Additional topics, summary and conclusion
  [slides] [video] [panopto] [feedback]
  

  • How to train better and focus on noise levels that matter most
  • How to sample faster using distillation and consistency models
  • Interpreting diffusion as distance minimization

  Suggested Readings:

  • Diffusion is spectral autoregression
  • Noise schedules considered harmful
  • The paradox of diffusion distillation
  • Interpreting and improving diffusion models from an optimization perspective