6.S183: A Practical Introduction to Diffusion Models / IAP 2025
Updates
-
Edited lecture videos are available on Youtube!
Course Description
Diffusion models are a class of generative models that iteratively refine noise into structured data. Although initially developed for image generation, they have been successful in many other domains such as robotics and molecular design. In this course we will introduce the basics of diffusion models and demonstrate how to build them from the ground up, culminating in a simple but powerful library to train diffusion models on custom data, as well as using state-of-the-art pretrained models for a variety of downstream tasks.
This is an introductory course targeted at students and researchers who wish to learn about diffusion models and explore their applications to new domains, or those currently working with diffusion models and want to understand how to effectively modify and adapt them for their specific applications.
Course Objectives
In the problem sets you will learn how to:
| Visualize diffusion sampling | Create visual illusions | Generate interpolations |
|---|---|---|
 |
 |
 |
After this course, you should be able to:
- Know what is a diffusion model, what are its components, and how they work together
- Understand the different perspectives and theoretical underpinnings of diffusion models
- Control the sampling process of pretrained diffusion models and use them for downstream tasks such as conditional generation, image restoration, and 3D model generation
- Design and train diffusion models on custom datasets
Prerequisites
Python programming, basic linear algebra, probability and mathematical maturity (e.g. 18.06, 18.02, 6.042). Experience with PyTorch will be helpful.
Grading
Grading with be P/D/F, 18 points and above required to receive credit.
- Attendance/participation (6 points)
- Problem Sets (12 points)
- Mini Project (4 points)
Problem Sets
There will be 2 short problem sets, assigned after lectures 1 and 3 and due in approximately 1 week. Each problem set will have a few theoretical/math questions, as well as a coding component. Most of the coding questions will not require GPUs. For any questions that require GPUs, we will provide a notebook that can be run on the free tier of Google Colab. The purpose of the problem sets is to help students check their own understanding and give them practice working with diffusion models. Grading will be coarse and mostly be based on completion.
Mini Project
During the second week students will have the opportunity to complete a mini project and apply what they have learnt about diffusion models to a problem of their interest. Potential projects include new applications using pretrained models or training diffusion models on custom datasets.
6.S184/6.S975
In the second half of IAP there will be another class “Generative AI with Stochastic Differential Equations - Theory and Practice of Flow and Diffusion Models” focusing on diffusion models through the lens of SDEs. Students who are interested in a deeper dive into the mathematics behind diffusion models should also consider taking this class.
