Hi 🤗 This repository contains all the code and exercise answers of the book Hands-On Generative AI with Transformers and Diffusion Models.
- Omar Sanseviero (X, LinkedIn, website): Omar Sanseviero was the Chief Llama Officer and Head of Platform and Community at Hugging Face, leading the developer advocacy engineering, on-device, and moonshot teams. Omar has extensive engineering experience working at Google in Google Assistant and TensorFlow Graphics. Omar’s work at Hugging Face was at the intersection of open source, product, research, and technical communities.
- Pedro Cuenca (X, LinkedIn): Pedro Cuenca is a machine learning engineer at Hugging Face working on diffusion software, models, and applications. He has 20+ years of software development experience in fields like internet applications and iOS. As a cofounder and CTO of LateNiteSoft, he worked on the technology behind Camera+, a successful iPhone app that used custom ML models for photography enhancement. He created deep-learning models for tasks such as photography enhancement and super-resolution. He was also involved in the development of and operations behind DALL·E mini. He brings a practical vision of integrating AI research into real-world services and the challenges and optimizations involved.
- Apolinario Passos (X, LinkedIn, website): Apolinario Passos is a machine learning art engineer at Hugging Face working across different teams on multiple machine learning for art and creativity use cases. Apolinario has 10+ years of professional and artistic experience, alternating between holding art exhibitions, coding, and product management, having been a head of product at World Data Lab. Apolinario aims to ensure that the ML ecosystem supports and makes sense for artistic use cases. He is also an artist that works with interactive installations using AI.
- Jonathan Whitaker (X, LinkedIn, website): Jonathan Whitaker is a data scientist and deep learning researcher focused on generative modeling. In addition to his R&D work at answer.ai, he focuses on sharing knowledge via the DataScienceCastnet YouTube channel and various free online resources he has created.
The book is available on:
To get the most out of this book, we recommend running the code examples as you read along. Experimenting with the code by making changes and exploring different scenarios will enhance your understanding. Working with transformers and diffusion models can be computationally intensive, so having access to a computer with an GPU is needed.
There are multiple online options that you can use, such as Google Colaboratory and Kaggle Notebooks. Most code should work on any Google Colab instance. We recommend you use GPU runtimes, which provide a T4 for free, specially for chapters with training loops.
There are many support utilities and helper functions used throughout the book. To access them, please install the genaibook package:
pip install genaibook
This will, in turn, install the libraries required to run transformers and diffusion models, along with PyTorch, Matplotlib, NumPy, and other essentials.
| Chapter | Colab |
|---|---|
| 1. An Introduction to Generative Media |  |
| 2. Transformers | |
| 3. Compressing and Representing Information | |
| 4. Diffusion Models | |
| 5. Stable Diffusion and Conditional Generation | |
| 6. Fine-Tuning Language Models | |
| 7. Fine-Tuning Stable Diffusion | |
| 8. Creative Applications of Text-To-Image Models | |
| 9. Generating Audio | |
| Appendix C. End-to-End Retrieval Augmented Generation | |