{"id":5646695,"date":"2022-10-12T10:21:14","date_gmt":"2022-10-12T14:21:14","guid":{"rendered":"https:\/\/lightning.ai\/pages\/?p=5646695"},"modified":"2023-03-07T17:24:35","modified_gmt":"2023-03-07T22:24:35","slug":"deploy-diffusion-models","status":"publish","type":"post","link":"https:\/\/lightning.ai\/pages\/community\/tutorial\/deploy-diffusion-models\/","title":{"rendered":"How to Deploy Diffusion Models"},"content":{"rendered":"

Diffusion Models in Production<\/h2>\n

tl;dr<\/em><\/span> In this tutorial, you\u2019ll learn how to deploy diffusion models at scale and build a text-to-image generator<\/h3>\n

Today, we released Muse<\/span><\/strong><\/a>: an entirely open-source text-to-image generator built on Stable Diffusion, and a blueprint for building intelligent cloud applications based on diffusion models. Give it a try!<\/strong><\/span><\/a><\/p>\n

If you\u2019re a part of the AI community, then chances are you\u2019ve probably seen the dizzying speed at which diffusion models (and their applications) are being developed. Diffusion models \u2014 a subcategory of deep generative models \u2014 are made up of forward and reverse diffusion stages to generate data similar to the data on which they\u2019re trained. At Lightning, we provide a way to deploy diffusion models at scale, which we cover below.<\/p>\n

A recent (and open-source) example of a diffusion model is Stability AI\u2019s Stable Diffusion<\/a><\/span><\/strong>, a text-to-image model that produces high-fidelity images by using 10 GB of VRAM on consumer GPUs to generate a 512x512px image in just a few seconds.<\/p>\n

These models are groundbreaking, but deploying them continues to be a bottleneck. Below, we show you how we built a React UI as well as a model server with FastAPI in order to deploy these models easily and quickly with a Lightning App.<\/p>\n

In the following sections, we discuss the process of building our first version of Muse, the challenges we faced, and the steps we took to solve them.<\/p>\n

<\/h2>\n

\u00b7 \u00b7 \u00b7<\/h2>\n

\"\"<\/a><\/h2>\n

\u00b7 \u00b7 \u00b7<\/h2>\n

Deploying Stable Diffusion with FastAPI and LightningApp<\/h2>\n

We implemented a StableDiffusionServe<\/code><\/span>\u00a0class as a <\/span>Lightning Work<\/span><\/a><\/strong><\/span>. \"tooltipWork is a building block for long-running jobs. Use it to download data or train and serve a model.<\/span><\/span><\/span> <\/span><\/p>\n

This Work downloads the Stable Diffusion checkpoint, loads the model, and defines a REST endpoint that returns a base64 image based on the given prompt<\/span>:<\/p>\n