vLLM and others

Lightning AI supports any inference engine like vLLM, ollama, SGLang and more.

To deploy one of these engines, use Lightning Deploy, enter the container name, and hit deploy. Here's an example where we deploy ollama.