Accelerating the LLM Life Cycle on the Cloud

Agenda

• Part 1: Building agents with third-party LLMs

  In this part, we’ll build agents using public APIs from Open AI, Mistral, etc…

• Part 2: Build agents with private models

  In part 2, we’ll show how to build the same agents with private models running on your own infrastructure with your own data. We’ll fly through the full model development lifecycle from preparing data to deploying and optimizing a model API.

  Step 1: Data preparation at scale

  This step will show how to download, process and optimize a massive open source dataset for training at scale. This is often an overlooked step that can improve model training speeds by at least 20x.

  Step 2: Continued pretraining for LLMs

  Next, we’ll continue pretraining a model on the dataset we created. You’ll learn how this is done on multi-node across 16 H100 GPUs with the latest tricks for multi-node training with fault-tolerance and more.

  Step 3: Finetune an LLM

  Once we’ve pretrained our model, we’ll finetune it to align it to answer questions in a way that is tailored to your industry (finance). In this example we’ll bias the model to sound more medical

  Step 4: Deploy model API

  We’ll deploy our model behind a high-performance API that auto-scales and can be plugged back into the agents.

  Step 5: Profile high-performance deployment

  We’ll benchmark the API and use our custom built profiling tools (in Studios) to highlight code bottlenecks that can be optimized to increase throughput, lower latency and maximize tokens/s speed.

  Step 6: Pipeline

  We’ll end with showing how this whole process can be automated with your favorite pipeline/workflow manager and the Studios SDK