---
title: LitServe Tutorial
description: LitLogger can also be used in inference-serving workflows through LitServe.
canonical: 'https://lightning.ai/docs/litlogger/stable/tutorials/litserve'
---
# LitServe Tutorial {#litserve-tutorial}

LitLogger can also be used in inference-serving workflows through LitServe.

![Comparing inference experiments side by side](https://pl-bolts-doc-images.s3.us-east-2.amazonaws.com/litlogger/experiment_comparison_side_by_side.png)

## Endpoint Logging {#endpoint-logging}

The LitServe example initializes a logger in `setup()` and records metrics
for each request in `predict()`.

```python
import time

import litgpt
import litlogger
import litserve as ls

class SimpleLitAPI(ls.LitAPI):
    def setup(self, device):
        self.llm = litgpt.LLM.load("EleutherAI/pythia-14m")
        self.experiment = litlogger.init(
            name="litserve-inference",
            metadata={"model": "pythia-14m"},
        )

    def decode_request(self, request):
        return request["prompt"]

    def predict(self, prompt):
        start = time.perf_counter()
        output = self.llm.generate(prompt, max_new_tokens=200)
        self.experiment["generation_time"].append(time.perf_counter() - start)
        self.experiment["input_tokens"].append(self.llm.preprocessor.encode(prompt).numel())
        self.experiment["output_tokens"].append(self.llm.preprocessor.encode(output).numel())
        return {"output": output}

    def encode_response(self, output):
        return {"output": output}

    def teardown(self, devices):
        self.experiment.finalize()

if __name__ == "__main__":
    server = ls.LitServer(SimpleLitAPI())
    server.run(port=8000)
```

## What This Workflow Covers {#what-this-workflow-covers}

- Request-time metric logging
- Inference latency tracking
- Token-count or request-shape metrics
- Deployment-oriented usage outside training loops

## Related Docs {#related-docs}

- [Workflows](/docs/litlogger/2026.08.28/guides/workflows.md)
- [Examples](/docs/litlogger/2026.08.28/guides/examples.md)

---

**Navigation**

- Parent: [Tutorials](/docs/litlogger/2026.08.28/tutorials.md)