---
title: Glossary
description: >-
  - 2D Parallelism — Combine Tensor Parallelism with FSDP (2D Parallel) to train
  efficiently on 100s of GPUs - Accelerators — Accelerators connect the...
canonical: 'https://lightning.ai/docs/pytorch/stable/more/glossary'
---
# Glossary {#glossary}

- [2D Parallelism](/docs/pytorch/2.6.6/reference/advanced/model_parallel/tp_fsdp.md) — Combine Tensor Parallelism with FSDP (2D Parallel) to train efficiently on 100s of GPUs

- [Accelerators](/docs/pytorch/2.6.6/reference/extensions/accelerator.md) — Accelerators connect the Trainer to hardware to train faster

- [Callback](/docs/pytorch/2.6.6/reference/extensions/callbacks.md) — Add self-contained extra functionality during training execution

- [Checkpointing](/docs/pytorch/2.6.6/reference/common/checkpointing.md) — Save and load progress with checkpoints

- [Cluster](/docs/pytorch/2.6.6/reference/clouds/cluster.md) — Run on your own group of servers

- [Cloud checkpoint](/docs/pytorch/2.6.6/reference/common/checkpointing_advanced.md) — Save your models to cloud filesystems

- [Compile](/docs/pytorch/2.6.6/reference/advanced/compile.md) — Use torch.compile to speed up models on modern hardware

- [Console Logging](/docs/pytorch/2.6.6/reference/common/console_logs.md) — Capture more visible logs

- [Debugging](/docs/pytorch/2.6.6/reference/debug/debugging.md) — Fix errors in your code

- [DeepSpeed](/docs/pytorch/2.6.6/reference/advanced/model_parallel/deepspeed.md) — Distribute models with billions of parameters across hundreds GPUs

- [Distributed Checkpoints](/docs/pytorch/2.6.6/reference/common/checkpointing_expert.md) — Save and load very large models efficiently with distributed checkpoints

- [Early stopping](/docs/pytorch/2.6.6/reference/common/early_stopping.md) — Stop the training when no improvement is observed

- [Experiment manager (Logger)](/docs/pytorch/2.6.6/reference/visualize/experiment_managers.md) — Tools for tracking and visualizing artifacts and logs

- [Finetuning](/docs/pytorch/2.6.6/reference/advanced/finetuning.md) — Technique for training pretrained models

- [FSDP](/docs/pytorch/2.6.6/reference/advanced/model_parallel/fsdp.md) — Distribute models with billions of parameters across hundreds GPUs

- [GPU](/docs/pytorch/2.6.6/reference/accelerators/gpu.md) — Graphics Processing Unit for faster training

- [Half precision](/docs/pytorch/2.6.6/reference/common/precision.md) — Using different numerical formats to save memory and run faster

- [Hooks](/docs/pytorch/2.6.6/reference/common/hooks.md) — How to customize the training, validation, and testing logic

- [Inference](/docs/pytorch/2.6.6/reference/deploy/production_intermediate.md) — Making predictions by applying a trained model to unlabeled examples

- [Lightning CLI](/docs/pytorch/2.6.6/reference/cli/lightning_cli.md) — A Command-line Interface (CLI) to interact with Lightning code via a terminal

- [LightningDataModule](/docs/pytorch/2.6.6/reference/data/datamodule.md) — A shareable, reusable class that encapsulates all the steps needed to process data

- [LightningModule](/docs/pytorch/2.6.6/core-api/lightning_module.md) — A base class organizing your neural network module

- [Log](/docs/pytorch/2.6.6/reference/visualize/loggers.md) — Outputs or results used for visualization and tracking

- [Metrics](https://torchmetrics.readthedocs.io/en/stable/) — A statistic used to measure performance or other objectives we want to optimize

- [Model](/docs/pytorch/2.6.6/reference/model/build_model.md) — The set of parameters and structure for a system to make predictions

- [Model Parallelism](/docs/pytorch/2.6.6/reference/advanced/model_parallel.md) — A way to scale training that splits a model between multiple devices.

- [Plugins](/docs/pytorch/2.6.6/reference/extensions/plugins.md) — Custom trainer integrations such as custom precision, checkpointing or cluster environment implementation

- [Progress bar](/docs/pytorch/2.6.6/reference/common/progress_bar.md) — Output printed to the terminal to visualize the progression of training

- [Production](/docs/pytorch/2.6.6/reference/deploy/production_advanced.md) — Using ML models in real world systems

- [Prediction](/docs/pytorch/2.6.6/reference/deploy/production_basic.md) — Computing a model's output

- [Pretrained models](/docs/pytorch/2.6.6/reference/advanced/pretrained.md) — Models that have already been trained for a particular task

- [Profiler](/docs/pytorch/2.6.6/reference/tuning/profiler.md) — Tool to identify bottlenecks and performance of different parts of a model

- [Pruning](/docs/pytorch/2.6.6/reference/advanced/pruning_quantization.md) — A technique to eliminae some of the model weights to reduce the model size and decrease inference requirements

- [Quantization](/docs/pytorch/2.6.6/reference/advanced/post_training_quantization.md) — A technique to accelerate the model inference speed and decrease the memory load while still maintaining the model accuracy

- [Remote filesystem and FSSPEC](/docs/pytorch/2.6.6/reference/common/remote_fs.md) — Accessing files from cloud storage providers

- [Strategy](/docs/pytorch/2.6.6/reference/extensions/strategy.md) — Ways the trainer controls the model distribution across training, evaluation, and prediction

- [Strategy registry](/docs/pytorch/2.6.6/reference/advanced/strategy_registry.md) — A class that holds information about training strategies and allows adding new custom strategies

- [Style guide](/docs/pytorch/2.6.6/reference/starter/style_guide.md) — Best practices to improve readability and reproducibility

- [SLURM](/docs/pytorch/2.6.6/reference/clouds/cluster_advanced.md) — Simple Linux Utility for Resource Management, or simply Slurm, is a free and open-source job scheduler for Linux clusters

- [Tensor Parallelism](/docs/pytorch/2.6.6/advanced/tp.md) — Parallelize the computation of model layers across multiple GPUs, reducing memory usage and communication overhead

- [Transfer learning](/docs/pytorch/2.6.6/reference/advanced/transfer_learning.md) — Using pre-trained models to improve learning

- [Trainer](/docs/pytorch/2.6.6/core-api/trainer.md) — The class that automates and customizes model training

- [Torch distributed](/docs/pytorch/2.6.6/reference/clouds/cluster_intermediate_2.md) — Setup for running on distributed environments

- [Warnings](/docs/pytorch/2.6.6/reference/advanced/warnings.md) — Disable false-positive warnings emitted by Lightning

- [Weight averaging](/docs/pytorch/2.6.6/reference/advanced/training_tricks.md#weight-averaging) — Stochastic Weight Averaging (SWA) or Exponential Moving Average (EMA) can make your models generalize better

---

**Navigation**

- Parent: [More](/docs/pytorch/2.6.6/more.md)