thunder.recipes.HFTransformers

class thunder.recipes.HFTransformers(show_progress=False, interpreter='thunder.jit', plugins=None)

Bases: BaseRecipe

Recipe tuned for Hugging Face transformers models.

Parameters:

• show_progress (bool, optional) – Forwarded to BaseRecipe.

• interpreter (str, optional) – Thunder interpreter to use.

• plugins (Iterable | None, optional) – Extra Thunder plugins.

__init__(show_progress=False, interpreter='thunder.jit', plugins=None)

Methods

__init__([show_progress, interpreter, plugins])
add_plugins(plugins)
apply(model)	Apply the recipe (compile the model) and patch generate / _sample so they work after tracing.
get_for_model(model)
register(key)
setup_config()	Enable NV-kernelised linear, matmul and SDPA ops on top of the base recipe’s debug configuration.
setup_executors()	Resolves executor names to concrete Executor objects.
setup_fuser()	Reconciles the requested fusion backend with self.executor_names.
setup_lookasides()	Swap out the warning helper when running under the non Thunder-FX interpreter.
setup_transforms()	Prepend the InplaceIndexCopyTransform to the default transform list.
validate(model)	Emit warnings (or errors) if model falls outside the supported transformer versions or base classes.

apply(model)

Apply the recipe (compile the model) and patch generate / _sample so they work after tracing.

Parameters:

model (transformers.PreTrainedModel) – The model to compile.

Returns:

Thunder-compiled model ready for inference.

Return type:

transformers.PreTrainedModel

setup_config()

Enable NV-kernelised linear, matmul and SDPA ops on top of the base recipe’s debug configuration.

Returns:

Thunder config dictionary augmented with nv_enable_* flags.

Return type:

dict[str, Any]

setup_lookasides()

Swap out the warning helper when running under the non Thunder-FX interpreter.

Returns:

list[thunder.core.recipe.Lookaside] | None

setup_transforms()

Prepend the InplaceIndexCopyTransform to the default transform list.

Returns:

transform list.

Return type:

list[thunder.Transform]

classmethod validate(model)

Emit warnings (or errors) if model falls outside the supported transformer versions or base classes.

Parameters:

model (transformers.PreTrainedModel) – Model instance to vet.

Raises:

ValueError – If model is not a PreTrainedModel.