Table of Contents

Shortcuts

thunder.distributed.FSDPType¶

class thunder.distributed.FSDPType(value)[source]¶

Bases: Enum

Specifies the sharding strategy to be used for FSDP in Thunder.

ZERO2[source]¶: Similar to torch.distributed.fsdp.ShardingStrategy.SHARD_GRAD_OP.

ZERO3[source]¶: Similar to torch.distributed.fsdp.ShardingStrategy.FULL_SHARD.

__init__()[source]¶

Attributes

`ZERO2`
`ZERO3`