Lightning AI
How to use xla_fsdp?
Fabric
yihuihe
September 15, 2023, 7:23am
1
could you give an example how xla_fsdp works on tpu? I couldn’t get it work. loss does not decrease
Related topics
Topic
Replies
Views
Activity
What is the batch size for distributed training fsdp?
0
164
April 23, 2024
FSDP for both pretrained teacher and trainable student
DDP/GPU
4
1083
April 18, 2023
Why `num_replica` != `world_size`?
DDP/GPU
0
101
May 22, 2024
Minimal fabric project with config file, resume and experiment tracker?
implementation help
2
397
December 25, 2023
Saving FSDP model with custom FSDPStrategy results in TypeError: cannot pickle 'module' object
1
267
March 22, 2024