Table of Contents

Shortcuts

thunder.distributed.set_skip_data_parallel_grad_sync¶

thunder.distributed.set_skip_data_parallel_grad_sync(value)[source]¶

Set whether to skip data parallel grad sync.

Parameters:

value¶ (bool) – Whether to skip data parallel grad sync.
value (bool) –

Return type:

Token

Returns:

A token that can be used to restore the previous value.