modalities.training package

Subpackages

• modalities.training.gradient_clipping package
  • Submodules
  • modalities.training.gradient_clipping.fsdp_gradient_clipper module
  • modalities.training.gradient_clipping.fsdp_gradient_clipper_config module
  • modalities.training.gradient_clipping.gradient_clipper module
  • Module contents

Submodules

modalities.training.activation_checkpointing module

modalities.training.training_progress module

class modalities.training.training_progress.TrainingProgress(num_seen_steps_current_run, num_seen_tokens_current_run, num_target_steps, num_target_tokens, num_seen_steps_previous_run=0, num_seen_tokens_previous_run=0)

Bases: object

Dataclass to store the training progress.

Attributes:

num_seen_steps_current_run (int): Number of seen steps in the current run. num_seen_tokens_current_run (int): Number of seen tokens in the current run. num_target_steps (int): Target number of steps. num_target_tokens (int): Target number of tokens. num_seen_steps_previous_run (Optional[int]): Number of seen steps in the previous run. num_seen_tokens_previous_run (Optional[int]): Number of seen tokens in the previous run.

Parameters:

• num_seen_steps_current_run (int)

• num_seen_tokens_current_run (int)

• num_target_steps (int)

• num_target_tokens (int)

• num_seen_steps_previous_run (int | None)

• num_seen_tokens_previous_run (int | None)

num_seen_steps_current_run: int

num_seen_steps_previous_run: Optional[int] = 0

property num_seen_steps_total: int

num_seen_tokens_current_run: int

num_seen_tokens_previous_run: Optional[int] = 0

property num_seen_tokens_total: int

num_target_steps: int

num_target_tokens: int

Module contents