PAI-TorchAcc

TorchAcc

  • Introduction
  • Installation
  • Quick Start

Functionality

  • Data Parallel
  • FSDP (Fully Sharded Data Parallel)
  • Data Bucketing

Tutorials

  • Flash Models
  • HuggingFace Transformers

Best Practices

  • Best Practices

API Reference

  • torchacc
    • torchacc package
      • Submodules
      • torchacc.accelerate module
      • torchacc.config module
      • Module contents
      • Subpackages
        • torchacc.core package
        • torchacc.dist package
        • torchacc.llm package
        • torchacc.ops package
        • torchacc.utils package
PAI-TorchAcc
  • torchacc
  • torchacc package
  • torchacc.ops package
  • View page source

torchacc.ops package

Subpackages

  • torchacc.ops.context_parallel package
    • Submodules
    • torchacc.ops.context_parallel.context_parallel_2d module
    • torchacc.ops.context_parallel.init_group module
    • torchacc.ops.context_parallel.ring_attn module
    • torchacc.ops.context_parallel.ulysses module
    • torchacc.ops.context_parallel.utils module
    • Module contents

Submodules

torchacc.ops.flash_attn module

torchacc.ops.scaled_dot_product_attention module

Module contents

Previous Next

© Copyright 2024, Alibaba Cloud.

Built with Sphinx using a theme provided by Read the Docs.