TorchAcc

TorchAcc

  • Introduction
  • Installation
  • Quick Start

Functionality

  • Data Parallel
  • FSDP (Fully Sharded Data Parallel)
  • Data Bucketing

Tutorials

  • Flash Models
  • HuggingFace Transformers

Best Practices

  • Best Practices

API Reference

  • torchacc

CONTRIBUTING

  • Contribute To TorchAcc
TorchAcc
  • Welcome to TorchAcc’s documentation!
  • View page source

Welcome to TorchAcc’s documentation!

TorchAcc

  • Introduction
    • Main Features
    • Model Performance
  • Installation
    • Docker images
    • Building from Source
  • Quick Start
    • Torch Native Task
    • Single GPU Acceleration with TorchAcc
    • Multiple GPUs Acceleration with TorchAcc

Functionality

  • Data Parallel
    • Torch Native Task
    • Single GPU Acceleration with TorchAcc
    • Data Parallel
    • Auto Mixed Precision (AMP)
  • FSDP (Fully Sharded Data Parallel)
    • Torch Native Task
    • FSDP
    • Checkpoint Save/Load
    • Configurable parameters
  • Data Bucketing
    • Introduction
    • How to use

Tutorials

  • Flash Models
  • HuggingFace Transformers
    • Environment Preparation
    • PyTorch Native Training
    • DeepSpeed Training
    • TorchAcc Training
    • Performance

Best Practices

  • Best Practices
    • Minimize the Calls to sync
    • Prefer AsyncLoader
    • Avoid Evaluating Tensors
    • Coordinate Gradient Accumulation with sync and AsyncLoader
    • Model Saving

API Reference

  • torchacc
    • torchacc package
      • Submodules
      • torchacc.accelerate module
      • torchacc.config module
      • Module contents
      • Subpackages

CONTRIBUTING

  • Contribute To TorchAcc
    • Building from source

Indices and tables

  • Index

  • Module Index

  • Search Page

Next

© Copyright 2024, Alibaba Cloud.

Built with Sphinx using a theme provided by Read the Docs.