https://pytorch.org/blog/introducing-accelerated-pytorch-training-on-mac/

* * *
 

  * Get Started
  * Ecosystem
    Tools

    Learn about the tools and frameworks in the PyTorch Ecosystem

    Ecosystem Day - 2021

    See the posters presented at ecosystem day 2021

    Developer Day - 2021

    See the posters presented at developer day 2021

  * Mobile
  * Blog
  * Tutorials
  * Docs
    PyTorch

    torchaudio

    torchtext

    torchvision

    TorchData

    TorchRec

    TorchServe

    PyTorch on XLA Devices

  * Resources
    About

    Learn about PyTorch's features and capabilities

    Community

    Join the PyTorch developer community to contribute, learn, and
    get your questions answered.

    Community stories

    Learn how our community solves real, everyday machine learning
    problems with PyTorch

    Developer Resources

    Find resources and get questions answered

    Events

    Find events, webinars, and podcasts

    Forums

    A place to discuss PyTorch code, issues, install, research

    Models (Beta)

    Discover, publish, and reuse pre-trained models

  * GitHub
  * 
    [                    ]
    X

 

May 18, 2022

Introducing Accelerated PyTorch Training on Mac

[logo-icon]

by PyTorch

In collaboration with the Metal engineering team at Apple, we are
excited to announce support for GPU-accelerated PyTorch training on
Mac. Until now, PyTorch training on Mac only leveraged the CPU, but
with the upcoming PyTorch v1.12 release, developers and researchers
can take advantage of Apple silicon GPUs for significantly faster
model training. This unlocks the ability to perform machine learning
workflows like prototyping and fine-tuning locally, right on Mac.

        [intro-graphic-accelerated-pytorch-training-revised]

Metal Acceleration

Accelerated GPU training is enabled using Apple's Metal Performance
Shaders (MPS) as a backend for PyTorch. The MPS backend extends the
PyTorch framework, providing scripts and capabilities to set up and
run operations on Mac. MPS optimizes compute performance with kernels
that are fine-tuned for the unique characteristics of each Metal GPU
family. The new device maps machine learning computational graphs and
primitives on the MPS Graph framework and tuned kernels provided by
MPS.

Training Benefits on Apple Silicon

Every Apple silicon Mac has a unified memory architecture, providing
the GPU with direct access to the full memory store. This makes Mac a
great platform for machine learning, enabling users to train larger
networks or batch sizes locally. This reduces costs associated with
cloud-based development or the need for additional local GPUs. The
Unified Memory architecture also reduces data retrieval latency,
improving end-to-end performance.

In the graphs below, you can see the performance speedup from
accelerated GPU training and evaluation compared to the CPU baseline:

                      [METAPT-002-BarGraph-02]

Getting Started

To get started, just install the latest Preview (Nightly) build on
your Apple silicon Mac running macOS 12.3 or later with a native
version (arm64) of Python.

You can also learn more about Metal and MPS on Apple's Metal page.

* Testing conducted by Apple in April 2022 using production Mac
Studio systems with Apple M1 Ultra, 20-core CPU, 64-core GPU 128GB of
RAM, and 2TB SSD. Tested with macOS Monterey 12.3, prerelease PyTorch
1.12, ResNet50 (batch size=128), HuggingFace BERT (batch size=64),
and VGG16 (batch size=64). Performance tests are conducted using
specific computer systems and reflect the approximate performance of
Mac Studio.

Docs

Access comprehensive developer documentation for PyTorch

View Docs

Tutorials

Get in-depth tutorials for beginners and advanced developers

View Tutorials

Resources

Find development resources and get your questions answered

View Resources
 

  * PyTorch
  * Get Started
  * Features
  * Ecosystem
  * Blog
  * Contributing

  * Resources
  * Tutorials
  * Docs
  * Discuss
  * GitHub Issues
  * Brand Guidelines

  * Stay up to date

  * Facebook
  * Twitter
  * YouTube
  * LinkedIn

  * PyTorch Podcasts

  * Spotify
  * Apple
  * Google
  * Amazon

  * Terms
  * |
  * Privacy

*
  

  * [                    ]
  * Get Started
  * Ecosystem
      + Tools
      + Ecosystem Day 2021
      + Developer Day 2021
  * Mobile
  * Blog
  * Tutorials
  * Docs
      + PyTorch
      + torchaudio
      + torchtext
      + torchvision
      + TorchElastic
      + TorchServe
      + PyTorch on XLA Devices
  * Resources
      + About
      + Community
      + Community stories
      + Developer Resources
      + Events
      + Forum
      + Models (Beta)
  * GitHub

To analyze traffic and optimize your experience, we serve cookies on
this site. By clicking or navigating, you agree to allow our usage of
cookies. As the current maintainers of this site, Facebook's Cookies
Policy applies. Learn more, including about available controls:
Cookies Policy.

[pytorch-x]