PyTorch 2.15 Brings Massive AI Performance Boost to Apple Macs

The PyTorch team at Hugging Face has released PyTorch 2.15, focusing heavily on enhancing the Metal Performance Shaders (MPS) backend. The primary goal of this massive update is to establish Apple Silicon Macs as a first-class, high-performance platform for running and training complex AI models, including those used for language, image, and video generation. Historically, running advanced models on MPS presented significant hurdles, often involving unsupported operations, performance bottlenecks, and unpredictable memory consumption. This new version addresses these limitations by replacing the general-purpose MPSGraph with highly optimized, dedicated Metal kernels, a process achieved through deep collaboration with the core PyTorch maintainers and the broader community.

One of the most impactful additions is the native support for FP8 (float8_e4m3fn) data types on MPS. FP8 allows developers to store model weights and activations using only one byte per value, effectively halving the memory footprint compared to BF16 or FP16. This memory efficiency is crucial for running large models locally on consumer hardware, and PyTorch 2.15 enables the use of these tensors with the highly optimized F.scaled_mm function, which performs scaled matrix multiplication directly on the GPU, maximizing the utilization of the Mac's dedicated hardware.

Performance gains are most dramatically visible in the realm of Large Language Models (LLMs) and Mixture-of-Experts (MoE) architectures. For MoE models, such as Qwen3.6-35B-A3B, the speedup is staggering, achieving a 5–6.5 times faster decoding rate compared to version 2.14 on an M3 Ultra chip. This massive acceleration is attributed to the implementation of a native grouped matrix multiplication (grouped matmul) kernel. Unlike previous versions, which might have required copying group offsets to the CPU and looping over groups, the new Metal kernels read these offsets directly on the GPU, significantly streamlining the decoding process. Furthermore, even dense LLMs benefit from specialized optimizations, including dedicated matrix-vector kernels for single-row linear layers and an impressive 8-fold speed increase for the softmax calculation over the vocabulary, which is critical for token sampling.

Beyond LLM decoding, PyTorch 2.15 delivers substantial improvements across fundamental linear algebra operations. The framework now offers faster implementations for complex mathematical solves, including LU and triangular solves, as well as faster small-matrix inversions and QR decomposition. These foundational improvements ensure that the underlying mathematical engine powering AI models is robust and highly efficient. The update also expands the scope of optimized operations to include video and audio processing, providing faster paths for convolutions, padding, dropout, and interpolation backward passes, which are common components in generative media pipelines.

Perhaps the most notable practical fix for generative media is the memory management for video decoding. Previously, decoding a video using Wan2.2's tiled VAE could push the driver memory usage to an astronomical 270 GiB. With PyTorch 2.15, this memory peak is drastically reduced to approximately 8 GiB, making high-resolution video generation feasible on consumer-grade Macs. Furthermore, the update addresses critical bugs in image editing workflows. Specifically, the Qwen-Image-2.1 VAE encoder, which previously corrupted images on MPS (as reported in tools like ComfyUI) due to padding issues, now functions correctly. An encode-decode round trip now achieves a PSNR of 29.95 dB, matching the CPU performance, a massive improvement from the previous 10.06 dB.

From a developer control perspective, the release provides public MPS streams and more reliable event timing, giving developers finer-grained control over the execution flow of their models. The architecture of the decoding phase—which involves both a prefill phase and a decode phase—has been optimized by connecting the general F.linear function directly to the specialized GEMV kernels, ensuring that the common single-row decoding input utilizes the fastest available path, leading to a geometric-mean speedup of 1.23× across various layer shapes. The implementation of grouped matmul is a prime example of this optimization, ensuring that uneven groups of tokens are handled efficiently in a single, GPU-native operation.

In terms of compatibility, users must ensure they are running macOS 15 or later and Python 3.11 or later to take advantage of these features. Looking ahead, the team has already signaled improvements for PyTorch 2.16, which promises to move attention mechanisms fully onto Metal and continue the migration of MPSGraph operations into dedicated, high-speed Metal paths, promising even greater performance gains for LLM decoding and overall model execution on Apple hardware. In summary, PyTorch 2.15 represents a foundational shift, transforming the Mac into a powerful, memory-efficient, and highly optimized platform for state-of-the-art AI workloads.