Meta Puts Google AI Chips to Work in Production, Shaking Up the Inference Market

For the first time, Google’s custom artificial intelligence hardware is powering production-level AI workloads at Meta. This development signals a tangible shift in how major tech companies deploy their most demanding computing tasks, moving beyond the long-standing dominance of a single chip supplier. While the move primarily affects enterprise infrastructure, it sets the stage for broader changes in how AI models are served and optimized for end-users over the coming year.

The core of this change involves the release of new, PyTorch-native versions of two major inference engines: SGLang and vLLM. Previously, running these engines on Google’s Tensor Processing Units, or TPUs, required developers to use Google’s own JAX framework. The new software builds allow these engines to run natively on PyTorch, the industry-standard library for AI development, without requiring extensive code rewrites. This lowers the technical barrier for organizations that rely heavily on PyTorch but want the performance benefits of Google’s specialized hardware.

Meta’s involvement is direct and operational. Zongwei Zhou, Meta’s lead for PyTorch-TPU integration, confirmed that the SGLang-TorchTPU engine is currently running in production. He described the achievement as the result of a close three-way collaboration between Meta, Google, and the team behind SGLang. This team is based at RadixArk, a startup founded by the creators of SGLang, which developed the engine in partnership with the two giants. Zhou noted that the new stack provides a remarkably mature environment for serving AI models.

Jana van Greunen, director of PyTorch engineering at Meta, emphasized the strategic importance of hardware flexibility. She stated that Meta cannot be constrained by a single hardware architecture and that the new TorchTPU backend enables the company to deploy its PyTorch workloads directly onto Google TPUs. Google has confirmed that it co-developed this specific backend with Meta’s engineering team, highlighting a deepening technical alliance between the two companies.

This is not Meta’s first experiment with Google’s hardware, but it is its most significant implementation. Google identifies the older PyTorch/XLA bridge as a predecessor to the new system. Industry reports previously indicated that Meta’s research teams used TPUs via this older bridge between 2020 and 2023 before canceling those contracts. The current push appears specifically tailored to address Meta’s preference to remain within the PyTorch ecosystem rather than migrating to JAX.

The physical location of these production chips remains somewhat opaque, though reporting suggests a structured commercial relationship. Discussions held last year reportedly involved Google Cloud TPU rentals beginning in 2026, with potential direct purchases scheduled for 2027. Meta is also reported to have signed a multi-year rental agreement, suggesting a long-term commitment to this infrastructure rather than a one-off trial.

The timing of this announcement is notable, as it arrived one day after Amazon revealed that it was open-sourcing TorchNeuron, its own native PyTorch backend for its Trainium2 and Trainium3 chips. Both moves appear strategically aligned to loosen the software lock-in that has benefited Nvidia. By making non-Nvidia hardware more accessible to the PyTorch community, these companies aim to intensify competition in the AI infrastructure market.

Performance metrics suggest the new hardware is highly competitive. According to Inferact, the startup founded by vLLM’s creators, a tuned vLLM setup on the seventh-generation Ironwood TPU achieved over 700 tokens per second for a single user while serving Moonshot AI’s open-weight Kimi K3 model. This test utilized 16 Ironwood chips with speculative decoding. In comparison, the baseline setup on 16 Nvidia GB200s reached 452 tokens per second. For smaller batch sizes without speculative decoding, the new kernels reportedly deliver 1.4 to 2 times the throughput of the GB200 baseline.

Looking ahead, the ecosystem is set to expand further. SGLang-TorchTPU will begin adding support for new models, starting with DeepSeek V4.1 Flash and Reflection Beam. RadixArk has stated that the goal is to achieve Day 0 TPU support for new open models by the first quarter of 2027, ensuring that new releases are immediately compatible with the hardware. Google, Meta, and RadixArk are scheduled to share additional details at the SGLang Summit in San Francisco on November 12 and 13.