Nvidia aims for the throne: The AI semiconductor war and game-changing Google TPUs

google's tensor processing units (TPUs), which have been shaking up the stock markets at home and abroad, have emerged as a new player in the artificial intelligence (AI) infrastructure market. with Google's recently unveiled next-generation AI model, Gemini 3.0, being praised for outperforming OpenAI's ChatGPT series, the world's attention is focused on the TPUs used to develop it. google's self-developed ASIC semiconductors (TPUs) have made a strong challenge to the AI accelerator market, which has been dominated by Nvidia's graphics processing units (GPUs) for more than 95% of the market.

this seismic shift is more than just a technology race; it's the crystallization of big tech's "go Nvidia" strategy, which centers on vertical integration and supply chain control. We take a deep dive into the rise of TPUs, the future of the AI semiconductor wars, and the implications of this upheaval for South Korean memory semiconductor companies.

GPUs are general-purpose SUVs, TPUs are sports cars on a dedicated circuit

Even if you don't know much about AI or computing, you've probably heard of Nvidia's Graphics Processing Units (GPUs), which were originally developed for image processing in games, video editing, and other applications, but have become the king of the AI era because of their ability to process thousands of operations in parallel, making them efficient for deep learning training. GPUs' biggest strengths are their versatilityandthe robustCUDA software ecosystemthat Nvidia has built. in the early stages of research and development and for smaller systems, the versatile GPU is still the best choice.

on the other hand, the Tensor Processing Unit (TPU), which Google spent more than a decade developing with Broadcom, was born different. TPUsare application-specific integrated circuit (ASIC) semiconductorsdesigned solely for accelerating deep learning computations. If a GPU is a general-purpose SUV with lots of performance, a TPU is like a customized sports car designed for maximum speed and electrical efficiency on a dedicated circuit.

a key driver for Google's focus on TPU developmentis optimizing total cost of ownership (TCO). when training and powering large-scale AI services like Gemini that are used by millions of people (the inference phase), using GPUs with a poor power-to-performance ratio would skyrocket costs. Considering that more than 90% of the total cost is incurred in the AI model execution and deployment (inference) phase, TPUs providethe highest speed and power efficiency, rather than general purpose, which dramatically increases the economic competitiveness of Google Cloud and its AI services.

a shadow on Nvidia's throne: TPU's technological leap forward

the overwhelming performance improvementsof thelatest generation,TPU v6e, aka Trillium, have helped Google TPUs become more than just an experimental chip, but a contender to GPUs.

Compared to its predecessor, v5e, Trillium delivers a roughly 4.6x increase in peak compute performance per chip, and most notably, a doubling of both HBM capacity and bandwidth, as well as inter-chip interconnect (ICI) bandwidth. data center network bandwidth per pod also increased by a factor of 4. These dramatic performance gains show that TPUs are designed to be overwhelmingly efficient inlarge-scale distributed computingenvironments, where thousands of chips are stitched together tocreateone giant supercomputer. when training a large language model (LLM), the speed of data communication between chips drives overall performance, and Trillium has eliminated this bottleneck to maximize scaling efficiency.

the 'post-Nvidia' era: big tech's multi-platform strategy

google's success has sent a strong signal to Nvidia's major customers. the more Nvidia dominates the AI chip market, the more GPU scarcity and high prices expose big techs to cost and supply chain risks. as a result, major Big Techs such as Amazon (AWS), Meta, and Microsoft have begun developing their own ASICs (AWS' Trinium, Meta's MTIA) or are considering adopting Google TPUs to diversify their supply chains.

the fact that Meta is considering a multi-billion dollar deployment of Google TPUs in its data centers in 2027, and AI startup Anthropic has announced plans to leverage up to 1 million TPUs, signals the beginning of a structural shift in the market.

these moves go beyond simply changing chips and reflect a fundamental shift in strategy for AI infrastructure. instead of choosing between GPUs and TPUs, big tech is moving to a "hybrid architecture" that allows them to selectively use the most efficient chip. a prime example of this is how Meta is preparing to enable AI models to run on different hardware, including CUDA (Nvidia), MTIA, and TPUs, through frameworks like PyTorch. This is breaking down the walls of proprietary software ecosystems (CUDA) and shifting the AI accelerator marketfrom a unipolar to a multipolar competition.

stock market reaction and new opportunities for Korean memory semiconductors

This tectonic shift in the AI industry was immediately reflected in the stock market. In the immediate aftermath of the TPU issue, shares of Broadcom, which co-developed the TPU with Alphabet (Google), surged, while Nvidia's stock price declined in the short term, suggesting that investors are increasingly wary of AI chip monopolies and value the commercial viability of ASICs.

it is South Koreanmemory semiconductor companies that areexpected to benefit the most. The rise of TPUs is expected to drive explosive growth in demand for memory semiconductors, including high-bandwidth memory (HBM).

TPUs, like GPUs, require 6-8 HBMs per chip, so rather than cannibalizing the GPU market, the growth of the TPU market will be an "additional demand source" that will drive up memory demand for the entire AI accelerator market. This will diversify the HBM supply risk that has been concentrated on Nvidia (GPUs) and provide a new growth axis for HBM technology leaders such as Samsung Electronics and SK Hynix.

in addition, the price of general-purpose D-RAM (DDR5) is continuously rising as the production of D-RAM is reduced to produce HBMs. In other words, the intensification of AI accelerator competition is creating structural favorable conditions to drive up prices across the memory semiconductor market, including D-RAM and NAND flash, beyond HBMs. This is the reason why domestic semiconductor stocks are rising.

conclusion: AI semiconductor second-act competition

while Nvidia will still reign supreme with its unrivaled technology and ecosystem, the success of Google's TPUs has officially declared a "multi-polar playing field" in the AI infrastructure market. We are entering an era where GPUs will be the general-purpose platform for developing new AI models and extensive experimentation, while TPUs and proprietary ASICs will be the dedicated engines for optimizing the TCO of large-scale learning and inference services.

in this chip war, Korean memory semiconductor companies can maximize their strategic value not as direct participants in the AI chip design race, butas neutral core providers that are essential to any AI chip. In this turbulent era driven by the ubiquity of AI technology and the expansion of the infrastructure market, strengthening HBM technology and packaging capabilities will be a long-term game changer for Korean companies. The battle for the AI infrastructure throne has just begun its second act.