"AI's Next Bottleneck Is the Wire Between the Chips — and Optics Are the Fix"

For the past two years the AI infrastructure story has been almost entirely about compute — how many GPUs, how much memory, how fast the next chip is. But a smaller, less glamorous layer is quietly becoming the thing that decides whether a cluster actually feels fast: the network between the chips. This month, Oxford-based photonics startup Salience Labs put a concrete marker on that shift, launching an all-optical 32-port switch built specifically for AI data centers, with 64-port and 128-port versions already on the roadmap.

To understand why this matters, you have to know how most AI clusters are wired today. A huge fraction of data-center networking still runs on a pattern called optical-electrical-optical, or OEO: a signal travels as light over fiber, gets converted to electrons so a chip can read and route it, then gets converted back into light to continue. Every one of those conversions costs time and energy, and at the scale of an AI cluster those costs compound into something structural. DataCenter News reports the switch is designed to replace that conversion in switching scenarios, routing signals entirely in the optical domain.

The numbers Salience Labs is putting forward are the interesting part. The switch has been tested at data rates up to 200G using 100 Gbaud PAM4 encoding, and it fits within a fraction of a rack unit — a real consideration when operators are giving more and more rack space over to compute and cooling. On performance, the company claims it can deliver up to an 80% improvement in "tokens per second per user," and on power, savings of up to 8× versus current OEO switching approaches, in part by eliminating transceivers that electronic packet switching would otherwise require.

That "tokens per second per user" phrasing is worth pausing on, because it signals a genuine change in what the industry is optimizing for. Raw throughput — bits moved per second — has been the default metric for networking forever. But an AI assistant's quality is measured in responsiveness: does the answer arrive in a way that feels conversational? That's a tail-latency problem, not a bandwidth problem, and tail latency is precisely what optical circuit switching claims to attack. The metric tells you the goalposts have moved from "move more data" to "make the model feel instant."

There's also a quieter historical echo here that I think gets under-covered. Telecommunications spent the 1990s and 2000s abandoning circuit switching for packet switching, and for good reason: packet switching handles bursty, unpredictable traffic far more efficiently. But AI workloads are different in kind. Distributed training and inference produce enormous, predictable, sustained flows between known nodes — traffic that looks a lot more like a dedicated circuit than a phone call. So this isn't a naive "back to the future" move; it's a workload-specific synthesis. Packet switching stays flexible at the edge, while the high-bandwidth core between compute nodes gets a dedicated optical path.

That's why the technology should be understood as complementary, not as a wholesale replacement for electronic switching. Optical circuit switching excels at predictable connectivity patterns and reduced switching overhead for specific traffic types, which is exactly the shape of scale-up and scale-out AI networking. But electronic packet switching still wins for dynamic, arbitrary traffic. The realistic future of the AI data center is a hybrid: optics where the traffic is known, electronics where it isn't.

The power angle is arguably the real strategic headline, more than any latency figure. The single hardest constraint on how large an AI cluster can get is power and thermal density — how many kilowatts you can cram into a rack and how you get rid of the heat. ABI Research forecasts global data-center capacity growing from 24.4 gigawatts to 147.1 gigawatts over the next decade, and every watt spent converting light to electricity and back is a watt not available for compute. If removing transceivers and OEO conversion really delivers anywhere near an 8× power reduction in the switching layer, that attacks the ceiling directly, not just the cost.

The market is responding accordingly. Dell'Oro Group projects that spending on switches for AI back-end networks will exceed $100 billion by 2030, and Salience Labs is only one of several suppliers betting that a meaningful slice of that will be optical. Its collaborations with Tower Semiconductor and Keysight Technologies are the tell that this is moving out of the lab and into an actual manufacturing-and-test supply chain — silicon photonics stops being a research curiosity exactly when foundries and test-equipment makers start building around it.

For anyone watching AI infrastructure, the takeaway is a reframing. The bottleneck in large-scale AI has quietly stopped being "how many flops can we buy" and started being "how fast and cheaply can we move the results of those flops around." Compute got all the attention because it was the scarce resource; now that the scarce resource is shifting to connectivity and power, the boring layer in the middle is where the next round of meaningful gains will come from.

Applied Materials has a good independent primer on the underlying physics and why optical circuit switches show promise for more energy-efficient AI, and the company's own site lays out the architecture in more technical depth for anyone who wants the datasheet-level view. The specific vendor matters less than the trend it represents: the wire between the chips, long an afterthought, is becoming the thing the whole system is built around.