AWS opts for simplicity: can modular, plug-and-play networking deliver?

In our recent series on interconnect topologies, Dolphin Research reviewed 2D fat-tree, switch-based fabrics represented by NVIDIA & AMD. We also covered 3D torus, direct-connect fabrics exemplified by Google.

$NVIDIA(NVDA.US) sets the benchmark. It owns the full stack across hardware, protocols, and software, delivering best-in-class performance via tight co-design. Its rapid, step-change cadence forces fast pivots across the supply chain.

$AMD(AMD.US) follows NVIDIA, but on a lagging generation. Helios copies the GB300 NVL72 fabric, yet targets Vera Rubin bandwidth. Key switches and other components are externally sourced, so even with Scale-Up bandwidth caught up, overall perf/cost still trails NVIDIA.

$Alphabet - C(GOOG.US) takes a different path. Leveraging superior software, Google uses a 3D torus topology and OCS to chain nearly ten thousand devices within a Scale-Up domain, optimizing cost for large-scale fabrics.

So how do players that lack strong competitiveness, but are racing to improve system-level delivery (especially networking), approach this?

This piece turns to the last CSP giant with a system-level solution: AWS. Its fabric strategy is: ship first, remediate later; buy vs. build; pick the cheapest viable option. In short, AWS is the least self-developed across the stack.

Based on these traits, we dig further:

1) Can standard networking protocols work?

2) Topology shift: from 3D torus direct-connect to switch-based

3) Where are the AWS-supply-chain opportunities?

The main text follows:

I. Can standard networking protocols work?

1) Scale-Up protocol: NeuronLink as a PCIe-based shell

1) Claimed in-house, but rooted in PCIe

AWS built NeuronLink (NL, analogous to NVLink) for its in-house Trainium (Trn, NVIDIA GPU analog), but the protocol is essentially PCIe-based. NLv4 aggregates 2,304 GB/s of bi-dir bandwidth per card (AWS says 2TB/s; vs. NVLink 6 & UALoE at 3.6TB/s), with the evolution below.

Similarly, AMD has relied on UALoE (UALink over Ethernet) as a bridge while its native UALink switch silicon is still in development. The parallel is clear for standard-based protocols.

However, AWS does not intend to build switch ASICs, making a non-PCIe path unlikely since custom protocols typically need custom silicon support. It could instead become a UALink buyer, as discussed later in the topology section.

2) What exactly is 'in-house' in Scale-Up protocols?

The industry often assumes in-house Scale-Up is required for high bandwidth and low latency. To judge AWS’s PCIe-based approach, we first need to see what Scale-Up protocols actually define. In short, a Scale-Up protocol must address three issues:

a. How fast the lanes run: chips exchange data via multiple lanes (each a Tx/Rx pair). Lane speed depends on SerDes quality, the hardest part to master.

b. Handling errors: how errors are detected and corrected along the link. c. What cross-chip features exist: e.g., whether direct, address-based remote memory access is allowed, which distinguishes Scale-Up from Scale-Out.

Among these, (a) is the most important. Items (b) and (c) are rules, while SerDes prowess is a physical implementation issue that directly dictates per-lane rates. NVIDIA self-develops 200G SerDes; AMD sources Broadcom TH6 to achieve 200G. Higher per-lane rates reduce the number of lanes needed for a given bandwidth.

Back to AWS: PCIe Gen6 lanes run at just 64G. To bridge the bandwidth gap, it either waits for PCIe lane rates to rise, or increases lane counts.

3) Are PCIe-based protocols any good? Short answer: no

PCIe was designed for CPU-to-device links (xPUs, NICs), not peer accelerators. Beyond slower per-lane rates vs. proprietary links, it has port-count limits and other constraints. See the table below.

a. Wait for PCIe to speed up?

Cadence lags. PCIe advances roughly every three years, vs. NVLink every two. PCIe also takes years from spec to deployment, leaving AWS reactive and behind.

b. Is lane-stacking viable?

It raises costs. Between generations, getting more bandwidth means adding lanes. That inflates backplane copper, connectors, and PCIe retimers, which drives up power and TCO.

c. Between a rock and a hard place, why choose PCIe?

To ship fast with minimal effort. Protocol layers can follow the standard, and skipping switch-ASIC R&D saves time. Clouds want to cut NVIDIA’s premium via in-house efforts yet must deliver nodes fast to anchor customers (e.g., Anthropic), making a PCIe shell pragmatic. Delivery trumps competitiveness here.

4) NLv5: UALink/NVLink may be the endgame

For the next gen, if NL still rides PCIe Gen7, the protocol layer looks even weaker.

Consider PCIe’s evolution. From 5.0 to 6.0, per-lane speed doubled via modulation changes. See below.

There are two ways to double rates: double the switching frequency of the voltage signal, which sharply raises copper loss and routing demands, or carry more bits per transition. NRZ uses two voltage levels to transmit 1 bit per transition; PAM4 uses four levels to transmit 2 bits per transition.

Gen7 cannot rely on another modulation leap and must boost frequency, which stresses materials and retimers and blows up costs. Thus, AWS has two paths: when switch ASICs move to UALink, align NL with UALink to raise bandwidth; or pivot to NVLink — AWS has already said Trn4 will support NVLink Fusion.

2) Scale-Out protocol: a 'laid-back' EFA

Given AWS’s 'Zen' stance on Scale-Up, its Scale-Out approach is even more hands-off. We will be brief.

a. Bandwidth flat for two generations. EFA is AWS’s variant of standard Ethernet. Two successive rack-level designs hold Scale-Out at 200Gb/s per card. By comparison, NVIDIA lifted GB300’s 800G to Vera Rubin’s 1.6T, and Google moved from v7’s 100G to v8t’s 400G.

b. Smaller Scale-Up domains should lower Scale-Out needs, yet AWS still under-provisions. Google’s near-10k Scale-Up domain reduces Scale-Out pressure, yet it quadrupled bandwidth. NVIDIA’s 72-card domain pushes significant traffic to Scale-Out, so NIC bandwidth needs are higher. AWS sits in the middle at 144 cards, yet keeps Scale-Out at just 200G, below Google.

c. Key switch ASICs are externally sourced. AWS builds Nitro NICs and white-box switches, but switch silicon is multi-sourced (Broadcom, Marvell). This underscores its buy-over-build stance.

<End>

Risk disclosure and disclaimer: Dolphin Research Disclaimer & General Disclosure

Dolphin AI DC Interconnect Series:

CPO

‘AI Hyper-Connectivity: Is AI sprinting toward optics?’ ‘Copper is not leaving: CPO — real opportunity or mirage?’

Fabric architectures

NVIDIA fabric: ‘AI-era DC Interconnects: Beyond single-die — team-play via networks; is there a China angle?’

AMD vs. NVIDIA in DC fabrics: ‘AMD vs. NVIDIA arm-wrestling: Is Helios ready?’

Google fabric: ‘Challenging NVIDIA’s dominance: why Google’s 'optical fabric'?’

Single names

Lumentum (Part I): ‘From optical veteran to 'omni water seller': what keeps Lumentum competitive?’

Lumentum (Part II): ‘Lumentum: capacity crunch — what other tricks remain?’