Simpson MFG

The rising cost of building Nvidia AI servers is pushing Taiwan manufacturers to ready more cash for operations as the cost of a single AI rack surges, media report:

Source: Dan Nystedt

HDDs are roughly 100x slower than SSDs and are more susceptible to vibration, so why are they still the go-to for cold data in AI data centers? One reason: capacity per dollar, as the cost advantage is overwhelming.

What sits in AI cold storage? Massive raw video, images, and text for model training, plus logs and user interactions from inference — large datasets with low access frequency and little need for speed, where the priority is cheap and dense.

HDDs fit that brief: unit storage cost is only 1/4–1/5 of SSDs, and capex to deliver 1EB is about 1/50 of SSDs, yielding huge savings at scale.

SSDs offer higher throughput and lower latency, but the price tag makes bulk cold storage uneconomic. Using SSDs for vast cold datasets would be prohibitively expensive. HDDs do have longer lead times (around 1 year), but cold data are stored for the long term, and forward planning mitigates delivery risks.

So despite the speed gap, as long as the cost edge holds, AI data centers won’t drop HDDs. In the near term, HDD remains the default choice for cold storage.

$Sandisk(SNDK.US) A simple analogy: Imagine a chef working on a tiny cutting board (HBM). Every time a customer adds a request—"No onions... Wait, add onions... Now make it vegetarian... And add a side dish"—these sticky notes (context/KV cache) pile up on the board. Eventually, the board is covered in notes, and the chef can't chop; this expensive chef just stands idle. The so-called "architectural refactoring" is simply establishing a reasonable kitchen workflow: Keep the most urgent notes on the cutting board (HBM), move "important but not immediate" notes to the prep table next to it (DRAM), and store the rest in a nearby filing cabinet/pantry (enterprise SSD). Then, you add a runner and an organizer (DPU + network) to fetch and place the right notes at the right time, allowing the chef to cook at full speed—this means higher throughput, lower per-token cost, and less wasted GPU time.$Micron Tech(MU.US)

Nvidia is working with SK Hynix on a brand-new AI solid-state drive (SSD) with 10x the performance of current AI server SSDs, a project code-named “Storage Next”, media report. A prototype is expected in 2026. It's being designed as a memory layer between an AI server’s memory and storage, while also preparing for the future needs of data centers in Space. The performance target of the new AI SSD is 100 million Input/Output operations per second (Wow!). Taiwan’s Phison is a co-developer. $NVIDIA(NVDA.US) #SKhynix #Phison #SSD #Semiconductors

Source: Dan Nystedt