The number that matters on Cerebras’ new rack is not the FLOPS

😁 Hello, super humans! Yesterday we looked at memory as a line item on an invoice. Today we look at it as a ceiling on your throughput, because Cerebras just shipped a rack whose whole argument is that the bandwidth number, not the FLOPS number, is what your users actually feel. Grab a coffee, there is a roofline calculation waiting for you further down.

📰 Quick Signals

• 🧠 AI: Gartner expects AI inference cost per agentic workflow to rise more than fivefold through 2028, arguing the rate of innovation is outpacing the cost curve rather than riding it down.
• 🤖 Robotics: an Intel commissioned study found 60 percent of leaders expect to operate robot fleets within five years while only 40 percent have a formal human and robot workforce strategy, with edge AI infrastructure named as a key gap.
• 💻 Programming: Python 3.15 reached release candidate 1, with the JIT reporting an 8 to 9 percent geometric mean speedup over the interpreter on x86-64 Linux and 12 to 13 percent on AArch64 macOS; rc2 lands September 1 and the final on October 1.
• ⚡ Electronics: Samsung raised pricing on new 4nm, 5nm and 8nm foundry orders by up to 15 percent, with its Pyeongtaek 4nm line reportedly running at full capacity, and SMIC confirmed increases of its own on last quarter’s earnings call.
• 📡 Telecom: Optus and Nokia hit 3.5 Gbps on a live Sydney site using 200 MHz of upper 6 GHz spectrum and a 768 element massive MIMO array, and found the coverage footprint roughly matches their existing 3.5 GHz 5G band.

🔍 The Big Story: Cerebras doubled both numbers, and only one of them changes your latency

If you serve a model and your users complain that it feels slow, the odds are overwhelming that no part of your accelerator is doing arithmetic at that moment. It is waiting for weights. Cerebras built a rack around that single observation, and the specification sheet reads very differently once you know what to look for.

What happened: on August 18 Cerebras introduced the CS-4, a rack scale system built from three of its new Wafer Scale Engine 3 Turbo processors. Each WSE-3T carries four trillion transistors and 900,000 cores across 46,225 square millimetres of silicon, with 44 GB of SRAM sitting directly on the wafer. Per wafer, compute doubles to 250 PFLOPS and memory bandwidth doubles to 43.2 petabytes per second. Across the rack that becomes 750 PFLOPS and 129.6 PB/s, with wafer to wafer latency as low as two microseconds. First shipments begin this quarter.

The details: the reason the bandwidth figure carries the story is arithmetic. Generating one token with a batch of one means reading every weight in the model exactly once and doing roughly two floating point operations per weight read. That is an arithmetic intensity of about one FLOP per byte, and a machine only reaches its peak FLOPS when intensity climbs above its ridge point, the ratio of peak compute to peak bandwidth. A high end GPU with roughly a petaflop of dense fp16 and around 8 TB/s of HBM has a ridge point near 125 FLOPs per byte, so single stream decode runs at well under one percent of the number printed on the box. The WSE-3T’s ridge point is closer to six, because the memory is SRAM on the same piece of silicon as the cores rather than stacks of HBM sitting off package. Cerebras is not claiming to have escaped the memory wall; it moved the wall. The company reports more than 4,400 tokens per second per user on GPT-OSS-120B, which it puts at up to 30 times a GPU based setup, alongside up to 10 times the throughput per watt of the CS-3.

flowchart LR
    subgraph GPU["Off-package memory"]
        C1["Compute cores"] -->|"about 8 TB/s"| H["HBM stacks"]
    end
    subgraph WSE["On-wafer memory"]
        C2["900,000 cores"] -->|"43.2 PB/s"| S["44 GB SRAM on the same wafer"]
    end
    GPU --> R["Ridge point: about 125 FLOP/byte"]
    WSE --> R2["Ridge point: about 6 FLOP/byte"]

The rest of the design follows from the same goal. Power conversion moved from roughly 50 millimetres away on a conventional board to about 0.5 millimetres from the processor, which nearly eliminates board level loss and lets the wafer clock higher. Off wafer I/O doubled to 2.4 Tb/s per wafer over standards based RoCE v2, and a second mode called Direct Wafer Links connects wafers without a switch, which is how Cerebras gets to two microsecond hops and claims support for models beyond 50 trillion parameters. That low latency path is aimed squarely at disaggregated serving, where a separate prefill engine chews the prompt and hands off the decode.

🗞️ More News

🧠 AI

• OpenAI open sourced Harness, the execution engine behind its Codex coding agent, under Apache-2.0, including the codex exec CLI, the Codex SDK and the app-server that runs the agent loop.
• OpenAI retires o3 from ChatGPT on August 26 at the end of its 90 day sunset window, so anything still pinned to it needs a target before tomorrow.
• Groq closed a 350 million dollar Series A to build out its AI inference cloud, keeping the deterministic-scheduling bet on the same side of the argument as today’s big story.
• Etched raised 700 million dollars for a rack scale inference system with co-designed chips, packages, boards, cold plates and interconnects, following 300 million announced last month.
• Nvidia will invest 1.5 billion dollars in and provide credit support for SoftBank’s SB Energy, which is building an 8 GW data centre campus in Ohio that will host Nvidia compute exclusively and be leased by OpenAI.
• Marvell will develop custom silicon for Google’s TPU ecosystem, potentially covering inference accelerators, NICs and memory interface controllers, with Google able to buy up to 12.2 billion dollars of Marvell shares.
• Cerebras and Callosum announced a partnership for heterogeneous agentic inference, splitting prefill and decode across different vendors’ hardware over the new low latency links.

🤖 Robotics

• IDC estimates US humanoid robot shipments could land 58 percent below its baseline by 2030 in a worst case where current FCC restrictions hold, noting China accounts for roughly 82 percent of global humanoid shipments today.
• TrendForce expects China’s robot market to reach about 2 billion dollars in 2026 with another 60 percent growth in 2027, while flagging profitability as the sector’s unresolved problem.
• Generalist AI showed GEN-1.5, a robot foundation model the company says can pick up a new manipulation task from a single demonstration.
• XPENG’s IRON humanoid pulled in what is being reported as a record physical AI funding round, another sign that the capital is chasing embodiment rather than chat.
• Amazon plans to extend Prime Air autonomous drone delivery to 500 US cities, which turns a pilot programme into an airspace coordination problem.

💻 Programming

• TypeScript 6.0 is the last major version of the compiler written in JavaScript; everything from 7.0 onward, in preview under the codename Project Corsa, runs on a new compiler written in Go.
• Canonical is backing a Bristol PhD project aimed at automating C to Rust translation, which is the part of the memory safety migration nobody wants to hand-write.
• An argument worth reading: AI coding assistants are pushing JavaScript’s share back up, because the tools are most confident in the language with the largest training corpus.
• A practical piece on why agent test suites need realistic data before production, since agents fail on the shape of real records rather than on the happy path fixtures.

⚡ Electronics

• Micron will spend 10 billion dollars over the next decade on Micron Research Labs in Boise, covering memory technologies, memory and compute architectures, packaging and manufacturing.
• IBM joined and cooled two cryogenic modules into a single environment, a step toward a modular ultra-cold system that could eventually link hundreds of quantum chips.
• NUS researchers demonstrated 0.8 nm amorphous carbon with a dielectric constant of 1.35 plus high breakdown strength and metal-ion barrier behaviour, aimed at the interconnect RC bottleneck.
• At Hot Interconnects, Arista demonstrated a 12.8 Tb/s, 64 channel liquid-cooled XPO optical module running 212 Gb/s per channel at about 130 W total.
• Coherent started customer sampling of 300 mm high thermal conductivity SiC substrates for AI infrastructure power stages.
• MIT engineers wired living bacteria into circuits that behave like transistors, printable onto a growth medium in a Petri dish.

📡 Telecom

• SoftBank and Ericsson tested an AI scheduler on a live Japanese 5G network, moving AI-RAN from lab slideware to production radio resource decisions.
• Following last week’s FCC testing authority, AST SpaceMobile’s UK direct-to-cell plans will run on Vodafone spectrum.
• ByteLens launched a platform for autonomous telecom fault repair, aiming at the gap between detecting a fault and someone actually rolling a truck.
• e& UAE is embedding agentic AI inside existing mobile and broadband product lines rather than selling it as a bolt-on service.
• EDOTCO built an Azure based AI tool to speed up tower site planning, one of the least glamorous and most expensive parts of a rollout.

👨‍💻 Code Corner

The roofline model answers the only hardware question that matters before you optimise anything: at your workload’s arithmetic intensity, are you limited by compute or by bandwidth? Everything above the ridge point is a FLOPS problem, everything below it is a memory problem, and almost all single-stream inference lives far below.

"""Roofline check: is this kernel compute-bound or memory-bound?"""


def ridge_point(peak_flops: float, peak_bw: float) -> float:
    """FLOPs per byte at which a machine stops being bandwidth-limited."""
    return peak_flops / peak_bw


def verdict(name: str, flops: float, bytes_moved: float,
            peak_flops: float, peak_bw: float) -> None:
    intensity = flops / bytes_moved
    ridge = ridge_point(peak_flops, peak_bw)
    bound = "compute" if intensity >= ridge else "memory"
    attainable = min(peak_flops, intensity * peak_bw)
    print(f"{name:>14}: {intensity:6.2f} FLOP/byte vs ridge {ridge:7.2f} "
          f"-> {bound}-bound")
    print(f"{'':>14}  ceiling {attainable / 1e12:10.1f} TFLOP/s "
          f"({100 * attainable / peak_flops:5.2f}% of peak)")


# Batch-1 decode through one 8192-wide fp16 linear layer: 2*N*N FLOPs of
# work, but 2*N*N bytes of weights streamed in. Intensity is about 1.
N = 8192
flops = 2 * N * N
bytes_moved = 2 * N * N

verdict("GPU + HBM", flops, bytes_moved, 1.0e15, 8.0e12)      # ~1 PFLOP/s, 8 TB/s
verdict("WSE-3T SRAM", flops, bytes_moved, 250e15, 43.2e15)   # 250 PFLOPS, 43.2 PB/s

Run it and the shape of the problem appears: both machines are memory-bound at intensity one, but the GPU reaches well under one percent of its peak while the wafer reaches roughly seventeen percent, because its ridge point sits about twenty times lower. Nobody escaped the memory wall; one of them moved it.

🧰 Toolbox

• Cerebras CS-4 datasheet: the full specification behind today’s headline numbers, useful even if you never buy one, as a reference set of bandwidth figures.
• Python 3.15 rc1 notes: the right moment to test your packages against the new JIT, while there is still time for a bug report to matter.
• TypeScript 6.0 announcement: worth reading now for the deprecations that Project Corsa will enforce rather than warn about.
• proteanTecs multi-die monitoring: per-chiplet telemetry from pre-assembly screening through in-mission operation, a good look at how chiplet systems get debugged.
• Chip industry earnings roundup: revenue and growth across roughly 80 chip companies in one page, the fastest way to sanity check a supply-chain assumption.
• Chips for Science report: input from more than 70 experts on opening advanced packaging facilities to low-volume, high-value research builds.

🔌 Component of the Week (rotating)

APS6404L (8 MB QSPI PSRAM): today’s big story is about bandwidth per byte of memory, so here is the same trade-off for about the price of a coffee. The APS6404L is a 64 Mb pseudo-static RAM in an 8-pin package that speaks plain SPI or quad SPI, handles its own refresh, and needs four data lines instead of the thirty-odd a parallel SRAM would demand. The 3 V part (APS6404L-3SQN) runs up to 133 MHz at 3.0 V, the 1.8 V part (APS6404L-SQH) up to 144 MHz, both across -40 to +85 °C. Do the arithmetic and quad mode at 133 MHz gives you roughly 66 MB/s, which is why an ESP32 or RP2040 framebuffer stored here feels fine for a 320 by 240 display and painful at 800 by 480: same lesson as the wafer, six orders of magnitude down. Single-unit pricing at the big distributors sits in the low single digits of dollars, so check current stock before you design it in. Start with the AP Memory SPI and QSPI product family and the APS6404L-3SQN datasheet.

📚 From the Blog

• Turning Pixels Into Something the AI Can Eat: the preprocessing stage between a camera and a model, and a good companion to today’s big story, because that pipeline is memory-bound long before it is compute-bound.
• Building Your First Neuron From Scratch: weights, bias, activation and one gradient step by hand, until the abstraction stops being magic.
• The Network Behind the Cameras: the unglamorous plumbing that moves video across a network without choking it.

😀 The Bot Says…

Forty-three petabytes per second of memory bandwidth, and the first thing anyone will run on it is a chatbot being asked to count the letter r. Somewhere a computer architect is staring at a ridge point chart and quietly opening a bottle.

---

That’s all for today! Run the roofline snippet with your own peak numbers and the intensity of the kernel you complain about most. Reply and tell us how far below the ridge point you landed.