Your agents are starving the CPU, not the GPU

😁 Hello, super humans! For three years the only question anyone asked about AI infrastructure was “can you get GPUs.” This morning the answer to a different question is getting uncomfortable: can you get cores? Agent loops spend most of their wall clock doing things a GPU cannot touch, and the bill for that is finally arriving.

📰 Quick Signals

• 🧠 AI: Agentic workloads have made the CPU the new bottleneck, with AMD reporting that seven of the eight stages in a realistic agent pipeline run entirely on the CPU.
• 🤖 Robotics: Schaeffler signed on to build and deploy hundreds of humanoid robots across its own factories, one of the larger single-customer commitments so far.
• 💻 Programming: Rust 1.98 goes stable on 20 August with C variadic functions finally stabilized and 128-bit integer support in inline assembly.
• ⚡ Electronics: Arm shipped its first in-house production processor in 35 years, a 136-core AGI CPU on TSMC 3nm aimed squarely at agent orchestration, co-developed with Meta.
• 📡 Telecom: Lockheed Martin’s NetSense reads RF disturbances in Verizon’s existing 5G network to detect and track drones, with no modified radios and no new hardware at the tower.

🔍 The Big Story: The CPU is the part of your agent stack nobody budgeted for

If your agent pipeline feels slow and your GPU utilization graph looks embarrassingly flat, you are not imagining it. The scarce resource in agentic AI turns out to be the one everybody stopped talking about in 2023.

What happened: IEEE Spectrum published a report on the CPU crunch yesterday, pulling together evidence that agentic workloads have shifted the bottleneck off the accelerator. Amazon Web Services told its own engineers to conserve CPU cycles, with one engineer saying capacity that used to appear in hours now takes days. Intel has sold out of server CPUs through at least the end of the year, AMD has doubled its server CPU forecast, and Arm, Qualcomm and Nvidia have each shipped or announced a CPU explicitly branded for agents. Madhu Rangarajan, AMD’s VP of compute and enterprise AI, put a number on it: “in our testing, seven of the eight stages in realistic agentic AI pipelines run entirely on the CPU.”

The details: The mechanism is not mysterious once you draw the loop. Model inference runs on the accelerator, but everything wrapped around it does not. Souvik Kundu, a senior staff research scientist at Intel, lists the CPU’s share as parsing the model’s output, deciding which tool to invoke, making the API call or running the code, collecting the result, and feeding it back. Safety guardrails pile on top: rule-based syntax and log inspection, plus sub-billion-parameter classifier models that stay on the CPU precisely because moving them to a GPU would cost more in latency than it saves in compute.

Then there is tokenization, which is the part most people underestimate. Tokenization is branchy, data-dependent sequential string manipulation, not matrix math, so it does not parallelize the way inference does. Euijun Chung, a PhD student at Georgia Tech, spelled out the cost in an arXiv paper on CPU-side inference bottlenecks: “if you have an ongoing sequence of, say, 100,000 tokens, and you have a tool result of a 1,000 tokens, the tokenizer will have to tokenize the whole sequence again. And you have to do tokenization at every agentic tool call.” Every tool call re-tokenizes the whole context. The paper found that when a server has too few cores it falls behind dispatching work, and the GPUs stall waiting for instructions. Adding cores cut time-to-first-token latency by roughly 1.5x to 7x at long sequence lengths; on Llama-8B, going from 5 cores to 32 cut latency by 4.45x.

flowchart LR
    A["User / parent agent"] --> B["Tokenize full context<br/>CPU, branchy, re-run every turn"]
    B --> C["LLM forward pass<br/>GPU"]
    C --> D["Parse output,<br/>pick a tool<br/>CPU"]
    D --> E["Run the tool:<br/>API call, shell, file I/O<br/>CPU"]
    E --> F["Guardrail check<br/>rules + small model<br/>CPU"]
    F --> G{"Done?"}
    G -->|"no"| B
    G -->|"yes"| H["Answer"]
    style C fill:#22C55E,color:#0B1117
    style B fill:#1FB6F5,color:#0B1117
    style D fill:#1FB6F5,color:#0B1117
    style E fill:#1FB6F5,color:#0B1117
    style F fill:#1FB6F5,color:#0B1117

Kundu’s own paper with Georgia Tech collaborators attacks the other half of the waste: the CPU sits idle during the GPU forward pass, and the GPU sits idle during tool calls. Their scheduling optimizations cut end-to-end agentic latency by up to 1.8x under sustained load. That is a real gain, and it is also chasing a target that keeps moving, because agents multiply their own work. OpenAI’s inadvertent Hugging Face incident had a single model firing off as many as 300 actions an hour, and each of those actions can spawn sub-agents making calls of their own.

🗞️ More News

🧠 AI

• AWS managers told engineers to cut compute usage and decommission idle EC2 instances, reallocating that capacity to paying customers.
• Amazon disputes that anything changed, saying the pressure sat in spot instances while contracted capacity held fine.
• Google retires three Imagen 4 model IDs today, and the replacement is not a drop-in swap: generate_images() is gone, so image generation moves to generate_content().
• An Intel and Georgia Tech paper proposes scheduling that overlaps CPU tool work with GPU inference, cutting end-to-end agentic latency by up to 1.8x under sustained load.
• OpenAI’s accidental Hugging Face incident saw a single model fire off as many as 300 actions an hour, a useful upper bound on how fast one agent can manufacture CPU work.
• The high-bandwidth memory squeeze that preceded this one is still running, and it is the reason your bill of materials moved before your architecture did.
• Attackers have started farming model hallucinations, registering the plausible-but-nonexistent package names that coding assistants invent.

🤖 Robotics

• RoboStore launched Robo Inc., a US-based manufacturing and systems integration company, with a 66,000 square foot Long Island facility due to be fully operational by Q1 2027.
• Wing and Walmart opened drone delivery across Greater Orlando out of the Apopka and Clermont Supercenters, reaching more than 50,000 homes with sub-30-minute drops.
• Corvus Robotics raised $18 million for autonomous warehouse drones, pitched less as flying robots and more as inventory visibility infrastructure.
• IEEE Humanoids 2026 runs 6 to 9 December in Santa Clara, chosen because 1X, Agility, Figure and Tesla’s Optimus team are all within driving distance of the convention center.

💻 Programming

• An npm worm published 2,234 poisoned versions across 444 package names starting 4 August, hitting keyv, cacheable, flat-cache and file-entry-cache.
• Microsoft’s teardown of the same campaign, which it calls ChainDrop, walks through the preinstall loader and the self-propagation step that republishes the malware under the victim’s own publishing token.
• Rust 1.98 also fixes a derived PartialOrd bug open since 2018, producing cleaner comparison code and smaller binaries.
• The Georgia Tech tokenization paper is worth reading in full if you maintain an agent framework, because the fix lives in your context handling, not your model choice.
• Microsoft’s Agent Framework hit 1.14.0 with a Mistral chat client, workflow checkpoint resume, and new hooks and storage options.

⚡ Electronics

• Nvidia’s Vera, its first CPU built for agents, is in full production and claims 1.8x faster task completion than x86, with Anthropic, OpenAI, ByteDance, CoreWeave and Oracle named as adopters.
• Qualcomm’s Dragonfly C1000 CPU and AI300 accelerator chase the same workload, with High Bandwidth Compute stacking roughly 768 GB of LPDDR on top of the compute tiles.
• Intel is reallocating PC production capacity to server CPUs amid tight wafer supply, which is how a data center shortage reaches a consumer price tag.
• AMD doubled its server CPU forecast to $120 billion, with its CEO saying the upcoming Epyc “Verano” was built purely for AI.
• imec and the University of Michigan launched Common Earth, a research push to remove rare earth elements and PFAS from chip manufacturing.

📡 Telecom

• NetSense starts selective pilots in the second half of 2026 with general availability planned for 2027, after a July live demonstration near Miami.
• Ooredoo is committing $800 million over five years to Zankore, an Indonesian AI compute platform built with Indosat, Nvidia and Nokia that aims to contract 1 GW of capacity in three years.
• Zankore’s first 200 MW lands in H1 2027 on Nvidia’s DSX AI Factory reference design, running GB300 NVL72 racks.
• Airtel is shifting away from fixed wireless access toward fiber as chipset cost inflation eats FWA margins, an early warning for anyone modelling 5G FWA in price-sensitive markets.

👨‍💻 Code Corner

Before you buy anything, find out where your agent’s time actually goes. Python’s standard library already splits wall-clock time from CPU time, so you can label every step of your loop as CPU-bound or waiting:

import time

def timed(label, fn, *args, **kwargs):
    """Report wall time, CPU time, and how CPU-bound one agent step is."""
    w0, c0 = time.perf_counter(), time.process_time()
    out = fn(*args, **kwargs)
    wall = time.perf_counter() - w0
    cpu = time.process_time() - c0
    share = cpu / wall if wall else 0.0
    print(f"{label:<18} wall {wall * 1000:8.1f} ms   cpu {cpu * 1000:8.1f} ms   {share:6.0%} CPU")
    return out

# Wrap each stage of one agent turn:
ids = timed("tokenize", tokenizer.encode, conversation_text)
reply = timed("model call", client.messages.create, model=MODEL, messages=msgs)
result = timed("tool call", run_tool, reply.tool_use)
timed("guardrail", policy_check, result)

Run it over twenty real turns. If tokenize and guardrail together own a third of your wall clock, more cores will buy you more throughput than a bigger accelerator will.

🧰 Toolbox

• Tokenization bottleneck paper: the Georgia Tech measurements behind today’s big story, including the core-count versus time-to-first-token curves.
• NVIDIA Vera CPU: the spec page for the chip Nvidia explicitly markets as the CPU for agents.
• Arm AGI CPU: 136 Neoverse V3 cores, 300 W, DDR5-8800 across twelve channels, PCIe Gen6, and over 45,000 cores per liquid-cooled rack.
• Socket keyv advisory: the affected version list, so you can pin or roll back before your next CI run.
• Rust 1.98 changelog: everything landing on Thursday, three days ahead of the release post.
• Modern home lab blueprint: a current, opinionated parts-and-software list if you want cores of your own instead of a rented instance.

🛠️ Build of the Week (rotating)

Home_LAB: a hands-on network, firewall and virtualization lab built entirely from decommissioned enterprise hardware.

• Difficulty: Intermediate
• Parts: Dell PowerEdge R740 running Proxmox, Cisco Catalyst 3850 switch, SonicWall NSA 2650, Cisco ASA 5525-X, UniFi access points
• Why we like it: it is the cheapest possible way to internalize today’s big story. A second-hand R740 gives you dozens of threads for roughly the price of a mid-range GPU cooler, and once you run an agent loop on it you will watch the tokenizer and the guardrail checks pin cores while the accelerator sits idle. Also, learning Proxmox, VLANs and firewall rules on gear you are allowed to break is still the best networking education available.

📚 From the Blog

• Turning Pixels Into Something the AI Can Eat: the preprocessing that turns raw camera frames into tensors, and a good companion to today’s big story, since that conversion is CPU work sitting in front of a GPU.
• Building Your First Neuron From Scratch: weights, bias, activation and one gradient step, written by hand before a framework hides any of it.
• The Network Behind the Cameras: the unglamorous plumbing that moves video across a network without choking it.

😀 The Bot Says…

Three years of “we need more GPUs,” and it turns out the thing holding up the whole parade was a for-loop parsing JSON. The most expensive silicon on earth, waiting politely for a string to finish being chopped into integers.

---

That’s all for today! Go instrument one agent loop and reply with the CPU percentage you find; I suspect it is higher than you expect.