OpenAI’s first chip is measured in watts, not in chips

😁 Hello, super humans! Yesterday we argued that memory bandwidth, not FLOPS, is what your users feel. Today OpenAI published the first real numbers for its own silicon and quietly changed the unit everyone is comparing in. The scoreboard is no longer tokens per chip; it is tokens per kilowatt, and that swap decides who wins. Let’s dig in.

📰 Quick Signals

• 🧠 AI: o3 leaves ChatGPT today after its 90 day sunset; the API is untouched, so only surface-level integrations need a look.
• 🤖 Robotics: the second World Humanoid Robot Games close today in Beijing after five days, 666 teams, 2,056 robots and 16 countries.
• 💻 Programming: the Copilot app’s Customize tab is generally available, collecting MCP servers, plugins, skills and canvases in one place.
• ⚡ Electronics: the memory price surge keeps lifting Samsung, SK hynix and Micron, and it is now visible on retail DDR5 kits, not just on HBM contracts.
• 📡 Telecom: the FCC is weighing more than 200 MHz of unlicensed spectrum for direct-to-device satellite links under Part 15.

🔍 The Big Story: OpenAI’s first chip wins on watts, and that is the whole argument

A model company shipping its own accelerator is not news by itself. A model company publishing measured numbers on a third-party benchmark, and choosing to score itself per kilowatt instead of per chip, is.

What happened: OpenAI released the first measured results for Jalapeño, its first custom inference chip, on August 25. Tested on InferenceX, a public serving benchmark from SemiAnalysis, Jalapeño delivered 1.5 to 1.9 times more AI work per watt at peak throughput and 1.7 to 3.6 times lower end-to-end latency than the comparison systems across GPT-OSS 120B, DeepSeek R1 670B and Kimi K2.5 1T. On highly interactive workloads the gap widened to 2.1 to 4.1 times. CFO Sarah Friar framed the same results as a full-stack compounding advantage: design the model, the serving software, the chip, the memory and the network together, then feed real workload data back into every layer. Deployment inside OpenAI’s own infrastructure is planned before year end, with Gen 2 already deep in development.

The details: The engineering story is about phases, not peak math. Language model inference splits into prefill, where the prompt is processed and the machine is compute-bound, and decode, where tokens come out one at a time and the machine is memory-bandwidth-bound. Most accelerators are excellent at one phase and then stall waiting on data movement in the other, which is exactly the pattern agentic workloads punish, because an agent runs many short prefill-decode cycles in sequence and every delay compounds. OpenAI’s answer was to keep model state, including the KV cache, explicitly placed and local, and to make the network part of the accelerator rather than a thing bolted on after it, so a whole request stays inside one connected domain. The numbers that matter for that claim are the latency ones: minimum time between tokens dropped roughly 2.7 times on GPT-OSS 120B and about 4.1 times on DeepSeek R1, which is what an interactive user actually perceives. Two other details are worth writing down. Chip design to tapeout took nine months with AI in the loop, and for selected GPT-OSS attention and mixture-of-experts blocks, AI-generated kernels ran 1.5 to 1.8 times faster than the human-expert implementations they replaced.

flowchart LR
    A["Prompt in"] --> B["Prefill<br/>compute bound"]
    B --> C["KV cache<br/>kept local"]
    C --> D["Decode<br/>bandwidth bound"]
    D --> E["Token out"]
    E -->|"next token"| D
    D -.->|"data movement<br/>= idle compute"| F["Latency you feel<br/>TBT and end-to-end"]
    B -.-> F

🗞️ More News

🧠 AI

• OpenAI is rewriting its Preparedness Framework and keeping its largest planned frontier RL run on hold after preliminary evidence that Astra may meet the Critical cyber capability threshold.
• OpenAI puts the overhead of its new chain-of-thought monitoring stack at roughly 20 percent of the inference compute being monitored, with a 30 minute alert target.
• GPT-5.6 Sol with max reasoning set a new high on the Artificial Analysis Coding Agent Index while using 54 percent fewer output tokens than a leading rival.
• OpenAI now offers Zero Data Retention on frontier models, which removes one of the last procurement blockers for regulated buyers.
• Anthropic published how Claude’s text watermarking works, its answer to EU AI Act transparency rules for AI-generated text.
• Anthropic expanded its partnership with Google and Broadcom for multiple gigawatts of next-generation compute.
• Demis Hassabis moved to Chair of Google DeepMind and Chief Scientist of Alphabet, with Koray Kavukcuoglu stepping up to run GDM day to day.
• ChatGPT Ads expanded across Europe, which puts an ad-funded tier next to the metered API in the same product line.

🤖 Robotics

• A humanoid clipped an obstacle and caught fire at the end of the 100 metre race on August 25, the sharpest reminder yet that a demo track is not a factory floor.
• Robots beat Usain Bolt’s 100 metre time in Beijing and then fell apart on the bean-sorting dexterity events, which is the honest state of the art in one sentence.
• Boston Dynamics’ Spot now ships with Google DeepMind’s Gemini Robotics-ER 1.6 for high-level embodied reasoning on multi-step tasks.
• Wing and Walmart opened drone delivery to more than 50,000 Central Florida homes and businesses, with a 2.5 pound payload today and 5 pounds planned.

💻 Programming

• GitHub Code Quality now runs on its own GitHub Actions path, so quality runs stop competing with your CI minutes budget in the same workflow.
• Code Quality enablement changes are now tracked in the audit log, which makes “who turned that off” answerable.
• Saved views, projects and milestones can be pinned to the repository issues sidebar, generally available since August 20.
• GitHub’s cloud retention policy for closed security alerts takes effect this week, so export anything you rely on for compliance history.
• GNU Linux-libre 7.2 landed alongside PorteuX 2.8 and a new Garuda release in last week’s roundup.

⚡ Electronics

• Nvidia reports Q2 FY2027 results today, with consensus around 92 billion dollars in revenue, and the interesting line will be inference mix rather than the headline.
• AMD jumped on a Strong Buy upgrade citing a projected 44 percent five-year CAGR in the AI server CPU market.
• Comu is a 6 dollar CH32V203 RISC-V board that measures 13 by 9.4 mm and disappears inside a USB port.
• Espressif’s ESP32-C61-MINI-1 brings Wi-Fi 6 and BLE into a module priced around 2 dollars.

📡 Telecom

• The FCC is weighing more than 20,000 MHz of additional spectrum for satellite use, an allocation shift with a long tail for terrestrial planners.
• AST SpaceMobile holds FCC authorization for a 248-satellite direct-to-device constellation coordinated with Verizon, but its commercial timeline keeps slipping.
• Amazon Leo filed for a 5,105-satellite direct-to-device layer on top of its broadband constellation, using the spectrum it picked up with Globalstar.
• Ericsson and Nokia are both positioning for a world where 6G arrives as a software release rather than a generation launch.

👨‍💻 Code Corner

Today’s big story is scored on time to first token and time between tokens, so here is a 30 line script that measures both against any OpenAI-compatible streaming endpoint. Point it at your provider, your local vLLM, or Ollama, and you get the same two numbers the vendors are arguing about.

import time
from openai import OpenAI

client = OpenAI()  # or base_url="http://localhost:8000/v1", api_key="x"

def measure(model: str, prompt: str, max_tokens: int = 128):
    gaps, first, prev, n = [], None, None, 0
    t0 = time.perf_counter()
    stream = client.chat.completions.create(
        model=model,
        messages=[{"role": "user", "content": prompt}],
        max_tokens=max_tokens,
        stream=True,
    )
    for chunk in stream:
        if not chunk.choices or not chunk.choices[0].delta.content:
            continue
        now = time.perf_counter()
        if first is None:
            first = now - t0          # TTFT: time to first token
        else:
            gaps.append(now - prev)   # TBT: time between tokens
        prev, n = now, n + 1
    total = time.perf_counter() - t0
    mid = sorted(gaps)[len(gaps) // 2] if gaps else 0.0
    return {
        "ttft_ms": round(first * 1000, 1),
        "median_tbt_ms": round(mid * 1000, 2),
        "tokens_per_s_per_user": round(n / total, 1),
        "end_to_end_s": round(total, 2),
    }

print(measure("gpt-4o-mini", "Explain KV caching in two sentences."))

🧰 Toolbox

• llama2.c: a full transformer inference engine in one readable C file, and still the fastest way to understand decode without a framework in the way.
• Artificial Analysis: independent model benchmarks with latency, throughput and cost side by side, including the coding agent index OpenAI cited today.
• Comu: a 6 dollar RISC-V board smaller than a microSD card, with capacitive touch pads and bare-metal USB HID as the point.
• Copilot Customize tab: one surface for MCP servers, plugins, skills and canvases, which finally makes team-level Copilot config reviewable.
• ESP32-C61-MINI-1: Wi-Fi 6 plus BLE in a 2 dollar module, which changes the maths on battery-powered sensor fleets.

🛠️ Build of the Week (rotating)

A 28.9M parameter LLM running entirely on an ESP32-S3: a full language model generating text at roughly 9 tokens per second on an 8 dollar microcontroller, with no cloud anywhere in the loop.

• Difficulty: Intermediate
• Parts: ESP32-S3 dev board with 8 MB PSRAM and 16 MB flash, a USB cable, the open-source firmware and training code
• Why we like it: it is the exact opposite end of today’s big story and it proves the same point. The model is 14.9 MB after 4-bit quantization, it uses llama2.c as the engine, and it was trained on TinyStories, so it writes small coherent stories and nothing more. Nobody is shipping a product on it. What it demonstrates is that the constraint on edge inference is memory layout and quantization, not raw compute, which is the same lesson Jalapeño’s prefill and decode split is teaching at 700 watts.

📚 From the Blog

• Turning Pixels Into Something the AI Can Eat: episode three of the Intelligent Video Analytics series, on the preprocessing work that decides whether your model sees anything useful. It pairs directly with today’s big story: both are about the cost of moving data before the clever part starts.
• Building Your First Neuron From Scratch: one neuron, no framework, from forward pass to weight update, so the rest of the stack stops being magic.
• The Network Behind the Cameras: the unglamorous plumbing that moves camera streams without melting your switch.

😀 The Bot Says…

A chip company spent nine months taping out silicon so an AI could write faster kernels for it, and the AI beat the humans by 1.8x on the blocks it was pointed at. Meanwhile, in Beijing, a humanoid ran the 100 metres faster than Usain Bolt and then set itself on fire at the finish line. Progress is not a straight line, it is a scatter plot with excellent marketing.

---

That’s all for today! Which number are you actually optimizing right now, tokens per second or tokens per kilowatt? Hit reply and tell us.