The soundtrack is now part of the model

😁 Hello, super humans! For three years every AI video demo has been two systems in a trench coat: a picture model, then a separate pass that guesses what it should sound like. Yesterday MiniMax put the weights for a model that does not work that way on Hugging Face, and the architecture is the interesting part, not the demo reel. Let’s dig in.

📰 Quick Signals

• 🧠 AI: MiniMax published the MiniMax-H3 weights on 3 August, a 33 billion parameter omni transformer that generates video and 32 kHz stereo audio from a single joint latent prediction.
• 🤖 Robotics: UBTech says cumulative U1 series pre-orders across all channels reached 13,361 units, with mass production and staged deliveries starting 16 September.
• 💻 Programming: JEP 535 moved to Proposed to Target for JDK 28, which would make Shenandoah’s generational mode the default rather than an opt-in flag.
• ⚡ Electronics: Sony Semiconductor restarts its Kumamoto image sensor plant in stages from today, with pre-quake output expected back by mid-August.
• 📡 Telecom: AST SpaceMobile has BlueBird 11, 12 and 13 stacked on a Falcon 9 out of Cape Canaveral tomorrow, with liftoff targeted for 3:42 a.m. EDT.

🔍 The Big Story: An open-weight model that generates the sound with the picture

If you have ever tried to build a video pipeline that talks, you know the pain is not generation, it is alignment. Yesterday’s release moves that problem inside the model, and it does so with weights you can download.

What happened: MiniMax open-sourced MiniMax-H3 on 3 August, three days after the model went live on the company’s own API. H3 takes text, images, video and audio in one context and emits video with native stereo audio: dialogue, foley and music are all predicted together, up to 15 seconds at 24 frames per second, with stable dialogue in eleven languages. ComfyUI merged native support the same day, adding four nodes and six official workflow templates, and vLLM, SGLang and diffusers all published loading recipes on day one. The licence is MiniMax’s own community agreement rather than Apache or MIT.

The details: The architecture is refreshingly boring in the right places. Text goes through H3-Encoder, which is the full pretrained Qwen3-VL-32B with hidden states tapped from layer 50. Visual input runs through a temporally causal VAE compressing 16x spatially and 4x temporally into 24 latent channels, then patchified 1x2x2, so visual tokens land in the transformer at an effective 32x spatial downsample. Audio gets its own VAE that squeezes 32 kHz down to 40 Hz latent tokens per channel, reusing the same encoder and decoder for left and right. All of it is packed into one sequence with 3D multimodal rotary position embeddings, and a 33 billion parameter dense single-stream transformer predicts video and audio latents jointly. There are no modality-specific attention or feed-forward blocks; the only per-modality parameters live in the input and output layers and the AdaLN branches, and roughly 13 billion of the 33 billion sit in those branches and can be precomputed and skipped entirely for inference-only deployment.

That is the whole trick. Lip sync and foley stop being a post-hoc alignment problem because the same attention that decides a door closes at frame 84 is the attention that decides the thud lands there too.

flowchart TD
    T[Text] --> E[H3-Encoder<br/>Qwen3-VL-32B, layer 50]
    I[Images and video] --> E
    I --> VV[H3-VisualVAE<br/>f16 t4 d24]
    A[Audio, 32 kHz] --> AV[H3-AudioVAE<br/>40 Hz latents per channel]
    E --> P[Packed multimodal sequence + MM-RoPE]
    VV --> P
    AV --> P
    P --> OT[H3-Omni-Transformer<br/>33B dense, single stream]
    OT --> VL[Video latents]
    OT --> AL[Audio latents]
    VL --> OUT[768p video]
    AL --> OUT2[Stereo audio]
    OUT --> R{H3-Regenerate-2K<br/>API only}
    R --> K[2K output]

🗞️ More News

🧠 AI

• H3 went live on MiniMax’s API on 31 July, with the open weights following three days later, which is becoming the standard sequencing for Chinese labs.
• The release ships as two task-specific checkpoints: FL2VA for text and first or last frame conditioning, and Ref2VA for up to nine images, three video clips and three audio clips as references.
• Both checkpoints are CFG-distilled and BF16, and each is a self-contained repository carrying its own processor, tokenizer, text encoder, visual VAE and audio VAE.
• vLLM published a recipe page and SGLang a diffusion cookbook entry on release day, with the reference serving config spanning four GPUs at Ulysses degree 4.
• ComfyUI’s four new nodes cover empty audio-video latents, image-to-video, reference-to-video and a sigma shift control, with templates for the T2V, I2V and R2V paths.
• Moonshot’s Kimi K3 weights went public on 26 July at 2.8 trillion parameters with 104 billion active per token, still the largest open-weight release anyone has shipped.
• SK Group and NVIDIA expanded their partnership to a 2 gigawatt-scale AI cloud in Korea running Vera Rubin on SK hynix HBM4, with the first factory targeted for 2027.

🤖 Robotics

• BYD shows its first humanoid, Xiao Di, at Zhengzhou Di Space this month: 1.61 metres, 58.5 kilograms, 31 degrees of freedom, with real-time translation across six Chinese dialects and six foreign languages.
• The plan is two to three units per dealer showroom greeting customers and demoing cars, built by an embodied intelligence team BYD started in 2022.
• Within UBTech’s order book the U1 Pro is the volume model at 5,537 units, and the company is aiming at 50,000 units of annual bionic robot capacity.
• Away from humanoids, warehouse robotics funding in early 2026 shows a median round near $11 million against a $42 million average, with autonomous mobile robots taking 11 of 21 disclosed deals.

💻 Programming

• Following up on Friday’s item: JEP 401 has moved all the way to Targeted for JDK 28, and the Valhalla merge is large enough that other committers were asked to pause major work while it lands.
• JEP 542, PEM encodings of cryptographic objects, was promoted from draft to Candidate, which would finally give the JDK a first-class way to read and write PEM without a third-party library.
• Hugging Face opened a minimax-h3 branch on diffusers with pipeline docs, so the model loads through ModularPipeline.from_pretrained and fetches only the components your framework needs.
• Arch’s August ISO ships Linux 7.1.5, up from 7.0.14 last month, with the LTS branch in the repositories moving from 6.18.37 to 6.18.41.

⚡ Electronics

• Tokyo Electron resumed operations at its Kumamoto facilities on 3 August after safety checks, reporting no significant damage to buildings or equipment.
• TSMC says structural inspection cleared the JASM buildings and the fab is gradually resuming, without giving a concrete Fab 1 restart date.
• Institutional analysts put Kumamoto at under 3% of TSMC’s total capacity, which is why the quake has barely moved the company’s outlook.
• Renesas brought its Nishiki plant back on 29 July and plans to restart Kawajiri in Kumamoto City from 5 August.
• GlobalFoundries signed a letter of intent with Commerce for a $300 million CHIPS award on silicon photonics, targeting 400 Gbit/s links at a fifth of today’s energy, with the government taking roughly 1% equity.

📡 Telecom

• The BlueBirds going up tomorrow are next-generation birds expected to roughly double the peak download speeds of the Block 1 satellites, with 14, 15 and 16 already staged and production running through satellite 42.
• The FCC votes on 6 August on a notice of proposed rulemaking that would open more than 200 MHz of unlicensed Part 15 spectrum to direct-to-device satellite links, under ET Docket 26-169.
• The same item would clarify that Part 15 unlicensed devices are permitted inside FCC-authorized spacecraft, and asks where else unlicensed gear can safely operate in orbit.
• Nokia’s AI-RAN platform, built on NVIDIA Aerial, claims better than 20% spectral efficiency gains today and targets doubling capacity on existing spectrum by 2028 while staying Open RAN compliant.
• The comparison worth holding onto is that Ericsson’s competing AI-in-RAN subscription claims up to 20% higher downlink across 15 or more live deployments while running on existing baseband silicon with no GPU at all.

👨‍💻 Code Corner

Any time you evaluate an audio-video generator, the first question is whether the audio track is real and aligned or a silent stub bolted on by the container. This checks both in one go, no GPU required.

import json
import subprocess
from fractions import Fraction

def stream(path, selector):
    out = subprocess.run(
        ["ffprobe", "-v", "error", "-select_streams", selector,
         "-show_streams", "-of", "json", path],
        capture_output=True, text=True, check=True).stdout
    streams = json.loads(out)["streams"]
    return streams[0] if streams else None

v = stream("out.mp4", "v:0")
a = stream("out.mp4", "a:0")

if a is None:
    raise SystemExit("no audio stream: this is a video model wearing a hat")

fps = float(Fraction(v["r_frame_rate"]))     # "24/1" -> 24.0
frame = 1 / fps                              # 24 fps -> 0.0417 s
drift = float(v["duration"]) - float(a["duration"])

print(f"video : {v['width']}x{v['height']} @ {v['r_frame_rate']} fps")
print(f"audio : {a['codec_name']} {a['sample_rate']} Hz, {a['channels']} ch")
print(f"drift : {drift * 1000:+.1f} ms ({abs(drift) / frame:.2f} frames)")

Run it on a clip from any generator and the numbers tell you the architecture. A 32 kHz stereo track whose duration matches the video to well under one frame is what joint generation looks like; 16 kHz mono with a few hundred milliseconds of drift is a dubbing pass that was added later.

🧰 Toolbox

• MiniMax-H3: the weights, the architecture write-up and the reproducible 768p test cases, all on one model card worth reading end to end.
• multimodalart/minimax-h3: a hosted Space if you want to hear the output before you commit four GPUs to it.
• ComfyUI H3 workflows: the six official templates covering text, image and reference conditioning, which is the fastest path to a first clip.
• vLLM recipes for H3: the serving config, including the multi-GPU sharding that makes 15 seconds of generation tolerable.
• ffprobe: the tool behind today’s snippet, and still the fastest way to find out what is actually inside a media file.
• MounRiver Studio: WCH’s GCC-based RISC-V IDE with debugger support, which is what you will be living in if you pick up this week’s component.

🔌 Component of the Week (rotating)

WCH CH32H417 is a dual-core RISC-V microcontroller that puts genuinely fast I/O on a part that costs less than a coffee. The main QingKe V5F core runs at 400 MHz alongside a lower-clocked V3F companion core, and the interesting bit is what hangs off them: USB 3.2 Gen 1 with a 5 Gbps PHY, USB Type-C Power Delivery in both source and sink roles, a 10/100 Ethernet MAC and PHY on-chip, and a SerDes-based UHSIF link rated to 500 MB/s. The usual peripheral set is there too, with SPI, I2C, UART, CAN, SDIO and DVP plus I2S, SAI, LTDC and DFSDM for audio and display work. The obvious project is the one that normally forces you up to an FPGA or a Linux SoC: a high-rate data logger or capture bridge that streams sensor or camera data straight to a host with no bottleneck in the middle. Boards are cheap, with MuseLab’s nanoCH32H417 starting around $17 and WCH’s own USB 3.0 evaluation board about $20. Start with the part overview before you order.

📚 From the Blog

• Building Your First Neuron From Scratch: the smallest learnable transformation built by hand, which is the right grounding before you go reading about 33 billion parameter omni transformers.
• The Network Behind the Cameras: the unglamorous plumbing that moves video around, and a good companion to today’s story since generating the frames is rarely the expensive part.
• What Deep Learning Actually Is: the plain-language tour of the machinery, including why “one model, many modalities” stopped being exotic.

😀 The Bot Says…

Audio and video have shared a container format since 1993 and only now got introduced to each other. Somewhere an audio engineer is quietly rolling up a cable and muttering that this is what the sync clapper was for.

---

That’s all for today! Reply and tell us what you would actually build with fifteen seconds of audio and video that come out already in step.