Agents Ate the Chatbot

The Numbers

By May 2026, 80.6% of sampled individual Codex users made at least one request estimated to exceed 30 minutes of human work; 70.2% made one exceeding an hour; and 25.6% made a request exceeding eight hours. At the 99th percentile of daily active OpenAI employees, users now regularly generate more than 60 hours of Codex agent turns per day — distributed across multiple parallel agents.

Non-developers Led the Transition

Engineers crossed into Codex-majority usage first, around December 2025. Legal, Finance, and Recruiting followed in April 2026, but their transitions were much faster once they started. The average lawyer or recruiter at OpenAI now generates more than 85% of output tokens on Codex rather than ChatGPT.

The task mix also shifted: over one-fourth of Codex work done by business-function employees was engineering or coding — tasks that previously required technical support. Agents lowered the cost of crossing skill boundaries.

Research Output Growth

Among active internal users, median Codex use in the Research org grew 56× between November 2025 and June 2026. Customer Support grew 32×; Engineering grew 27×. The gap between departments is narrowing as non-technical adoption accelerates.

"As Codex became more powerful and parallelizable, users moved from only asking Codex for one answer at a time into increasingly orchestrating multiple agent tasks over the course of a day."

What Jalapeño Is (and Isn't)

Jalapeño is an inference ASIC — it handles serving finished models to users, not training. This is the layer where OpenAI spends the most compute dollars in production. Keeping Nvidia for training while owning inference silicon allows OpenAI to tune each layer for cost and speed without a wholesale hardware transition.

The Significance

The nine-month development timeline, aided by AI, is being described internally as potentially "the fastest ASIC development cycle ever achieved." Owning the silicon, model, and product simultaneously creates a compounding efficiency edge: improvements at one layer amplify benefits at the others. This is the Google TPU playbook applied to LLM inference, now with AI-accelerated chip design on top.

Other Stories in Today's Rundown

The same issue covered Anthropic and OpenAI jointly joining a $500M initiative to accelerate research on common cold and respiratory virus prevention — an early indication of AI labs redirecting resources toward hard biomedical problems. The Fable 5 comeback story (see topic 4) also got coverage.

How It Works

Claude Tag differs from conventional chatbot integrations in three ways: it's multiplayer (multiple humans can hand off a thread mid-conversation), contextual (it reads channel history and connected data sources rather than starting fresh each time), and asynchronous (it can schedule follow-ups across hours or days in ambient mode).

Who It's For

Claude Tag targets Enterprise and Team plan customers. Admins can create multiple Claude identities with scoped permissions, spending limits, and separate activity logs — so the Legal team's Claude doesn't share access with the Engineering team's Claude. Currently Slack-only; other platforms are planned.

Internal Usage at Anthropic

Beyond the 65% product-code stat, Anthropic says usage has spread to metrics analysis, customer support triage, and debugging. The framing — a "team member," not a tool — is a deliberate positioning shift that directly mirrors what Microsoft 365 Copilot and Google Workspace AI have been pitching to enterprise buyers.

Background

The order, covered in depth earlier this week, requires Anthropic to block access to Fable 5 and Mythos for users identified as foreign nationals — including many researchers at top US universities and employees at non-US subsidiaries of American companies. The policy was framed as a national security measure but has been criticized for unclear implementation criteria.

Expert Pushback

The 100+ signatories to the counter-letter argue that export controls on already-deployed models are ineffective: the capability knowledge is already in the world, and restricting access to US-developed models accelerates foreign adoption of Chinese alternatives like GLM-5.2 — potentially the opposite of the intended geopolitical effect.

Comeback Path

Details on the "comeback" are sparse, but the framing suggests Anthropic may be negotiating a compliance framework — possibly involving API key attestation or enterprise verification — that would allow access to be restored under conditions acceptable to the US government. SpaceX's acquisition of Cursor and its deep defense-sector relationships may also create a precedent for "trusted operator" pathways.

What Changed

Previously, Gemini's computer use capability was a separate model offering. Baking it into the main Flash model means developers building with Gemini 3.5 Flash now get agentic computer control as a built-in tool rather than an add-on. This reduces latency from model-switching and simplifies deployment.

Use Cases Highlighted

Google's blog points to continuous software testing (an agent runs your test suite across browser states) and enterprise automation across professional applications. The model can analyze UI interfaces, return categorized feature lists, and audit documentation for accessibility issues — all without human guidance between steps.

Safety Approach

Adversarial training specifically targets indirect prompt injection — the attack where a malicious actor embeds instructions in a webpage the agent reads, redirecting its actions. Enterprise safeguards let admins add confirmation gates for sensitive actions like file deletion or form submission.

Access: Gemini API, Gemini Enterprise Agent Platform, a Browserbase demo environment, and a GitHub reference implementation.

Three Security Challenges Identified

• Detection evasion: Advanced models may hide reasoning through "oversight awareness" or opaque internal processes
• Harmful execution potential: High-risk actions like cyber attacks require real-time prevention, not post-hoc analysis
• Misalignment risks: Even well-trained agents may misinterpret user goals in ways that aren't apparent until damage is done

The Framework

DeepMind frames this as treating deployed agents as "potentially compromised users" inside a system — borrowed directly from zero-trust network security. Supervisors (which are themselves AI systems) monitor action traces in real time. The roadmap also published "Three Layers of Agent Security" guidance for individual agents, multi-agent systems, and ecosystem-wide resilience.

The honest acknowledgment here: alignment is one layer, not the whole answer. Defense-in-depth assumes that alignment will sometimes fail and builds detection and response capability for when it does.

Eight Tools

• Model discovery: Search the live catalog with filters for price, context length, provider
• Benchmarking: Third-party scores from Artificial Analysis and Design Arena
• Pricing intelligence: Compare cost and latency across providers in real time
• Test inference: Send trial prompts to models and receive cost breakdowns
• Documentation lookup: Full-text search across OpenRouter docs
• Usage tracking: Monitor remaining credits and trending models

Why This Matters

An agent that selects a model based on training-data knowledge may pick something that's been deprecated, or miss a model that's now 40% cheaper and faster. This MCP plugs live market state into the agent's decision loop. The ability to actually run test prompts and get cost breakdowns before committing is particularly useful for multi-model routing setups.

Schneier's Argument

"If a company hired human writers to write its summaries, that company would be liable for inaccuracies in those summaries."

The corollary: exempting AI from the same standard creates a perverse incentive to replace qualified professionals with AI systems while offloading the risk of mistakes to users. Schneier argues this isn't just bad policy — it actively harms consumers and distorts the labor market.

The German Court Ruling

The ruling targeted Google's AI Overviews specifically, but the legal reasoning — deployer-as-principal — applies broadly to any AI-generated output presented to users as authoritative. If adopted by US courts or regulators, it would make AI hallucinations an ordinary product liability question rather than a novel legal frontier.

The practical implication: AI products that present outputs as factual (summaries, citations, medical or legal guidance) face materially higher legal exposure than products that present outputs as suggestions or drafts.

The "Software Factories" Frame

The conference organizers describe the current moment as the transition from artisanal software development (individual engineers writing individual features) to industrial-scale software production — where agents handle implementation and humans handle architecture, quality control, and business logic. The keynote framing echoes Andrew Ng's three-loop model (agentic loop, developer loop, user feedback loop) published in The Batch this week.

Who Was There

The AI Engineer World's Fair is the largest conference for AI application builders. Day 1 included individual conference talks from Omnara, OpenProse, Agentic AI Foundation, and Results Gen (covered separately in topics 10–13), plus the Latent Space live recording with OpenAI's Mark Chen (topic 14).

The full Day 1 keynotes stream is available at the link above.

The Core Argument

Sehgal's framing treats the execution log as the "source of truth" artifact — equivalent to what git commits are for code. An agent that produces structured, reproducible logs can be debugged by replaying its trace, improved by fine-tuning on successful traces, and monitored by comparing traces to expected patterns. An agent whose behavior lives only in the model's reasoning is a black box until something goes wrong.

This connects to the verification-horizon problem raised in today's HuggingFace papers (see topic 20): if you can't verify what the agent did, you can't improve it systematically. Logs are the verification substrate.

The interface-specification insight is similar to why test-driven development works: you're forced to define success criteria before implementation. Recursive agents encode that discipline structurally — each parent agent must write a clear specification for its children, and failures at the leaf level propagate up the tree with context about where the ambiguity originated.

Weitekamp's talk is particularly relevant alongside Angie Jones' argument (topic 12) that engineers should build systems rather than code — recursive agent trees are one concrete implementation of that philosophy.

Jones positions this as a skill evolution, not job replacement: the engineers who will thrive are those who understand how to specify systems — what the agent should optimize for, what it should refuse, how it should escalate, and what feedback loops it should incorporate. This is closer to systems design than to traditional software engineering.

The counter-risk she implicitly acknowledges: if every engineer shifts to "systems thinking," who ensures the implementation code the agents generate is correct, secure, and maintainable? This is the quality-control problem that conferences like AI Engineer are increasingly focused on.

The talk addresses a subtle but important evals problem: when a benchmark persona is based on a character who appears extensively in training data (Hamilton's cast is heavily discussed and quoted online), models can fake fidelity to that persona by recalling training data rather than demonstrating genuine persona-following capability. The evals measure memorization, not generalization.

This connects to the broader verification-horizon problem (topic 20): as models become more capable, contamination of this kind becomes harder to detect because the model's output becomes more fluent and confident — making it harder to distinguish "understood the character" from "memorized the lyrics."

Context

Mark Chen leads research at OpenAI and was one of the principal architects of the o-series reasoning models. A live interview at AI Engineer is unusually public for someone in his position — most deep technical conversations from OpenAI research leadership happen inside the company or in academic papers, not at industry conferences.

Likely Topics

Based on the episode title and Chen's public positions: the o1 development story (particularly how chain-of-thought reasoning was stabilized at scale), what evaluation metrics OpenAI uses internally that aren't publicly visible, and Chen's personal view on whether the current trajectory of capability improvements continues or hits a wall. The "scaling laws" framing suggests the conversation addressed the Chinchilla / compute-optimal training debates directly.

Full episode available at the YouTube link above.

This is a productive counter-narrative to the pure productivity framing dominant in most AI coding coverage. The concern isn't that AI coding is slow or bad — it's that it may be eroding the kind of deep system understanding that makes engineers effective at debugging novel problems, architecting large systems, and making good judgment calls under ambiguity.

The episode likely draws on early evidence from engineering teams who have been using AI coding heavily for 12–18 months — enough time for early signals of skill atrophy or changed mental models to become visible. Full episode at the link above.

SpaceX's acquisition of Cursor — one of the dominant AI coding editors — is a signal that defense-adjacent aerospace companies are building AI software stacks internally. The acquisition creates a "trusted operator" pathway for AI tools used in classified or defense contexts, and potentially insulates Cursor from future export controls.

The OSS segment likely covers GLM-5.2 (which narrowly outperformed Claude Opus on PostTrainBench at approximately 25% of the API cost) and the growing gap between open-weights and proprietary models that has been narrowing through Q2 2026.

Full episode at the link above.

Ada Palmer is a University of Chicago history professor and science fiction author whose work spans the Italian Renaissance, the history of censorship, and speculative fiction. The episode title suggests the conversation covers how rulers and institutions offered forgiveness strategically — not as genuine mercy, but as a mechanism to get opponents to self-identify and stand down before being quietly neutralized.

Dwarkesh's podcast regularly draws explicit connections between historical power dynamics and current AI governance debates. This episode is likely no exception, with the "forgiveness as trap" framing potentially mapping onto current AI safety discussions about oversight and corrigibility.

The K2 series uses a mixture-of-experts architecture that has proven competitive with much larger dense models on coding tasks. The "Code" designation on K2.7 suggests it's a fine-tuned or post-trained variant optimized specifically for code generation and debugging, following the pattern of code-specialized releases from Qwen, DeepSeek, and others.

Full hands-on demo from AICodeKing available at the link above.

A world model for agents is a significant architecture piece: rather than an agent making decisions based purely on current observations, a world model lets it simulate what would happen if it took various actions — enabling true lookahead planning. This is more commonly seen in game-playing AI (like MuZero) being applied to general language-based agent tasks.

The HuggingFace paper "Qwen-Image-Agent: Bridging the Context Gap in Real-World Image Generation" (also from today) appears to be from the same Qwen team, suggesting a broader multi-modal agent push from Alibaba this week.

OPID: Skill Distillation from Trajectories

The key insight: after a policy completes a trajectory (successfully or not), that trajectory contains rich information about which intermediate decisions were critical and which were filler. OPID extracts "hierarchical hindsight skills" from these trajectories — episode-level skills capture global workflows; step-level skills target critical local decisions. These skills are injected back as dense token-level supervision during the next training pass.

Results across ALFWorld, WebShop, and Search-based QA: +8.6 to +12.8 points over baseline GRPO. Critically, it achieves this with ~40% less training data than the comparison approaches, and skills learned during training are not needed at inference time — no runtime overhead.

The Verification Horizon

This paper argues that reward design for coding agents is entering a new regime: as generators (models) improve, the job of verification becomes proportionally harder. Traditional automated testing can be gamed; rubric-based judges can be flattered; user feedback doesn't scale; agentic evaluators are approximate. Every verifier remains "only a proxy for human intent" — and optimization pressure will eventually exploit the gap.

The implication isn't that RL for coding agents is futile — it's that verification infrastructure needs to co-evolve with the generator, rather than being treated as a solved problem.

The β/ρ Problem

The field typically uses pairwise error correlation (ρ) to decide whether model combination helps — if models fail on different queries, combining them should help. But the paper proves mathematically that pairwise correlation is "blind to the joint failure rate (β) that actually bounds orchestration gains." You can have low pairwise correlation among models that still all fail on the same hard queries.

Two Distinct Regimes

• Ceiling-bound (math/code): Real co-failure tails cap gains; orchestration offers little benefit regardless of strategy
• Realizability-bound (science): Minimal co-failure exists, but learned routers still capture almost none of the available gains

Practical Implication

Adding more models to a routing pool doesn't reliably help — and may actively hurt by introducing models that add cost without reducing failure rate. The lever is failure-mode diversity (models that fail on genuinely different query types) and market churn (the best model today is not the best model next quarter). This is a significant challenge for multi-model architecture decisions.

What Makes This Benchmark Different

Unlike most LLM benchmarks, CoffeeBench is adversarial in an economic sense: your agent competes against other agents (run by reference models) whose goals partially conflict with yours. You can't game it by producing fluent text — you need to actually negotiate better prices, manage inventory, and avoid cash-flow problems. This makes it a better proxy for real-world agentic task performance in contested environments.

The Idle-Drift Problem

Haiku's failure mode is particularly interesting: it didn't produce incoherent outputs or crash — it reasoned correctly about what it could do, then chose not to do it. The authors don't have a clear explanation for why. This suggests capability and agency are not the same thing: a model can be capable of acting while systematically failing to initiate action in ambiguous multi-step situations.

Missing Strategies

No models discovered collusion, coordinated pricing, or long-horizon strategic positioning. This places current frontier models at an "intelligent opportunist" level in economic environments — responsive to immediate conditions but not capable of sustained strategic planning across weeks of simulated time.

Karpathy has been one of the clearest public communicators about AI progress and architecture. His articulation of "Software 2.0" (neural networks replacing hand-coded logic) proved prescient. A "next paradigm" claim from him is worth tracking — it likely involves the shift from pretraining-dominated capability to reinforcement learning, world models, or some form of continuous learning that doesn't require retraining from scratch.

Theo's commentary tends to ground abstract claims in their practical implications for developers and startups. Full video at the link above.

This experiment directly probes the gap between instruction-following (do this specific thing) and goal-directed behavior (achieve this outcome via whatever means you find). The results — whatever they are — illustrate both what current agents can do autonomously and what constraints still matter for keeping autonomous financial actions safe and bounded.

Full experiment results in the video above.

The Dario relatability question has been floating in AI circles since his essays on responsible scaling and economic transformation — he speaks with unusual frankness about existential risks but in a register that feels academic rather than accessible. Contrast with Sam Altman, who leans into founder-tech-optimist framing that reads as more personally warm even when making similarly large claims.

The Cursor/SpaceX video arrives as the acquisition story is developing. Someone who was early to Cursor as a tool or investment is in a good position to comment on whether the SpaceX acquisition was the right outcome for the product.

Pincus's core claim is that enterprise AI is currently absorbing most AI investment because the unit economics are clearer and faster — a known buyer, a known use case, a measurable ROI. Consumer AI faces the harder challenge of behavior change, discovery, and monetization in a world where foundation model providers also compete as consumer products.

The contrarian case he's making is essentially a temporal arbitrage argument: low competition now → category creation → premium exit when the category matures. This logic worked for Zynga itself (social gaming in a market no major publisher was taking seriously) and for a long list of YC companies that built through winters.

This is squarely in the "Rust rewrites the JS toolchain" trend alongside Biome (ESLint + Prettier), Oxc (transpiler), and Turbopack (bundler). The appeal is: fewer moving parts, faster cold starts, and one install step instead of three. Whether it achieves the same level of ecosystem compatibility as established tools is the usual question for consolidating binaries.