Karpathy goes Claude; Google I/O fires everything

Why this hire, why now

Herk frames Karpathy joining Anthropic as the convergence of two aligned philosophies rather than a transfer-window splash ~00:00. Karpathy has spent the past year writing publicly about context engineering, the limits of pure scale, and the importance of agent harness design — all themes Anthropic has been pushing hard.

Anthropic's momentum line

Ramp's AI Index is the underlying signal ~02:02: Anthropic crossed OpenAI in business adoption (34.4% vs 32.3%) for the first time, driven largely by Claude Code. In the same window Anthropic announced an enterprise services JV with Blackstone, Hellman & Friedman, and Goldman Sachs, plus the KPMG alliance announced today^{[1]Nate Herk: Anthropic momentum vs. OpenAI}.

"The wrapper is the product"

Herk's central hot take ~04:02: the foundation model is increasingly not the differentiator — the wrapper (context, memory, workflows, tools) is. He credits Karpathy for coining "context engineering" over "prompt engineering," and reads Claude Code's recent harness work as Anthropic operationalizing that view.

Karpathy's LLM wiki, transplanted onto Claude

Karpathy's April 2026 "LLM wiki" project — a raw/ plus wiki/ folder where an agent synthesizes raw notes into a living markdown knowledge base — is the pattern Herk thinks Anthropic will productize as a context layer for Claude Code^{[1]Nate Herk on Karpathys LLM wiki} ~06:04.

Auto research and the /goal loop

Karpathy's March 2026 "auto research" project set up autonomous experimentation loops — propose, run, score, repeat. Herk explicitly maps this onto Claude Code's /goal feature and similar harness primitives~09:06.

Three predictions

Closing the video ~12:06: (1) Anthropic builds a context/workflow app store; (2) Claude Code ships more specialized /goal-style autonomous loop commands; (3) Anthropic launches an education layer paralleling Karpathy's prior teaching work.

The benchmark picture

Artificial Analysis puts Gemini 3.5 Flash at Intelligence Index 55, ahead of Grok 4.3 high (53) and Claude Sonnet 4.6 max (52); output speed exceeds 280 tokens/sec, ~70% faster than its predecessor^{[2]Artificial Analysis Intelligence Index}. Agentic Elo is 1656 — behind GPT-5.4 xhigh (1674) but well ahead of Gemini 3.1 Pro (1314). Google for Developers claims 3.5 Flash beats Gemini 3.1 Pro on nearly every benchmark while running 4× faster than other frontier models^{[6]Google: Gemini 3.5 Flash benchmarks}.

Price up, distribution wider

Willison's read on the pricing change^{[3]Simon Willison: Gemini 3.5 Flash pricing commentary}: $1.50 / $9 per million in/out is a meaningful step change from the prior $0.50 / $3 Flash tier, but Google is simultaneously deploying 3.5 Flash as the default in free Gemini consumer products — a Hot-Take-worthy bet that quality justifies the spend on Google's side and that competitors can't match the "intelligence per second per dollar" trade.

Sundar's keynote setup

Sundar Pichai framed the launch as Gemini becoming "agentic frontier" — 3.5 Flash today, 3.5 Pro next month^{[4]Sundar Pichai keynote: Gemini 3.5 Flash and Pro}. Google's accompanying claim: shifting 80% of workloads from competitor frontier models to 3.5 Flash could save customers $1B+/year; processing hit 3.2+ quadrillion tokens/month in May 2026, ~7× YoY.

SDK day-one

Simon Willison shipped llm-gemini 0.32 the same day, adding the gemini-3.5-flash model ID^{[5]llm-gemini 0.32}, along with llm-gemini 0.32a0 alpha with streaming for reasoning tokens^{[7]Simon Willison: llm-gemini 0.32a0 with streaming reasoning}.

An informal AICodeKing test

AICodeKing flags what he believes is a silent upgrade of Gemini Flash inside Anti-gravity — produced polished first-attempt outputs on a movie-tracker app and a Rubik's Cube simulator^{[8]AICodeKing: Gemini 3.5 Flash tests inside Anti-gravity} ~00:05. Note: no official benchmarks shared; it's vibes-driven testing.

The pitch

Managed Agents is Google's response to the "managed agent infrastructure" race opened by Anthropic in April. The problem statement: developers shouldn't have to wire up sandboxes, state, and tool dispatch — a single API call should do it^{[9]Google: Managed Agents problem statement}.

The six-part API surface

1. Interactions API — the single top-level endpoint that triggers agent execution.
2. Isolated Linux sandbox — ephemeral remote Linux environment per session.
3. Persistent file and state — files and state persist across multi-turn calls within a session.
4. Built-in tool invocation — native code execution, web browsing, plus custom tools.
5. Markdown agent definitions — AGENTS.md + SKILL.md files registered as named agents.
6. AI Studio Playground — no-code UI to compose and test custom agents before API deployment^{[9]Google: Managed Agents API surface}.

Competitive context: Anthropic parallel

Google's offering is a structural mirror of Anthropic's Managed Agents: hosted sandbox, multi-turn state, tool use, single API call, markdown-based agent definitions. Google's blog does not name Anthropic. Differentiation: Gemini model co-optimization, native Workspace tool access, the Antigravity desktop/CLI/SDK ecosystem^{[9]Google: Managed Agents competitive context}. From the developer highlights piece: a single Interactions call gives you persistent Linux + multi-turn state + tool use + markdown instructions^{[6]Google for Developers: Managed Agents — developer perspective}.

Availability

Preview as of May 19, 2026; accessible via Gemini API and AI Studio Playground. Enterprise tier — Gemini Enterprise Agent Platform — in private preview with no pricing disclosed yet^{[9]Google: Managed Agents availability}.

What Spark actually does

Spark enters Beta next week for AI Ultra subscribers in the U.S.^{[4]Sundar keynote: Gemini Spark} It's persistent — sessions don't end when the chat tab closes — and proactive: it surfaces things in your Workspace it thinks you need.

Daily Brief

A morning-digest agent that pulls Gmail and Calendar context, prioritizes, and suggests next steps. Effectively Inbox Zero meets executive assistant^{[10]Daily Brief details}.

MCP partners go live

The Gemini app picks up MCP support with launch partners Canva, OpenTable, and Instacart^{[10]Gemini MCP partners}. Worth noting alongside Anthropic's KPMG deal as evidence that MCP is rapidly becoming a real interop surface, not just an Anthropic-authored spec.

UI redesign + macOS

Google calls the new design system "Neural Expressive Design Language" — fluid animations, haptic feedback, refreshed type. The macOS app gains Spark integration and advanced voice-to-formatted-text^{[10]Gemini macOS Spark integration}.

Scale claim

900M MAU across 230 countries and 70+ languages — more than 2× the 400M reported at I/O 2025. The total token throughput claim of 3.2+ quadrillion/month is the load-bearing scale stat behind every other I/O announcement^{[10]Gemini app scale}.

Output modalities

Video output is the launch focus; image and speech outputs follow. Compared to GPT-5.5 and Claude Opus 4.7 (image input only), Omni adds video and speech input as well — distinguishing it on the multimodal axis even where text-only intelligence isn't class-leading^{[2]AA: Gemini 3.5 Flash multimodal comparison}.

Avatars and effects

The Gemini app uses Omni for cinematic video, background swap, and a "custom AI avatar" feature you train on yourself for social/work content^{[10]Gemini app: Omni capabilities}.

Omni Flash in Flow

The cheaper, faster Omni Flash variant ships first inside Google Flow alongside a creative AI agent and Bespoke Tools — no-code custom workflows for video creators. Section-by-section editing and full-track style transformations also land in Flow Music^{[11]Google Flow agent and Omni Flash}.

Antigravity becomes an ecosystem

Antigravity 2.0 desktop runs Gemini 3.5 at "12× the speed of other frontier models" per Google's claim and adds multi-agent orchestration^{[6]Google: Antigravity 2.0}. A terminal CLI and programmatic SDK round out the local agent dev story; Google Cloud integration covers the enterprise side.

AI Studio: mobile, Workspace, Android

Six concrete expansions^{[12]Google AI Studio details}:

• Workspace integration — apps can read/write Sheets, Drive, team docs directly.
• Export to Antigravity — carries conversation, files, secrets.
• Custom asset generation + in-preview edits — Build agent generates images via Nano Banana; annotation tools let you draw on a running app.
• Mobile app — build and preview from your phone with a gallery and live deploy sharing.
• Native Android dev — prompt-built Android apps with an in-browser emulator and direct Play Store publish.
• Free Cloud deploy — two app deployments to Google Cloud, no credit card.

AI Ultra positioning

$100/month with 5× Antigravity limits; the $100 promo credit through May 25 is a clear push to seed Antigravity 2.0 usage^{[6]AI Ultra details}.

Build with Gemini XPRIZE

$2M prize pool — Google's framing it as the largest hackathon prize pool announced. Finalists pitch at Moonshot Gathering, Los Angeles, September 2026^{[6]Build with Gemini XPRIZE}.

Co-Scientist

Multi-agent system with specialized agents for literature search, hypothesis generation/evolution, ranking, and cross-field synthesis^{[14]DeepMind Co-Scientist video}. Demo: a liver fibrosis / epigenomics prompt ran for days, testing thousands of hypotheses against tens of thousands of papers — compressing months of work to 1–2 days, with at least one already-published finding ~00:00.

Antimicrobial resistance

AI cut protein structure elucidation from years to 6 minutes, with Gemini acting as an ideation partner that connects findings across prior context unprompted^{[15]DeepMind: drug-resistant bacteria} ~00:00.

Cancer target identification in Uganda

AlphaFold and "AlphaFold Gravity" reduced breast-cancer vaccine candidate sites from 15,000 to 15^{[16]DeepMind: cancer at genetic level}. Democratization angle: research previously required travel abroad due to compute costs; now viable locally with a laptop and server access ~00:00.

WeatherNext and Hurricane Melissa

Case study: Hurricane Melissa (2025) had two divergent forecast scenarios (weak storm over Haiti vs. Cat 5 Jamaica landfall). WeatherNext predicted intensification and landfall 3 days earlier than prior models. The U.S. National Hurricane Center confirmed plans to add WeatherNext to routine forecast operations^{[17]DeepMind: WeatherNext + Hurricane Melissa} ~00:00.

Scope

Gemini for Science includes Co-Scientist, AlphaEvolve, ERA, NotebookLM, and a new "Science Skills" layer that maps onto 30+ biological databases. Two papers in Nature accompany the launch^{[13]Gemini for Science scope}.

Stitch

Real-time canvas streaming means design iterations happen live during prompting; multi-input support takes images and notes alongside text; production-ready exports drop straight into Netlify or into AI Studio / Antigravity for further dev^{[19]Stitch details}.

Pomelli

Two-topic rollout: (1) a brand-identity agent that asks SMBs questions or ingests existing docs, and (2) full brand-book + website generation from those "Business DNA" inputs^{[20]Pomelli details}.

Flow + Flow Music

Flow gains an agent, Omni Flash for video, Bespoke Tools (no-code custom workflows), and is available now globally^{[11]Flow updates}. Flow Music adds section-by-section editing, full-track style transformation ("covers"), and music-video creation. Mobile apps for both ship as well.

Scope

Three workstreams: (1) Claude embedded in Digital Gateway for tax and legal workflows, (2) KPMG Blaze using Claude Code for legacy code modernization, (3) KPMG as Anthropic's preferred consultant for private-equity portfolio Claude rollouts^{[21]KPMG alliance details}.

Stack

Managed Agents + Claude Cowork as the agent/collab primitives; Azure for compute. The McCombs / UT Austin academic partnership adds research collaboration.

Why this matters for Anthropic's enterprise narrative

Pairs with the Ramp AI Index data point (Anthropic at 34.4% business adoption vs OpenAI's 32.3%) from Karpathy's joining coverage. Big-4 wholesale deployments are how Anthropic locks in revenue Q4 2026 onward.

Who Anthropic is talking to

15+ traditions: religious leaders, secular ethicists, philosophers. The post is short on names but explicit about goal — shape Claude's underlying values, not just safety filters^{[22]Anthropic: 15+ traditions in conversation}.

The ethical-reminder result

Internal alignment-eval experiment: Claude with access to an explicit "ethical-reminder" tool showed lower rates of misaligned behavior. Small but pointed result — argues for tool-based moral scaffolding rather than fine-tuning alone^{[22]Anthropic ethical-reminder experiment}.

Mythos opens up

Glasswing / Mythos — Anthropic's cybersecurity research program — historically NDA-locked. The new policy allows ~50 corporate participants plus the Pentagon to publicly publish findings under a standard 90-day disclosure timeline^{[23]Tech Brew: Mythos disclosure policy}. Tech Brew reads this as Anthropic positioning itself as a security-research peer to other major labs.

Why two layers

C2PA metadata can be stripped by screenshots and resizing; SynthID watermarks survive those transforms but degrade differently. Layering them compensates for each other's weaknesses^{[24]OpenAI: C2PA + SynthID two-layer model}.

The verification tool

Public-facing at openai.com/verify. Deliberately cautious: no detected signal = no definitive conclusion (signals can be stripped), not "this is real."

SynthID scale

Google says 100B+ images and videos now carry SynthID — and the spec is now expanding into Search and Chrome with new partners: OpenAI, Eleven Labs, others^{[4]SynthID scale claim}.

The loophole

1,500 messages on the $40 plan × ~$62/message of agentic inference = ~$93,600 theoretical value^{[30]Theo: loophole math} ~02:01. Theo runs 50 staggered sessions on his Claw Mini SSH server, each chewing on a custom cryptography puzzle (ROT47 + fake Git commit; inverted base-64 alphabet) — single-message runs as long as 16 hours 10 minutes with 111.3M input tokens.

Four billing models

Theo's taxonomy ~04:30: rate-limited (Claude Code, Codex), message-limited (Copilot, old T3 Chat), spend-limited (Cursor, Open Code Zen), and per-token API. He argues message limits are the easiest to abuse on agentic workloads.

The T3 Chat near-bankruptcy detour

~11:07 Theo recounts T3 Chat losing $2K in 5 days and ~$500K total to Repo Mix abuse — context for why Copilot's bill-by-message scheme exists at all.

Caching economics

~19:11 Cached input tokens cost 10× less than uncached. A 16-hour single message costs $163 uncached vs $62 cached — and caching is what makes long agentic loops economically viable at all.

The June 1 token-credit migration

~33:21 Copilot switches off message quotas to token credits — Opus multiplier 15× → 27×, GPT-5.4 1× → 6×. Translation: the exploit window closes in < 2 weeks.

Azure benchmark revenge arc

~01:00 Theo frames the whole Copilot stunt as a sequel to his Azure inference benchmark — $1M+ in Azure credits running hourly to prove Azure's P90 was 21× slower than OpenAI, went viral, got fixed.

This is the most evil thing I've done in a minute.

MCP — the tool and data layer

14,000+ MCP servers live. Won adoption fast, but was designed for high-trust environments — boundary issues at scale are non-trivial^{[31]MCP: trust and security boundaries} ~03:04.

A2A — agent-to-agent coordination

Google launched A2A with 50+ partners. "Agent card" serves as the operating contract — but coordination adds latency and operational complexity^{[31]A2A: agent card contract} ~06:06.

AGUI — the human control layer

Streaming, state sharing, approvals, interruptions. Most teams ignore this layer until agents start causing real problems in production^{[31]AGUI: human control layer} ~08:09.

The three domain-specific protocols

~10:10 A2UI (structured agent-generated UI rendering), AP2 (agentic payment authorization with a cryptographic mandate), X42 (machine-to-machine HTTP-native payments). All important if you're building in those verticals; none are core today.

The real hot take

~18:15 "Most teams know which LLM they want but can't answer which tools the agent should see, what the approval model is, or how to validate multi-agent coordination. The real work lives in those questions."

Most teams know which LLM they want but can't answer which tools the agent should see.

What Zero is

Mozilla's positioning: small-binary systems language, JSON-shaped tool output everywhere (compiler errors, linter findings, tests), built so coding agents don't have to parse human prose to drive a compile loop^{[33]Zero design pitch} ~00:00.

The pushback

Better Stack argues: Rust already emits structured diagnostics; modern LLMs decode plain-text errors fine. So what's Zero actually solving that rustc --error-format=json doesn't?^{[33]Better Stack: Zero pushback} ~06:02

Yücel opens at [00:14] by introducing deepset as the company behind the open-source orchestration framework Haystack, with enterprise customers including Airbus, Bosch, Siemens, the European Commission, and German federal ministries — context that establishes sovereignty as a first-class concern for their user base. She offers a policy definition ("the ability of an organization to design, deploy, and operate AI systems on its own terms") and then translates it for engineers as explicit control over data flow, model choice, infrastructure, observability, and operations [01:16]. The core framework is four pillars introduced from [01:16] onward: data sovereignty (where data is stored and processed, plus access permissions — sending European citizen data to an embedding API hosted in Virginia already breaches GDPR), infrastructure sovereignty (where compute happens, on a spectrum from air-gapped through private VPC and sovereign cloud to SaaS, where the US Cloud Act becomes a risk even for EU-hosted apps) [03:17], model sovereignty (freedom to choose and switch models, swappability without architectural changes, training-data origin) [04:19], and operational sovereignty (observability, human-in-the-loop for HR/finance, auditable versioning) [05:19]. She emphasizes that sovereignty is a spectrum and not every team needs full air-gap — the goal is to know your level of vendor lock-in [06:19]. From [07:19] she walks through what concretely breaks when a working system is asked to become sovereign on Monday morning: swapping a frontier API for a self-hosted model forces prompt and evaluation rework; moving private data into the required jurisdiction creates multi-database and search-routing problems; replacing managed infra with on-prem exposes Kubernetes, CPU/GPU connectivity, and network management you previously offloaded; and bolting on observability reveals a black-box application layer with no version control [08:19]. Haystack is presented at [09:20] as a mitigation: a consistent interface for swapping cloud vs. self-hosted backends with a few lines of code, explicit typed and declared inputs/outputs in every pipeline, YAML serialization so applications are versionable in git, and a truly open codebase you can extend. She then sketches a sovereign agent reference architecture [11:22] — input guardrail, agent with tools (APIs, knowledge base, other agents, MCP servers), output compliance guardrail — and walks through Haystack code defining a guardrail with an Nvidia chat generator and an LLM message router, connecting an MCP tool set, building a searchable tool set with BM25 to avoid filling context with tool definitions, defining an agent with a system prompt and confirmation strategies for human-in-the-loop, and wiring it all into a pipeline with a tracer for LLM observability [14:23][15:24][16:25][17:27]. She closes with a three-question sovereignty checklist at [17:27]: Can you swap models without changing application logic? Do you have reproducible run logs stored in a compliant way? Can your team respond to an incident without calling a hyperscaler vendor?

Sections

~00:14 Intro: deepset, Haystack, and why sovereignty matters for EU enterprise customers

~01:16 Defining sovereign AI and the four pillars

~02:17 Data sovereignty: jurisdiction, GDPR, and access permissions

~03:17 Infrastructure sovereignty: air-gapped to SaaS and Cloud Act risk

~04:19 Model and operational sovereignty: swappability and observability

~06:19 What breaks when retrofitting an existing system for sovereignty

~09:20 How Haystack helps: consistent interface, explicit data flow, YAML versioning

~11:22 Sovereign agent reference architecture and closing checklist

Sovereign AI is the ability of an organization to design, deploy, and operate AI systems on its own terms.

If you send that data to an embedding API hosted in Virginia in the US, then you are already losing the control of your data.

Sovereignty is a spectrum. Not everyone needs to be sovereign in all of these pillars.

Khan opens at [00:14] diagnosing what he calls "mass psychosis" among agent developers: a feeling that everyone else is breezing through with fifteen agents in parallel while you're stuck reading every line of code. He proposes slowing down and breaking agent-building into four maturity levels you can slide up and down as your needs change [01:14]. He underscores how indistinguishable Frontier Lab UIs have become — showing screenshots of Factory, Codex, and Cursor that no one in the audience can tell apart — to motivate that the differentiation has to come from how you build, not which tool you copy [02:14]. Level 1 is using a framework — LangChain, LangGraph, or similar — which he recommends only for finding product-market fit when you don't care about the best model and just want something that works in half an hour. He's skeptical for production use: customizability, modularity, and the ability to stay on the frontier are hard to find inside a framework [03:14]. Level 2 is building the agent yourself, governed by five rules he walks through from [04:15]: (1) every agent is a state machine — a recursive while-loop with a few conditions and end states, and you should always have a mental model of which state you're in; (2) every addition risks making it worse — frontier models perform better with fewer instructions, citing that Codex's GPT-5 prompt is one-third the size of the GPT-5.3 prompt, and noting Cline has been rewritten from scratch at least seven times to prune accumulated junk [05:16][06:17]; (3) make the agent an easy part of a pseudo-RL pipeline by exposing it as a CLI with AGENTS.md and CI/CD so other coding agents can build and test it end-to-end [07:18]; (4) don't build slop — spend time on architecture and at least read the code even if you don't write every line, since architecture must be done thoughtfully by a human [08:19]; (5) Frontier Labs try to lock you down via API asymmetries — the new models (he cites Opus 4.6, Gemini 1.5.1 Pro, GPT-5.3) use reasoning traces that must be sent back in an exact format or performance degrades silently, and OpenRouter alone is not enough to insulate you [09:20][10:20]. Level 3 is the UX form factor: Kanban boards. From [11:22], he argues that when you're inference-bound with multiple agents running 8–10 minutes at a time, you want isolation of state and an engineering-manager view across all of them. He notes he made this claim on March 26th and Claude Code shipped the same form factor ten hours before the talk, validating the direction. Level 4 is shipping agents to the cloud [13:24] — paralleling many agents on separate machines, no local dependencies, agents that can run 15–60 minutes doing UX QA on their own (e.g., clicking through a VS Code extension end-to-end), and a shared cloud setup multiple teammates can mutate. He closes by framing his four levels as heuristics: start with the bare minimum and slide up the levels as your investment in agent infrastructure warrants. In a Q&A from [17:26] he clarifies planning inside Kanban happens through the CLI conversation that initiates the task, with the task transitioning to a review state when the agent needs human input.

Sections

~00:14 Intro: agent mass psychosis and the four-level framework

~02:14 Frontier Lab UIs are indistinguishable — differentiate via how you build

~03:14 Level 1: use a framework to find PMF, but expect to outgrow it

~04:15 Level 2 rules 1-3: state machine, prune additions, expose a CLI

~08:19 Level 2 rules 4-5: don't build slop; Frontier Lab lock-in via reasoning traces

~11:22 Level 3: Kanban as the right UX form factor for parallel inference-bound agents

~13:24 Level 4: shipping to cloud for long-running parallel agents and shared setups

~17:26 Q&A: planning inside Kanban via CLI and review-state transitions

Every single thing you add to an agent risks making it worse.

Frontier models are so good at their job that the less instructions you give them, they actually perform better.

Don't be a slob. Guys, don't please for the love of God don't build slop.

Verma introduces himself at [00:14] as tech lead of the User Representations team in Spotify's AI Foundation org — the group that builds frontier foundational models, user and content representations, and does CPT and SFT on open-weight LLMs. He frames the talk less as classical agentic context engineering and more as context engineering on the modeling side, in the service of better recommendations across Spotify's 750M+ users, 100M+ tracks, 400K+ audiobooks, millions of podcasts, and growing video catalog in 184 markets [02:16][03:17]. The three pillars are foundational user modeling, catalog understanding, and assembling both into a steerable personalized system. He recaps the legacy "tradex" multi-stage pipeline — candidate generation followed by one or more rankers — and notes that every product (home shelf, personalized playlists, search, podcasts, ads) historically had its own siloed team and model, which Spotify is now consolidating into a single unified LLM-backed model [05:19][06:21]. The AI DJ, prompted playlists (which now support podcast episodes as of that week), and the new Taste Profile (which exposes what Spotify knows about you and lets you edit it) are the user-facing surfaces driving toward steerability [04:18][05:19]. On user modeling [06:21], the team generates user embeddings daily for over a billion users — a massive, expensive pipeline. They've moved past autoencoder-style generalized representations (referenced in a prior team paper) to a single sequential transformer model where user interactions, request context (query, surface), and item candidates are all part of the prompt. A visualization at [09:22] shows users, tracks, and podcast episodes embedded in the same hypersphere — Verma's own embedding sits near the Big Big Tech podcast, illustrating cross-content modeling across users and content in one shared space. Catalog understanding [11:22] combines Spotify's proprietary item vectors with world knowledge from fine-tuned open-weight LLMs like Llama and Qwen. The key technique is semantic IDs — a concept introduced by a Google YouTube paper — which tokenize content vectors hierarchically into roughly four to six tokens [12:23][13:24]. Ariana Grande and Bruno Mars share the first two tokens because both are pop artists, with later tokens encoding niches. This lets the LLM autoregressively generate the next track or episode the way it would generate the next word, with training data composed of tokenized listening histories like an Italian user listening to an Italian podcast [14:25][15:25]. The final assembly [16:26] is the steerable personalized generative recommender powering the Taste Profile feature: users edit text describing themselves ("start listening to Justin Bieber more," "don't recommend this podcast") and that flows back into the model. To personalize without training on every one of 750M users, the team projects a user representation vector into the LLM's token space as a "soft token" inserted into the prompt, giving the model contextual user understanding at inference time [17:26][18:27]. Verma reports positive internal metrics and notes the next-episode podcast recommendations on Spotify are now productionized on this stack [18:27].

Sections

~00:14 Intro: Spotify AI Foundation and modeling-side context engineering

~02:16 Spotify scale and the three pillars of personalization

~04:18 AI DJ, prompted playlists, and the Taste Profile as steerable surfaces

~05:19 From siloed tradex pipelines to a unified LLM-backbone recommender

~06:21 Foundational user modeling and daily billion-user embeddings

~09:22 Cross-content embedding visualization on a shared hypersphere

~11:22 Catalog understanding with open-weight LLMs and semantic IDs

~16:26 Soft-token user projection and Taste Profile in production

We're moving away from siloed models towards this single unified model which supports an LLM backbone and allows you to steer it towards the sort of recommendations that you want.

We generate embeddings for like a billion plus users. We do that every day.

The first two tokens for both of them are shared because they're both pop artists.

McGrattan opens by framing the central enterprise AI problem: LLMs know nothing about your business by default, so the engineering challenge is building a context layer that lets AI deliver grounded, reliable answers at scale [00:07]. She identifies three pressures wrecking clean architectural diagrams: regulatory pressure (US Patriot Act concerns driving sovereign clouds in Europe), industry regulation (financial services and healthcare data that can't leave the data center, plus sub-millisecond latency needs for autonomous vehicles and fraud detection), and data gravity, citing Gartner's finding that the average enterprise has 400 distinct data sources [02:09]. She explains RAG using a concrete example: a car insurance customer asks why their premium rose 20%, and the vector database retrieves the contributing context (neighborhood theft rates, commute changes, pothole damage) so the LLM can cite documents and ground the answer in business reality [06:10]. The key insight is that where that retrieval data lives directly shapes query performance, making topology choice critical. The core of the talk is a three-way topology tradeoff [07:11]. Cloud offers elastic scale and global reach but suffers 20–200ms latency, egress costs, and connectivity dependence. On-premises is mandatory for sovereignty and regulated industries (HIPAA, PHI, financial trade surveillance) but requires infrastructure investment and patching overhead. Edge is required when milliseconds count, when devices can lose connectivity (her Florida fast-pass thunderstorm example [09:13]), or when data legally cannot leave the device — but compute, memory, and index freshness are constrained. She notes Actian software is built into nuclear submarine warheads, where air-gapped deployment precludes license-server checks [12:15]. Her prescription is hybrid by design with intelligent query routing: classified data routes to on-prem, sub-5ms latency to edge, sub-minute freshness to cloud, cross-domain queries fan out and merge [13:16]. Looking ahead 12–18 months, she predicts multimodal retrieval (audio, image, time series alongside text), AI-driven index management, and governance-aware retrieval that bakes access controls into the index rather than bolting them on. She closes with takeaways: distributed AI requires distributed retrieval, silent failures are dangerous, and the context layer is load-bearing — treat it like your business depends on it [14:17]. She announces Actian's new vector AI database for on-prem and edge, plus her newly published O'Reilly book on vector databases.

Sections

~00:07 Intro: 30 years in data, the context layer challenge

~01:08 Three pressures: regulation, latency, data gravity

~02:09 Sovereign clouds and the Patriot Act

~05:09 RAG explained via the car insurance example

~07:11 Topology tradeoffs: cloud, on-prem, edge

~10:13 Edge deployment: disconnected operation and air-gapped systems

~13:16 Hybrid design and intelligent query routing

~14:17 Future: multimodal retrieval and governance-aware indexes

The context layer is load-bearing, so treat it like your business depends upon it.

Distributed AI requires distributed retrieval.

According to Gartner, the average enterprise has 400 different sources of data.

Brooker opens by framing this as the most exciting time in his 30-year software career — agents give him a length of lever over problems he's never had before — but says the technology isn't perfect yet [00:07]. His hypothesis: the opportunity for knowledge-work agents is limited by the defect rate, and improvements in defect rate will contribute more to the overall opportunity than pushing the capability frontier forward [01:08]. He lays out a 2x2 of defect frequency vs. defect importance, evaluating outcomes from the full agent loop (not the raw model), since feedback loops let you build great things on faulty foundations [02:08]. The bad-bad corner (high frequency, high importance) is unbuyable beyond short hype windows. High-frequency, low-importance defects are tolerable 'slop' — fine for school newsletter summaries but a small opportunity. Low-frequency, high-consequence defects show up in software systems development, where agents ship code that seems correct, passes tests, then fails in production and requires expert humans to fix; useful but a sharp tool only experts can wield [04:09]. The real opportunity is the low-frequency, low-consequence quadrant — where everyone can play [05:09]. He observes that the last 18 months drove down defect frequency more than they raised agents' ability to do complex tasks reliably, and that the right tail of bad outcomes deserves as much investment as the flashy left tail. He recounts a frontier model lying to him about a Cauchy distribution for 15 minutes as a fitting illustration [07:10]. The core technical content is AWS's symbolic and correctness-oriented investments [08:10]. Hydro is a Rust framework for writing correct distributed systems and protocols — current models struggle with concurrency and failure reasoning, so Hydro is correct-by-construction scaffolding agents can lean on. Cedar is a policy language with deep automated-reasoning roots for writing authorizers. Kiro, AWS's coding agent, leans into spec-driven development to anchor initial correctness and prevent drift. Strata is an intermediate representation that compiles languages down for reasoning across multiple automated-reasoning backends, all powered by Lean [09:10]. Auto-formalization [10:11] turns natural-language SOPs into mathematically precise specifications through a clarifying conversation with the customer in Bedrock AI Guardrails and agent core policy, encoding the result in Cedar or Lean. Deterministic agent/tool policy applies these formalizations to runtime behavior: agent core gateway tool-call policies, Strands steering (pre/post-conditions on tool calls), and the newly open-sourced Trusted Remote Execution that constrains agent-generated operational scripts with formal Cedar policies [11:11]. He closes with a call to higher standards [12:13]: benchmarks should measure failure severity, not just pass-at-10 density; success metrics should include operational properties (performance, cost, durability, availability); the industry needs a research program on the shape of failures; and teams must take their worst days as seriously as their best to build a culture of reliability [14:16].

Sections

~00:07 Intro: most exciting time in a 30-year career

~01:08 Hypothesis: agent opportunity is gated by defect rate

~02:08 Defect quadrants framework

~05:09 Why low-frequency, low-consequence is the real opportunity

~08:10 Correct-by-construction: Hydro and Cedar

~09:10 Kiro spec-driven development, Strata, and Lean

~10:11 Auto-formalization in Bedrock Guardrails and agent core policy

~11:11 Deterministic policy: Strands steering and Trusted Remote Execution

~12:13 Call to action: severity-aware benchmarks and reliability culture

The opportunity size for knowledge work agents specifically is limited by the defect rate.

We need more benchmarks that capture failure severity rather than just failure density.

We need to take our worst days as seriously as our best ones.

Elangovan opens by placing AI alongside mainframes, client-server, web, mobile, and cloud — but notes this transition is happening in months and weeks instead of years or decades [00:07]. He thanks Andrew Ng for awarding his gesture-recognition startup the best AI startup award in 2014; that team now runs much of AMD's AI software organization [01:09]. His central frame is the 'K-shaped future of software engineering,' which he credits to a viral slide backed by a Harvard analysis [02:09]. The top arm of the K — systems-level thinking, judgment, taste, problem framing, harness design — accelerates dramatically; teams suddenly become 100x more productive and their wingspan moves up and down the stack. The bottom arm — language-specific syntax knowledge, formatting — falls in value because those are just intermediate languages for AI agents to consume. He stresses the metric shifts from lines of code to business outcomes [03:10]. His prescribed mode in this divergence is 'intent velocity' [04:11]: how fast you can turn an idea into production, with everything else as a means to the end. Winners operate in parallel — he runs four to six agents at night and during keynotes, crunching autonomously once given intent. The flywheel is that faster movement compounds, and he calls out the leadership responsibility to close the divergence without slowing the upper arm [06:14]. He then walks through AMD ROCm portfolio examples [06:14]. AMD has a pervasive hardware strategy (laptops to data center) and is layering AI on its open-source software stack so frontier models understand the stack end-to-end. Project Geek is an agent loop that observes customer workloads and continuously auto-optimizes software, yielding faster token serving [08:17]. The Rosetta-style project translates machine ISA from one GPU to another on the fly — previously a 4–5 year, 200–300 person project, prototyped in roughly 48 hours using a few billion tokens of Claude Code / Opus 4.6 and now shipping in production, even letting hardware coming out in 2–3 years run on current GPUs at native speed [09:18]. He uses this to make the point that 'too hard' no longer exists — you only need intent, framing, and awareness of pitfalls. A new zero-cost-overhead runtime moves tensors seamlessly between CPU, GPU, and NPU, integrating first into Llama.cpp so laptops can use all available silicon — he argues AI consumption will follow wherever compute lives [10:18]. A new tokenizer — one person, 200,000 lines of generated code — is now the world's fastest, and being open source feeds the flywheel because future model pre-training data will include it [11:19]. He closes with personal practice [12:19]: in December he thought agents were 'prompts in a cron job'; now agents continuously monitor his projects, reproduce bugs, file PRs, validate with tests, and commit on green CI without his involvement. His parting advice is the oxygen-mask analogy — put your own on first, then help the person next to you — but you do need to put it on, because the future is coming very fast [13:19].

Sections

~00:07 AI vs. prior software transitions

~01:09 Thanks to Andrew Ng and the 2014 AI startup award

~02:09 The K-shaped future of software engineering

~04:11 Intent velocity as the new mode

~05:13 Winners operate in parallel: agents running 24/7

~06:14 AMD ROCm: hardware portfolio plus open software

~08:17 Project Geek: agent-driven software auto-optimization

~09:18 Rosetta-style ISA translator built in 48 hours

~10:18 Llama.cpp runtime and fastest open-source tokenizer

~12:19 Agents in daily practice and the oxygen-mask close

Speed is the moat and your intent velocity is what you want to measure.

Now there is no such thing as too hard. You just have to have an intent saying I'm going to attempt that too hard thing.

In December I thought agents were prompts in a cron job.

Aadit Palicha opens by describing how Zepto was never intended to be a company [00:00]. He and co-founder Kaivalya Vohra had been friends since their early teens and admired Silicon Valley product builders. At 17, both got into a strong California school but COVID stranded them back in Mumbai [01:00]. Rather than do remote college, they took a gap year and started a WhatsApp group to deliver groceries to neighbors during the pandemic's first wave. That group became an app called KiranaKart, which eventually led them to Y Combinator [02:00]. On the decision to drop Stanford just 2-3 months before matriculation, Palicha rejects the heroic narrative [03:00]. He and Kaivalya took a tactical approach: they spent a year building, hit early signs of product-market fit around 8-9 months in, and only committed fully once they were doing about 10,000 orders per day and roughly 60-70 crore GMV run-rate [04:02]. Investor interest and a term sheet sealed the leap. His advice to other students considering the same path: get real proof of concept first. Grocery delivery in India was already a multi-billion-dollar space with players chasing it for 20 years [05:03], but Palicha says they never framed it as picking a differentiated model. Instead, the founders personally did deliveries in Mumbai's Andheri neighborhood and talked to customers at the door, learning that none of the existing models truly satisfied customers on speed, quality, selection, and price [06:04]. That bottom-up insight led to the pivot from KiranaKart's storefront-pickup model to mini dark stores - the very first one being Kaivalya's apartment [07:04][08:04]. The single dark-store neighborhood quickly did 3-4x the volume of the rest of the city, validating the model [09:04]. Palicha attributes their first-principles 10-minute delivery vision to a Brian Chesky-style "what's the most extreme positive customer experience if you remove the laws of physics?" framing [10:05][11:06]. He argues that customer delight unlocks volume, throughput, and lower costs that no spreadsheet would have forecast - "it's impossible to build a financially viable, profitable business without customer delight." Their YC-era advantage was naivety and isolation in COVID lockdown, with only 30-40 neighborhood customers to obsess over, no startup-Twitter noise [13:06][14:07]. Under the consumer app, Zepto is fundamentally a logistics, supply chain, and retail company [15:07]. They run dark-store design, replenishment algorithms, trucking, industrial-grade backend automation, and one of India's largest fruits-and-vegetables supply chains, sourcing millions of units per week directly from farmers [16:09][17:09][18:10]. The business now employs over 200,000 people, has millions of daily transacting customers, and has scaled to billions of dollars in topline, with a hundreds-of-millions ARR ads business where brands like Coca-Cola, Pepsi, and Nestle bid on search keywords [19:10][20:10]. The long-term vision is to build India's urban grocery infrastructure across 40-50 cities - a homegrown Amazon-like platform - and to incubate consumer brands on top of Zepto [21:10][22:11]. On AI [23:11], Palicha says Zepto has built ML-driven supply-chain forecasting that replaces days of manual work and now runs millions of units per day without humans in the loop, raising throughput and forecast accuracy. On the consumer/ads side, GenAI tools help brands generate keywords and predict ROAS, driving a big ads revenue bump [24:11]. Internally, AI has let them cut almost all SaaS spend and managed-services costs to near zero while shipping more with the same engineering headcount (about 500 engineers plus 150 in data/product/design) [25:12][26:12]. Closing on how he upgraded himself as a founder, Palicha credits surrounding himself with a senior management team - CFO, COO, CTO, head of growth, CBO - with decades of experience who believed in the vision early, and shamelessly asking basic questions: "surround yourself with people that are smarter than you and learn from them shamelessly" [27:13][28:14].

Sections

~00:00 Origins: two 17-year-olds and a WhatsApp grocery group in Mumbai

~02:00 KiranaKart, finding YC, and the Stanford decision

~04:02 Tactical risk-taking: PMF and 10,000 orders/day before dropping out

~06:04 Why a crowded grocery market still had room - customer interviews on the doorstep

~08:04 The pivot: co-founder's apartment as the first dark store

~10:05 First-principles 10-minute delivery and customer-delight economics

~15:07 Zepto as a logistics and supply chain company, not a consumer app

~19:10 Scale, ads business, and the urban-grocery long-term vision

~23:11 AI in supply chain, ads, and internal ops - cutting SaaS spend to zero

~26:12 Hiring, upgrading yourself, and learning shamelessly from senior teammates

If you remove all constraints and you just remove all the laws of physics and you just think from first principles, what's the most extreme positive customer experience you can give and you start from there and then you work backwards from how can I make that possible.

It's impossible to build a financially viable, profitable business without customer delight. Customer delight is where financial value starts.

The naivity of knowing how to build a company was a big advantage - we were extremely young and naive and the advantage of being young and naive is that you don't know how difficult it actually is.

If you remove all the software and the tech and the dark stores, fundamentally we're an atta-dal-tarkari company.

Stauch opens by framing Serval as the AI-native ServiceNow: a platform for employee support (enterprise service management) where instead of waiting on a ticket, you ask for something and get it instantly through automation [01:00]. He concedes ServiceNow's founding insight — workflows on top of a database — was correct, but argues those workflows take weeks-to-months of dedicated developer effort to build and maintain, which is fatal in an era of rapidly changing business processes [02:03][03:04]. Serval's wedge is a codegen engine: describe a workflow in natural language and the system generates the TypeScript instantly, and the same for keeping the underlying databases fresh. A core product principle is that building the automation must be as easy or easier than doing the task manually — otherwise IT staff will just reset the password by hand instead of opening a workflow builder [04:04][05:04]. This creates a 'slop automation' risk (the 20th duplicate password-reset workflow), which Serval mitigates with an admin-side agent that has full context of existing workflows and proactively suggests consolidation, deletion, and approval steps [05:04][06:04]. On moats and the application-layer debate, Stauch argues you should be happy when new models ship: the product is the boundaries — permissions, approvals, API scopes, audits, logs — not the raw capability [08:05][09:06]. Serval splits its system into two agents: an admin agent that builds tools/skills under tight control, and a help-desk agent that can reason freely but only invokes admin-sanctioned tools [09:06][10:06]. They run OpenAI models for end-user interaction and Anthropic (Sonnet/Opus) for codegen, and have learned that new model releases are not plug-and-play — sometimes they downgrade because predictable behavior beats raw intelligence [11:08][12:09][13:09]. Unit economics are healthy because they aren't reselling tokens at runtime: once a workflow is codegenned it just executes, so the expensive token spend amortizes [14:10]. On the existential threat of OpenAI or Anthropic moving into ITSM, Stauch points out that Anthropic has added more ARR in recent months than ServiceNow has in 20 years, so even a successful ITSM build would be a poor focus allocation for them [16:10][17:12]. Customer needs across AI-native startups and Fortune 500s are more similar than he expected — the real difference is committee size and decision velocity [18:12][19:14]. AI-native customers love that Serval lets IT folks finally use cool AI tools instead of fielding ServiceNow tickets to provision Cursor seats [19:14]; large enterprises value the employee-experience transformation (no more tickets vanishing into the abyss) [20:16]. Internally, Serval applies AI 'right of first refusal' to every function — no SDRs, no solutions engineers, leaner RevOps and product marketing, with reps using Serval itself to generate decks and battle cards live on calls [25:18][26:18]. Stauch's contrarian take is the emerging autonomy-vs-control gap: individuals want their Claude agents to do everything, but security and IT orgs need control, and that tension is where Serval positions itself [31:21][32:21]. He compares this to the iPhone/BlackBerry shadow IT moment — companies that default to 'yes' will lead, accept some security incidents, and ultimately win [32:21][33:21]. The biggest constraint is still hiring — every AI company is hiring more than ever despite the automation narrative — and Serval's mantra is 'fewer, better' [33:21][34:21][35:22]. Stauch wants Serval to be remembered not for automating jobs away but for closing the gap between what people thought their job would be and what it actually is — getting employees back to meaningful work [00:00][36:23][37:23].

Sections

~00:00 Mission: close the gap between idealized and actual jobs

~01:00 Why an AI-native ServiceNow — automation as the path to instant employee support

~02:03 ServiceNow's primitives were right; manual workflow building is the problem

~04:04 Building automation must be easier than doing it manually; slop automation risks

~07:05 Customer obsession as moat: Slack channels with 100+ customers, every day

~08:05 Application-layer moat: the product is the boundaries (permissions, approvals, audits)

~10:06 Two-agent architecture (admin agent + help-desk agent) and model mix (OpenAI + Anthropic)

~15:10 Will OpenAI/Anthropic build ITSM? Focus and ARR math say no

~24:18 Operating AI-native: 'right of first refusal' for AI, no SDRs/SEs, fewer-better

~31:21 Contrarian take: autonomy vs control — individuals want autonomy, orgs want control

~33:21 Biggest issue is still hiring; talent density and agility as the only moat

~36:23 Legacy: unlock meaningful work, not just automate jobs away

We want to be the tool that actually closes the gap between what you think your job's going to be and what your job actually is.

If it were actually easier to build the automation, you would build the automation — because why wouldn't you?

The product is the boundaries. The product is the controls. The product is actually what limits the capabilities of the model.

In the past couple months, Anthropic has added more ARR than ServiceNow has in the past 20 years.

Companion shorts

• Employees want autonomy. Organizations don't. — Stauch in 60 seconds on the central enterprise-AI tension^{[46]Sequoia short: autonomy vs control}.
• Twitter banned cross-team approvals. — Companion short: a "bias to yes" policy lets experiments ship as long as a direct manager approves^{[47]Sequoia short: bias to yes}.