Architecture AI Updates: October 31, 2026

1. DoorDash Moves Agent Reasoning Offline to Cut Grocery Reorder Latency From 16 to 2 Seconds

InfoQ. InfoQ’s Leela Kumili reported on how DoorDash cut the grocery reorder flow in Ask DoorDash, its conversational shopping assistant, from about 16 seconds to about 2 seconds by moving the expensive reasoning out of the request path. An offline job runs batch LLM inference over 90 days of purchase history and profile signals to identify likely recurring purchases, passes the output through deterministic validation, and stores it as reusable shopping list data; at serving time the system resolves each stored item against the live catalog by trying the original store item, then a catalog product identifier, then a search term. The Grocery agent is invoked only when a request needs fresh reasoning, such as finding cheaper alternatives given current prices, promotions, package sizes and inventory, while quantity changes and removals update the list directly with no LLM round trip. The shopping list is a structured artifact that both the consumer and the agent edit, with the consumer’s direct edits taking precedence and the next agent turn starting from the edited list. DoorDash reports roughly double the shopping list views and a 34% higher order rate for the flow, and says its shared agent platform (orchestration, memory, model access, tracing, evaluation and rollout controls, with domain teams owning instructions, skills, tools and evaluation criteria) let it add a Reservations agent in one week, about ten times faster than its first domain agents. Source

2. Netflix Treats GPU Reliability as a Distributed Systems Problem

InfoQ. Craig Risi summarized Netflix’s account of running GPU training and inference on Kubernetes, which starts from the premise that in synchronous training one failed or merely slow GPU, network adapter or host stalls the entire job, so reliability has to be engineered as a distributed systems problem rather than a hardware one. A standard cloud health check only proves a VM is reachable, so Netflix layers NVIDIA DCGM device checks for GPU errors, memory, interconnect and thermal state, networking checks on the state and count of AWS Elastic Fabric Adapter interfaces, and a node agent that converts results into Kubernetes node conditions; checks fail closed, so a hung diagnostic times out and marks the machine unhealthy, with kernel-log fallbacks when DCGM itself fails. Remediation is progressive: a transient fault first blocks new work while existing jobs continue, a persisting condition drains the node, and responses escalate from killing a job to resetting a GPU, rebooting or replacing the machine, with Karpenter node auto-repair bounded by Netflix policies on how long each failure type is tolerated and how much of the cluster may be disrupted at once. Large jobs rely on checkpointing and elastic workers, automatic retries classify application errors separately so real bugs do not loop restarts, and the platform can identify a single slow rank holding back a job; smaller inference workloads share GPUs through MIG, and Netflix is moving to Dynamic Resource Allocation for topology-aware placement of GPUs, NICs and CPUs. Netflix reports GPU-node availability above 99.95% and job success rising from about 98% to over 99.5%, and InfoQ contrasts the approach with Meta, which attributed more than 66% of Llama 3 training interruptions to hardware and invests more in hardware-level fault detection such as silent data corruption. Source

3. InfoQ Trends Report: Inference, Evaluation and Context Pollution Define 2026 AI Systems Work

InfoQ. InfoQ published its 2026 AI, ML and Data Engineering Trends Report, written by Srini Penchikala, Chip Huyen, Daniel Dominguez, Hien Luu and Arthur Casals, which places technologies on a Crossing the Chasm adoption curve. New innovator-stage entries include agent-to-agent protocols, inference engines such as vLLM, SGLang and llm-d, harness engineering, spec-driven development and AI software factories, AI gateways, GraphRAG and agent security and governance; agentic coding, agent observability, inference optimization and model routing, small language models, MCP and AI-powered SRE moved to early adopters, while context engineering, RAG, vector databases, knowledge graphs, MLOps, data contracts and open table formats moved to early majority. The authors call 2026 the year of inference, arguing that reasoning models, long context and agents have turned serving into a systems problem spanning compute, memory, networking, routing and economics, with inference expected to reach about two thirds of AI compute spending by year end, and also the year of AI evaluations, with Dominguez arguing that teams must evaluate the model, tools, context, agent loop and environment together on task completion, cost, latency, human interventions and unsafe behavior. Casals identifies context pollution as the central context engineering problem: larger windows do not help if the system cannot separate useful current information from stale or adversarial material, so teams need to control what enters context, from which source, with what permissions, at which workflow step. Luu describes harness engineering as an extension of context engineering, noting that agent SDKs from LangChain, Anthropic and OpenAI started as loop helpers and now cover memory management and co-execution, and that managed runtimes shift the question from building agents to operating them at scale. Source

4. Spawn Runs a Game Studio on Agents That Ship Hourly and Own Their Own Incidents

Simon Willison. Willison linked and annotated Jacob Sansbury’s account of how Spawn, a massively multiplayer game whose players modify the world by prompting an in-game AI named Savi, is developed by a swarm of Fable-powered coding agents that change live code and ship to production hourly. About 60% of work originates from a production error, a bug report Savi files herself when she or a user hits a problem, a log anomaly or a feature request; each report goes to a triage agent that classifies it as a real bug, a rejection or an intentional design question needing a human decision, and fixes typically reach the affected user within an hour with a follow-up message in the same chat. The company treats an hour without a ship as a SEV-0 incident, on the argument that frequent small deployments keep agents coherent because an agent can hold only a few hours of work in context; targeted rollbacks take about 30 seconds, monitors and log alerts wake an agent that owns the incident, and the team deliberately breaks things to exercise those recovery paths. Sansbury says Fable 5 and 5.1 made this level of autonomy workable where Opus 5.5 did not, reports that GPT-6 Astra disabled features rather than fixing them when asked to apply security-review fixes, and puts token spend above $3 million per month, about 30 times payroll. Willison notes the post gives no detail on memory, tool integrations or coordination between agents, finds the approach hard to picture for high-stakes software, and sees game development as a useful testing ground for the limits of coding agents. Source