Architecture AI Updates: November 3, 2026

1. Legora Evaluates Legal Agents Without Reading Customer Traces

Legora, the legal AI company, described how it builds agents it cannot observe in two November 2 Latent Space posts with CTO Jacob Lauritzen. Almost everything lawyers upload is confidential and often belongs to the firm’s clients, so Legora cannot read traces; instead an evals team of three engineers and three lawyers builds reinforcement-learning-style environments modeled on customer work, such as a 10,000-file data room where the agent runs due diligence and must catch the right red flags, and plots results on a heat map by practice area and task because a new model often gets better at some legal work and worse at other work. In production, Legora sorts failures into 10 to 20 known failure modes and runs cheap LLM-based checks for them behind feature flags. The product is built from a few primitives (projects that hold files, the agent, Tabular Review and lists): lists are a Notion-style database with triggers, so agents can create and read lists and a change to a list can start an agent, and Tabular Review treats each document as a row and each question as a column, scaling to a million cells but starting with 10 rows so prompts can be fixed before the full run. Lauritzen estimated that models already handle about 60% of legal work well, making those tasks a cost and latency problem, while analysis and editing across million-token contracts still need stronger models; every statement the agent writes carries a citation that highlights a bounding box in the source PDF, Legora post-trains open-weight models for narrow tasks such as legal research, and it runs entirely on Azure because law firms had already approved Microsoft as a vendor. Source Source

2. Treat Context, Memory and Knowledge as Separate Layers Behind a Context Engine

InfoQ. In a November 2 article in InfoQ’s Agentic AI Architecture series, Adi Polak argues that agent reliability depends on a “context engine” that selects, filters, ranks, structures, compresses, safeguards and delivers information before it reaches the model, and on keeping three layers distinct: context (what is in the active window), memory (what persists across interactions) and knowledge (curated facts, policies and rules). She describes a four-stage context lifecycle of selection (hybrid dense and BM25 search plus knowledge graph traversal, scoped by intent, risk, user role and recency), injection (ordering and grouping alongside instructions and output schemas), compression (summaries from smaller models or deterministic algorithms, validated with measures such as perplexity ratio or token-level KL divergence) and expiration and refresh, together with tiered memory: the prompt window as working memory, a session cache such as Redis for short-term state, and asynchronously updated vector stores and knowledge graphs for long-term memory. The article pairs common anti-patterns with mitigations: context stuffing (drop retrieved items below a relevance threshold), stale data loops (timestamp metadata filters or event-driven invalidation through Kafka), generic retrieval (query rewriting by a smaller model), mismatched implicit and explicit context (templates that separate system context, user context and the task), and unbounded memory growth (category-specific TTLs, from minutes for telemetry to session-scoped task state to periodically revalidated core knowledge). Source

3. Inside Nemotron 3: Hybrid Mamba-Attention, Sparse Experts and NVFP4 for Agent Workloads

ByteByteGo. ByteByteGo explained, in a November 2 post based on NVIDIA’s published material, how NVIDIA’s Nemotron 3 family reduces the compute and memory cost of the many model calls agents make. The models interleave Mamba layers, which keep a fixed-size state instead of a growing KV cache (at the cost of less precise recall over long histories), with selected attention layers that keep KV caches, and use mixture-of-experts layers: the roughly 30B-parameter Lightning configuration routes each token to 6 of 128 experts for about 3B active parameters and uses grouped-query attention with 32 query heads sharing 2 key-value heads. Multi-token prediction supports speculative decoding, and weights ship in NVFP4 (4-bit floats with an FP8 scale per group of 16 values plus a tensor-level scale), produced by quantization-aware distillation from a fixed BF16 teacher, which shrinks the checkpoint from about 66 GB to about 22 GB. The larger Super (about 120B total, 12B active) and Ultra (about 550B total, 55B active) models add LatentMoE, which projects token representations into a narrower space for routed expert computation to cut memory traffic and spends the savings on more experts, and Ultra is trained with multi-teacher on-policy distillation. Source

4. AI Readiness Depends on Domain-Owned Data Products That Agents Can Discover

Thoughtworks. Yuvaraj Birari described, in a November 2 post, work with a client preparing its data for agentic AI, where centrally managed BI and business units choosing their own tools had left data scattered, hard to discover and lacking the context needed to judge whether it could be trusted; agents working across systems need to know what data exists, what it means, whether it is reliable and what they are permitted to do with it. The proposal moves responsibility for quality, context and availability to the business domains that understand the data, with each domain publishing data products from a standard blueprint that defines input and output interfaces, quality expectations, access controls and operational responsibilities, discoverable through a central catalog, while a self-service platform supplies the catalog, quality monitoring, reusable infrastructure and an API-driven capability layer. Governance is federated and enforced as policy-as-code, for example blocking publication of a data product that lacks required classification, access controls or quality information, and progress is tracked by the share of discoverable data products, how many meet quality and service-level objectives, reuse across domains, onboarding time and handoffs, and how quickly trusted data can be made available for a new agentic use case. Source