Architecture AI Updates: October 22, 2026

1. Prompt Caching vs. Semantic Caching: Different Layers, Different Failure Modes

ByteByteGo. ByteByteGo compares the two common ways to cache LLM work. Prompt caching reuses KV states from the prefill stage for an exact shared prefix, but the model still generates a new answer. Semantic caching returns a stored answer when the embedding of an incoming question is close enough to a previous one. Prompt caching only works on exact matches, so one early timestamp or request ID breaks reuse for everything after it. Using Anthropic’s pricing (cache reads at 0.1x the input rate, writes at 1.25x or 2x), the article shows one write plus one read of a $1 prefix costing $1.35 instead of $2. Semantic caching carries more risk: negations, dates, plan tiers, and permissions can sit close together in embedding space while changing the correct answer, and similarity scores are not probabilities. The article therefore calls for metadata scoping, invalidation by TTL, versioning, or dependency tracking, and a metric for how many hits were actually appropriate, not just the hit rate. Its recommended stack has three layers: exact-match cache, then a scoped semantic cache, then an LLM call that uses prompt caching. Source

2. Debugging With Agents: Collect Evidence First, Use Controls, Give Agents Raw Data

Martin Fowler. In his October 21 fragments, Fowler points to Cindy Sridharan’s long essay on debugging code you no longer understand, which draws on three recent bug hunts done with AI agents. Her method separates a bottom tier of raw facts from an upper tier of hypotheses. The agent first gathers evidence about how the system behaves, using a control (a comparable case without the bug) to pick out the signals that matter. It then ranks hypotheses by how well they fit those signals and checks the strongest ones against every fact, reading code only as a last resort. Fowler highlights two architectural points. Agents, unlike humans, do better with more raw data, and dashboard-style aggregation can destroy the signal they need, which argues for richer logging. Humans still add the most value by choosing controls, setting constraints, and rejecting hypotheses that have not been validated: in all three bugs the agent could have found the root cause, but it did not set up a control until asked. Source

3. InfoQ Launches an Agentic AI Architecture Article Series

InfoQ (Rafał Gancarz). InfoQ has started a five-part series that treats agentic AI architecture as its own discipline in software architecture, noting that in this field “last year’s best practice is this year’s antipattern.” Two articles are available now: Mallika Rao’s piece on agents as the next step after microservices, where agents decompose decisions the way microservices decompose functionality, and a three-tier enterprise framework for designing and delivering agentic systems from Subash Natarajan and Ahilan Ponnusamy. Three more are scheduled: Karthik Ramgopal on how agent harnesses evolved from chains to graphs to code (week of October 26), Adi Polak on memory, knowledge, and context engineering (week of November 2), and Gancarz on open challenges (week of November 9). Source