Architecture AI Updates: August 31, 2026

1. Dynamic CPU/GPU Co-Scheduling for Local MoE Inference

InfoQ / FreeToken (UC Berkeley, MIT). Writing on InfoQ, Olimpiu Pop reports on FreeToken, a research inference engine that runs frontier Mixture-of-Experts models on consumer hardware by replacing static expert offloading with dynamic co-scheduling that splits token computation between CPU and GPU based on real-time interconnect throughput. Its design overlaps PCIe weight streaming with active computation through fast weight formatting and double buffering, and adds semantic anchor checkpointing so intermediate states can be reused when an agent edits a prompt or injects a tool response rather than recomputed from scratch. The pattern matters because it treats memory bandwidth and interconnect limits, not raw GPU compute, as the primary architectural constraint for on-device agentic workloads. Source