NVIDIA AI Updates: September 19, 2026

1. AIPerf Replaces GenAI-Perf Because the Benchmark Client Was the Bottleneck

NVIDIA introduced AIPerf, a ground-up rewrite of GenAI-Perf for measuring LLM inference performance. The problem it solves is that GenAI-Perf ran on Perf Analyzer in a single process, so at high concurrency the benchmarking client became the limiting factor and the numbers measured the harness rather than the server. AIPerf splits the work across processes: worker processes generate load, separate record-processor services handle results, and the pieces coordinate over ZMQ. Source

2. It Reports TTFT, ITL, Latency, and Throughput With Full Percentile Breakdowns

NVIDIA ships AIPerf with four core metrics, time to first token, inter-token latency, request latency, and output token throughput, each broken out at p25 through p99, plus GPU telemetry when DCGM or pynvml is available. It covers more than 15 endpoint types including chat, image generation, and NIM rankings, reads datasets in ShareGPT, Mooncake, Baseten, and WEKA AgentX formats, and can synthesize variable workloads with tunable input and output token distributions. Arrival patterns are configurable as constant, Poisson, or gamma, and the tool deploys multi-node on Kubernetes. Source