Faster AI teams: our NeurIPS 2026 paper on latency-aware multi-agent workflows

Our paper “Optimize Once, Execute Fast: Latency-Aware Multi-Agent Workflow Learning for Recurrent Queries” (with Enpei Zhang, Feiyu Qu, Zheng Huang, Dawei Zhou, and Yujun Yan) has been accepted at NeurIPS 2026.

Many deployed AI systems are now teams of models: one plans, one drafts, one checks. Each handoff adds delay, and for workloads like customer support, where similar queries arrive thousands of times a day, that delay adds up. Our method, LAWA, represents a workflow as a dependency graph and edits it: steps run in parallel when neither needs the other’s output, and steps that add time but little value are removed. Each edit provably never lengthens the critical path, and a small learned predictor rejects edits likely to hurt accuracy, so the whole search takes a couple of seconds and never calls the models themselves. Across seven benchmarks, the optimized workflows were fastest on six and had the best overall accuracy ranking, about 14 seconds instead of 40 on coding tasks. The method can also take a workflow built by another tool and simply make it faster.

🔗 Links:

📝 Blog post for a broader audience
📄 arXiv:2610.14481
🏛️ NeurIPS 2026 page
💻 Code on GitHub