Multi-Agent System for Incident Resolution
Agents consolidate context across structured and unstructured sources for 500+ incidents a day
A multi-agent system that connects to live and historical incident data, then compares, summarises and consolidates it, cutting the manual overhead of gathering context before an incident can be resolved.
- Microsoft TaskWeaver
- Vanna.ai
- RAG
- Text-to-SQL
- Vector databases
- AWS
- Python
- SQL
Problem
Incident resolution at a telecommunications operator ran at 500+ incidents a day. The resolution itself was rarely the slow part; assembling the picture was. Relevant context sat across structured operational databases and unstructured historical records, and an engineer had to query several systems by hand before they could judge whether an incident resembled something already solved.
Approach
The task splits cleanly along how the data is stored, which is what made a multi-agent design the right shape rather than an aesthetic choice:
- Text-to-SQL, built on Vanna.ai, for the structured operational data, where the answer is a query result and correctness is checkable.
- Retrieval over the unstructured historical records, where the answer is a precedent and relevance is a ranking problem.
- A coordinating agent that decides which sources a given question needs and consolidates their answers into one summary.
Splitting these meant each agent could be evaluated against its own notion of correct. A single agent covering both would have had one accuracy number hiding two very different failure modes.
Text-to-SQL is the risky half: a syntactically valid query against the wrong column returns a confident, wrong number. Generated queries were constrained to a known schema and validated before execution rather than trusted on sight.
Result
A working multi-agent system deployed on AWS, combining RAG and text-to-SQL under prompt-engineered coordination, giving engineers a consolidated view of an incident’s context instead of a set of systems to search.
What I’d do differently
I would invest earlier in a fixed evaluation set of real incident questions with known-good answers. Prompt changes were assessed on a handful of examples for longer than was comfortable, which makes it hard to tell a genuine improvement from a lucky sample.