United KingdomPublic sector2025 – ongoingScottish Government
Consult AI analysis of consultation responses
Scottish officials used AI to group more than 2,000 consultation responses with results close to human reviewers
Problem
UK governments ran around 500 public consultations a year. Civil servants read and categorized written responses by hand before drafting policy, which took substantial staff time.
Approach
The Scottish Government tested Consult on responses about regulating non-surgical cosmetic procedures such as lip fillers. The UK government’s Incubator for AI (i.AI) built the generative AI tool as part of its Humphrey suite. Consult proposed themes from more than 2,000 responses to six open questions and assigned responses to them, while government experts checked and refined the themes. For this first live use, officials also reviewed every response manually to compare the results.
Results
Takeaway
Officials checked every response by hand during the trial and measured the tool’s agreement with human reviewers at an F1 score of 0.76. Annual savings of 75,000 days and about £20 million were projections for use across government, with further evaluation planned before expansion.
Categories
- Size
- Large enterprise
- Industry
- Public sector
- Business function
- Internal work and knowledge managementStrategy and organization
- Innovation type
- Process innovation
- AI technology
- Generative AI
- Adoption
- Co-developed with a partner
- Evidence
- Company's own claim
Sources
Source summaries
The UK Department for Science, Innovation and Technology reported that Consult’s first live trial covered more than 2,000 responses to six questions in Scotland. Its F1 score was 0.76, and differences from human reviewers barely changed the ranking of themes. The department projected annual savings of 75,000 days and about £20 million across 500 consultations. The tool remained in trial pending further evaluation.
Holyrood reported that the Scottish public health minister welcomed the Consult trial and participating officials said it saved substantial time. People continued to review the results. The government’s savings figures were projections.
i.AI’s evaluation found a multi-label F1 score of 0.76 and little change in theme rankings compared with human reviewers. Median review time was 23 seconds per response. The evaluators identified theme approval and detection of missing themes as areas for improvement.
Full details are in the original sources.