NetherlandsPublic sector2013–2021Dutch Tax and Customs Administration
Childcare benefits risk classification model
A Dutch fraud-risk algorithm that used nationality helped wrongly accuse tens of thousands of parents and brought down the government
Problem
The Dutch tax authority pays childcare benefits to working parents and wanted to catch incorrect or fraudulent claims early. Checking every claim by hand was not possible.
Approach
From 2013 the authority used an algorithmic risk classification model to score childcare benefit applications and flag those to check. According to Amnesty International, not holding Dutch nationality was one of the risk factors, so non-Dutch applicants received higher scores, and the system adapted over time with little meaningful human oversight. Tens of thousands of parents and carers, mostly on low incomes, were wrongly treated as fraudsters and ordered to repay benefits. The scandal led the Dutch cabinet to resign in January 2021.
Results
Takeaway
The harm came from combining a risk score that used nationality with enforcement that treated a flag as proof. Because the model set the targets and staff followed them, the bias spread across years of decisions before it was found.
Categories
- Size
- Large enterprise
- Industry
- Public sector
- Business function
- Finance, risk and legal
- Innovation type
- Process innovation
- AI technology
- Predictive analytics and ML
- Adoption
- Built in-house
- Evidence
- Independent study
Sources
Source summaries
Presents Amnesty's findings that an algorithmic system introduced by the Dutch tax authorities in 2013 to flag incorrect and fraudulent childcare benefit claims used Dutch nationality as a risk factor, giving non-Dutch nationals higher scores. Says the system had a self-learning element with no meaningful human oversight. States that tens of thousands of mostly low-income parents and carers were falsely accused of fraud and that the scandal brought down the government in January. Calls for a ban on using nationality and ethnicity in risk scoring.
Amnesty described how the tax authorities used an algorithmic decision-making system to build risk profiles of childcare benefit applicants in order to detect inaccurate and potentially fraudulent claims early. It concluded that using nationality as a risk factor led to discrimination and racial profiling, and described the Netherlands as an early mover in automating social security enforcement.
Full details are in the original sources.
