United StatesTechnology and telecom2021 – ongoingNGL Labs
AI content moderation for anonymous messages
NGL agreed to a $5 million settlement and an under-18 ban after regulators challenged its AI moderation claims
Problem
Anonymous messaging exposed NGL users to bullying and threats. The company marketed the app to teenagers with assurances about safety.
Approach
NGL said its moderation used deep learning and pattern matching to filter harmful messages. The FTC and Los Angeles district attorney alleged that bullying and threats still reached users. The proposed settlement would ban access for users under 18 and prohibit misleading AI claims, subject to court approval.
Results
Takeaway
The settlement covered privacy and subscription practices as well as moderation claims. Its payment was not a penalty solely for AI failures.
Categories
- Size
- Small business
- Industry
- Technology and telecom
- Business function
- Customer service
- Innovation type
- Process innovation
- AI technology
- Predictive analytics and ML
- Adoption
- Built in-house
- Goal
- Quality and risk
- Evidence
- Regulator, audit or court
Sources
Source summaries
The FTC alleged misleading moderation claims alongside privacy and subscription violations. It announced settlement terms requiring payments and an adult-only service, subject to court approval.
Full details are in the original sources.