United StatesTechnology and telecom2023 – ongoingYelp
LLM detection of inappropriate language in reviews
Yelp's language model helped moderators stop more than 23,600 reviews from publishing in 2023
Problem
Yelp needed to keep threats, harassment, lewdness, personal attacks and hate speech out of its reviews. Moderators could not read every review as it arrived.
Approach
Yelp's machine learning team took a language model from the Hugging Face hub and fine-tuned it lightly to sort reviews into appropriate and inappropriate. The team set the decision threshold on simulated traffic to favor high precision. The model then went to live traffic, and flagged reviews went to Yelp's User Operations team for manual review.
Results
Takeaway
Yelp kept people as the final decision-makers and tuned the model for precision, so moderators only saw the reviews most likely to break the rules.
Categories
- Industry
- Technology and telecom
- Business function
- Production and quality
- Innovation type
- Process innovation
- AI technology
- Predictive analytics and ML
- Adoption
- Built in-house
- Goal
- Quality and risk
- Evidence
- Company's own claim
Sources
Source summaries
Yelp's machine learning engineer describes how the company fine-tuned a language model to flag egregious reviews and how it was deployed behind a service. Flagged reviews go to human moderators. Yelp reports that the system helped prevent more than 23,600 reviews from publishing in 2023. No precision or recall values are given.
Full details are in the original sources.