South KoreaTechnology and telecom2020–2021Scatter Lab
Iruda AI chatbot
A startup's chatbot trained on users' private chats was pulled within weeks and fined by the privacy regulator
Problem
Scatter Lab, a Seoul startup, wanted a chatbot that could hold natural, friendly conversations in Korean. It already held large volumes of KakaoTalk conversations that users had uploaded to its relationship-analysis apps, Science of Love and TextAt.
Approach
The company trained the Iruda model on about 9.4 billion KakaoTalk sentences from some 600,000 users of its other apps and released the chatbot on Facebook Messenger in late 2020. Within a month it drew more than 750,000 users, but in early January 2021 it was criticised for hateful and discriminatory replies and for exposing personal data, and it was taken offline. In April 2021 the Personal Information Protection Commission found eight violations of the privacy act, including using names, phone numbers and addresses without deleting or encrypting them, and imposed penalties of KRW 103.3 million.
Results
Takeaway
The chatbot's fluency attracted users quickly, but training-data handling caused the failure. Consent for one service did not cover reuse in another product; the regulator treated this as the first AI-company sanction for data handling.
Categories
- Size
- Small business
- Industry
- Technology and telecom
- Business function
- IT and R&DCustomer service
- Innovation type
- Product and service innovation
- AI technology
- Conversational and voice AI
- Adoption
- Built in-house
- Evidence
- Regulator, audit or court
Sources
Source summaries
Reports the Personal Information Protection Commission's decision of 28 April 2021 against Scatter Lab, totalling KRW 103.3 million (a KRW 55.5 million surcharge and KRW 47.8 million in fines). It lists eight violations, five tied to the Science of Love and TextAt apps, two to Iruda and one to code posted on GitHub. It notes that about 9.4 billion KakaoTalk sentences from roughly 600,000 users were used without removing names, phone numbers or addresses.
The Future of Privacy Forum explains the regulator's decision. It says Iruda reached more than 750,000 Facebook Messenger users in under a month before controversy in early January 2021 led to its removal. It details the findings, including the reuse of data beyond its original purpose, the addition of 100 million messages to the response database, GitHub uploads containing personal details and data from over 200,000 children under 14 collected without parental consent.
Full details are in the original sources.
