Skip to main content

Amazon discovered a 'high volume' of CSAM in its AI training data but isn't saying where it came from

By

The National Center for Missing and Exploited Children said it received more than 1 million reports of AI-related child sexual abuse material (CSAM) in 2025. The "vast majority" of that content was reported by Amazon, which found the material in its training data, according to an investigation by Bloomberg. In addition, Amazon said only that it obtained the inappropriate content from external sources used to train its AI services and claimed it could not provide any further details about where the CSAM came from. 

"This is really an outlier," Fallon McNulty, executive director of NCMEC’s CyberTipline, told Bloomberg. The CyberTipline is where many types of US-based companies are legally required to report suspected CSAM. “Having such a high volume come in throughout the year begs a lot of questions about where the data is coming from, and what safeguards have been put in place.” She added that aside from Amazon, the AI-related reports the organization received from other companies last year included actionable data that it could pass along to law enforcement for next steps. Since Amazon isn’t disclosing sources, McNulty said its reports have proved “inactionable.”

"We take a deliberately cautious approach to scanning foundation model training data, including data from the public web, to identify and remove known [child sexual abuse material] and protect our customers," an Amazon representative said in a statement to Bloomberg. The spokesperson also said that Amazon aimed to over-report its figures to NCMEC in order to avoid missing any cases. The company said that it removed the suspected CSAM content before feeding training data into its AI models. 

Safety questions for minors have emerged as a critical concern for the artificial intelligence industry in recent months. CSAM has skyrocketed in NCMEC's records; compared with the more than 1 million AI-related reports the organization received last year, the 2024 total was 67,000 reports while 2023 only saw 4,700 reports. 

In addition to issues such as abusive content being used to train models, AI chatbots have also been implicated in several dangerous or tragic cases involving young users. OpenAI and Character.AI have both been sued after teenagers planned their suicides with those companies' platforms. Meta is also being sued for alleged failures to protect teen users from sexually explicit conversations with chatbots.

This article originally appeared on Engadget at https://ift.tt/F0Afveq

from Engadget is a web magazine with obsessive daily coverage of everything new in gadgets and consumer electronics https://ift.tt/F0Afveq
via IFTTT
Labels:

Comments

Post a Comment

Popular posts from this blog

Instagram accidentally reinstated Pornhub’s banned account

By
After years of on-and-off temporary suspensions, Instagram permanently banned Pornhub’s account in September. Then, for a short period of time this weekend, the account was reinstated. By Tuesday, it was permanently banned again. “This was done in error,” an Instagram spokesperson told TechCrunch. “As we’ve said previously, we permanently disabled this Instagram account for repeatedly violating our policies.” Instagram’s content guidelines prohibit  nudity and sexual solicitation . A Pornhub spokesperson told TechCrunch, though, that they believe the adult streaming platform’s account did not violate any guidelines. Instagram has not commented on the exact reasoning for the ban, or which policies the account violated. It’s worrying from a moderation perspective if a permanently banned Instagram account can accidentally get switched back on. Pornhub told TechCrunch that its account even received a notice from Instagram, stating that its ban had been a mistake (that message itse...
Post a Comment
Read more

The Nintendo Switch has been the US’s bestselling console for 23 straight months

By
Photo by James Bareham / The Verge It’s been a good two years for the Nintendo Switch. According to Nintendo, the gaming tablet has been the bestselling console in the US for 23 straight months. And according to data from the NPD Group, it just had its best October ever, moving 735,926 units of both the Switch and Switch Lite in the US. The company says that represents a 136 percent increase compared to last year. To date, the Switch has sold 22.5 million units in the US, and last week Nintendo revealed that more than 68 million units have been sold globally . “We’re excited about our momentum,” says Nick Chavez, Nintendo of America’s SVP of sales and marketing. Chavez puts the company’s big October down to two main factors. One is a better supply of stock; this year in particular, it’s often been hard to find a Switch on store shelves. This has only been exacerbated by increased demand due to a combination of the pandemic and the breakout success of Animal Crossing: New Horizons . ...
Post a Comment
Read more

Richard Branson is renting out his private estate on his second Caribbean island for the first time at $25,000 a night. Take a look inside the 3-villa compound.

By
The Branson Estate. Virgin Limited Edition Richard Branson's private Caribbean island, Moskito Island, is opening to the public for the first time. Guests can rent Branson's entire personal estate - which can house 22 guests in three villas - starting at $25,000 per night. It's just 2.5 miles from his other private island, Necker Island. See more stories on Insider's business page . Richard Branson, billionaire business magnate and Virgin Group founder, is opening up his private estate on his second Caribbean island to the public for the first time. Richard Branson in Australia in November 2019. Getty Images/Don Arnold/WireImage Branson, who's worth $4.9 billion and whose businesses ventures span airlines, luxury travel, space travel, telecom, and more, bought Moskito Island for a reported $10 million in 2007. Moskito Island is not to be confused with Branson's  other private Caribbean island, Necker Island, which Branson bought in 1978 and wh...
Post a Comment
Read more