San Francisco City Attorney David Chiu announced a first-of-its-kind lawsuit today targeting 16 of the world’s most-visited websites that create and distribute non-consensual deepfakes.

The lawsuit, filed in San Francisco Superior Court, alleges that these websites violate state and federal laws prohibiting deepfake pornography, revenge pornography, and child pornography.

The mainstream media often portrays any sort of AI porn as a deepfake, but the reality is, there is a vast difference between the two, and that difference all comes down to one word – CONSENT.

A deepfake is not the same as regular Ai porn. A deep fake is an image or video that has been altered and manipulated to misrepresent someone as doing or saying something that was not actually done or said.  The material frequently merges a victim’s face with a body in a pornographic video. Generative artificial intelligence models can also create audio, videos and images that are entirely fake but look and sound realistic. They “fake” the image (or video), to make someone is doing something that they didn’t do, such as having sex with someone that they’ve never even met.

This is something that, as a whole, the adult industry has been against, as those who appear in a deepfake haven’t consented. Consent is always key.

“This investigation has taken us to the darkest corners of the internet, and I am absolutely horrified for the women and girls who have had to endure this exploitation,” said City Attorney David Chiu. “Generative AI has enormous promise, but as with all new technologies, there are unintended consequences and criminals seeking to exploit the new technology. We have to be very clear that this is not innovation—this is sexual abuse.”

The rapid development and availability of sophisticated AI models have enabled bad actors to misuse the technology, training it on pornographic and child sexual abuse materials. The result is the creation of realistic, nonconsensual pornographic images of real, identifiable women and girls. These AI-generated images have caused immeasurable harm, affecting everyone from celebrities to middle school students.

One particularly alarming incident occurred in February 2024, when AI-generated nude images of 16 eighth-grade students circulated among their peers at a California middle school. These images, virtually indistinguishable from real photographs, have been used to extort, bully, and humiliate victims. The Federal Bureau of Investigation (FBI) has also reported an increase in extortion schemes involving nonconsensual AI-generated pornography.

Victims of this digital exploitation often find little recourse. Once their deepfaked images are distributed, controlling their own image becomes nearly impossible. The defendants in this lawsuit operate websites that allow users to upload clothed images of real people and generate realistic pornographic versions. Some sites specifically promote the nonconsensual nature of their services, with one website stating, “Imagine wasting time taking her out on dates when you can just use [website] to get her nudes.”

The lawsuit, filed on behalf of the People of the State of California, seeks the removal of the defendants’ websites, injunctive relief to prevent further unlawful conduct, and civil penalties. The legal action is based on violations of California’s Unfair Competition Law, as well as state and federal laws against deepfake and child pornography.

The case, People of the State of California v. Sol Ecom, Inc., et al., is a pivotal moment in the fight against AI-generated exploitation. The lawsuit is not only aimed at holding these websites accountable but also serves as a broader call to address the societal implications of AI misuse.

The City Attorney’s Office encourages anyone who has been a victim of nonconsensual deepfake pornography or has relevant information to come forward. Victims and witnesses can contact the office through its consumer complaint web portal or hotline at (415) 554-3977.

As technology continues to evolve, so too must the legal frameworks that protect individuals from its misuse. This lawsuit represents a significant step in addressing the exploitation of women and girls through AI-generated pornography and underscores the need for swift action against those who profit from such abuse.