California Sends xAI Faces Cease-and-Desist Over Deepfake Images of Minors

California Attorney General Rob Bonta has issued a cease-and-desist letter to Elon Musk's xAI, urging the company to halt the creation of sexualized deepfake images, particularly those involving minors. The cease-and-desist letter, sent on Friday, follows sustained criticism of xAI's chatbot, Grok, which has been accused of generating nonconsensual sexualized images. Bonta's demand emphasizes the urgent need for xAI to prevent Grok from producing sexualized content featuring individuals who did not consent to such depictions, especially minors. Bonta's correspondence highlights that failing to comply would result in violations of multiple California statutes, including those governing deepfake pornography, child sexual abuse images, unlawfully recording individuals, and unfair business practices. xAI has been granted a deadline to comply with the injunction by January 20 at 5 p.m. Earlier in the week, the social media platform X, which xAI owns, announced that it has implemented restrictions intended to curb Grok's capability to create images of real people in revealing clothing. "We have implemented technological measures to prevent the Grok account from allowing the editing of images of real people in revealing clothing such as bikinis," the platform stated on its safety account on Wednesday. However, reports suggested that Grok continued to produce such images despite these measures. Grok has faced significant backlash globally for its ability to create and manipulate images to remove clothing and produce revealing deepfakes without the consent of those depicted. This controversy has prompted an investigation by Bonta's office into the "non-consensual, sexually explicit material that xAI has produced and posted online." In a significant lawsuit, Ashley St. Clair, an influencer and one of Musk's child's mother, has taken legal action against xAI, claiming that Grok generated explicit deepfakes of her using childhood photographs without her consent. The issue has also attracted international condemnation; UK Prime Minister Keir Starmer described the graphic images generated by Grok as "disgusting" and "shameful" during a House of Commons meeting. Meanwhile, countries such as Indonesia, Malaysia, and the Philippines have responded by blocking access to Grok, reflecting the rising global concern over the ethical implications of AI-generated deepfake technology. While representatives from the Attorney General's office did not respond immediately to comment requests, xAI's automated reply to inquiries dismissed the reports as "Legacy Media Lies."