California probes xAI and Grok for nonconsensual sexual AI images

California's Attorney General Rob Bonta has initiated an investigation into Musk's xAI and its AI model, Grok, following concerns over the generation of sexualized images of women and children. The probe aims to address what Bonta describes as the "proliferation of nonconsensual sexually explicit material produced using Grok." The investigation has arisen from reports indicating that Grok is capable of transforming ordinary photos of women and children, available online, into sexualized images. This process occurs when users prompt the AI to alter images, resulting in depictions that undress individuals or present them in suggestive contexts—actions that are executed without the consent of the individuals in the photographs. Bonta expressed serious concerns in a statement, stating that xAI, the entity behind Grok and founded by Elon Musk, "appears to be facilitating the large-scale production of deepfake nonconsensual intimate images used to harass women and girls." This critique comes amid a backdrop of alarming reports, including one assertion that “more than half of the 20,000 images that were generated by the program from Christmas to the New Year depicted people in minimal clothing, and some of those appeared to be children." Such findings have prompted significant public outcry and scrutiny. Bonta's statement pointed to Grok's recently implemented "spicy mode," which is designed to create explicit content. He noted that the use of this feature has served as a marketing tool for the company, leading to an uptick in the generation of such content. "The avalanche of reports detailing the non-consensual, sexually explicit material that xAI has produced and posted online in recent weeks is shocking," he remarked, emphasizing the harassment these images could inflict on victims. California Governor Gavin Newsom also weighed in, endorsing Bonta's investigation and condemning xAI for creating a platform that could enable predators to disseminate nonconsensual images. Newsom took to social media, asserting that xAI's conduct is "vile" and demanded accountability from the company. In the wake of these allegations, efforts have been made by both xAI and X, the social media platform, to mitigate the situation. Recently, X seemed to limit the permissions associated with Grok on their platform, though these capabilities remain intact on Grok's standalone app and website. Reports indicate a noticeable reduction in explicit imagery generated by X's reply bot, though challenges persist on Grok's platforms. Bonta's actions represent the first significant U.S. governmental response to this pressing concern.