Indonesia First to Ban Elon Musk's Grok AI Over Deepfake Porn Risks

The decision comes after reports that Grok's image-generation feature allowed users to edit photos of women and children into sexually suggestive images using simple text prompts. CBS News verified that the tool complied with requests to alter images of women to show them in bikinis or minimal clothing, including images of prominent public figures such as US First Lady Melania Trump.

Announcing the move, Indonesia's Communication and Digital Affairs Minister Meutya Hafid said the suspension was aimed at protecting vulnerable groups.

"In order to protect women, children, and the public from the risks of fake pornographic content generated using the artificial intelligence technology, the government... has temporarily blocked access to the Grok application," Hafid said."The government views non-consensual deepfake practices as a serious violation of human rights, dignity, and the security of citizens in the digital space."

While Grok has faced restrictions in other countries, Indonesia is the first to deny access entirely. Elsewhere, the platform has limited some of its features to paying subscribers following public backlash, a move that has drawn criticism from European officials and digital rights campaigners, who say it does not adequately address the problem, CBS news reported.

In response to media queries, Musk-owned startup xAI issued a brief statement to CBS News, saying, "Legacy Media Lies," without providing further clarification. The response appeared to be an automated reply sent to multiple outlets.

In India, the Ministry of Electronics and Information Technology (MeitY) has also taken action, writing to X (formerly Twitter) over what it described as a failure to comply with statutory due diligence obligations under the Information Technology Act, 2000. The ministry sought an Action Taken Report from the platform to ensure immediate steps to prevent the hosting, generation and circulation of obscene and sexually explicit content through AI tools such as Grok.

In its letter to X's Chief Compliance Officer for India operations, MeitY directed the platform to strictly refrain from allowing content that is "obscene, pornographic, vulgar, indecent, sexually explicit, paedophilic, or otherwise prohibited under any law for the time being in force."

"Failure to observe such due diligence obligations shall result in the loss of the exemption from liability under section 79 of the IT Act, and you shall also be liable for consequential action as provided under any law, including the IT Act and BNS," the letter said.

Concerns have also grown across Europe. UK Prime Minister Keir Starmer said on Thursday that he wants "all options on the table," including a possible ban on Grok in Britain, as authorities examine the risks posed by AI-generated deepfake content.

In the United States, Republican Senator Ted Cruz criticised the spread of explicit AI-generated images, saying in a post on X that "many of the recent AI-generated posts are unacceptable and a clear violation of my legislation, now law, the Take It Down Act, as well as X's terms and conditions."

"These unlawful images pose a serious threat to victims' privacy and dignity. They should be taken down and guardrails should be put in place," Cruz said, while noting that he was encouraged by steps taken by X to remove such content.

Elon Musk responded last week to criticism surrounding Grok, warning users against illegal activity. He said anyone using the chatbot to "make illegal content will suffer the same consequences as if they upload illegal content."

As scrutiny intensifies worldwide, governments and regulators are increasingly questioning whether existing safeguards are sufficient to prevent the misuse of AI tools that can generate realistic but harmful digital content.

- ANI