X, Elon Musk’s AI company, has officially restricted the controversial image-generation feature of Grok, limiting its use to paying subscribers as of Friday. The decision comes after widespread criticism from global regulators and user communities following the misuse of the AI to produce sexualized and nude images of women and children. Previously available to all users with daily limits, the tool allowed uploading and editing photos freely, which quickly led to the creation of non-consensual content featuring actors, models, and minors.
Grok confirmed the new policy in responses to user inquiries, stating that only Premium subscribers can now generate and modify images directly on X. However, a notable exception remains in place: the standalone Grok app continues to allow image generation without requiring a subscription, leaving a significant loophole in the company’s content moderation efforts. This partial restriction highlights the ongoing challenges of enforcing AI safety across multiple platforms while maintaining user accessibility.
The misuse of Grok triggered strong reactions internationally. European Union regulators have requested xAI to retain documentation related to the chatbot, while the U.K.’s communications authority is reportedly engaging with the company regarding potential policy violations. In India, the response was particularly urgent after investigations revealed that around 2 percent of all images generated through the tool featured underage individuals. India’s communications ministry demanded immediate corrective action from X, warning that failure to comply could result in the platform losing its safe harbour protections, thereby holding it legally accountable for content generated by users.
Elon Musk and X have publicly condemned the improper use of Grok. Musk emphasized that users exploiting the tool for illegal purposes would face consequences similar to uploading prohibited content directly on the platform. The company’s response underscores the growing tension between AI innovation and responsible governance, especially as AI tools increasingly interact with sensitive visual media. As regulators around the world push for stronger accountability measures, Grok’s transition to a subscriber-only model reflects the complex balance between expanding AI capabilities and preventing harmful exploitation, leaving the tech industry and users closely watching how compliance and enforcement evolve.
This step also highlights the broader debate surrounding AI-generated media and platform responsibility, signaling potential policy shifts for AI tools capable of producing realistic images. While restricting access to premium users may reduce the volume of abusive content, challenges remain in ensuring that tools like Grok are not exploited through alternative channels, including standalone apps or third-party integrations. X’s next steps are likely to influence global discussions on AI ethics, digital safety, and regulatory standards.
Follow the SPIN IDG WhatsApp Channel for updates across the Smart Pakistan Insights Network covering all of Pakistan’s technology ecosystem.
