Attackers tried to clone Gemini by prompting it 100,000 times

Copying a successful product has been a practice as long as tools and technologies have existed, but chatbots are a special case. Competitors can’t pull them apart, but they can ask the AI as many questions as you like in an attempt to figure out how it works. According to a new report from Google, that’s exactly how some actors have been trying to clone Gemini. In one case, Google says a single campaign sent more than 100,000 prompts to the chatbot, in what it describes as a large-scale model-extraction attempt.

The findings come from Google’s latest Threat Intelligence Group report (via NBC News), which outlines a rise in so-called “distillation” attacks. In simple terms, that means repeatedly querying a model to study how it responds, then using those answers to train a competing system. Google says this activity violates its terms of service and amounts to intellectual property theft, even though the attackers are using legitimate API access rather than breaking into its systems.

One campaign highlighted in the report specifically targeted Gemini’s reasoning capabilities. While Gemini doesn’t normally expose its full internal “chain of thought,” Google says attackers tried to coerce it into revealing more detailed reasoning abilities. The scale of the prompts — over 100,000 in this case — suggests an effort to replicate Gemini’s ability to reason across different tasks and even in non-English languages. Google says its systems detected the activity in real time and adjusted protections to prevent internal reasoning details from being exposed.

While Google declined to name suspects, it says most of the extraction attempts appear to have come from private companies and researchers seeking a competitive edge. John Hultquist, chief analyst at Google’s Threat Intelligence Group, told NBC News that as more businesses build custom AI systems trained on sensitive data, similar cloning attempts could become more common across the industry.

Beyond model extraction, the report also outlines other ways Gemini has been misused. Google describes instances of threat actors experimenting with AI-assisted phishing campaigns and even malware that calls Gemini’s API to generate code on the fly. In each case, Google says it disabled associated accounts and updated safeguards to limit further abuse.