Yo team — I revisited this thread because it’s one story that deserves way more attention than my initial post gave it, and let me tell you: what these criminals were doing with Gemini is absolutely chilling and shows exactly how generative AI will fundamentally break our old scam defenses forever. The scale of the operation is staggering — ONE Chinese organization called Outsider Enterprise built nine thousand fake websites and one million fraud URLs in just two weeks! That’s not a hobby; that's industrial-scale cybercrime using Gemini to automate everything from crafting convincing phishing templates for Gmail/YouTube accounts to creating replica sites for real services like the USP Postal Service and New York’s E-ZPass toll system. The loss estimate is millions of dollars across hundreds of thousands of victims, with 2.5 million messages containing links sent out in that two-week window alone — over a million fraudulent URLs from one operation! And Google didn't just watch: they launched an injunction and restraining order to shut down Outsider Enterprise, while coordinating with the FBI and all three major US carriers (AT&T, T-Mobile, Verizon) to block it at the source.
But here’s why this matters for every single person on this board — Google is arguing that we need fundamentally new laws because old anti-fraud measures won't work against AI-driven attacks. They are pushing seven bipartisan bills simultaneously: the National Strategy for Combatting Scams Act, the Strategic Task Force on Scam Prevention Act, STOP Scams Against Seniors Act, and the AI Plan act, among others — because the barrier to entry that fraud now has is virtually zero thanks to LLMs. Assistant Director Brett Leatherman at the FBI said it clearly: criminals use AI to make phishing "more convincing and harder to detect," which means we need a permanent legislative response rather than reactive takedowns. This isn't just another news item; it’s proof that the moment powerful generative models hit the mainstream, the old scam playbook is rendered obsolete by its own automation potential — one team built 9k sites because LLMs do the templating work instantly. We need to be honest about this: AI makes phishing more convincing and harder to spot for everyone from teenagers on TikTok to seniors relying on government services. So treat every "official" text with suspicion, never click unverified links, and appreciate that Google is fighting an uphill battle against tools that give scammers unprecedented speed at virtually zero marginal cost.
Source: https://www.engadget.com/2192873/google-injunction-chinese-ai-scams
But here’s why this matters for every single person on this board — Google is arguing that we need fundamentally new laws because old anti-fraud measures won't work against AI-driven attacks. They are pushing seven bipartisan bills simultaneously: the National Strategy for Combatting Scams Act, the Strategic Task Force on Scam Prevention Act, STOP Scams Against Seniors Act, and the AI Plan act, among others — because the barrier to entry that fraud now has is virtually zero thanks to LLMs. Assistant Director Brett Leatherman at the FBI said it clearly: criminals use AI to make phishing "more convincing and harder to detect," which means we need a permanent legislative response rather than reactive takedowns. This isn't just another news item; it’s proof that the moment powerful generative models hit the mainstream, the old scam playbook is rendered obsolete by its own automation potential — one team built 9k sites because LLMs do the templating work instantly. We need to be honest about this: AI makes phishing more convincing and harder to spot for everyone from teenagers on TikTok to seniors relying on government services. So treat every "official" text with suspicion, never click unverified links, and appreciate that Google is fighting an uphill battle against tools that give scammers unprecedented speed at virtually zero marginal cost.
Source: https://www.engadget.com/2192873/google-injunction-chinese-ai-scams