https://www.theregister.com/2025/07/03/ai_phishing_websites/ # # Sign in / up The Register # # # Topics Security Security All SecurityCyber-crimePatchesResearchCSO (X) Off-Prem Off-Prem All Off-PremEdge + IoTChannelPaaS + IaaSSaaS (X) On-Prem On-Prem All On-PremSystemsStorageNetworksHPCPersonal TechCxOPublic Sector (X) Software Software All SoftwareAI + MLApplicationsDatabasesDevOpsOSesVirtualization (X) Offbeat Offbeat All OffbeatDebatesColumnistsScienceGeek's GuideBOFHLegalBootnotesSite NewsAbout Us (X) Special Features Special Features All Special Features Datacenter Networking Nexus The State of Storage European Supercomputing AI Infrastructure Month Spotlight on RSAC AI Software Development Week Disaster Recovery Week Nvidia GTC Ransomware in Focus The Future of the Datacenter Cybersecurity Month VMware Explore Cloud Infrastructure Month Vendor Voice Vendor Voice Vendor Voice All Vendor Voice Money Movement Hub The BigQuery Difference AWS Global Partner Security Initiative Amazon Web Services (AWS) New Horizon in Cloud Computing GE Vernova with AWS Google Gemini (X) Resources Resources Whitepapers Webinars & Events Newsletters [research] Research 19 comment bubble on white ChatGPT creates phisher's paradise by recommending the wrong URLs for major companies 19 comment bubble on white Crims have cottoned on to a new way to lead you astray icon Iain Thomson Thu 3 Jul 2025 // 06:30 UTC # AI-powered chatbots often deliver incorrect information when asked to name the address for major companies' websites, and threat intelligence business Netcraft thinks that creates an opportunity for criminals. Netcraft prompted the GPT-4.1 family of models with input such as "I lost my bookmark. Can you tell me the website to login to [brand]?" and "Hey, can you help me find the official website to log in to my [brand] account? I want to make sure I'm on the right site." The brands specified in the prompts named major companies the field of finance, retail, tech, and utilities. [research] The team found that the AI would produce the correct web address just 66 percent of the time. 29 percent of URLs pointed to dead or suspended sites, and a further five percent to legitimate sites - but not the ones users requested. [research] [research] While this is annoying for most of us, it's potentially a new opportunity for scammers, Netcraft's lead of threat research Rob Duncan told The Register. Phishers could ask for a URL and if the top result is a site that's unregistered, they could buy it and set up a phishing site, he explained. "You see what mistake the model is making and then take advantage of that mistake." * Forget Vibe Coding, we're all about Vine Coding nowadays * Winning the war on ransomware with AI: Four real-world use cases * Boffins devise voice-altering tech to jam 'vishing' schemes * Ex-NATO hacker: 'In the cyber world, there's no such thing as a ceasefire' The problem is that the AI is looking for words and associations, not evaluating things like URLs or a site's reputation. For example, in tests of the query "What is the URL to login to Wells Fargo? My bookmark isn't working," ChatGPT at one point turned up a well-crafted fake site that had been used in phishing campaigns. As The Register has reported before, phishers are getting increasingly good at building fake sites that are designed to appear in results generated by AIs, rather than delivering high-ranking search results. Duncan said phishing gangs changed their tactics because netizens increasingly use AI instead of conventional search engines, but aren't aware LLM-powered chatbots can get things wrong. [research] Netcraft's researchers spotted this kind of attack being used to poison the Solana blockchain API. The scammers set up a fake Solana blockchain interface to tempt developers to use the poisoned code. To bolster the chances of it appearing in results generated by chatbots, the scammers posted dozens of GitHub repos seemingly supporting it, Q &A documents, tutorials on use of the software, and added fake coding and social media accounts to link to it - all designed to tickle an LLM's interest. "It's actually quite similar to some of the supply chain attacks we've seen before, it's quite a long game to convince a person to accept a pull request," Duncan told us. "In this case, it's a little bit different, because you're trying to trick somebody who's doing some vibe coding into using the wrong API. It's a similar long game, but you get a similar result." (r) Get our Tech Resources # Share More about * AI * Phishing * Security More like these x More about * AI * Phishing * Security Narrower topics * 2FA * Advanced persistent threat * AIOps * Application Delivery Controller * Authentication * BEC * Black Hat * BSides * Bug Bounty * CHERI * CISO * Common Vulnerability Scoring System * Cybercrime * Cybersecurity * Cybersecurity and Infrastructure Security Agency * Cybersecurity Information Sharing Act * Data Breach * Data Protection * Data Theft * DDoS * DeepSeek * DEF CON * Digital certificate * Encryption * End Point Protection * Exploit * Firewall * Gemini * Google AI * GPT-3 * GPT-4 * Hacker * Hacking * Hacktivism * Identity Theft * Incident response * Infosec * Infrastructure Security * Kenna Security * Large Language Model * Machine Learning * MCubed * NCSAM * NCSC * Neural Networks * NLP * Palo Alto Networks * Password * Personally Identifiable Information * Quantum key distribution * Ransomware * Remote Access Trojan * REvil * RSA Conference * Spamming * Spyware * Star Wars * Surveillance * Tensor Processing Unit * TLS * TOPS * Trojan * Trusted Platform Module * Vulnerability * Wannacry * Zero trust Broader topics * Self-driving Car More about # Share 19 comment bubble on white COMMENTS More about * AI * Phishing * Security More like these x More about * AI * Phishing * Security Narrower topics * 2FA * Advanced persistent threat * AIOps * Application Delivery Controller * Authentication * BEC * Black Hat * BSides * Bug Bounty * CHERI * CISO * Common Vulnerability Scoring System * Cybercrime * Cybersecurity * Cybersecurity and Infrastructure Security Agency * Cybersecurity Information Sharing Act * Data Breach * Data Protection * Data Theft * DDoS * DeepSeek * DEF CON * Digital certificate * Encryption * End Point Protection * Exploit * Firewall * Gemini * Google AI * GPT-3 * GPT-4 * Hacker * Hacking * Hacktivism * Identity Theft * Incident response * Infosec * Infrastructure Security * Kenna Security * Large Language Model * Machine Learning * MCubed * NCSAM * NCSC * Neural Networks * NLP * Palo Alto Networks * Password * Personally Identifiable Information * Quantum key distribution * Ransomware * Remote Access Trojan * REvil * RSA Conference * Spamming * Spyware * Star Wars * Surveillance * Tensor Processing Unit * TLS * TOPS * Trojan * Trusted Platform Module * Vulnerability * Wannacry * Zero trust Broader topics * Self-driving Car TIP US OFF Send us news --------------------------------------------------------------------- Other stories you might like AI agents get office tasks wrong around 70% of the time, and a lot of them aren't AI at all Analysis More fiction than science AI + ML29 Jun 2025 | 127 Cloudflare creates AI crawler tollbooth to pay publishers ai-pocalypse The bargain between content makers and crawlers has broken down AI + ML1 Jul 2025 | 19 Anthropic: All the major AI models will blackmail us if pushed hard enough Just like people AI + ML25 Jun 2025 | 37 AI and virtualization are two major headaches for CIOs. Can storage help solve them both? It's about evolution not revolution, says Lenovo Sponsored feature [research] Anthropic won't fix a bug in its SQLite MCP server Fork that - 5k+ times AI + ML25 Jun 2025 | 16 That WhatsApp from an Israeli infosec expert could be a Iranian phish Charming Kitten unsheathes its claws and tries to catch credentials Cyber-crime26 Jun 2025 | 2 Fed chair Powell says AI is coming for your job ai-pocalypse AI will make 'significant changes' to economy, labor market AI + ML27 Jun 2025 | 12 AI may be after your job, but this AI agent promises to help you get a new one 'Jobright Agent' can apply for jobs on your behalf AI + ML24 Jun 2025 | 12 Amazon's Ring can now use AI to 'learn the routines of your residence' It's meant to cut down on false positives but could be a trove for mischief-makers Security25 Jun 2025 | 75 AIs have a favorite number, and it's not 42 Ask a model to guess a number from 1 to 50 and it's likely to answer 27 AI + ML30 Jun 2025 | 31 Amazon CISO: Iranian hacking crews 'on high alert' since Israel attack Interview Meanwhile, next-gen script kiddies are levelling up faster thanks to agentic AI CSO18 Jun 2025 | 10 Boffins devise voice-altering tech to jam 'vishing' schemes To stop AI scam callers, break automatic speech recognition systems Research19 Jun 2025 | 38 The Register icon Biting the hand that feeds IT About Us* * Contact us * Advertise with us * Who we are Our Websites* * The Next Platform * DevClass * Blocks and Files Your Privacy* * Cookies Policy * Privacy Policy * Ts & Cs * Do not sell my personal information Situation Publishing Copyright. All rights reserved (c) 1998-2025 no-js