Research analyzing web traffic and AI search outputs reveals that generative AI tools and search assistants, such as ChatGPT, are increasingly guiding users to fraudulent websites and harmful online schemes.
AI Search Tools and Emerging Risks
The study shows a growing issue in how large language models gather, organize, and present web links. Unlike conventional search engines, which depend on domain authority and proven indexing methods to exclude harmful content, AI search tools condense extensive web data. This process sometimes involves collecting and highlighting unreliable or compromised sources containing phishing links or deceptive online stores.
Recent insights from Eset, a leading European cybersecurity firm, raise alarms about users relying on AI tools for online searches. Many assume that links from AI responses are safe, but reports indicate that ChatGPT’s results have directed users to dangerous websites previously flagged as unsafe.
Among the identified sites were counterfeit online stores, cryptocurrency scams, and fake login pages. Some AI-generated results also connected users to harmful code capable of stealing personal information.
Unquestioning Trust in Digital Guidance
“The danger lies in the unquestioning trust we place in these tools’ responses,” notes Scott Leman, Eset NZ country manager. “A user seeks help from an AI tool, follows its provided link, and lands on a fraudulent login page, scam site, or harmful download.”
Read Also: Small firms overestimate AML compliance readiness
Since the recommendation originates from a trusted tool, users may be less inclined to verify the destination’s authenticity or safety. This misplaced confidence offers a significant opportunity for malicious actors, who design fraudulent sites to be easily discovered by AI scrapers.
To distort crawling data, attackers flood the web with keyword-rich, AI-generated content aimed at rapid ranking and inclusion in automated scraping processes. This content then redirects users to harmful pages or domains through links, documents, QR codes, or fake websites requesting sensitive information. These links appear in AI summaries as legitimate suggestions for customer support, bookings, or shopping platforms.
Scammers also insert fake contact details and links into seemingly authentic forum posts or press releases, which AI tools compile and present as reliable solutions.
Narrowing Response Time for Businesses
“Historically, businesses could assume they had time to address a newly discovered security flaw,” Leman explains. “That buffer is vanishing. AI can swiftly analyze a new security patch, reverse-engineer the vulnerability it addresses, and transform that knowledge into an effective attack.”
