Hackers influence ChatGPT and Gemini to direct users to scam centers
Which summary reads better? Pick one — models revealed after.Both summaries are AI-generated.
Search-integrated LLMs are surfacing attacker-planted URLs and phone numbers as authoritative "customer support" contacts, meaning model output now inherits the poisoning of whatever web content it retrieves. If you ship any RAG or search-augmented feature, treat retrieved contact details, links, and action-triggering strings as untrusted input—validate against allowlists before displaying or acting on them, because your model will confidently hand users straight to scam infrastructure.
Hackers manipulated both ChatGPT and Gemini to redirect users to scam centers, demonstrating vulnerabilities in LLM-based systems that can be exploited through prompt injection or adversarial inputs. This breach highlights the critical need for robust input sanitization, adversarial testing, and ongoing monitoring in production deployments to prevent malicious exploitation and ensure user safety.