AI-Generated Avatars Enable Online Scams at Scale
Scammers are using AI-generated avatars to lure consumers in with a compelling narrative, before tricking them into downloading malware under the guise of ‘free’ access to commercial software products.
A YouTube video of an influencer with thousands of followers, promises to show you a workaround to get access to the premium tier of Claude for free, just by installing a piece of software. It sounds too good to be true, and it is. The influencer doesn’t exist, neither do their hundreds of engaged commentators, and the software creates a backdoor in your computer.
AI-generated avatars enable playbook for running online scams at scale
Duping users into installing malware by promising free or cracked premium software is one of the oldest scams on the internet. But where threat actors used to rely on a malicious URL and a paragraph of text convincing targets to click the link, the advent of generative AI has rewritten the playbook and enabled a far more effective delivery mechanism for several types of scam.
Generative AI has made it easier for threat actors to weaponize narrative through the creation of highly realistic synthetic influencers. Videos featuring these gen-AI avatars can now run to several minutes in length, giving the scammer more opportunity to lure the victim in with a convincing story.
Longer form synthetic videos deliver compelling scam narrative
This attack leverages the dissemination strategy we document in the attack framework accompanying the Narrative Kill Chain. Where deepfake-enabled scam narratives are introduced and distributed through high-velocity social platforms like YouTube, short-form video content is structurally advantaged by algorithmic ranking systems, benefiting from recommendation engines optimized for engagement rather than verification.
In one example, our Threat Intelligence team found a scammer using a bought or compromised YouTube channel with over 10 thousand followers, pushing malicious links to almost 90k views.
In a number of videos hosted by the same channel, fully synthetic avatars claim to offer free access to premium versions of software including Claude, ChatGPT, and AutoTune.
Victims are directed to fake installers hosted on multiple accounts on GitHub and SourceForge, which install a Deno JavaScript runtime backdoor known as DinDoor. This backdoor then installs a remote access trojan (RAT), allowing the attacker to exfiltrate data including from crypto wallets and browsers and even allows remote control of the compromised machine.
The fraudsters also appear to make good use of bots to automate and inflate the number of comments and engagements on the posted videos. The same comment structure is seen time and again, expressing support for the AI-generated protagonist and ‘confirmation’ that the technique works. But there are also a number of comments from real users claiming to have tried the technique and failed, meaning that numerous people have already been caught by the scam.
Synthetic protagonists appeal to pity to sell deceptive goods
In another deepfake-driven scam technique scam we recently uncovered, AI was used to generate hyper-realistic synthetic protagonists with disabilities, old people, even people with terminal illnesses, with the intent of convincing large numbers of consumers to spend their money on what they think are unique, hand-made items.
But the items in question are mass produced and drop-shipped from a factory, and the social media videos are mass produced by AI-generators to follow a templated and well-structured narrative that fraudsters know will appeal to a broad audience.
Some of the social media profiles of these fictional people have hundreds of thousands of followers and some of the posts have millions of likes, demonstrating just how effective the deepfake-drive scam technique is at going viral.
