Humans or AI? Who or what can scam you more effectively?
BYU research finds AI knows how to use a personal touch to dupe people
A text from your neighborhood gardening club friend Ben offering a deal on seeds might seem too specific to be a scam. But that kind of personalization is exactly what can make phishing messages convincing.
Scammers are doubling down on an evolving method called spear phishing — using personal information such as a job, coworker, hobby, social media posts or even voice — to create compelling messages that encourage people to click on a link. Some 90% of corporate hacks originate from spear phishing attacks.
To make matters worse, scammers are now tasking AI agents with drumming up deceptive phishing attacks en masse. Recent BYU research examined AI-generated deception across text messages and emails and found they can be even more convincing than human-created scams.
- AI-generated messages fooled people 28% of the time; human-authored 21%
- 80% of the time AI performed the same or better than humans in generating a click
- Messages mentioning a coworker are 2.3x more likely to be clicked on than messages from generic organizations
- Basic information from company websites, LinkedIn profiles, social media accounts, and organizational directories is enough to create highly persuasive messages
“Our research is just the latest to show how sophisticated the current state of AI is,” said Jerson Francia, a BYU cybersecurity PhD student and co-author on several papers on the topic.
To check a suspicious message:
- Visit websites by typing the address yourself instead of clicking a message link.
- Contact the person or organization through a phone number or method you already know is legitimate.
- Never give passwords, ID numbers, financial info, or security codes to an unexpected request.
- Report suspicious messages to your workplace, school, etc.
The research, led by BYU cybersecurity professor Derek Hansen, found participants struggled to determine whether a message was written by AI or a person, correctly identifying the source only 52% of the time. And AI matched or outperformed humans in generating a click in 80% of cases.
In other words, you may no longer be able to recognize AI-generated phishing simply by how a message is written.
Text messages and emails that reference personal details such as participants’ jobs, hobbies, workplaces or social media activity can be harder to distinguish from legitimate messages. As more details about people’s lives become readily available online, AI can quickly gather that information and use it to create messages that appear trustworthy.
Messages that included job-related personal details were the most convincing. References to a person’s workplace or occupation made the messages feel more relevant and credible, showing how even basic professional information can be used to make a scam more persuasive.
“This study opened my eyes to just how good AI is at creating messages that use personal information about individuals,” Hansen said. “AI can reduce the time and effort required to create personalized spear-phishing messages, thus making them more effective and more common.”
AI can amplify the persuasive power of personalization by combining publicly available details to produce a compelling message in seconds. Without AI, scammers would have to manually search for information about their targets and compile those details into a convincing message, making the process more time-consuming and difficult to carry out at scale.
Most messages from unknown numbers or emails can easily appear as a scam when it doesn’t apply to you personally. But when it involves someone you know by name or a favorite hobby, Francia recommends verifying unexpected messages through a separate, trusted channel before responding or clicking on a link.
“If you receive a text message from an unknown number, don’t trust it, even if it mentions personal details about your life,” Francia said. “We need to more thoroughly verify the authenticity of messages through other means, rather than relying solely on the content of the message itself.”
Undergraduate students who helped with the research include Matthew Taylor, Rebekah Cornelius, Nathan Seneca, Spencer Smith, Shydra Murray and Malaya Canite. Other BYU faculty involved include Benjamin Schooley and Gregory Snow.
