Hijacking a Phone Call with AI is ‘Fairly Easy’, says IBM Researcher

An IBM security researcher proved how frighteningly simple it would be for a hacker to hijack phone conversations using readily available AI tools.

Security researcher Chenta Lee posted a report detailing how he and a group of colleagues built a tool to intercept and manipulate a live phone conversation. In the simulated case, a bot inserted itself into a call between two persons, in which one shared his bank account number in order to receive a payment. As a “man-in-the-middle”, the bot was able to monitor the exchange, intercept the responses and, using mimicked voices, trick one of the parties into depositing money into a different bank account.

“We were able to modify the details of a live financial conversation occurring between the two speakers, diverting money to a fake adversarial account (an inexistent one in this case), instead of the intended recipient, without the speakers realizing their call was compromised,” Chenta Lee explained in his report.

While the bot would require installation of malware in one of the phones involved or the use of a compromised voice over IP (VoIP) in order to work, the rest of the hijacking process came off smoothly. In the demonstration, there was no way to differentiate between the voices of the human callers and their AI counterparts. The only real signs of deceit came from lagged responses due to latency. These were patched with additional phrasing to make the pauses in the conversation seem natural enough. 

Mr. Lee described the process of building this proof of concept as “surprisingly and scarily easy,” underscoring the large potential that generative AI tools have to cause havoc among individuals and organizations.

“It was fairly easy to construct this highly intrusive capability, creating a significant concern about its use by an attacker driven by monetary incentives and limited to no lawful boundary,” he warned.

AI-cloned voices are one of many nightmare possibilities enabled by the successful and continued development of large language models (LLMs). Enough tools are out there that make voice-cloning a relatively easy task. So much so that US authorities have already taken action to stop the fire from spreading. 

Nevertheless, IBM’s proof of concept shows that AI-mimicked voices can be quite effective in a context where most individuals don’t expect fraud to happen and in which a hijacking is difficult to identify. 

“It feels different; as if the sky’s the limit with this new stuff. And the IBM proof of concept is an example of that,” commented Erik Avakian, Technical Counselor at Info-Tech Research Group. “If we can mimic or spoof audio, or even video now, then what’s real? This takes it to a different level.”

The specific tool developed by Mr. Lee and company is not available for widespread use. However, it should not surprise anyone if a similar bot already exists out there in the darkest corners of the marketplace.

“Generally, if tools are created in a lab environment to test security measures, a version can and will be duplicated by criminals,” warned Michael Scheumack, Chief Innovation Officer at identity theft protection firm IDIQ. “Malware, wi-fi hi-jacking, spoofing calls and the ability to mimic another device have all been used for years to gain some kind of control of a victim’s device.”

A calamity in the making

Any tool with the capabilities of IBM’s proof of concept spells potential doom for unsuspecting business organizations. Cybercriminals already craft highly sophisticated phishing attempts targeting top executives or other relevant members of a company.

Providers of BPO and other third-party services are particularly vulnerable. Since the latest AI explosion began in late 2022, digital security experts warned that the sector has become a favorite target of hackers due to the amounts of sensitive data handled; data which generally belongs to consumers or whole organizations.

Earlier this year, Bank of America pointed to outsourcing giant Infosys as the culprit of a massive data breach which affected over 57,000 of its customers. A couple of bug-hunters who had done security work for Apple broke into the company’s systems through an unidentified CX provider, stealing millions worth of Apple products and services.

The call center industry is no stranger to the prevalence of tech-savvy phone scammers. Experts already warned of AI’s capabilities to cause even more trouble for businesses and customers who interact with call center services. IBM’s concept of an automated phone call hijacker provides a glimpse at possible ordeals to come.

Vendor managers and procurement officers tend to be –or they should be– vigilant of vendors’ security credentials. Periodical security assessments, certification evaluation and clearly stated security demands in SLAs are basic practices in vendor risk management. In a time of increasing risks, extra steps should be taken to ensure security.

“Any time you outsource a service to a vendor, you’re just extending that [cybersecurity] to someone else,” said Erik Avakian. “We need to ensure that their security is at least following the policies that we [as an organization] have in place, if not more.”

Humans, stay vigilant

IBM’s proof of concept underscores one of the harsher truths in cybersecurity: humans are the weakest link in the security chain. 

There’s no shortage of reports out there pointing to the human element as the common denominator in most cybersecurity incidents. Now that bots capable of hijacking live phone calls and mimicking human voices exist, targeting the members of an organization becomes an even more promising avenue of attack for nefarious actors seeking to break into an organization’s systems.

“Humans are usually the weakest link [in cybersecurity]. There’s phishing, but also vishing and mishing and much more,” commented Erik Avakian. “The human has to do so much more to stay vigilant.”

The best policy in such an environment is to keep one’s eyes open. Best practices recommended by cybersecurity experts tend to include the taking of extra steps to make sure communication attempts and interactions are genuine.

“The first step in helping protect yourself against AI scams is critical thinking”, advised Michael Scheumack. “Most scams rely on forcing the victim into fight or flight where there is a strong sense of urgency. Don’t trust everything you see, hear or read. Make sure you verify the source.”

Safe words among groups of individuals –be that families, companies or government organizations– can be of great use, said Mr. Scheumack. Strong passwords, multi-factor authentication and updating devices can also help, as well as keeping an eye on the usage of personal data and one’s own identity. 

IBM’s proof of concept paints a scary prospect for the future of organizational and individual digital security. However, experts insist that such is the name of the game. Every time a new disruptive technology comes into being, security experts have little option but to stay ahead of nefarious actors.

Mr. Lee himself closes his report on a bittersweet note, calling for increased vigilance around the harmful potential of AI without outright arguing for a stop to the technology’s development. It all comes down, he states, to keeping our eyes and our communication channels open and clear.