One Click to Deceive: The Rise of Modern Cyber Scams

AI-driven phishing and cyber scams are becoming more convincing and increasingly mainstream, enabling attackers to impersonate real people and bypass "common sense" defences. This article will highlight the impact and effects within society, alongside indicators and ethical considerations involved.

As societal reliance on personal technology grows, cybercrime has surged alongside it, specifically AI-driven fraud. AI phishing and social engineering techniques have soared in late 2022, following the public release of ChatGPT, and now account for over $25 billion in annual losses in the global economy. In addition, 82.6% of phishing attacks utilise AI generation to bypass security filters. Although AI offers profound technological advancements, it also lowers the barrier to entry for cyber fraud.

AI phishing utilises a machine learning algorithm to enable machines to learn from data, recognise patterns, solve problems, and make decisions with minimal human intervention. AI phishing has now unlocked new capabilities of incorporating synthetic voice cloning, audio filtering, and real-time deepfake videos to execute far more convincing attacks.

Traditionally, phishing attacks occur when you receive a deceptive message online from an attacker claiming to be a trusted source, but they are designed to create a sense of urgency. Once you click the embedded link or download the attachment from the message, sensitive data such as login credentials or financial information can be obtained for malicious purposes or malpractice. Phishing is driven by social engineering, using psychological manipulation to trick targets in order to obtain personal information.

Recently, two significant advancements in AI phishing have created an immense threat to society, allowing cybercriminals to generate hyper-realistic attacks at an unprecedented scale and speed.

• Semi-automated message creation — Traditionally, a human takes around 30 minutes to create one phishing attack, but by using large language models that are trained by massive data sets to understand, summarise and generate text, these AI systems can produce different variations of attacks within the same amount of time. This changes the game through rapid efficiency.
• Timely and contextual targeting — As a phishing attack develops and expands, AI allows attackers to include real-time news and developments within the conversation or phishing messages. This causes attacks to appear more believable and less detectable.

In early 2024, Arup's Hong Kong branch fell victim to the most expensive AI phishing scam to date. An employee joined a video call to discuss a confidential transaction with their UK-based CFO, only to interact with an AI-generated deepfake mimicking the executive's voice and appearance. As the employee believed that the call was legitimate, this AI scam ultimately resulted in a loss of $25.6 million USD after the execution of numerous financial transfers.

These attacks have the power to significantly impact society as a whole by, eroding trust in effective digital communication, increase fear within communities and relationships, financially burden households and organisations, strain workplaces and institutions, etc.

In general, when encountering scams and social engineering techniques, you should be aware of spelling errors, awkward tone, odd context and any suspicious details within the message, which often reveal the phishing attempt. Furthermore, social engineering patterns like requesting to keep the matter confidential or instructions to avoid escalation, e.g. "there is no need to call anyone" or "verify with me directly", isolate the victim from regular security procedures.

Essentially, to reduce the risk of being a target is to simply limit the use of sharing information; although this may slow down operations, it mitigates the greater threat. The key idea of AI phishing is that they personalise messages to make them appear more convincing through public or leaked information. Reducing your public profile is not about disappearing from the internet. Rather, it is about being intentional with what you share and avoiding oversharing.

However, large language models have the ability to scrape any of these indicators by producing native level messages in a natural tone. Additionally, the presence of a victim's digital footprint and social media can exacerbate the situation in which AI can create personalised and convincing messages by exploiting this information. This further accelerates AI's ability to automate deception at scale.

It is quite transparent that AI phishing entirely corrodes all moral principles and cannot be justified under any circumstances. Attackers may justify their actions as a "side hustle" or "lesson for better security", but these rationalisations neglect the intentional deception and overall harm.

The social costs of AI phishing include abolishing consent through impersonation and targeting vulnerable groups (the elderly, people under stress, online communities, and people with limited access to verification sources), and impacting trust and social cohesion within society. Ultimately, society will bear a staggering cost. A world where everything must be verified, and nothing can be assumed.

Thank you for reading! This article is a part of SEVA Charity, an organization striving to raise awareness regarding important issues of today. Contact us at sevahongkong@gmail.com if you wish to write as well! Check us out here: https://www.instagram.com/sevahongkong/