&w=3840&q=80)
The more we advance and rely on technology, the more important the question of security becomes. Some forms of cybersecurity threats have been around for a very long time – things like malware, phishing and SQL injection – and we’ve gotten used to them and how we handle them, both preventatively and on the go.
But, of course, as cyber security improves, so does innovation in cybersecurity exploits. More often than not, it’s the latter that’s a step ahead of the former. In recent years, however, AI is taking this to an even higher level.
Granted, AI / LLMs have presented privacy and security concerns since the beginning of the AI era in late 2022, with experts being consistently vocal about such concerns. We were also able to see real-life privacy scandals very early on which helped people be more conscious of issues such as entering sensitive business information into publicly available tools such as ChatGPT.
But irresponsible use of AI tools presents risks that are significantly less serious than the new, more directed and sophisticated forms of cybersecurity attacks that AI is enabling.
The new forms of AI-enabled cybersecurity threats
When it comes to AI-enabled threats, there are generally two rough categories. The first one includes exploits that were previously possible to do, but required too much time, effort and often specific skills to be consistently convincing and ultimately worth it for attackers. With AI, bad actors can now easily and reliably execute these at scale.
The other category includes threats associated with the automated nature of how AI agents work, specifically their privileged execution and their inability to distinguish between trusted and untrusted content in the sources they consume.
AI deepfakes & voiceovers
Deepfakes were a problem before AI, mostly in the socio-political context. The creation of deepfake videos required a certain amount of effort and skill level – barriers that AI has now mostly removed.
Bad actors are able to feed AI publicly available photos to create hyperrealistic deepfake videos – or alter existing ones – in order to achieve a wide range of possible goals, from essentially harmless ones such as chasing online engagement, to more serious ones such as creating sensitive and/or inappropriate content based on real individuals.
For the average person, AI voiceovers are just as if not even more worrying. It’s now possible to create a reliable AI voiceover from minimal original audio. So, if there’s a recording of you available publicly online – let’s say from a podcast or from a company social media clip – a bad actor could easily obtain the audio, then use it to create an AI voiceover and potentially extract money from your friends or relatives.
New, sophisticated forms of phishing at scale
In recent years, we’ve been seeing a rise of more and more sophisticated phishing attacks. Brand impersonations that were easily spotted before now appear much more legitimate. And they come from places that you use regularly, such as your local postal service.
If you’re already expecting a shipment when you get a scam message from someone pretending to be the delivery service, you’re much more likely to automatically click on the link and inadvertently give attackers your personal information.
Privacy-related phishing scams can go even further, though. I once received an email from a supposed data protection company alerting me that my personal information had been exposed, and to click on the provided link to start the process of remedying it. The ingenuity here was downright sinister – and I nearly fell for it.
An even more serious phishing attempt happened very recently at the company level. Our CEO explains it best, but to put it briefly, the attacker acted as a potential client looking for a development partner. Following communication with the sales representative which seemed completely harmless, the attacker sent a Git repository that the sales rep should then pass on to a developer.
The catch was that the repo contained a malicious script that would automatically execute and steal information from the computer of the developer who ran it. We were very fortunate to catch it in time, no doubt owing to our sales reps’ close involvement in client projects and consequent adequate experience to become suspicious.
People have reported similar phishing attempts from people pretending to be either looking for or offering employment. The strategy was basically the same: a Git repo containing a malicious script, that was sent out either by the “employer” as a task for the job seeker, or as part of their CV by the “job seeker” to the employer interviewing them.
Before AI, scams like these were too time-consuming and skill-intensive to perform at scale, and scale is of course needed to yield desirable results and justify the effort for the bad actors. Now with AI, it’s much easier, if not trivial, to perform such phishing attacks at scale.
Prompt injection & log poisoning
Prompt injection is an exploit reminiscent of the long known SQL injection, but with crucial differences. Affecting sites and applications coded with the help of AI, prompt injection works by inserting a malicious prompt that an LLM cannot distinguish from genuine user-generated prompts, and so executes it without question.
LLMs analyse information from a variety of publicly available sources, including Git repositories. Someone with an otherwise commendable profile and several useful repos could inject a malicious script inot a seemingly harmless repository, which an LLM would then execute on its own, without approval or intervention from the actual user.
Log poisoning is a similar type of attack associated with the OpenClaw AI coding assistant. Here the attacker injects malicious code into log files, which an AI agent will then use as a resource, resulting in the execution of this malicious code.
These two exploits are particularly dangerous since they both take advantage of one of the biggest benefits of working with AI agents: the efficiency of automation. Threats like these underscore the critical importance of both human oversight and proper guardrails for the AI-assisted development workflow.
How to defend against these new forms of security exploits & minimise risk
Since attacks like phishing and other forms of human engineering work by exploiting human nature, one of the most reliable forms of defence against them is awareness of and familiarity with these types of attacks, which helps individuals quickly recognise and avoid them.
Proper cyber security education goes a long way here, ideally done following ISO standards, performed by certified cybersecurity experts who can also address more specific questions and use cases.
Another essential practice is responsible use of AI. People need to be mindful about not entering sensitive business information into public LLM tools. Proposed solutions need to be carefully reviewed by humans before entering production, not only for adequacy but also to minimise the risk of specific exploits such as prompt injection.
Responsible AI use also involves using tools which are more security minded, such as NVIDIA’s NemoClaw or amazee’s amazeeClaw over OpenClaw. Additionally, guardrails need to be set up to avoid risks associated with the automated nature of AI agents; these guardrails should prioritise carefully defining trusted sources for the agent as well as their privilege, i.e. how much access they have.
Other technical considerations to minimise security risks include:
- Setting up two-factor authentication (2FA) for logins
- Configuring devices for maximum safety
- Other data protection measures, such as data encryption and having strong backups
Lastly, individuals should consciously manage any personal data which they share or which is otherwise available online. This ranges from essentials like having strong passwords, using trusted providers and sources, and regularly clearing cache, to more active measures such as reaching out to data brokers on a regular basis to request removal of any personal data they’ve gathered.
In closing
As cybersecurity threats evolve, so should the ways in which we ensure and maintain online safety. AI is now opening up a lot of opportunities for bad actors to exploit more easily than ever, but conscious awareness and proactive consideration remain the primary defence against them.
Training on cyber security best practices and on responsible AI use is key to help employees be mindful of the various potential threats and consequently critical about how they access and share information digitally.
These should be particularly important considerations for digital agencies and consultancies that regularly work with client information, potentially under strict NDAs and/or in a white-label setting. Since they are responsible for proper handling of the clients’ information and business secrets, adequate security practices are an absolute must in client-agency relationships.
