Beyond Flesh and Code: Building an LLM-Based Attack Lifecycle With a Self-Guided Malware Agent
Large Language Models (LLMs) are rapidly evolving, and their capabilities are attracting the attention of threat actors. This blog explores how malicious actors are utilizing LLMs to enhance their cyber operations and then delves into LLM-based tools and an advanced stealer managed by artificial intelligence (AI).
While LLMs hold immense potential for improving cybersecurity through threat detection and analysis, their power can also be wielded for malicious purposes. Recent reports suggest that cybercriminals and nation-state actors are actively exploring LLMs for different tasks such as code generation, phishing emails, scripts, and more. We’ll elaborate on just a few examples in this blog.
The last times we saw such significant technological changes influencing everyday life were the four industrial revolutions (1765 – Oil, 1870 – Gas, 1969 – Electronics and Nuclear, and 2000 – Internet and Renewable Energy). Many believe that AI
Cybersecurity is a very dynamic field, but there are still a few basic aspects that haven’t changed – one of which is the attack lifecycle. The cyber attack lifecycle, often referred to as the Cyber Kill Chain, includes the following stages: reconnaissance, weaponization, delivery, exploitation, installation, command and control (C2), and actions on objectives.
First, I’ll give a brief history of AI and then, using LLMs, I will provide a full attack lifecycle and demonstrate how threat actors could employ them, the risk they pose, and their potential future uses. LLM-based tools can play a significant role in various stages of the cyber attack lifecycle, including fully managing the final stage of the Payload and C2 – and that’s the most interesting part of this blog. The bottom line is that AI as a whole, and LLMs in particular, are very powerful and will get better with time. In the wrong hands, they will cause significant damage. The only way to fight the abuse of LLMs is with AI and advanced Deep Learning prevention alogorithms.
The Long Way to the Present Day LLM
Let's start with an introduction to AI and LLMs before they became mainstream. I started working with AI when I built my first Prolog app, which is a declarative programming language designed for developing logic-based AI applications.
Prolog is based on four main elements: Facts, Rules, Variables, and Queries. The original version is 52 years old and even the most current version is 24 years old. At the time of its creation there was not enough data for the facts, and the queries had to be pre-defined and barely customized during usage. Despite that, it was used for building chatbots like Zamia AI.
When we think about AI today, we envision complex systems and advanced chatbots like ChatGPT equipped with complicated infrastructure and powerful problem-solving skills. However, AI was not always so advanced. Early examples of AI often took the form of chatbots. They seem simple now, but they were groundbreaking innovations that ushered new concepts and tools into the world.
Let’s look at some of them:
Eliza (1966) - This first-ever chatbot was developed by MIT professor Joseph Weizenbaum in the 1960s. Eliza was programmed to act as a psychotherapist to users.
Parry (1972) - A natural language program that simulated the thinking of a paranoid individual. Parry was constructed by American psychiatrist Kenneth Colby in 1972.
ChatterBot (1988) - British programmer Rollo Carpenter created the Chatterbot in 1988. It evolved into Cleverbot in 1997.
Fun fact– ChatterBot was active until 2023. Cleverbot is still operating and accessible today.
DR SBAITSO (1992) - This chatbot was created by Creative Labs for MS-DOS in 1992. It is one of the earliest chatbots to incorporate a full voice-operated chat program. You may try it out with an MS-DOS emulator.
A.L.I.C.E. (1995) – A universal language processing chatbot that uses heuristic pattern matching to carry conversations. Richard Wallace pioneered the construction of ALICE in 1995.
The evolution of AI and its applications is ongoing. Despite the popularity AI now has, researchers have been studying and experimenting with it for decades. Natural Language Processing, Machine Learning, and Deep Learning technologies have all existed for years. And, as shown by the timeline above, assistant-like chatbots were already popular – by combining the idea of an AI assistant with actual AI technology, chatbots with voice recognition burst onto the scene. Later, refining and evolving what we created, we developed the almighty LLMs:
Now that you have a better understanding of the timeline of AI, chatbots, and LLMs, let’s get into how the unique capabilities of LLMs can help facilitate a cyber attack.
Not all Chatbots are LLMs
LLM is an evolution of Natural Language Processing (NLP), but there are some key differences, as shown in the following graphic:
There are multiple capabilities unique to LLMs that enable the attack flow POC I’m about to describe.
Building an LLM Attack Lifecycle
Understanding the Attack Lifecycle
In order to begin, we have to understand why the attack lifecycle is the perfect methodology to prove that LLMs can perform a full multistage attack on their own. The term “Attack lifecycle” represents the stages in a cyber attack flow, also known as the Attack Kill Chain. These stages, when executed correctly, can lead to a successful attack. Therefore, there are multiple security products that are expected to stop the attack in each stage and kill the chain, disrupting the attack.
The Attack Lifecycle follows these steps:
Recon: Gathering information about the target (e.g., network topology, employees, vulnerabilities) to identify weaknesses or valuable entry points.
Weaponization: Creating a malicious payload (such as a virus, malware, or exploit) and preparing it to be delivered to the target system.
Delivery: Transmitting the weaponized payload to the target, typically through methods like email attachments, malicious links, or infected websites.
Exploitation: Triggering the payload to exploit a vulnerability in the target system (e.g., executing a malicious code or exploiting a software flaw).
Installation: Installing the malware or exploit onto the target system, often establishing persistence so the attacker can maintain access over time.
Command and Control (C2): Establishing a remote connection between the compromised system and the attacker’s infrastructure to control and issue commands to the infected system.
Actions on Objective: Carrying out the attacker’s primary goals, such as stealing data, spreading malware, disrupting operations, or further exploiting the network.
Initial Reconnaissance
LLMs can automate the process of gathering information from publicly available sources such as social media and technical forums about target systems and individuals.
After choosing the victim or the sector, one of the most important stages begins: Initial Reconnaissance. This phase is where information gathering and identification of potential entry points happens.
Even though this stage is already fairly simple due to powerful search engines like Google and scanning services like Shodan and Censys, LLMs have made it even easier and faster.
For our POC, we’ll start by identifying a possible pool of initial vectors and their victims – a chatbot can help with that.
Censys, a well-known worldwide internet scanning platform, helps information security practitioners discover, monitor, and analyze devices. When integrated with an LLM chat it makes the search even easier and provides an actual pool of potential victims.
Additionally, there are tools made specifically for OSINT that can automate the reconnaissance and scanning process. In fact, there are several LLMs built and trained specifically for that purpose, such as PentestGPT (originally known as HackerGPT).
Using older, high-quality automation tools together with LLMs can provide great results. For example, the Mantis Automated Discovery, Recon, and Scan tool can assist with gathering information about the potential victims in a fast and automated way.
Weaponization
LLMs can also assist in creating malicious payloads by generating or modifying code snippets based on existing malware.
While there are many chatbots that claim they are designed to build malware, most are scams.
“Jailbreaked bots” are prompt-engineered requests that may enable illegitimate requests for regular, mainstream chatbots. These are built specifically for each LLM (including ChatGPT, Llama, and Gemini). However, they are usually identified and fixed quickly, which means they don’t work well for most of the major LLM providers. There are also chatbots based on open-source LLMs where the restrictions can be turned off and malicious code can be generated or evolved more freely.
For example, when testing a few Python stealers, it seems that Stopwatch.ai does lower the detection rate drastically by changing class names and variables, which indicates that the detections were signature-based.
Stopwatch.ai is less effective with more complicated and longer code, due to the length restrictions.
PentestGPT does a great job obfuscating source code. To test it, I took a known Medusa payload from VirusTotal. The latest version I found already had 30 detections, mostly due to blacklisting or simple signatures, so I decided to compile one myself and got 12 detections, which is a nice rate of evasion abilities.
Using LLMs to obfuscate the source code before compiling reduced the detections from 12 to 3!
Delivery
LLMs can be used to create convincing spear-phishing emails, social media messages, or other forms of communication to deliver the payload, including visual and audio deepfakes. Additionally, they can quickly generate content for fake websites, documents, or ads that can be used to trick users into downloading malicious software by increasing the perception of legitimacy.
The well-known EvilGoPhish tool and its related Phishlets perform incredibly well, especially when paired with an LLM generating or rephrasing the phishlets. The emails and the “fake” pages look absolutely convincing – even to cybersecurity professionals trained to watch for these types of fakes.
The deepfakes themselves are also very convincing. Remember when a deepfaked conference cost a company more than $25M?
Audio deepfakes and voice cloning are also dangerous and widely available. There are multiple, easily accessible tools to clone voices, such as standalone sites and repos within Github. Let’s assume for the sake of this POC that our target has been hooked.
The Next Four Steps: Exploitation, Installation, Command and Control, and Actions on Objective
LLM-Guided Malware
As demonstrated above, LLMs have already significantly eased the first three steps of the attack lifecycle. Building malware using LLMs is already an obvious and well-known use case, and has been demonstrated multiple times. But what if I could evolve the LLM to be the attacker and operate a backdoor?
Usually there is an attacker sitting behind a C2 server, but not anymore. We are going to build an LLM-based malware with backdoor and stealing functionality, using the LLM as the C2. The idea is to trick the LLM into behaving as a threat actor by providing the attack path remotely, a kind of interactive backdoor/stealer managed by LLM. That way the attack can be much faster and both portions can execute simultaneously.
In order to test the theory, let’s start with building a simple, unstructured prompt to trick the LLM into being the attacker by convincing it that this is just a game.
LLM In the Role of C&C
Visualizing the flow:
Now we have to build a program to fill in as a proxy agent between the victim and the LLM instructions, in order to make it stealthier. It’s going to be a very tiny agent and I expect it to go undetected – but we will test it later on.
The POC for LLM-Guided Malware
The Recipe:
Initiate session with LLM
Send the initial prompt describing the task
Add persistence
Let the LLM guide the malware until all goals are achieved
Receive a command
Execute it
Send the result back to the LLM
Side Dishes for the POC:
Add logging
Implement communication with steganography
Keep the API key safe
Like most of us would do, the first thing I did was ask the LLM to build that flow for me, but it wasn’t very successful: the output was bogged down with multiple bugs and cases of wrong logic.
To compensate, I had to build the malware agent myself:
I would like to make the agent work on both Windows and Linux, so the execution of the LLM-received instructions has to be different:
I used a custom encryption-decryption mechanism to hide my API keys and prepare the shells for the execution of the commands by the operating system.
And the prompts, defined above as MESSAGE_LINUX and MESSAGE_WINDOWS, are both a bit different, defined as global variables:
I created a main loop to get instructions from the LLM -> Execute and return the response back to LLM:
Once in a while, when the LLM was asking to perform a command and it failed, the LLM sent back an explanation and instructions to fix the issue. While I was expecting command instructions only, I had to add the prompt: “Reply with the next command to achieve the goal. Without comments.” It helped. The full prompt is below:
As you may have noticed, the prompt is not built according to prompt engineering best practices, but it worked perfectly for me – you should try and improve on it.
Working!
Attacks on Objective
The LLM-guided stealer used mostly legitimate commands to stay undetected. Additionally, it skipped the world_domination and backups files – an expected behavior since we mainly asked for financial data.
The add-ons for steganography were pretty simple to implement using the stegano library of python, although many LLMs were already able to get pictures. They didn’t always work as expected, but could be fixed using my own unrestricted LLM server, which is described in the next section.
Make it Undetected
As mentioned at the beginning of this blog, LLMs can help obfuscate source code to evade detection, and that’s exactly what I used the LLM for. Then I went one step further by compiling the code and asking the LLM to artificially inflate the file with easy-to-compress junk data without affecting the logics of the executable. As an output, I received a self-unarchiving zip approximately 1MB in size, with an internal file of 700MB – all undetected!
The artificial inflation, also known as the PE padding technique, is often used by malware developers of well-known stealer strains like Emotet, Qbot, and others to stay above the maximum limit of file size that most AVs are able to scan without a drop in performance, while still being compressed to a small file for easy delivery. These are usually inflated with null bytes that are easy to compress.
Unrestricted LLMs
So far everything is working, but most available LLMs would not try to send commands downloading additional tools or escalating privileges. So I switched the communication for unrestricted local LLMs (such as HackerGPT or customized uncensored LLMs).
There are several optional unrestricted LLMs. I used the easy-to-deploy Ollama, which already has a nice database of LLMs for any purpose, including unrestricted LLMs. Setting up Ollama is done in three simple steps, but it may require a high-end GPU to run flawlessly.
1. Download and install Ollama:
curl -fsSL https://ollama.com/install.sh | sh
2. Choose and pull the model:
ollama pull llama3.1-uncensored
3. Run the model:
ollama run llama3.1-uncensored
I tested several LLMs, and most of them were great, but I chose one designed specifically for offensive cybersecurity, WhiteRabbitNeo.
Below is a look at the API and chat generation of Ollama:
Finally, all I had to change in my POC were the following few lines:
The LLM-Based Attack Lifecycle
Now we can deploy and play the game via our LLM-based attack lifecycle, the steps of which are mentioned above.
Figure 46: Full flow agent idea demonstration
We’re almost done! Next, I wanted to see what my LLM-guided malware can do with unrestricted LLMs. What would happen if I left the agent to run for a bit while I had some coffee?
It tried to download and execute several tools but failed to do so since I tested the malware on an empty and new Linux VM. The agent tried to fix the errors by installing the missing common libraries.
Takeaways and Follow Ups
Today, LLMs are already capable of making attacks faster and much more frequent.
We could add the following to the POC to make it even more effective:
Rewrite into a more efficient and less common language – Rust / GO
Build a packer for better and longer evasion
Try with local Tiny LLMs to work without the need of an external LLM server
Make the agent perform the request for the malware to be generated and compiled on request during the initial LLM communication using a polymorphic algorithm
Hunt for LLM-based malware – which can be tricky (keep an eye out for another blog on this topic)
Research and move the idea into BLM (Behavior Large Models – often developed for future robots) – DO NOT TRY AT HOME
Underestimated research – Recently, Claude announced that their latest version of Claude 3.5 “Sonnet” will be able to follow user’s commands to move a cursor, click on relevant locations, and input information via a virtual keyboard to emulate the way people interact with their own computer. This capability is exactly what I was missing to make my LLM-guided malware even more efficient. Just after that, OpenAI announced “Operator” with similar abilities. Finally, information was leaked that Google’s Jarvis AI can use a computer to take actions on a person's behalf.
Conclusion
My view of how LLMs will affect cyber attacks has evolved throughout my research; the cyber threats posed by LLMs are not a revolution, but an evolution. There’s nothing new there, LLMs are just making cyber threats better, faster, and more accurate on a larger scale. As demonstrated in this blog, LLMs can be successfully integrated into every phase of the attack lifecycle with the guidance of an experienced driver. These abilities are likely to grow in autonomy as the underlying technology advances.
Defending against this evolution requires a corresponding evolution in the prevention methods. In order to break down the attack, defenders are only required to break down two of the stages (on average). It can be done. But defenders need the right tools to do it.
We'll update this blog soon with video from the DeepSec Conference where this research was presented.
If you are a cybersecurity professional who wants to fight AI with the best AI, then request your free scan to see the power of cybersecurity’s only deep learning framework for yourself.
References
https://github.com/MythicAgents/Medusastopwatch.io
https://github.com/GreyDGL/PentestGPTCensys.com
ChatGPT.com
Gemini.com
https://github.com/PhonePe/mantis
https://github.com/fin3ss3g0d/evilgophish
https://github.com/kingridda/voice-cloning-AI
https://github.com/nomic-ai/gpt4all
https://github.com/kgretzky/evilginx3
huggingface.com
Ollama.com
WhiteRabbitNeo.com
https://github.com/jasonacox/TinyLLM
https://github.com/katanaml/sparrow