I wrote this article in Romanian, in 2014, and I decided to translate it, because it is a very detailed introduction in the exploitation of a “Stack Based Buffer Overflow” on x86 (32 bits) Windows.

Introduction

This tutorial is for beginners, but it requires at least some basic knowledge about C/C++ programming in order to understand the concepts.

The system that we will use and exploit the vulnerability on is Windows XP (32 bits – x86) for simplicity reasons: there is not DEP and ASLR, things that will be detailed later.

I would like to start with a short introduction on assembly (ASM) language. It will not be very detailed, but I will shortly describe the concepts required to understand how a “buffer overflow” vulnerability looks like, and how it can be exploited. There are multiple types of buffer overflows, here we will discuss only the easiest to understand one, stack based buffer overflow.

Introduction to ASM

In order to make sure all C/C++ developers will understand, I will explain first what happens with a C/C++ code when it is compiled. Let’s take the following code:

#include <stdio.h>
int main() 
{ 
    puts("RST rullz"); 
    return 0; 
}

The compiler will translate the code into assembly language, which will be translated later into machine code, that can be understood by the processor.

The ASM generated code will look similar to the following one:

PUSH OFFSET SimpleEX.??_C@_09GGKPFABJ@RST?5rullz?$AA@ ; /s = "RST rullz"
CALL DWORD PTR DS:[<&MSVCR100.puts>] ; \puts
ADD ESP,4
XOR EAX,EAX
RETN

It is not required to understand it at this time.

This ASM code will be assembled in machine code, such as the following:

68 F4200300    PUSH OFFSET SimpleEX.??_C@_09GGKPFABJ@RST?5rullz?$AA@ ; /s = "RST rullz"
FF15 A0200300  CALL DWORD PTR DS:[<&MSVCR100.puts>] ; \puts
83C4 04        ADD ESP,4
33C0           XOR EAX,EAX
C3             RETN

We can see a series of bytes: 0x68 0xF4 0x20 0x03 0x00 0xFF 0x15 0xA0 0x20 0x03 0x00 0x83 0xC4 0x04 0x33 0xC0 0xC3. On the right, we can see the instructions that were assembled to those bytes. In order words, the processor will read the bytes and process them as assembly code.

The processor does not understand the C/C++ variables. It has its own “variables”, more specifically, each processor has its own registers where it can store data.

A few of those registers, are the following:

• EAX, EBX, ECX, EDX, ESI, EDI – General purpose registers that store data.
• EIP – Special register: the processor executes each instruction, one by one (such as ASM code). Let’s suppose the first instruction is available at the address 0x10000000. One instruction can have one or more bytes, let’s suppose it has 3 bytes. Initially, the value of this register is 0x10000000. When the processor will execute the instruction, the EIP value will be 0x10000003.
• ESP – Stack pointer: We will detail this later. For the moment it is enough to mention that a special data region, called stack, will be used by the program, and this register holds the value of the address of the top of the stack. We also have EBP register, which holds the base of the current stack memory.

All these registers can store 4 bytes of memory. The “E” comes from “Extended” as the processors on 16 bits had only registers that could store 16 bits, such as AX, BX, CX, DX. On 64 bits, the registers can hold 64 bits: RAX, RBX etc.

A very important concept that needs to be understood when it comes to assembly language is the stack. The stack is a way to store data, piece by piece (4 bytes pieces of data) where each new added piece is placed on the top of the last one. When the data is removed from the stack, it is removed from the top to the bottom, piece by piece.  Or, how a teacher from college used to tell us, the stack is similar to a stack of plates: you can add one only at the top, and you remove them one by one from the top to the bottom.

The stack is used at the processor level (on 32 bits) because:

• local variables (inside functions) are placed on the stack
• function parameters are also placed on the stack

There are also two things that we need to take care when we work with ASM:

• the processors are little endian: more exactly, if you have a variable x = 0x11223344, this will be stored in memory such as 0x44332211.
• when we add a new element (4 bytes piece of memory) on the stack, the value of the ESP will be ESP-4! This is important, as the “stack grows to 0”.

We have two ASM instructions that we can use to work with the stack:

• PUSH – Will place a 4 bytes value on the stack
• POP – Will remove a 4 bytes value from the stack

For example, we can have the following stack (left is the address, right is the value):

24 - 1111
28 - 2222
32 - 3333

The address on the left will be smaller when we will add new items on the stack. Let’s add two new elements:

PUSH 5555
PUSH 6666

The stack will look like this:

16 - 6666 
20 - 5555 
24 - 1111 
28 - 2222 
32 - 3333

The easiest way to understand this is to consider that ESP, the registers that holds the top of the stack, is to think about it as a “how much space has the stack left to add new elements”.

As we already discussed, PUSH and POP instructions work with the stack. The processor executes instructions in order to do its job and each instruction has its own role. Let’s see some other instructions:

• MOV – Stores data to a register
• ADD – Does an addition
• SUB – Does a substraction
• CALL – Calls a function
• RETN – Returns from a function
• JMP – Jumps to an address
• XOR – Binary operations, for example XOR EAX, EAX is the equivalent of EAX=0
• INC – Increments the value by 1 (x++)
• DEC – Decrements the value by 1 (x–)

There are a lot of other instructions, but these are the most common and easy to understand. Let’s see a few examples:

ADD EAX, 5     ; Adds the value 5 to the EAX register. It is EAX = EAX + 5
SUB EDX, 7     ; Substracts 7 from the value of EDX register. Such as EDX = EDX - 7
CALL puts      ; Calls the "puts" function
RETN           ; Returns from the function
JMP 0x11223344 ; Jumps to the specified address and execute the instructions from there
XOR EBX, EBX   ; The equivalent of EBX = 0
MOV ECX, 3     ; The equivalent of ECX = 3
INC ECX        ; The equivalent of ECX++
DEC ECX        ; The equivalent of ECX--

It should be pretty easy to understand. Now we can also understand what the processor does to print our message.

• PUSH OFFSET SimpleEX.@_rst_@ – I replaced the longer string with something simple. It is actually a pointer to the memory location where the “RST rullz” message is placed in memory. The instruction adds on the stack the addres of our string. As a result, the value of the ESP register will be ESP – 4.
• CALL DWORD PTR DS:[<&MSVCR100.puts>] – Calls the “puts” function from the “MSVCR100” (Microsoft Visual C Runtime v10) library, used by Visual Studio 2010. We will detail later how this instruction works, but before we call a function, we have to add the parameters on the stack (first instruction).
• ADD ESP, 4 – Since the first instruction will substract 4 bytes from the ESP register value, by doing this we retore those 4 bytes.
• XOR EAX, EAX – This means EAX = 0. The value returned by a function will be stored in the EAX register (we have return 0 at the end of the code).
• RETN – As we specified the return value with the previous instruction, we can safely return from the “main” function.

In order to understand better how function call works, let’s take the following example:

#include <stdio.h>
int functie(int a, int b) 
{ 
    return a + b; 
}
int main() 
{ 
    functie(5, 6); 
    return 0; 
}

The “main” function will look in ASM code like this:

PUSH EBP
MOV EBP,ESP
PUSH 6
PUSH 5
CALL SimpleEX.functie
ADD ESP,8
XOR EAX,EAX
POP EBP
RETN

The “functie” function will look like this:

PUSH EBP
MOV EBP,ESP
MOV EAX,DWORD PTR SS:[EBP+8]
ADD EAX,DWORD PTR SS:[EBP+C]
POP EBP
RETN

Note: Visual Studio is smart enough to automatically have the result of the addition (5 + 6 = 11). For tests, you can completely deactivate the compiler optimizations from Properties > C++ > Optimization.

We can see some common instructions for both functions:

• PUSH EBP – At the beginning of the functions
• MOV EBP, ESP – At the beginning of the functions
• POP EBP – At the end of the functions

Well, these instructions have the role to create “stack frames”. They have the role to separate the function calls on the stack, so the EBP and ESP registers (the base and the top of the stack) will contain the stack memory area that is used by the currently called function . With other words, using these instructions, the EBP register will hold the address where the data (local variables) used by the current function begins and the ESP register will holds the address where the data used by the current function ends.

Let’s start with the function that does the addition.

• MOV EAX,DWORD PTR SS:[EBP+8]
• ADD EAX,DWORD PTR SS:[EBP+C]

Don’t be scared about the “DWORD PTR SS:[EBP+8]” stuff. As we previously discussed between EBP and ESP we can find the data used by the function. In this case, this data represents the parameters of the function. The parameters are available on the stack and there are relative to the EBP address, at the EBP+8 and EBP+C (0xC == 12).

Also, in ASM, the square braces are used such as “*” in C/C++ when it comes to pointers. As *p means “the value at the address p”, [EBP] means “the value at the address of the EBP register”. It is required to do this because the EBP register contains an address of memory as a value and we need the value that is stored at that memory location.

Other thing to notice is that the “DWORD” specifies that at the specified address there is a 4 bytes value. There are a few types of data that specify the size of the data:

• BYTE – 1 byte
• WORD – 2 bytes
• DWORD – 4 bytes (Double WORD)

The SS (Stack Segment), DS (Data Segment) or CS (Code segment) are other registers that identify different memory regions/segments: stack, data or code, and each of those locations has its own access rights: read, write or execute.

So what those two instructions do? First instruction will place the value of the parameter “a”, the first parameter, in the EAX register. The second instruction will add the value of the second parameter “b”, to the EAX register. So, in the end, the EAX register will contain the “a+b” value and this value will be returned by the function on RETN.

Let’s go now to the function that calls the addition function.

PUSH 6
PUSH 5
CALL SimpleEX.functie
ADD ESP,8

We remember that the function call is “functie(5, 6)”. Well, in order to call a function, we have to do the following:

1. Put the parameters on the stack, from right to left, so first 6, second 5
2. Call the function
3. Clear the space allocated for the parameters (4 bytes * 2 parameters)

So, we place the two parameters on the stack (32 bits or 4 bytes each parameter): first we add 6 to the stack, followed by 5, we call the function and clean the stack. In order to clean the stack, we just add 8 to the ESP value (the two parameters size) in order to restore it to the value before the function call. We previously discussed that it is possible to use POP instruction to remove data from the stack, but in this case, there would be two POP instructions. If we would call a function with 100 parameters, we would have to do 100 POP instructions, and we can do it easier and faster with a single “ADD ESP” instruction.

Note: It is important, but not for the purpose of this article: there are multiple ways to call a function, knwon as “calling conventions”. This method, which requires to place the parameters from the right to the left and clean the stack after the function call is called “cdecl”. Other functions, such as the functions from the Windows operating system, called Windows API (Application Programming Interface) use a different calling convention called “stdcall”, which also requires to place the function parameters on the stack from right to the left, but the cleaning of the stack is done inside the function that is called, not after the “CALL” instruction.

It is also important to understand that when we call a function using the “CALL” instruction, the address of the instruction following the “CALL” instruction is placed on the stack. For example:

00261013 | PUSH 6 ; /Arg2 = 00000006
00261015 | PUSH 5 ; |Arg1 = 00000005
00261017 | CALL SimpleEX.functie ; \functie
0026101C | ADD ESP,8

On the left we can see the memory addresses where the instructions are stored. The PUSH instructions have each 2 bytes. The CALL instruction, available at the address 0x00261017 has 5 bytes. So, the address following this instruction in 0x0026101C (which is 0x00261017 + 5). This is the address that will be pushed on the stack when the CALL instruction is executed.

Before the CALL instruction, the stack will look like this:

24 - 0x5
28 - 0x6
32 - 0x1337 ; Anything we have before the PUSH instructions

After the exection of the CALL instruction, the stack will look like this (the address values are simple to be easier to understand):

20 - 0x0026101C ; The address of the instruction following the CALL instruction
                ; We need to save it in order to be able to know where to return after the function code is executed
                ; This is also code the "return address"
24 - 0x5
28 - 0x6
32 - 0x1337     ; Anything we have before the PUSH instructions

After the return address is placed on the stack, the execution will continue with the function code. The first two instruction, called function “prologue”, are used to create the stack frame for the function called:

PUSH EBP
MOV EBP,ESP

After the PUSH instruction, the stack will look like this;

16 - 32         ; The value of EBP before the function call
20 - 0x0026101C ; The return address, where we will go back at the RETN instruction
24 - 0x5
28 - 0x6
32 - 0x1337     ; Anything we have before the PUSH instructions

After “MOV EBP, ESP” instruction, the EBP will have the value the top of the stack. It is important to note that if we would use local variables inside the function, they will be placed on the stack.

Let’s modify the function to this:

int functie(int a, int b)
{
    int v1 = 3, v2 = 4;
    return a + b;
}

We have now two local variables which are initialized with the values 3 and 4. The new code of the function will contain some new code:

SUB ESP,8                  ; Allocate space on the stack for the two variables, 4 bytes each
MOV DWORD PTR SS:[EBP-4],3 ; Initialize the first variable
MOV DWORD PTR SS:[EBP-8],4 ; Initialize the second variable

The stack will contain now:

08 - 4          ; Second variable
12 - 3          ; First variable
16 - 32         ; The value of EBP before the function call
20 - 0x0026101C ; The return address
24 - 0x5
28 - 0x6
32 - 0x1337     ; Anything we have before the PUSH instructions

As a conclusion, it is important to remember the following:

• local function variables are placed on the stack
• the return address is also placed on the stack

If you have any question, before proceeding with the stack based buffer overflow, make sure you have the answers to your questions in order to properly understand the subject.

You can continue this article with the second part.