8 min read

Binary Exploitation: 64-bit Buffer Overflow Attack

Binary Exploitation: 64-bit Buffer Overflow Attack

Hello security folks, before I start let me first introduce myself. I am jarvis0p, a college student in daylight and cyber security learner during shadows. This write up is going to be complete beginners friendly, a guide to perform 64-bit buffer overflow attack. Along the way you ll learn some fundamentals of assembly, payload creation, and getting code execution.

Setup Environment

Before we start, lets setup our working environment to perform this attack and make sure in anyway it is not going to harm our system. For this, I want to you to use a remote Virtual Private Server (VPS). You can achieve this easily by firing up an Ubuntu Machine on AWS  EC2 instance. After initializing, connect to the instance and run below commands to install necessary packages.

sudo apt update
sudo apt install gcc gdb checksec
bash -c "$(curl -fsSL https://gef.blah.cat/sh)"

There is one more thing you need to configure before we proceed. By default all Linux kernels are configured with a memory randomization protection known as Address Space Layout Randomization or ASLR. This is a security feature which randomizes the memory addresses at which various components of a program, including the stack, heap, shared libraries, and the executable itself. This randomization makes it more challenging for attackers to predict the memory addresses of specific functions or data structures, making it harder to execute successful buffer overflow attacks. This mechanism can be implemented on three levels.

  • 2 = Full Randomization
  • 1 = Conservative Randomization
  • 0 = No Randomization

Run this command to see the current value of randomization.

sysctl kernel.randomize_va_space

You ll get it default value set to be 2. Yes, linux kernels are very protective!!!. So lets turn it off completely to 0 so that we don't have to deal with complex low level programming.

sudo sysctl kernel.randomize_va_space=0

With this we are all set with our environment.

Vulnerable Code

Take a look at this C code

#include <stdio.h>
#include <string.h>

void secret() {
    printf("You have successfully executed the secret function!\n");

void vulnerable_function(char *input) {
    char buffer[64];
    strcpy(buffer, input);
    printf("Buffer content: %s\n", buffer);

int main() {
    char input[128]; 

    printf("Enter input: ");
    if (scanf("%127s", input) != 1) {
        printf("Failed to read input.\n");
        return 1;


    return 0;

This C program have three functions: main, vulnerable_function and secret. The main() function is reading array of characters as input. The vulnerable_function is then called with this input as argument. In this function a 64 bytes buffer array has been defined and then it is calling a literally very dangerous function strcpy() to copy bytes form input variable to buffer. The problem here with this function is that while copying it does not care about size of the buffer and keeps doing it even though the size 64 bytes is filled. The copying of extra bytes now start to mess around with other stacks in the programs memory. So suppose if we pass lots of 'A' to the input of the program, it will start overwriting other stacks and pointers ex stack pointer ESP, base pointer EBP. But there is one more secret function which is actually never been called in the code or program never intends to call it, but by the end I ll show you that using buffer overflow vulnerability how an attacker can execute this secret function.

Disable securities

Now let see this in live action and compile the program & run it with say 10 A's as input

gcc vuln.c -o vulnbin

Okay, everything went smoothly, no problem. But we want to create problem, so this time try it with 100 A's. For that you can use our good old friend python3 to print these A's like this:

python3 -c 'print("A"*100)'

Ooops, looks like someone's hurt. Yess Linux and gcc does not like to be messed around. It says stack smashing detected so basically we are caught red hands. Here I want to you to use a tool that I earlier made you to install. It is designed to check the security of a binary.

checksec --file=vulnbin

This shows how securely the program was compiled. You ll see full relro, stack canary found, NX enabled, Position Independent executable (PIE), etc. I recommend to dig into these terms later but for now we atleast get that these green colors shows that this was the thing which was earlier detecting stack smashing. So to make these in red or I mean to say turn off these security protections, compile it like this and then again run the program with 100 A's.

gcc -fno-stack-protector -z execstack -no-pie vuln.c -o vulnbin

Overflow begins

Finally this time we get different output. It says Segmentation fault. Its clear that we managed to break the program. Lets get into details of exactly what and where it broke. You can easily achieve this by using a tool dmesg which logs all kernel errors in it. You will need root permission for that so put sudo and pipe its output to tail command which will only show important last 5 lines of it.

sudo dmesg | tail

Take a look at the last line, here it says about our program vulnbin that some fault occured at instruction pointer ip at mem address 4011f0. Hmmmmm. This is not enough details but at least we get the idea that ip broke at somewhere 0x4011f0.  

Okay Finally its time to get our hands dirty with gdb. We will be using gdb gef (GDB enhanced feature) to assist us in exploit development. Fire up gdb with specifying the binary.

gdb vulnbin

You ll see that you have entered into a new command line. To get your way through you can type help and enter to see all commands of gdb gef. But before we run the program here, lets have deep dive in our binary and its functions address in memory. Run this inside gdb gef.

info functions

You can see all the functions used in the code are here. But we are interested in the secret function, so note its mem address i.e. 0x0000000000401196.

Now lets run our program by simply typing r and enter. This time we will again provide 100 A's as input to create buffer overflow scenario.

I know this is lot, even for me, but at least we can look around for things that we are familiar with, like esp or eip.

$rsp   : 0x00007fffffffe2d8  →  "AAAAAAAAAAAAAAAAAAAAAAAAAAAA"
this means that the actual value of $rsp (stack pointer) was  0x00007fffffffe2d8 but because we supplied input so long that it started interfering with the stack pointer and filled it with all A's

Same goes with $rbp (base pointer), and other stacks. What I want you to notice is that, it does not knocked out $rip (instruction pointer) address. Even though you supply any number of A's it may not effect it at all but this is what we want to achieve because instruction pointer controls which function to execute. If this was a 32-bit binary it would have worked, but why is this happening with x64 bit? Stuck? Lets take help. Take a quick read of this post and learn how calling convention works for x64 bit binary.

x64 calling convention
Learn about the details of the default x64 calling convention.

Hunt the offset

This is were gdb gef will assist us and make our work easy to find the exact number of A's or call it as offset after which we can put our payload which will overwrite ip address.

Run this command inside gdb to get a cyclic string of 100 bytes

pattern create 100

Now run the program using r command and copy paste the pattern as input. Then run this command to get value of offset bytes:

pattern search $rsp

Gef will find offset for you, 72 in this case with little endianess

Payload generation

We now know that 72 bytes is the exact number of stuffing we have to do to reach to instruction pointer and may be overwrite it. Here comes the part were we play with our input/payload to observe the behaviour of stacks and pointers. So lets try 72 A's and then 6 B's (exactly six because of calling convention of 64-bit ELF binary) and see what it does with the $rip. Again you can get this done by python3

python3 -c 'print("A"*72 + "B"*6)'

copy the payload and run it inside gdb.

Finlly we are able to overwrite instruction pointer. `$rip   : 0x424242424242`. (value of B in hex is 42). So this should be our payload structure:

72 A's + (address which we want to write in $rip)

Note we already saved the address of secret function and this address we will put after 72 A's. This will overwrite instruction pointer to the secret function address which will force it to move to the function and execute  it. So here is our final payload to execute secret function. Keep in mind I am on little endian arch, so writing secret function address in reverse order up to 6 bytes.

72 A's + "\x96\x11\x40\x00\x00\x00"

Since these hex may be non characters so we can not copy paste it, we have to pipe this as stdin to the binary. This time instead of using print function, I am using sys.stdout module to output non printable characters in bytes format.

python3 -c 'import sys; sys.stdout.buffer.write(b"A"*72 + b"\x96\x11\x40\x00\x00\x00")' | ./vulnbin

Yaaay, We successfully executed secret function.


If you think about root cause of this whole shenanigan, it all started due to strcpy() function which was blindly doing its work. Although we had to disable some security measures to see the attack it does not mean a hacker can not break those, it will only be more challenging but not impossible.

Author: Krishna Jaishwal
Twitter: https://twitter.com/jarvis0p1