Improper restriction of operations within the bounds of a memory buffer

Improper restriction of operations within the bounds of a memory buffer occurs when a program reads from or writes to memory locations outside a buffer's intended boundaries. This happens when code fails to validate that data access stays within the allocated space, allowing operations to spill into adjacent memory regions.Retry. It is an overarching vulnerability class and also the root cause of many memory safety vulnerabilities, such as buffer overflows and memory leaks. It occurs often in languages that require the programmer to manage their own memory (such as C, C++ and assembly), when read or write operations on memory occur in locations that are input-controlled and unchecked.

Depending on the application, it can have drastic consequences, such as the program crashing, leaking sensitive data, and even remote code execution when an attacker takes control of the program's flow via function pointers.

In this lesson, you will learn about the dangers of improper restrictions on operations in memory and how to protect your application from such issues by writing memory-safe code. We will step into the shoes of Jessica, a wannabe hacker student aiming to dominate her computer science class server by ruining everyone else's homework and showing them who's boss.

Jessica's latest introduction to programming homework was to create a "calculator" server program called calcd. The whole class needed to write, compile and test this on the shared Linux server to learn C++. It seemed so simple that she thought it was a joke. Most of it was template code written by the teacher, Mr. Simmons, who also teaches math and electrical engineering. The whole assignment seemed to just be Simmons' way to show-off of his impressive newfound C++ "skills" thanks to ChatGPT.

The requirement was pretty simple, and only operations on two numbers needed to be implemented. Once she filled in all the empty functions (add, sub, mul, div and savefile), it looked like this on the terminal:

And the file calcd.txt stores the results:

Interestingly, she realized that the program would not process beyond two numbers. Also, it would crash if nothing was entered. This tipped her off that there must be some kind of memory related bug in the template code that she could potentially exploit.

She got so bored that she started poking around the home directories of other students on the server. She noticed something strange: for automatic marking purposes, a script (calcd_start.sh) is made for each student that assigns them a high number port to host their calcd programs:

The source code template calcd.cpp is available at every student's home directory for their modification, and is made intentionally unreadable except by the user; but because the program doesn't compile initially (it's missing the code you need to fill in), the actual compiled binary is available under default permissions, meaning anyone can read anyone else's compiled C++ code. Neat.

The output of ls -l in her own home directory reveals interesting permissions.

Knowing that she now has access to binary programs written by other students, and having found a deadly flaw in the template code that would affect any student who didn't patch it (everyone else but her, of course), she came up with an evil exploit which generates a payload that allows her to execute code as other students (or even Mr,=. Simmons) on the school server:

With this exploit in hand, all she has to do to target anyone on the school server is to host a script that she wants the victim to run in /tmp/hack.sh, then generate the exploit and point the payload at the target student's UID when their calcd server is running (e.g. the UID 1003 which belonged to her frenemy Linda):

Now, with the backdoor command added to ~/.profile of the target user, next time they login, their shell will run whatever Jessica's heart desires.

How did Jess come up with this clever exploit? Let's take a look at the template code Mr. Simmons provided to all the students:

The only functions that the students were required to fill in was commented with "HOMEWORK", which were the five private functions that were called by the class's calculate method. All of the parsing of input was done by the template code. The problem was that to ease the difficulty of the task for students, only two numeric inputs were expected (e.g. 2 + 2 instead of 1 + 2 + 3), but the loop that appends space-separated elements will keep appending more data to the inputs array until the number of elements run out:

Since a function pointer (op) was used to select what function to call, two potential issues arise. First, if no valid operation is found in the input, op remains uninitialized, which would cause undefined behavior when dereferenced. This explains the segmentation fault Jessica observed with empty input. However, her actual exploit takes a different approach: by providing valid-looking input that overflows the inputs array, she overwrites the op pointer with the address of savefile. The key insight is that the buffer overflow gives her control over which function gets called and with what arguments, turning a crash into a precise exploit.

inputs[input_count++] = element;

Jessica could simply find the address of the function she wanted to call (in this case, savefile), and use the buffer overflow to overwrite the op pointer to that function's address and cause it to be used with her two inputs (which is the file she wishes to write to, the victim's .profile file in their home directory, and the string she wants to write, /tmp/hack.sh).

Just to be safe, Jess wrote out the pointer to savefile multiple times in the input to make sure that it reaches the target op pointer she wished to overwrite, separated by spaces. She also carefully crafted the file path and the backdoor command to contain no spaces (/tmp/hack.sh), by putting any bigger commands she wishes to execute in that script. Otherwise the input would be treated as a separate element by the split(input, ' ') call and mess up her exploit.

There is a variety of files that she could write to for demonstrating impact, including calcd.cpp which could cause the source code of the victim's homework to be damaged. Note that in order for her exploit to work, the victim would actually need to have implemented the savefile function correctly to write the output to the file.

Since the setup uses the user's UID as the port to start the calcd server with netcat, all she had to do was to generate the exploit with the correct savefile function offset by loading the target's calcd binary, then fire it at the appropriate port.

By the end of the week, Jess was bored again; this time around she shared her findings with the rest of the class, and showed them how to fix it.

NOTE: In order for Jessica's exploit to work well, the Linux server the code is running on must have certain exploit mitigations disabled. This includes ASLR (Address Space Layout Randomization), which randomizes memory addresses every time a program runs. In addition, the flag -no-pie must be passed to the G++ compiler in order to produce a non-position independent executable that starts at the same entry point every time in order for her hardcoded savefile address to work.

The resulting impact of such a wide class of vulnerability is highly context-specific. In most cases, it will cause the program to behave in undefined ways and crash. In the case of Jessica's vulnerable homework program, it enabled her to write arbitrary content to any file the user can access on the system, which subsequently led to account takeover on the Linux server.

When improper restriction of operations relates to data read by the program and returned to the user, sensitive data in memory, such as encryption keys, passwords, and authentication cookies, might be leaked. Having a leak primitive can also help attackers bypass memory security features like ASLR, since they can leak addresses of pointers in memory. Those addresses can be used to calculate an offset to where the attacker wants to go, such as achieving remote code execution by constructing a chain of functions to jump to until they reach a function that allows them to execute arbitrary code (a technique known as ROP, or Returned Oriented Programming).

In this case, the culprit was the design of the procedure that parses the input. As this homework calculator only requires the student to handle two numbers, there was no need to write a loop that takes any number of elements separated by a space.

While bounds checking of the input_count variable can be performed inside the loop to prevent writing past the input array, the simplest way to implement the calculator parsing functionality safely is to only accept valid input. By making the distinction between which position of the input is the number and which is the operation, no loop is needed at all:

While this would make a pretty limited calculator (only handling 2 numbers at a time), it was the original specification of the assignment to begin with, which meant that being "fancy" with a loop that handles more than 2 elements when it wasn't needed was the over-engineering root cause of the bug.

Memory safety vulnerabilities remain among the most critical security issues in software today, consistently appearing in the CWE Top 25 Most Dangerous Software Weaknesses. Understanding how improper memory operations lead to exploitable conditions is essential for writing secure code, particularly in languages like C and C++ where manual memory management is required. To deepen your knowledge, explore the resources below.

• The CWE page for this vulnerability type
• Hands-on guide to buffer overflow, a subclass of this vulnerability, by Snyk
• OWASP overview of buffer overflow vulnerabilities
• Microsoft's guide to memory safety and secure coding practices