Integer Overflow Attack and Prevention

Integer Overflow Attack and Prevention

Software threats have seen exponential growth over the last few years. Software and application-related vulnerabilities have opened the door for a lot of damage to different organizations and individuals. Below is a brief insight categorized on the severity of vulnerabilities identified in applications:


Source: EdgeScan 2020 Vulnerability Statistics Report


These vulnerabilities have a negative impact on the confidentiality, integrity, and availability of the asset. Some software/programming related attacks include: 


  • Buffer Overflow 
  • Format String Attack
  • Integer Errors
  • Integer Overflow Attack


Apart from the above there are many other attacks which are related to programming and software. Our main focus in this article will be integer overflow attack; how it works, how it can lead to exploitation, and steps we can take to prevent it.


Introduction to Integer Overflow

Integer overflow, also known as wraparound, occurs when an arithmetic operation outputs a numeric value that falls outside allocated memory space or overflows the range of the given value of the integer. Mostly in all programming languages, integers values are allocated limited bits of storage. 


For example, we have a 16-bit integer value which may store an unsigned integer ranging from 0 to 65535, or signed integer ranging from -32768 to 32767. So, during an arithmetic operation, if the results require more than the allocated space (like 65535+1), the compiler may:


  • completely ignore the error caused, or
  • abort the program.


Most compilers will ignore the overflow and store unexpected output or error. This will result in various attacks such buffer overflow which is the most common attack and leads to executing malicious programs or privilege escalation.


The C Standard [ISO/IEC 9899:2011], states:


  • A computation involving unsigned operands can never overflow, because a result that cannot be represented by the resulting unsigned integer type is reduced modulo the number that is one greater than the largest value that can be represented by the resulting type.


Let’s take an example to better understand the scenario. The program below has an integer overflow vulnerability and will lead to a buffer overflow.



The above code calculates student grades against random numbers of students. If we consider a 32-bit computer architecture, an integer overflow will occur when the value of unsigned integer exceeds 230 – 1. 


If a value 230 + 1 is used, the calculated size of the student array which is passed to the malloc is 230 multiplied by 4, as the size of int is 4 bytes. The calculated value, 232+4, exceeds the maximum size of the unsigned integer. The for loop will treat the 4 byte allocated buffer as if it stored the array value of 230 bit integer and will write the data out of bounds for the specified buffer thus leading to a buffer overflow.


Let’s consider another example where we try to store a large value variable into a smaller one:



Here’s the output:



We can see that since each assignment causes the value to exceed the default value type, so it is truncated in order to the store it in the assigned variable


Now that we have a basic understanding of integer overflow, let’s consider the risks and vulnerabilities associated with an integer overflow attack.


Integer Overflow Attack 

The 2020 CWE Top 25 Most Dangerous Software Weaknesses ranks integer overflow as the 11th most dangerous software weakness and is assigned an id of CWE-190.


This weakness is categorized as critical because they are relatively easy to locate and exploit. The exploitation leads to complete takeover of the system, data theft, data exfiltration, or preventing an application to run correctly. This error is usually introduced in the implementation of software lifecycle. Below is a brief insight about how this attack impacts the CIA Triad, which is a very important consideration in terms of information security:




Availability Technical Impact: DoS: Crash, Exit, or Restart; DoS: Resource Consumption (CPU); DoS: Resource Consumption (Memory); DoS: Instability

This weakness generally leads to undefined behaviour, causing crashes. The chance of experiencing infinite loops is very high. The risks of overflow increases by using loop variables.

Integrity Technical Impact: Memory Modification 

If the value in data is important basic data corruption has taken place where as if integer overflow results in other vulnerabilities such as buffer overfull more data/memory corruption will occur

Confidentiality, Availability, Access Control Technical Impact:  Unauthorized Execution of Code or Commands; Protected Mechanism is bypassed

The vulnerability can lead to buffer overflows which can allow the adversary to execute arbitrary code


Risks Associated with Integer Overflow Attack

There are over 1113 vulnerabilities in the Common Vulnerability Exposure (CVE) database that are associated with integer overflow attacks. Out of many vulnerabilities pointed in CVE, buffer overflow attacks are very common and the chosen favorite of hackers and adversaries. 


Buffer overflows occur when a developer does not sanitize or validate the user input before allocating space for it in the buffer. Integer overflow leads to the execution of buffer overflow vulnerability which allows the attacker to gain shell and elevate his privileges once this vulnerability is exploited. The validation checks are actually disabled by the integer overflow vulnerability thus resulting in execution of buffer overflow.


One example of integer overflow which led to buffer overflow was found in OpenSSH (3.3) (Older Version). Here’s a code snippet that demonstrates this:




On the off chance that nresp has the value 1073741824 and sizeof(char*) has its normal value of 4, the result of the operation nresp*sizeof(char*) overflows, and the argument to xmalloc() will be 0. Most malloc() executions will take into account the allocation of a 0-byte buffer, making the ensuing loop cycles flood the heap buffer response allowing the adversary to use arbitrary code 


Preventing Integer Overflow Attacks

Due to the undefined behavior of integer overflow, they are hard to detect and debug. We need to ensure that we apply the right techniques and strategies during the requirement, implementation, architecture and design phases while developing a certain program, software or application. Let’s go through these phases step-by-step to ensure that we apply the right mitigation techniques.


Requirement Phase

We need to ensure that all rules and protocols are defined strictly for all out-of-bound operations. The programs need to be able to handle exceptions related to out-of-bound values. Select a programming language that can handle exceptions that arise due to this vulnerability. A compiler or programming language that checks for out of bound operations is preferable.


Architecture & Design 

It is recommended to use libraries and frameworks that are used in secure coding, and best practices in order to avoid undefined or erroneous behavior while handling numbers. Packages like SafeInt or IntergerLib can be used in order to handle this type of behavior. Ensure that security checks are duplicated on both client and server sides as there are techniques that can bypass security checks at the client side.


Implementation Phase 

As pointed out earlier, make sure to provide input validation for numbers that are entered by the user. Define clear ranges in case of signed numbers by providing minimum and maximum values for the desired integer type. Clearly understand the programming language framework that uses when numbers are handled. 


You should also closely analyze truncation, byte size manipulation, type casting and how the programming language handles numbers that are too large or small. If you observe any compiler errors, make sure to analyze them carefully and eliminate them by implementing appropriate secure coding practices.


There are many detection methods for vulnerabilities like integer overflows. Here are a few:


Manual Analysis 

A detection method involving human analysis through penetration testing, tools, etc., which allows a tester to test and interact with live sessions and report the issues more effectively.


Black box testing 

Black box testing is a method in which software functionalities are tested without the knowledge of its code, internal paths or structure of the software. Certain inputs are given to the program and tested for outputs.



An automated software testing method that focuses on providing invalid commands, expressions or random numbers to the program in order to analyze the behavior of the software or application.


Manual Static Analysis 

This technique involves analyzing the source code of the application or software to test potential flaws or weaknesses.


Automated Static Analysis 

Automated Static Analysis involves testing of a program for vulnerabilities without actually executing the program.


In Conclusion

Always remember that attackers are always looking for ways to exploit potential vulnerabilities. Even the smallest loophole left while coding the application or software can lead to devastating results, from total system failure or application takeover to financial loss, customer dissatisfaction and loss of trust. Using best practices and secure coding techniques can mitigate these risks. 

Muhammad Luqman / About Author

Information Security Enthusiast | Ethical Hacker | CEH | MS Infosec