At some point in our careers, we all get asked to write tests. If not unit tests, then it is integration, sanity, regression, interface, or beta/acceptance testing. Whatever the case, as a developer, testing is an inescapable part of software development.
For many of us, when we’re given such a task, we often implement them with a barebones understanding, sometimes writing them as a last point of thought rather than the first thing in the coding process. We apply our mechanical thinking processes, based on what we’re told as being good practice, often without first examining the context and its long-term usefulness as an actual test.
The point of testing software is to discover defects before it gets pushed into production. But sometimes, we can’t properly detect a defect without having awareness of what it might be. This is how testing software security can fall through the cracks.
When testing becomes the last port of thought, security as a factor in software development gets left an unconsidered detail, if it even exists at all. This is because writing test code is not the same as writing normal code. The way you see how things fit together gets inverted.
What do we mean by inverted? The expected result of the test often comes first, before the process of getting to it. A good test is small because it allows the developer to conceptualize all the logic at once. It is a diagnostic tool, leading it to be as simple and obvious as possible. However, while conceptually simple, it is not as easy to implement.
Good developers, bad tests
When you write a test, the point is to keep your subsequent test readers in the test and not lead them through a rabbit hole maze of code and the rest of the test suite.
If a test breaks, the reader should be able to diagnose the problem based on the test suite, rather than jump around reading all the related ancillary code.
Here is a common test pattern we often see in test suites:
The test is clearly vague. To the uninitiated developer on the team, they will have to find out where johndoe comes from. Also, there is no answer as to why the expected return is supposed to be 100.
So the developer has to go looking elsewhere in the test suite for answers. The point of having tests is so that bugs can be easily detected and diagnosed — not wrapped in a mystery that is the test suite.
The above is an example of a bad test. Bad tests written for security purposes will do the minimum to protect your data, let alone secure it from potential attacks, memory, and data leaks.
Security unit tests are important, too
The value of a test is defined by the following criteria — impact, diagnostic value, consistency, and success rate. When a test is fragile, it often requires a lot of maintenance, leading you and your team to spend more time on test units rather than the actual code-building process. Fragile tests can also be time-intensive to run, based on how they are constructed. This can dissuade developers from running them on a regular basis to catch potential issues.
But what makes a good security test? Here’s a list to get you started.
Testing Security Controls
Security controls sit outside the code and are therefore often missed as a testing factor. Security controls and security components are how your application responds to different types of protocol calls.
It is easy to boot up an API — but how often do you test incoming loads and their security statuses? Active and passive attacks on API calls wrapped in HTTPS can easily mask their way into your database if your security protocols are not implemented properly. This can be done through passing SQL injection patterns through things like user inputs, forms, and manipulating parameters to mount path traversal attacks.
Sometimes, writing test code is not enough. Sometimes, you have to mimic the potential attacks in order to see how your application and infrastructure will respond. One way to achieve this is through Jenkins OWASP Zapper, which is an automated tool that allows you to mimic proxy attacks in a test environment.
Memory Behavior
It is not hard for an attack to lock up your SQL database tables. Unmanaged buffer overflows and unmonitored memory executions can lead to leaks, unidentified behavior, and glitches that can lead attackers to hijack the execution flow for their purposes.
While garbage collection deals with most issues regarding memory, sometimes leaks can still happen because there is an actual bug in the code that didn’t properly kill the process. This is because garbage collection only deals with things it knows that will not be used.
However, there are automated tools that you can use to perform memory checks based on your desired setup and scenarios. ABASH for example, is a string expansion error detection tool that gives you the option to perform insertion errors, allowing you to properly reveal potential weaknesses and security vulnerabilities.
For JavaScript-based applications, Closure Compiler is a tool that allows you to remove dead code, check the syntax and create warnings against common JavaScript pitfalls. This is a good one to have because it’s easy for JavaScript applications to be an ad-hoc way with minimal controls put into place for standardization and general cohesiveness. This is a major contrast to traditional programming languages such as Java and the C programming family.
Incident recovery testing
Let’s imagine that something has happened. You might have an automated backup somewhere — but when was the last time you ran the recovery process? Most of the time, this doesn’t happen until after the event has occurred.
The process of data recovery is generally a stressful situation. It gets even more stressful if you find that your backups are not actually valid. The solution is to test the backups.
Testing the recovery process includes covering the physical recovery, virtual recovery using virtual environments, data recovery, and full data recovery. It is often not feasible to expect that we test our backups every time a backup is created. This is due to the time it takes to perform the task and the frequency of backups being made. However, it is still good practice to include backup and recovery testing into a regular security and vulnerabilities test schedule.
When you only test the main system or just the code, you are putting your data at risk. Not testing if your backups actually work is taking a gamble on if you have any backups at all.
Conclusion
Testing security is a continuous process. It’s not a test-it-once-and-be-done kind of deal. Why? Because every time something changes, there is a chance for vulnerabilities to appear and grow. Even if your application or software never changes, the techniques, external services and server updates can sometimes be enough to invalidate your previous protocols, or accidentally reset them back to default settings. When you have a routine security testing schedule, these potential issues are easily caught and rectified before they become a major problem.
Running security tests can have its pitfalls such as the time required to perform the processes required. However, it’s better that we wreak havoc on our own systems than an unknown entity.
