SAST: You Can’t Improve What You Can’t Measure
Protecting your custom applications and data is a never-ending task. It seems like the burden on application architects, designers, and developers has only increased in the world of Continuous Integration/Continuous Deployment (CI/CD). Teams have to find the right mix of tools like Dynamic Application Security Testing tools (DAST) and Static Application Security Testing tools (SAST), or they risk letting problem get loose in production.
These tools are powerful, and they can provide a wealth of information about your application’s security posture. That’s a good thing, but it’s not without its drawbacks. For example, I wonder if you’ve had the same experience I have with SAST reporting tools: Do they give you the information you need when you need it? Do their reports help you improve your processes as fast as you want?
It’s an old aphorism, but it’s true: you can’t improve what you can’t measure. In this series, I’m going to share how to pull software development metrics out of Fortify’s Static Code Analyzer (SCA) and its companion product Software Security Center (SSC). By the end of the series, you’ll:
- Understand the basic tables involved
- Have a set of queries you can customize
Fortify SCA and SSC
Have you used Fortify’s tools? If you have, then you can probably skip this section. I’m going to describe how Fortify packages its functionality so the reporting piece will make more sense. Context is important!
Micro Focus markets these tools, and you can find their documentation for the products on their site.
Fortify Static Code Analyzer
Fortify SCA is the heart of the product. It’s the piece that looks at your source code and finds potential vulnerabilities. To accomplish that, it uses rulepacks that describe the rules it can apply to a variety of program languages. For the examples I’ll use later, we’ll focus on Java and PHP, but it also supports .Net languages like C#, Python, and C/C++, as well as many others.
SCA doesn’t just look for strings like “password” (though it does do some basic pattern matching). It also looks for how modules/classes pass data back and forth, or how data coming from a UI element like a web form goes through an application to the database. It does this by looking at the application from the perspective of Kingdoms. This thoughtful approach to static code might be SCA’s biggest advantage.
You can read output from SCA using Audit Workbench or SSC.
SCA scans the code in two passes:
- Translation: SCA converts the source code into an interim language and stores it in a temporary location. This is important because it allows SCA to analyze an application that’s written in many different languages and present the results all at once. An application that’s partly written in Java, partly in C++, partly in C#, and partly in PHP is not a problem for SCA because of this step. Translation doesn’t put much demand on the CPU or RAM.
- Analysis: Once the code has been translated, SCA can apply the rulepacks. This is where it analyzes not only the source code file by file, but how all of the programs interact. This step can be very demanding of a computer’s CPU and RAM. For large codebases (in the range of hundreds of thousand of line of source code), I’ve seen SCA consume 60Gb of RAM on a Linux Server configured with 64Gb of RAM. That same scan pegged the CPU at 100% for the duration of the analysis.
The result of the scam is an *.FPR file. You can load that into a provided tool called Audit Workbench (not described in this series, but you can find materials on line), or you can use the next tool I’ll describe.
Fortify Software Security Center
SSC is an application that comes with SCA. SSC runs under Apache Tomcat, and it relies on a relational database management system. The examples I’ll show will be based on Microsoft SQL Server 2014 SP2, but it supports other RDBMes if you prefr something like MySQL. I’ve had very good luck with Microsoft’s database.
SSC is a tool for collaborative vulnerability remediation. It starts when you load an FPR file from SCA into SSC. You can do that manually or automatically using an SCA tool called fortifyclient. SSC provides several services:
SSC gives developers, developer managers, and senior managers ways to look at the scan results.
- Developers can log in and review the potential vulnerabilities that SCA found for their code. SSC provides an explanation of each potential vulnerability. It describes potential solutions and includes links to additional information.
- Once a developer reviews the potential vulnerability, they can decide what to do with it. They might decide it’s a false positive, in which case they might mark it as “Not an Issue.” They might decide that it is indeed a problem and mark it “Exploitable.” At this stage, I’d suggest you not suppress the issue (which is an option) because suppressions expire at a period that depends on how large and active your codebase is.
- Developer managers can log in and review reports. Reports can show basic trends (e.g., potential vulnerabilities are down 5%) or groupings (e.g., show the potential vulnerabilities that fall into the OWASP Top 10 cagtegories).
- Company management can log into the dashboard to get a general idea of the application portfolio’s vulnerability trends.
Fortify Integrated Development Environment (IDE) Tools
Some developers prefer to work strictly within their IDE, and Fortify provides plug ins for IntelliJ, Visual Studio, and Eclipse that will let developers do these things:
- Review the potential vulnerabilities from SSC by querying the database and displaying the results within their IDE. Clicking on a potential vulnerability in the SSC pane will display the source code to edit in the IDE’s editor.
- Initiate an SCA scan from within the IDE. Developers can then use Audit Workbench to review the scan or upload the results into SSC.
The IDE tools are very useful for small to mid-sized codebases. However, in larger codebases, I’ve seen performance lag.
Fortify CloudScan
Remember how I said that the SCA translate stage has low resource demands, but that the analysis stage can be stressful for RAM and CPU? The latter can be a problem in a CI/CD environment where you’re trying to do everything from the build servers.
Fortify recognized this and provides a tool called CloudScan. I intend to write after it in a future series. In short, it:
- Leaves the translate step on the build server, which packages the combined results of the translation step
- Implements a queue management server to which the build server submits the translation package
- Implements an analysis server that accepts the package and conducts the SCA analysis stage; it also outputs the FPR file
- Provides for the queue management server reporting the fail/success message from the analysis so the build server (or other process) can retrieve the FPR
What Kinds of SSC Reports Will We Create?
In the SSC section above, I mentioned that SSC has some management reports. I want to stress that this series is not a slam on the functionality already in SSC. Far from it — I want to clearly state that many of those reports are very useful!
But there are several reports that I’d like as a manager, or that my team would like, that just aren’t in SSC. So, I want to show you how I’ve used SSC’s data in SQL Server to produce those reports.
SSC has some great reports! But sometimes, you just want more…
What kinds of reports? Here’s a sample of what I intend to show you:
- Trending by number of general verified/potential vulnerabilities for an application over time
- Trending by number of OWASP top 10 verified/potential vulnerabilities for an application over time
- Number of new potential vulnerabilities introduced between two scans
- Number of potential vulnerabilities closed between two scans
If these reports sound interesting to you, or if you have other needs that you might be able to address with a general understanding of the SSC database schema, I hope you’ll join me for my next post!
