Is OWASP Benchmark Any Good?

A short analysis of the OWASP Benchmark project towards comparing the SAST tools

OWASP Benchmark

The OWASP Benchmark Project is a Java test suite designed to verify the speed and accuracy of vulnerability detection tools.

A single test case represented with an Java source file and a complementing XML file.

A Simple Analysis

// Simple if statement that assigns constant to bar on true condition
int num = 86;
if ((7 * 42) - num > 200) bar = "This_should_always_happen";
else bar = param;
DangerousMethod(bar); // a synthesized sink
String bar = "alsosafe";
if (param != null) {
java.util.List<String> valuesList = new java.util.ArrayList<String>();
valuesList.add("safe");
valuesList.add(param);
valuesList.add("moresafe");

valuesList.remove(0); // remove the 1st safe value

bar = valuesList.get(1); // get the last 'safe' value
}
DangerousMethod(bar); // a synthesized sink
SeparateClassRequest scr = new SeparateClassRequest(request);
String param = scr.getTheValue("BenchmarkTest00051");
...
String[] args = {a1, a2, "echo " + param};

ProcessBuilder pb = new ProcessBuilder(args);
public String getTheValue(String p) {
return "bar";
}
String algorithm = benchmarkprops.getProperty("cryptoAlg1", "AES/ECB/PKCS5Padding");
# This file contains various property values used by various test cases in the OWASP Benchmark
cryptoAlg1=DES/ECB/PKCS5Padding
cryptoAlg2=AES/CCM/NoPadding
hashAlg1=MD5

A Short Critique

A classified benchmark test cases of FlowBlot.NET against SAST tools.
Cookie cookie = new javax.servlet.http.Cookie("SomeCookie", str);

cookie.setSecure(false);
cookie.setHttpOnly(true);

Conclusion

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store