[Owasp-leaders] [Owasp-board] OWASP Benchmark project - potential conflict of interest

Andre Gironda andreg+owasp at gmail.com
Thu Nov 26 20:12:51 UTC 2015

On Thu, Nov 26, 2015 at 12:09 PM, Josh Sokol <josh.sokol at owasp.org> wrote:
> I would be happy to provide an update.
> Matt Konda and Dave Wichers, the Benchmark Project Leader, had a
> conversation a few weeks back.  To summarize their conversation, Dave
> acknowledges the currently lack of diversity in his project and it is his
> sincere desire to drive more people to it to help.

>From my perspective, this is a core project that has the potential for
the best outcomes. Every appsec program -- every infosec program --
leads with tool(s) instead of people. Business owners and app owners
want business-as-usual portal(s) for the everyday uninitiated portal
user. I emphasize my parenthetical use of the plural (i.e., (s)'s)
because many times only one tool is chosen, or [at best?] chosen for a
few quarters and then migrated entirely to a new [often worse?] tool.

What both Aspect and Contrast have contributed should be encouraged
more. These vendors are _contributing_ forward-looking solutions that
get to the root cause of obstacles in application security.

So what do we give them? A reward? No -- we give them more obstacles.
What is that about? The vendors who have a seat to the table but never
show up (and those that complain in the title of their blog post but
then surreptitiously point out that they actually already resell
Contrast): they are in the wrong here.

> Josh Sokol and Jeff Williams, the CTO of Contrast, had a conversation a few
> weeks back.  To summarize their conversation, Jeff believes that the work
> that Dave is doing on the Benchmark is a game changer in that it gives OWASP
> the power in dictating what these tools need to be finding.  He wants the
> Benchmark to be successful and understands that it needs to be diverse in
> order to be trusted.

I hope we all believe in Jeff's message. The primary problem I see
isn't around the marketing -- it's that the Benchmark page on Tool
Results makes no mention of other IAST solutions, such as Synopsys
Seeker, Secure Decisions, or HPE Security's SecurityScope. As an early
IAST adopter who was consulting for one of these primary vendors on
many of the first-ever large-installation IAST projects, I can attest
to the need for both greater competition (e.g., demonstrating need and
delivering the IAST solutions) as well as more attention (e.g.,

Worse still, is the attitude (and, really, culture) against ad-hoc
testing, even when aligned with threat assessments and other critical
pieces of OpenSAMM (or other appsec program model and framework).
Power users (no matter their background) are heretics when compared to
the general-user population. Leaders rule by authority and not by
[technical] competency. Ad-hoc tests and testers, who know the error
rates of tools, are shut down and not allowed to speak.

Organizations must allow flow to occur in their appsec and appdev
integration programs. They must set clear goals using the OWASP
frameworks (OpenSAMM) and the guidance from OWASP on off-the-shelf
(packaged, FOSS, or not) appsec technology (Benchmark). Ad-hoc testers
and power users must have the right tools to perform each task.
Technology should be mapped to the process and people in order to be
additive to the value chain -- not subtractive.

Thank you,

More information about the OWASP-Leaders mailing list