[Owasp-leaders] Listing commercial products in OWASP projects

Calderon, Juan Carlos (GE, Corporate, consultant) juan.calderon at ge.com
Tue Jan 20 10:56:01 EST 2009

I agree, as long a deep research of tools is done and all of them are
published that is ok,  while we not only mention those we know at first
hand (Which I think is huge effort) also adding an invitation in the
guide page asking readers to add more tools is a good idea, I guess.

About comparison is not that you cannot compare them, of course you can
do it, for internal purposes. But in our experience and AFAIK. What you
cannot do, is to publish the results of the comparison without
permission of the vendor, that (ironically) is the purpose of the
comparison in our case. 

This is my experience in the past, but, it was before acquisition of
SPIDynamics and Watchfire so maybe the policies have changed by now.
Also this might not be applicable to all the vendors, anyway, if they
cannot be benchmarking a not of "Not possible to benchmark" or similar
might work and maybe self-promote a benchmark for one of those tools.

Juan Carlos Calderon

-----Original Message-----
From: owasp-leaders-bounces at lists.owasp.org
[mailto:owasp-leaders-bounces at lists.owasp.org] On Behalf Of Pavol Luptak
Sent: Martes, 20 de Enero de 2009 07:33 a.m.
To: jeff.williams at owasp.org; owasp-leaders at lists.owasp.org
Cc: dvstein at gmail.com
Subject: Re: [Owasp-leaders] Listing commercial products in OWASP

On Mon, Jan 19, 2009 at 01:21:36PM -0500, Jeff Williams wrote:
>    I suggest that we should follow the same rule that is in force for
>    meetings.  The basic principle is that you can't use OWASP to
promote your
>    product (outside of an authorized commercial opportunity - like an
>    advertisement, conference sponsorship, etc...)  Mentions of vendor
>    products are allowed on the website, but only in the context of a
>    particular problem, and only when a broad survey of the available
>    commercial and open-source tools is performed.

It sounds absolutely logical for me.

But what about a new vendor-unbiased, open OWASP project that would
compare various commercial and free penetration tools, scanners (all
high-quality stuff we daily use :-)

Because at the present time (when sometimes it is impossible to obtain
from vendor free/trial version) it is very difficult to compare various
penetration tools and decide which is better in which category....

And because almost all available (opensource + commercial) used tools
would be mentioned in this project, this project should be

We can benchmark these tools against OWASP WebGoat (or create some other
test samples).
And of course - if this "testing tool comparison/benchmark project"
should be vendor/product unbiased, it should be covered by some unbiased
open foundation
(OWASP?) not by a private company.

Juan Carlos also mentioned than vendors are very reluctant to
benchmarking (some EULAs can contain clauses that prohibit you to
benchmark some products).
Personally I don't know any "non-comparable" security tool, but if they
really exist, we simply would not involve these tools in the final
benchmark/comparison (personally I would never buy a product that could
not be compared with the other one :-)

Remember, our original OWASP mission is "to have more secure web
applications in the world", so we can use this "open benchmark project"
to support/promote _ALL_ high-quality tools (and in this case it doesn't
matter if they are commercial or not).
I think that everybody would appreciate this project :-)

What do you think ? Is it compliant with OWASP rules?

Pavol Luptak, CISSP, CEH
OWASP Slovakia chapter leader

More information about the OWASP-Leaders mailing list