[Owasp-leaders] A neutral Benchmark

johanna curiel curiel johanna.curiel at owasp.org
Sat Nov 28 01:10:11 UTC 2015

I think that OWASP should not be publishing results.

Agree, the person publishing the results is Johanna et al.
Also with a disclaimer: Johanna's opinions do not represent in any way
OWASP endorsing or not the tool. This initiative is solely carried on by
Johanna etc...

Fact is that due to the dependency of a XML output report of the findings,
I can totally assert that this tool cannot compare 1 on 1 any SAST/DAST
tools against each other, therefore the claims done by Contrast are totally
Contrast dominates SAST & DAST in Speed and Accuracy?

This is so false😂....


[image: Inline image 1]

On Fri, Nov 27, 2015 at 9:00 PM, Josh Sokol <josh.sokol at owasp.org> wrote:

> I really like this idea, Johanna, and it seems inline with Dave's
> suggestion of having an Advisory Board for the project.  The one thing that
> I do think that we need to steer clear from, however, is in publishing the
> results of the tests conducted with the Benchmark.  If others want to test
> and publish their personal results, that's not something we can stop, but
> in an effort to be vendor-neutral, I think that OWASP should not be
> publishing results.
> ~josh
> On Fri, Nov 27, 2015 at 4:16 PM, johanna curiel curiel <
> johanna.curiel at owasp.org> wrote:
>> Hi Dave
>> >>I don¹t have licenses to any of these tools and so far, no one has
>> stepped
>> up and offered to run any of these tools against the Benchmark.
>> I think that the Contrast marketing campaign hurt the participation of a
>> promising project before it could take off.
>> For every specific xml output report , you need to create a parser in
>> order to produce the reports. Without their collaboration or people with
>> licences to test, you won't get their input
>> As a neutral party with no conflict of interests in this project, I think
>> we can request licenses to these vendors and with the participation of
>> other volunteers that have no commercial ulterior motives to this. I have
>> added Ali Ramzoo, who is also part of the  OWASP Research initiative
>> We could indeed:
>>    - Promote that the project is under a neutral research initiative
>>    - Ask for licenses,
>>    - Deploy them in a VM we can all have access to
>>    - Verify if the tools can produce an XML output report (if not you
>>    cannot parse)
>>    - Discuss with them our findings privately before publishing our
>>    findings
>>    - We have also to be very conscious that if the XML report does not
>>    generate all the findings in their tool (as the case of ZAP with Fuzzing)
>>    then we need mention this very clear. Otherwise you can hurt the reputation
>>    of the tool.
>> This is how I can help this project and try to create a neutral clean
>> view of a tool which I believe has potential but it needs to shake off all
>> the publicity around Contrast
>> Regards
>> Johanna
>> _______________________________________________
>> OWASP-Leaders mailing list
>> OWASP-Leaders at lists.owasp.org
>> https://lists.owasp.org/mailman/listinfo/owasp-leaders
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.owasp.org/pipermail/owasp-leaders/attachments/20151127/0b7c9bf7/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: Screenshot 2015-11-27 21.06.49.png
Type: image/png
Size: 526209 bytes
Desc: not available
URL: <http://lists.owasp.org/pipermail/owasp-leaders/attachments/20151127/0b7c9bf7/attachment-0001.png>

More information about the OWASP-Leaders mailing list