[Owasp-board] Contrast Marketing / OWASP Benchmark Project

johanna curiel curiel johanna.curiel at owasp.org
Wed Oct 21 20:11:18 UTC 2015


Hi Paul and all,

++++Martin, thats why don't wait too long to make an statement.

*Contrast Enterprise, which the OWASP **Benchmark demonstrated is both fast
and accurate, is a natural choice to augment or **replace existing SAST and
DAST solutions*

Benchmark cannot 'demonstrate' anything because is far from being stable or
considered to be able to 'benchmark' SAST/DAST tools


I did a quite intensive Project Review with other members of the review
team (including Abbas) and it was clear that the project is in an *infancy*
stage . A vendor doing these  kind of claims, in my opinion , false
statements about a tool that has not been proven, using OWASP name seems
wrong and that is the reason why Simon was concerned.

Please read the review here:
https://drive.google.com/file/d/0B3BoOR0oMwssOHMtdElqOUpKbW40RHpKbTVkTnRKRjVQUERB/view?usp=sharing

As you can see , the tool is far from being considered as something that
can 'Benchmark' tools. This kind of statements seems very wrong and create
false expectations.

I spoke to Dave Winchers and told him that I would like to do a 'separate'
research of the project which I will lead and we can compare findings.
During our discussion was clear that if the tool being benchmark does not
produce output results but is able to do automates tests, then Benchmark
cannot 'benchmark' all the capabilities.

I really like this project, excellent but I'm concerned how is being
misused for the benefit and profit using OWASP name under a Project, doing
false claims.

This is what I see as the major issue

Regards

Johanna

On Wed, Oct 21, 2015 at 3:48 PM, Paul Ritchie <paul.ritchie at owasp.org>
wrote:

> Michael, Jim, All:
>
> As this topic develops, I've heard back from Matt K on his direct 1x1
> contact, and I look forward to Josh's comments after talks with their CTO.
>   Also, I re-read their most recent commercial whitepaper, carefully.
> https://cdn2.hubspot.net/hubfs/203759/Contrast_Benchmark092215.pdf
>
> 1.  *Re:  Adjustments needed to Brand Usage guidelines*:   In my opinion,
> No, not based on this one example.
> >>  Jim already pointed out this clause:
>
>    - The OWASP Brand must not be used in a manner that suggests that The
>    OWASP Foundation supports, advocates, or recommends any particular product
>    or technology.
>
> 2.   *Re:  Paul to communicate with Contrast & Community on proper, or
> best practice use of OWASP brand & logo* = YES.
> >>  Noreen sent out several emails, a couple months ago....but time for a
> refresh & reminder.
>
> >>  While the language of the commercial whitepaper is "carefully" crafted
> to say simply "here are our results from the OWASP Bench Mark
> test"....there is no overt statement of OWASP support or endorsement.
>
> >>BUT, I DO take issue with the posting of our Logo on the side of their
> Whitepaper, which they call a 'Business Case on the OWASP Benchmark
> project", and I take issue that no "About OWASP" section was included in
> this whitepaper.
>
> >>  If Contrast is going to say this in their whitepaper:  *Contrast
> Enterprise, which the OWASP **Benchmark demonstrated is both fast and
> accurate, is a natural choice to augment or **replace existing SAST and
> DAST solutions*
> ....THEN, they should have provided an 'About OWASP' section describing
> our nonprofit, vendor neutral position or a disclaimer that OWASP does not
> endorse or support.
>
> Best Regards, Paul Ritchie
> OWASP Executive Director
> paul.ritchie at owasp.org
>
>
> On Wed, Oct 21, 2015 at 11:24 AM, Michael Coates <michael.coates at owasp.org
> > wrote:
>
>> I don't think the brand usage policy needs adjusting. It seems to
>> correctly capture this situation as not in line with the policy. But let me
>> know if you're seeing something that's being missed and would warrant a
>> brand usage policy update.
>>
>>
>> --
>> Michael Coates | @_mwc <https://twitter.com/intent/user?screen_name=_mwc>
>> OWASP Global Board
>>
>>
>>
>>
>>
>> On Tue, Oct 20, 2015 at 11:52 PM, Jim Manico <jim.manico at owasp.org>
>> wrote:
>>
>>> Michael,
>>>
>>> Good analysis and resolution suggestions. Do you think the brand usage
>>> policy needs adjusting as well?
>>>
>>> --
>>> Jim Manico
>>> Global Board Member
>>> OWASP Foundation
>>> https://www.owasp.org
>>> Join me in Rome for AppSecEU 2016!
>>>
>>> On Oct 20, 2015, at 10:11 PM, Michael Coates <michael.coates at owasp.org>
>>> wrote:
>>>
>>> I think we definitely have an issue here.
>>>
>>> 1)It is quite clear, at it's current state, the project has a conflict
>>> of interest.
>>>
>>> The concept of the project could be great. But a conflict is present in
>>> it's current state. As a result, everything achieved by the project is
>>> under a shadow. I also don't think there's any disagreement about conflict
>>> of interest either (see chapter leader response
>>> http://lists.owasp.org/pipermail/owasp-benchmark-project/2015-October/000031.html).
>>> This isn't to say that anyone is purposively influencing results, but a
>>> "conflict of interest' is about relationships and impartiality, not about
>>> specific actions. As a result I feel the conflict of interest is here and
>>> should be acted on.
>>>
>>> Suggested action:
>>> 1a - we should label the project as such (idea: a banner across the wiki
>>> page with "outstanding issues: conflict of interest)' just like wikipedia
>>> does.
>>> 1b - we should ask the project committee to consider updating the
>>> project maturity process such that a project cannot advance out of the most
>>> initial phase if a conflict of interest is present.
>>>
>>>
>>> 2) Branding
>>> The quotes you mention are not inline with our branding requirements. I
>>> also don't believe the logo is to be used on vendor literature.
>>>
>>> https://www.owasp.org/index.php/Marketing/Resources
>>> The OWASP Brand must not be used in a manner that suggests that The
>>> OWASP Foundation supports, advocates, or recommends any particular product
>>> or technology.
>>>
>>> Suggested action:
>>> 2a - Paul to reach out to Contrast to discuss how to work with OWASP
>>> correctly.
>>>
>>>
>>> Thoughts from others?
>>>
>>>
>>> --
>>> Michael Coates | @_mwc
>>> <https://twitter.com/intent/user?screen_name=_mwc>
>>> OWASP Global Board
>>>
>>>
>>>
>>>
>>>
>>> On Tue, Oct 20, 2015 at 11:39 AM, Josh Sokol <josh.sokol at owasp.org>
>>> wrote:
>>>
>>>> Did anyone see this?
>>>>
>>>> https://cdn2.hubspot.net/hubfs/203759/Contrast_Benchmark092215.pdf
>>>>
>>>> It is a vendor "Whitepaper" using the OWASP Benchmark Project along
>>>> with the Foundation brand and logo.  A couple of choice quotes from the
>>>> whitepaper:
>>>>
>>>> "Any product that doesn’t score highly on the OWASP Benchmark puts
>>>> organizations at serious risk of missing major vulnerabilities in their
>>>> real-world applications and generating lots of false alarms."
>>>>
>>>> "Contrast Enterprise, which the OWASP Benchmark demonstrated is both
>>>> fast and accurate, is a natural choice to augment or replace existing SAST
>>>> and DAST solutions."
>>>>
>>>> ~josh
>>>>
>>>>
>>>> _______________________________________________
>>>> Owasp-board mailing list
>>>> Owasp-board at lists.owasp.org
>>>> https://lists.owasp.org/mailman/listinfo/owasp-board
>>>>
>>>>
>>> _______________________________________________
>>> Owasp-board mailing list
>>> Owasp-board at lists.owasp.org
>>> https://lists.owasp.org/mailman/listinfo/owasp-board
>>>
>>>
>>
>> _______________________________________________
>> Owasp-board mailing list
>> Owasp-board at lists.owasp.org
>> https://lists.owasp.org/mailman/listinfo/owasp-board
>>
>>
>
> _______________________________________________
> Owasp-board mailing list
> Owasp-board at lists.owasp.org
> https://lists.owasp.org/mailman/listinfo/owasp-board
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.owasp.org/pipermail/owasp-board/attachments/20151021/72464605/attachment.html>


More information about the Owasp-board mailing list