[Owasp-board] [Owasp-leaders] OWASP Benchmark project - potential conflict of interest

psiinon psiinon at gmail.com
Mon Nov 30 17:40:46 UTC 2015


>
>
> In the short term, It is better to remove it from OWASP, leaving the door
> open for its return (in a future when some of the independence and quality
> issues have been solved)
>
>
I would be delighted to welcome an independent, high quality Benchmark
project back into OWASP :)




> Specially when recently we made David Rook remove this much more benign
> 'commercial content' from OWASP
>
> Dinis
>
> On 30 November 2015 at 17:17, psiinon <psiinon at gmail.com> wrote:
>
>> I'd like to start by saying that I actually _like_ the Benchmark project.
>> Myself and other ZAP developers have made some contributions to it, and
>> we have used (and will continue to use) it to make ZAP better.
>> I think these sort of testing applications are very valuable to all
>> security tools, and I'd like to thank Dave and his team for the significant
>> amount of effort involved in developing and open sourcing it.
>>
>> But I dont think it should be an OWASP project.
>> I do not think that a vendor led project can ever objectively evaluate
>> competing commercial and open source projects.
>> I do not think that just saying 'pull requests welcomed' makes a project
>> vendor neutral.
>> I do not think that a project as mired in controversy as the Benchmark
>> project can ever recover to become truly independent.
>>
>> I am very disappointed in the Boards handling of this affair.
>>
>> Ideally I'd like Dave to understand how much damage this project has done
>> and to withdraw it as an OWASP project, while still maintaining it as a
>> very valuable vendor led open source resource.
>>
>> Failing that I really hope that the Board comes to its senses and ejects
>> the Benchmark project before even more damage is done.
>> At the _very_ least it should flag the project as being 'in dispute' (as
>> Kevin suggested) while a more detailed evaluation is performed.
>>
>> However I'm rapidly loosing loosing faith that the Board will do the
>> right thing and protect OWASP's image in the way that they should have
>> already done.
>> Members - please make your voices heard before more people and projects
>> leave OWASP.
>>
>> Simon
>>
>> On Sat, Nov 28, 2015 at 5:14 AM, Jim Manico <jim.manico at owasp.org> wrote:
>>
>>> WAFEC does not "do vendor assessment"; they define a comprehensive
>>> standard built by many vendors and let the community use that standard to
>>> measure tools on their own. Just a FYI, I was involved in the early version
>>> of this project. (Things may have changed since my involvement, I'm sure
>>> Tony has more details here)
>>>
>>> Johanna's comments on this issue lead me to believe that the damage done
>>> to both OWASP and DHS is even more destructive that I thought. It saddens
>>> me to see this level of abuse just to sell product.
>>>
>>> --
>>> Jim Manico
>>> Global Board Member
>>> OWASP Foundation
>>> https://www.owasp.org
>>> Join me in Rome for AppSecEU 2016!
>>>
>>> On Nov 28, 2015, at 2:40 AM, Josh Sokol <josh.sokol at owasp.org> wrote:
>>>
>>> One of the ideas that Andrew proposed was actually approaching WAFEC to
>>> learn more about how they do vendor assessment in a neutral way.  It's
>>> great to hear that we have a resource here already that we can leverage.  I
>>> wasn't aware of your affiliation.
>>>
>>> ~josh
>>> On Nov 27, 2015 2:47 PM, "Tony Turner" <tony.turner at owasp.org> wrote:
>>>
>>>> I sincerely hope so. That's not the impression I got from others
>>>> comments. Personally I haven't used the tool at all, but as I'm the project
>>>> lead for another product evaluation project (WAFEC) I'm very sensitive to
>>>> the need of collaboration with many different vendors. There really has to
>>>> be a very high level (almost paranoid level) transparency with how vendors
>>>> are approached, worked with, how requirements for evaluation are defined,
>>>> and how metrics are derived.
>>>>
>>>> It appears the project team is attempting to address these last 2
>>>> somewhat but I'd like to see more specifics, and the lack of information on
>>>> how they are addressing vendor communication, participation and
>>>> transparency seems a bit concerning. Lastly, it is my opinion that project
>>>> leadership should not belong to anyone working for or with a
>>>> partnership/ownership stake for any vendor being evaluated. I think this is
>>>> a flawed model and should transition to a vendor neutral party.
>>>> On Nov 27, 2015 3:16 PM, "Josh Sokol" <josh.sokol at owasp.org> wrote:
>>>>
>>>>> I don't know what qualifies as "significant" in your mind, but my
>>>>> understanding is that there have been contributions from other vendors:
>>>>>
>>>>> https://www.owasp.org/index.php/Benchmark#tab=Acknowledgements
>>>>>
>>>>> Still, Dave would like more, but he can't force them to help.
>>>>>
>>>>> ~josh
>>>>>
>>>>> On Fri, Nov 27, 2015 at 1:45 PM, Tony Turner <tony.turner at owasp.org>
>>>>> wrote:
>>>>>
>>>>>> While I can appreciate that they started with Contrast, if there
>>>>>> hasn't been significant effort to include other vendors it's a worthless
>>>>>> benchmark. It's easy to state you haven't gotten support from other vendors
>>>>>> and that's fine, but until you do there's really nothing to release. Why
>>>>>> was it ever upgraded? Talking about the results without an accurate
>>>>>> comparative analysis is akin to snake oil.
>>>>>> On Nov 27, 2015 1:49 PM, "Josh Sokol" <josh.sokol at owasp.org> wrote:
>>>>>>
>>>>>>> Thank you for the links to those articles.  The first one discusses
>>>>>>> the strengths and weaknesses of the different methods of evaluating for
>>>>>>> application vulnerabilities.  The section on the Benchmark seems wholly
>>>>>>> appropriate to me.  That seems like an excellent description of what the
>>>>>>> project is designed to do.  I see some metrics in there about which tools
>>>>>>> are more effective on which types of vulnerabilities, but I don't see him
>>>>>>> straight up saying "The OWASP Benchmark proves that Contrast is better".
>>>>>>> This seems like statements made based on some level of testing and
>>>>>>> research.  Honestly, I don't see any OWASP brand abuse in that article.
>>>>>>> Whether it's in good taste or not at this stage in the project is certainly
>>>>>>> debatable, but if you look at the brand usage guidelines (
>>>>>>> https://www.owasp.org/index.php/Marketing/Resources#tab=BRAND_GUIDELINES),
>>>>>>> I don't see any violations.  We need to govern to policy here which is why
>>>>>>> Paul and Noreen are evaluating changes to the guidelines and our
>>>>>>> enforcement policies to make abuse more difficult.
>>>>>>>
>>>>>>> The second article is a competing vendor's reaction to the first.
>>>>>>> He makes some good points about the issues with Benchmark, but he also says
>>>>>>> that he hopes that it will be improved over time, and Dave has committed to
>>>>>>> that.  What I don't see is the vendor saying "...and Veracode has committed
>>>>>>> resources to help make the Benchmark more accurate across all tool sets".
>>>>>>> The Benchmark page is pretty clear that it does it's best to provide a
>>>>>>> benchmark without working exactly like a real-world application.  Maybe
>>>>>>> some more disclaimer text about where the project is at today would be in
>>>>>>> order to validate some of Chris' concerns, but I hardly see this as "brand
>>>>>>> abuse" or a reason to demote the project.
>>>>>>>
>>>>>>> Please consider that I have spoken with both Dave and Jeff on this
>>>>>>> topic and read much of the discussions around it before formulating my
>>>>>>> opinion.  I doubt that you have done the same so I'm not sure how you can
>>>>>>> claim that you have researched the issues and all parties involved when you
>>>>>>> haven't even spoken with the two people whom you are accusing of
>>>>>>> impropriety.  I have no bias here.  I am simply speaking with the
>>>>>>> individuals involved, looking at the currently OWASP policies and
>>>>>>> guidelines, and helping to determine our next steps.
>>>>>>>
>>>>>>> ~josh
>>>>>>>
>>>>>>> On Fri, Nov 27, 2015 at 12:22 PM, johanna curiel curiel <
>>>>>>> johanna.curiel at owasp.org> wrote:
>>>>>>>
>>>>>>>> >>While I agree with you that there has been some brand abuse, it
>>>>>>>> was abuse by Contrast (specifically their marketing department), and not by
>>>>>>>> "these gentlemen" as  you state.
>>>>>>>>
>>>>>>>> Really? ..'some brand abuse'..this is more than brand abuse
>>>>>>>>
>>>>>>>> Josh , please read also the article written by Jeff
>>>>>>>>
>>>>>>>> http://www.darkreading.com/vulnerabilities---threats/why-its-insane-to-trust-static-analysis/a/d-id/1322274
>>>>>>>> ?
>>>>>>>>
>>>>>>>> And Veracode's reaction including others in Twitter
>>>>>>>>
>>>>>>>> https://www.veracode.com/blog/2015/09/no-one-technology-silver-bullet
>>>>>>>>
>>>>>>>> My strong advice is to research the issues and all the parties
>>>>>>>> involved before making statements
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On Fri, Nov 27, 2015 at 2:07 PM, Josh Sokol <josh.sokol at owasp.org>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Jim,
>>>>>>>>>
>>>>>>>>> A concern was expressed to the Board and, frankly, I am insulted
>>>>>>>>> by you saying that this was "brushed under the rug".  The Board delegated
>>>>>>>>> Matt to talk with Dave and they had a lengthy conversation on the subject.
>>>>>>>>> The Board delegated me to talk with Jeff and we had a lengthy conversation
>>>>>>>>> on the subject.  If you do not trust in our abilities to read people, ask
>>>>>>>>> the right questions, and provide honest feedback about our conversations,
>>>>>>>>> then that's a bigger issue that we should take offline.  After our
>>>>>>>>> conversations, we took the time to call a special two-hour session of the
>>>>>>>>> Board in order to discuss this subject (and only this subject).  We spoke
>>>>>>>>> about all facets of the issue at hand, about the challenges and possible
>>>>>>>>> solutions, and concluded on some very concrete next steps.
>>>>>>>>>
>>>>>>>>> While I agree with you that there has been some brand abuse, it
>>>>>>>>> was abuse by Contrast (specifically their marketing department), and not by
>>>>>>>>> "these gentlemen" as  you state.  Unless you can point to some sort of
>>>>>>>>> evidence showing that Jeff and/or Dave first-hand abused the brand, then I
>>>>>>>>> believe that you are speaking with your heart instead of with your head.  I
>>>>>>>>> appreciate your passion, but I label this as conspiracy theory because
>>>>>>>>> without evidence to support your claims, I cannot accept it as anything
>>>>>>>>> other.
>>>>>>>>>
>>>>>>>>> ~josh
>>>>>>>>>
>>>>>>>>> On Fri, Nov 27, 2015 at 11:39 AM, Jim Manico <jim.manico at owasp.org
>>>>>>>>> > wrote:
>>>>>>>>>
>>>>>>>>>> Josh,
>>>>>>>>>>
>>>>>>>>>> I stand by my comments and perspective, but I'm disheartened that
>>>>>>>>>> you consider my presentation of facts (and the concerns of many active
>>>>>>>>>> members of our community) as a "conspiracy theory".
>>>>>>>>>>
>>>>>>>>>> In my experience, these kind of comments border on insults and
>>>>>>>>>> only cause folks to harden their opinions.
>>>>>>>>>>
>>>>>>>>>> Once again I feel these gentlemen got away with a kind of brand
>>>>>>>>>> abuse that is very hurtful to the OWASP community but I am at a loss as to
>>>>>>>>>> how handle or prevent these kinds of mishaps - especially when board
>>>>>>>>>> members like yourself seem willing to - from what I see - brush it under
>>>>>>>>>> the rug.
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> Jim Manico
>>>>>>>>>> Global Board Member
>>>>>>>>>> OWASP Foundation
>>>>>>>>>> https://www.owasp.org
>>>>>>>>>> Join me in Rome for AppSecEU 2016!
>>>>>>>>>>
>>>>>>>>>> On Nov 27, 2015, at 7:23 PM, Josh Sokol <josh.sokol at owasp.org>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>> Admittedly, this was my gut reaction at first as well.  I began
>>>>>>>>>> linking all of these companies, people, and projects together in my mind
>>>>>>>>>> (there are some loose links there) and painted a big conspiracy picture
>>>>>>>>>> similar to what Jim and Dinis have stated.  But, after speaking directly
>>>>>>>>>> with Jeff, and hearing about the conversation that Dave and Matt had, I've
>>>>>>>>>> changed my mind.
>>>>>>>>>>
>>>>>>>>>> I think it begins with the project itself.  If you aren't sold on
>>>>>>>>>> the idea of the Benchmark, then you'll never be able to get to the same
>>>>>>>>>> place.  My original line of thinking was that it was just a bar for vendors
>>>>>>>>>> to compare their tools against eachother, but that's a bit myopic.  We are
>>>>>>>>>> in an industry where things evolve very quickly.  As a customer of these
>>>>>>>>>> tools, I know firsthand that something that a tool does today may not be
>>>>>>>>>> the case a week from now.  Likewise, new features are being added daily and
>>>>>>>>>> I need a point-in-time metric to be able to gauge continual effectiveness.
>>>>>>>>>> Cool, right?  But not a game changer.  The game changer part comes when you
>>>>>>>>>> realize that by developing and evolving the tests that go into the
>>>>>>>>>> Benchmark, we are moving the bar higher and higher.  We (OWASP) are
>>>>>>>>>> effectively setting the standard by which these tools will be compared.  A
>>>>>>>>>> tool that receives a lower score on the Benchmark today knows exactly what
>>>>>>>>>> they need to work on in order to pass that test tomorrow and we already
>>>>>>>>>> have examples of tools that have made improvements because of their
>>>>>>>>>> Benchmark score (Ask Simon about ZAP's experience with the Benchmark).  I
>>>>>>>>>> don't think that anyone can argue that the Benchmark project isn't being
>>>>>>>>>> effective when OWASP's own tools are being driven forward as a result of
>>>>>>>>>> using it.
>>>>>>>>>>
>>>>>>>>>> But, but, but, Dave and Jeff own Aspect and have stock in
>>>>>>>>>> Contrast and Jeff is the Contrast CTO and Contrast got good scores so it's
>>>>>>>>>> a conspiracy right?  Is there some code that allows Contrast to use the
>>>>>>>>>> Benchmark?  Absolutely.  Can you really blame Dave for starting his testing
>>>>>>>>>> on the effectiveness of the Benchmark with a tool that he owned and is
>>>>>>>>>> familiar with?  If I were going to start a similar project, there's no
>>>>>>>>>> question in my mind that I would begin my testing with the tools that I
>>>>>>>>>> have available to me.  That said, is there code that allows other tools to
>>>>>>>>>> use the Benchmark?  Absolutely.
>>>>>>>>>>
>>>>>>>>>> Regarding "Dave has a history of breaching his duty to be vendor
>>>>>>>>>> neutral", while I cannot comment on his past actions, I can judge what
>>>>>>>>>> we've seen recently.  Matt saw a presentation from Dave on the Benchmark at
>>>>>>>>>> a conference in Chicago.  He said that he felt that the message was
>>>>>>>>>> appropriate and while IAST tools were mentioned as receiving higher scores,
>>>>>>>>>> it wasn't a "Contrast is the best" type of message, more of a generality.
>>>>>>>>>> I saw a very similar (if not the same) talk by Jeff at LASCON 2015 and the
>>>>>>>>>> message was exactly the same.  I watched the talk expecting some sort of
>>>>>>>>>> impropriety, but found none.  So, perhaps Dave has abused some privilege
>>>>>>>>>> granted to him in the past, but what I've seen from him at this point, with
>>>>>>>>>> respect to the Benchmark, has been appropriate.
>>>>>>>>>>
>>>>>>>>>> You have a very good point with respect to the Contrast marketing
>>>>>>>>>> message around the Benchmark.  It's been completely absurd, over the top,
>>>>>>>>>> and, in my personal opinion, intolerable.  In fact, I experienced the same
>>>>>>>>>> thing that you talked about with them at LASCON 2015 where they stood in
>>>>>>>>>> front of the door of the room Jeff was speaking in and scanned attendees as
>>>>>>>>>> they went into the talk.  I agree that these types of aggressive marketing
>>>>>>>>>> tactics cannot be tolerated at OWASP.  In addition, we have seen several
>>>>>>>>>> marketing messages from them effectively implying that OWASP endorses
>>>>>>>>>> Contrast.  Clearly this is not OK.  I've spoken with Jeff about it and we
>>>>>>>>>> agreed that it is not in the Benchmark's best interest to have this
>>>>>>>>>> aggressive Contrast marketing around it at such an early stage.  He has
>>>>>>>>>> said that he is not responsible for Contrast's marketing team, but that he
>>>>>>>>>> would speak with the people who are.  I haven't seen a single message from
>>>>>>>>>> them since so I'm guessing that he's made good on this promise.  While
>>>>>>>>>> that's an excellent start, OWASP's takeaway here should be that we need to
>>>>>>>>>> do a better job with our brand usage guidelines both in terms of the
>>>>>>>>>> wording and enforcement.  There are many other companies out there that use
>>>>>>>>>> the OWASP brand and I think that we agree that selective enforcement
>>>>>>>>>> against Contrast is not the right answer.  Paul and Noreen are actively
>>>>>>>>>> working on this.  Either way, I think that implying that activities from a
>>>>>>>>>> vendor's marketing department means that the project is not objective is
>>>>>>>>>> not inappropriate.  If we feel that the project is not objective, then
>>>>>>>>>> separate measures need to be taken to drive contribution diversity into
>>>>>>>>>> it.  That I absolutely agree with and the message from Dave was that he
>>>>>>>>>> would love to have more contributors to his project.  But, seeing as we
>>>>>>>>>> cannot force people to work on it, this becomes a matter of "put up or shut
>>>>>>>>>> up".  The same goes for the experts that you said reviewed the code.  If
>>>>>>>>>> they feel that it is somehow skewed towards Contrast, they have the power
>>>>>>>>>> to change that.  Now, if someone tries to participate and Dave tells them
>>>>>>>>>> "No thanks", then I agree we have a problem, but I don't hear anyone
>>>>>>>>>> inferring that happened.
>>>>>>>>>>
>>>>>>>>>> Please, let's drop the conspiracy theories and focus on the
>>>>>>>>>> tangible things that we can do to help an OWASP project to be more
>>>>>>>>>> successful.  Help find more participants to drive diversity, update our
>>>>>>>>>> brand usage guidelines to prevent abuse, enforce them widely, etc.  Thank
>>>>>>>>>> you.
>>>>>>>>>>
>>>>>>>>>> ~josh
>>>>>>>>>>
>>>>>>>>>> On Thu, Nov 26, 2015 at 4:24 PM, Jim Manico <jim.manico at owasp.org
>>>>>>>>>> > wrote:
>>>>>>>>>>
>>>>>>>>>>> Dinis,
>>>>>>>>>>>
>>>>>>>>>>> Like a rare celestial moment when all the planets plus Pluto are
>>>>>>>>>>> aligned, I just read your email on the future of OWASP projects thinking,
>>>>>>>>>>> "Dinis is spot on".
>>>>>>>>>>>
>>>>>>>>>>> Reflecting on projects I manage or work on...
>>>>>>>>>>>
>>>>>>>>>>> The Java Encoder and HTML Sanitizer are likely best moved to
>>>>>>>>>>> Apache now that they have reached a measure of adoption and maturity.
>>>>>>>>>>> Apache would be a much better long term custodian. Perhaps the same for
>>>>>>>>>>> AppSensor, but not my project - just thinking out loud.
>>>>>>>>>>>
>>>>>>>>>>> Other similar defensive projects are still being noodled on, so
>>>>>>>>>>> OWASP is a decent home for these research efforts.
>>>>>>>>>>>
>>>>>>>>>>> The whole tools category is also something to consider.
>>>>>>>>>>> Dependency Check and of course ZAP are some of the best projects that OWASP
>>>>>>>>>>> offers, are they best served where they are today? Both have rich
>>>>>>>>>>> communities of developers but I don't see the foundation doing much to
>>>>>>>>>>> support these efforts.
>>>>>>>>>>>
>>>>>>>>>>> ASVS has the opportunity to effect massive change, I would to
>>>>>>>>>>> love to see major investment and volunteer activity here. Pro tech writer,
>>>>>>>>>>> detailed discourses on each individual requirement, etc. If I was king (and
>>>>>>>>>>> I am not, at all) I would invest in ASVS on a 6 figure scale. (And who
>>>>>>>>>>> started ASVS? Jeff, Dave and Boberski, hat tip to such a marvelous idea).
>>>>>>>>>>> Or maybe moving ASVS to the W3C or IETF would help it grow?
>>>>>>>>>>>
>>>>>>>>>>> The Proactive Controls was a pet project but as we approach 2.0
>>>>>>>>>>> we have several active/awesome volunteers working on it. We will be making
>>>>>>>>>>> the doc "world editable" to make contributions easy. OWASP seems like a
>>>>>>>>>>> good home for such an awareness doc. Same with T10, especially if community
>>>>>>>>>>> edits are welcome.
>>>>>>>>>>>
>>>>>>>>>>> Anyhow, I'm with you on this Dinis. Once a project starts to
>>>>>>>>>>> reach production quality, spinning off the project as an external project
>>>>>>>>>>> or moving it to a different foundation where managing production software
>>>>>>>>>>> or formal standards is their thing seems realistic.
>>>>>>>>>>>
>>>>>>>>>>> I don't have all the answers here, but your email certainly
>>>>>>>>>>> resonated with me.
>>>>>>>>>>>
>>>>>>>>>>> Aloha,
>>>>>>>>>>> --
>>>>>>>>>>> Jim Manico
>>>>>>>>>>> Global Board Member
>>>>>>>>>>> OWASP Foundation
>>>>>>>>>>> https://www.owasp.org
>>>>>>>>>>> Join me in Rome for AppSecEU 2016!
>>>>>>>>>>>
>>>>>>>>>>> On Nov 26, 2015, at 11:26 PM, Dinis Cruz <dinis.cruz at owasp.org>
>>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>> Jim's reading of this situation is exactly my view on the value
>>>>>>>>>>> of the Contrast tool and how it has been 'pushing' the rules of engagement
>>>>>>>>>>> to an very 'fuzzy' moral/ethical/commercial limit :)
>>>>>>>>>>>
>>>>>>>>>>> As per my last email, a key problem here is the 'perceived
>>>>>>>>>>> expectation' of what is an OWASP project, and how it should be consumed.
>>>>>>>>>>>
>>>>>>>>>>> If you look at the OWASP benchmark as a research project, then
>>>>>>>>>>> the only way it could be making the kind of claims it makes (and have
>>>>>>>>>>> credibility) is if it had evolved from OWASP, with its own (diverse)
>>>>>>>>>>> community
>>>>>>>>>>>
>>>>>>>>>>> On 26 November 2015 at 21:01, Jim Manico <jim.manico at owasp.org>
>>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> I have a different take on this situation but my opinion is the
>>>>>>>>>>>> "minority opinion". I will respect the rest of the boards take on this, but
>>>>>>>>>>>> here is how I see it.
>>>>>>>>>>>>
>>>>>>>>>>>> First of all, Jeff has stated that he feels I am attacking him
>>>>>>>>>>>> personally from a past personal grudge, and frankly I do not fault him for
>>>>>>>>>>>> that perspective since we definitely have history with conflict. So it's
>>>>>>>>>>>> fair to take my opinion on this with a grain of salt.
>>>>>>>>>>>>
>>>>>>>>>>>> I look at this situation from the perspective of a forensic
>>>>>>>>>>>> investigator.
>>>>>>>>>>>>
>>>>>>>>>>>> 1) The Benchmark project had Contrast hooks and only Contrast
>>>>>>>>>>>> hooks in it when I reviewed it so this leads me to believe that the project
>>>>>>>>>>>> was clearly built with Contrast in mind from the ground up, at least in
>>>>>>>>>>>> some way.
>>>>>>>>>>>> 3) Dave has a history of breaching his duty to be vendor
>>>>>>>>>>>> neutral. He was gifted with a keynote in South Korea a few years ago, and
>>>>>>>>>>>> used that opportunity to discuss and pitch Contrast, on stage, during a
>>>>>>>>>>>> keynote - with Contrast specific slides. This is just supporting evidence
>>>>>>>>>>>> of his intention at OWASP to push Contrast in ways that I think are against
>>>>>>>>>>>> the intentions and goals of our foundation.
>>>>>>>>>>>> 3) Other experts have reviewed the project and felt that many
>>>>>>>>>>>> of the tests were very slanted and almost contrived to support Contrast. I
>>>>>>>>>>>> can drag those folks into this conversation, but I do not think that would
>>>>>>>>>>>> help in any way. So it's fair to call this point heresy.
>>>>>>>>>>>> 4) I do not see this project as revolutionary, at all. Every
>>>>>>>>>>>> vendor has their own test suite tuned for their tool. As the benchmark
>>>>>>>>>>>> stands today, I see it as just another vendors product-specific benchmark.
>>>>>>>>>>>> Mass collaboration from many vendors is not just a "nice to have" but a
>>>>>>>>>>>> base requirement to get even close to useful for objective tool measurement.
>>>>>>>>>>>> 5) Jeff stating that his Marketing people went over the line is
>>>>>>>>>>>> also an admission that - well, they went over the line. By the same token
>>>>>>>>>>>> Jeff was in his booth at AppSec USA surrounded by benchmark marketing
>>>>>>>>>>>> material, discussing this to prospects and he even asked me and Mr Coates
>>>>>>>>>>>> to wade into this debate and support Dave. So to say he was not involved
>>>>>>>>>>>> and it was only his marketing people seems a stretch at best.
>>>>>>>>>>>> 6) The Contrast marketing team was wandering around the
>>>>>>>>>>>> conference zapping folks to get leads, and I asked them to stay in their
>>>>>>>>>>>> booth, which is standard conference policy. These folks know better but are
>>>>>>>>>>>> again going over the line to sell product at OWASP. There is a better way
>>>>>>>>>>>> (like focusing on product capability and language support, have consistent
>>>>>>>>>>>> + stellar customer service, have a humble and gracious attitude to all
>>>>>>>>>>>> prospects and customers, actively participate in OWASP in a vendor neutral
>>>>>>>>>>>> and community supportive way, etc).
>>>>>>>>>>>>
>>>>>>>>>>>> Please note, I think Contrast is a decent tool, I've offered to
>>>>>>>>>>>> resell in the past, and I have recommended it in certain situations - even
>>>>>>>>>>>> after this situation arose. I'm stating this out of honestly and desire to
>>>>>>>>>>>> put my cards on the table. I truly want Jeff and Dave to be successful.
>>>>>>>>>>>> They have dedicated their lives to AppSec and if anyone should win
>>>>>>>>>>>> big-time, I hope it's them. I even told Jeff I hope he hits the mother load
>>>>>>>>>>>> and donates a little back to OWASP.
>>>>>>>>>>>>
>>>>>>>>>>>> However, my instinct and evidence tell me that they both went
>>>>>>>>>>>> over the line in the use of the OWASP brand to sell product.
>>>>>>>>>>>>
>>>>>>>>>>>> Now, Jeff makes a good point. We as a board and staff are very
>>>>>>>>>>>> poor at enforcing brand management policy and it's not fair to single out
>>>>>>>>>>>> Contrast, when many other vendors violate the brand, IMO. Just google OWASP
>>>>>>>>>>>> and watch the ads fly that use the OWASP name to sell product.
>>>>>>>>>>>>
>>>>>>>>>>>> Also, any and every request that was made of Dave to adjust the
>>>>>>>>>>>> project for the sake of vendor neutrality was taken very seriously.
>>>>>>>>>>>> Regardless of Daves past intentions, he is clearly trying to do the right
>>>>>>>>>>>> thing moving forward.
>>>>>>>>>>>>
>>>>>>>>>>>> I look to "postels principle" in this situation (this is
>>>>>>>>>>>> otherwise known as the "robustness principle" and dates back to the
>>>>>>>>>>>> creation of TCP) . This is paraphrased as, "Be liberal in what you take
>>>>>>>>>>>> from others but be conservative in what you dish out". So I think it's
>>>>>>>>>>>> critical that OWASP and any OWASP resource present itself in a strict
>>>>>>>>>>>> vendor neutral way. But unless OWASP wants to be much more "even" in the
>>>>>>>>>>>> enforcement of brand policy across the board to all violators, we should be
>>>>>>>>>>>> fairly lax in the enforcement of these issues from the outside world.
>>>>>>>>>>>>
>>>>>>>>>>>> I am trying to be objective here. My trigonometry teacher once
>>>>>>>>>>>> told me "I'd fail my mother" when I asked him if he would ever fail me (I
>>>>>>>>>>>> was an A student). If my mother owned a security company and tried the same
>>>>>>>>>>>> stunt, I'd have the same opinions about her actions as well.
>>>>>>>>>>>>
>>>>>>>>>>>> So what next? Well hello from the other side. I'm going back to
>>>>>>>>>>>> listening to Adele's new album where I can sit in my deep feelings and
>>>>>>>>>>>> reflect upon what the OWASP foundation has done to enrich my life. I would
>>>>>>>>>>>> much rather keep out of this (and any other conflict laden situation at
>>>>>>>>>>>> OWASP), but I feel it's my responsibility to speak up.
>>>>>>>>>>>>
>>>>>>>>>>>> Aloha,
>>>>>>>>>>>> --
>>>>>>>>>>>> Jim Manico
>>>>>>>>>>>> Global Board Member
>>>>>>>>>>>> OWASP Foundation
>>>>>>>>>>>> https://www.owasp.org
>>>>>>>>>>>> Join me in Rome for AppSecEU 2016!
>>>>>>>>>>>>
>>>>>>>>>>>> On Nov 26, 2015, at 9:09 PM, Josh Sokol <josh.sokol at owasp.org>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>> I would be happy to provide an update.
>>>>>>>>>>>>
>>>>>>>>>>>>    - Matt Konda and Dave Wichers, the Benchmark Project
>>>>>>>>>>>>    Leader, had a conversation a few weeks back.  To summarize their
>>>>>>>>>>>>    conversation, Dave acknowledges the currently lack of diversity in his
>>>>>>>>>>>>    project and it is his sincere desire to drive more people to it to help.
>>>>>>>>>>>>    He also acknowledges the issues with Contrast's extreme marketing around
>>>>>>>>>>>>    the project and feels that it is in everyone's best interests for them to
>>>>>>>>>>>>    curb it back.  While he does have an ownership stake in Contrast, he works
>>>>>>>>>>>>    at Aspect and has no control over the marketing messages that they are
>>>>>>>>>>>>    putting out there.  From the Board perspective, there has been no evidence
>>>>>>>>>>>>    of any impropriety on Dave's part and it should be our goal to drive more
>>>>>>>>>>>>    diversity into the project to support Dave.  Dave appears to be sincere in
>>>>>>>>>>>>    his desires to create a tool where OWASP can tell vendors what we expect
>>>>>>>>>>>>    from their tools.  If the main issue is that only members of Aspect are
>>>>>>>>>>>>    working on it, then the best thing that we can do is try to get him some
>>>>>>>>>>>>    outside assistance.  We are also asking that the project be opened up to
>>>>>>>>>>>>    commits via Git so that outsiders can push commits to it.
>>>>>>>>>>>>    - Josh Sokol and Jeff Williams, the CTO of Contrast, had a
>>>>>>>>>>>>    conversation a few weeks back.  To summarize their conversation, Jeff
>>>>>>>>>>>>    believes that the work that Dave is doing on the Benchmark is a game
>>>>>>>>>>>>    changer in that it gives OWASP the power in dictating what these tools need
>>>>>>>>>>>>    to be finding.  He wants the Benchmark to be successful and understands
>>>>>>>>>>>>    that it needs to be diverse in order to be trusted.  He recognizes that
>>>>>>>>>>>>    Dave is trying to do that and does not want the marketing message from
>>>>>>>>>>>>    Contrast to interfere with his efforts.  Jeff felt that the "Lab" status
>>>>>>>>>>>>    granted to Benchmark meant that it was ready for mainstream adoption, that
>>>>>>>>>>>>    it had 21k tests, and was almost a year old, and didn't see anything wrong
>>>>>>>>>>>>    with marketing their results, but has agreed to talk to their marketing
>>>>>>>>>>>>    team to get them to lay off that message for now.  From the Board
>>>>>>>>>>>>    perspective, we have come to the realization that our brand usage
>>>>>>>>>>>>    guidelines need an overhaul to clarify what is and is not allowed.  We have
>>>>>>>>>>>>    made a few proposals and have reached out to Mozilla to gain more insight
>>>>>>>>>>>>    on their guidelines and even ask for assistance.  Noreen and Paul are
>>>>>>>>>>>>    taking lead on these efforts.
>>>>>>>>>>>>    - There is a note in the notes that the Board was supposed
>>>>>>>>>>>>    to follow up with an open letter to the community and companies involved
>>>>>>>>>>>>    describing our review and actions.  I don't think that has happened so I
>>>>>>>>>>>>    will remind the person who took on that action item.
>>>>>>>>>>>>
>>>>>>>>>>>> I'm happy to answer any questions that you may have.
>>>>>>>>>>>>
>>>>>>>>>>>> ~josh
>>>>>>>>>>>>
>>>>>>>>>>>> On Thu, Nov 26, 2015 at 11:55 AM, Tobias <
>>>>>>>>>>>> tobias.gondrom at owasp.org> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> There have been several conversations on that matter and a
>>>>>>>>>>>>> dedicated call. Unfortunately for personal reasons I could not attend the
>>>>>>>>>>>>> last call as it was at 04:00am my local time, but all other board members
>>>>>>>>>>>>> did participate.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Could please one of my fellow board members give an update.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Best, Tobias
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> On 26/11/15 18:04, Timo Goosen wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>> I would also like to know the answer to Simon's question. We
>>>>>>>>>>>>> need to get rid of bad apples in OWASP in my opinion, there are too many
>>>>>>>>>>>>> people just using the OWASP "name" or "brand" to improve their own
>>>>>>>>>>>>> financial situation or career.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Regards.
>>>>>>>>>>>>> Timo
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Thu, Nov 26, 2015 at 1:13 PM, psiinon <psiinon at gmail.com>
>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Paul, and the rest of the board,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Its been over 2 months since I raised this issue.
>>>>>>>>>>>>>> Whats happening?
>>>>>>>>>>>>>> Has the board even discussed it?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Cheers,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Simon
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Tue, Oct 20, 2015 at 10:00 PM, Paul Ritchie <
>>>>>>>>>>>>>> paul.ritchie at owasp.org> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Eoin, Johanna, All:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> In an earlier email, Josh Sokol mentioned that he will be
>>>>>>>>>>>>>>> speaking in the next day or 2 to their CTO, while at LASCON, as a
>>>>>>>>>>>>>>> representative of the OWASP Board.  Following that feedback, the Board has
>>>>>>>>>>>>>>> action to take the next steps.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Just an FYI that all comments are recognized and action is
>>>>>>>>>>>>>>> being taken.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Paul
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Best Regards, Paul Ritchie
>>>>>>>>>>>>>>> OWASP Executive Director
>>>>>>>>>>>>>>> paul.ritchie at owasp.org
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Tue, Oct 20, 2015 at 1:54 PM, johanna curiel curiel <
>>>>>>>>>>>>>>> johanna.curiel at owasp.org> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Time for owasp to do a public statement and put a clear
>>>>>>>>>>>>>>>> story regarding this abusive behavior of Owasp brand
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On Tuesday, October 20, 2015, Eoin Keary <
>>>>>>>>>>>>>>>> eoin.keary at owasp.org> wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Folks,
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> The project should be immediately shelved it's simply bad
>>>>>>>>>>>>>>>>> form.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> This is damaging to OWASP, the industry and exactly what
>>>>>>>>>>>>>>>>> OWASP is not about.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> There is a clear conflict of interest and distinct lack of
>>>>>>>>>>>>>>>>> science behind the claims made by Contrast.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Eoin Keary
>>>>>>>>>>>>>>>>> OWASP Volunteer
>>>>>>>>>>>>>>>>> @eoinkeary
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> On 7 Oct 2015, at 3:53 p.m., johanna curiel curiel <
>>>>>>>>>>>>>>>>> johanna.curiel at owasp.org> wrote:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> At the moment we did the project review, we observed that
>>>>>>>>>>>>>>>>> the project did not have enough testing to be considered in any form as
>>>>>>>>>>>>>>>>> 'ready'  for benchmarking, neither that it had yet the community adoption,
>>>>>>>>>>>>>>>>> however technically speaking as it has been classified by the leaders, the
>>>>>>>>>>>>>>>>> project is at the beta stage.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Indeed , Dave had the push to have the project reviewed
>>>>>>>>>>>>>>>>> but it was never clear that later on the project was going to be
>>>>>>>>>>>>>>>>> advertisied this way. That all happend after the presentation at Appsec.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> I had my concerns regarding how sensitive is the subject
>>>>>>>>>>>>>>>>> of the project ,but I think we should allow project leaders to develop
>>>>>>>>>>>>>>>>> their communication strategy even if this has conflict of interest. It all
>>>>>>>>>>>>>>>>> depends how they behave and how they manage this.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> On Tuesday, October 6, 2015, Michael Coates <
>>>>>>>>>>>>>>>>> michael.coates at owasp.org> wrote:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> It's not really that formal to add to the agenda, just a
>>>>>>>>>>>>>>>>>> wiki that we add in the text.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> I think you can safely assume it will get the appropriate
>>>>>>>>>>>>>>>>>> discussion.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> On Oct 6, 2015, at 7:16 AM, psiinon <psiinon at gmail.com>
>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Really?? Its not on the agenda yet for the next meeting??
>>>>>>>>>>>>>>>>>> How does it get added to the agenda?
>>>>>>>>>>>>>>>>>> And that was a formal request if that makes any
>>>>>>>>>>>>>>>>>> difference :)
>>>>>>>>>>>>>>>>>> I'm all in favour of getting the facts straight before
>>>>>>>>>>>>>>>>>> any actions are taken, hence my request for an 'ethical review' or whatever
>>>>>>>>>>>>>>>>>> it should be called.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Cheers,
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Simon
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> On Tue, Oct 6, 2015 at 3:07 PM, Michael Coates <
>>>>>>>>>>>>>>>>>> michael.coates at owasp.org> wrote:
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> First step is to get all of our information straight so
>>>>>>>>>>>>>>>>>>> we're clear on where things are at.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> This was not on the board agenda last meeting and is
>>>>>>>>>>>>>>>>>>> also not on the next agenda as of yet (of course it could always be added
>>>>>>>>>>>>>>>>>>> if needed).
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> We are aware that people have raised questions though.
>>>>>>>>>>>>>>>>>>> I'm hoping we can get a clear understanding of all the facts and then
>>>>>>>>>>>>>>>>>>> discuss if changes are needed.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> On Oct 6, 2015, at 1:52 AM, psiinon <psiinon at gmail.com>
>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Hey Michael,
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Is the board going to take any action?
>>>>>>>>>>>>>>>>>>> Were there any discussions about this controversy in the
>>>>>>>>>>>>>>>>>>> board meeting at AppSec USA?
>>>>>>>>>>>>>>>>>>> If not will it be on the agenda for the meeting on
>>>>>>>>>>>>>>>>>>> October 14th?
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Cheers,
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Simon
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> On Tue, Oct 6, 2015 at 8:25 AM, Michael Coates <
>>>>>>>>>>>>>>>>>>> michael.coates at owasp.org> wrote:
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Simon
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> I posted the below message earlier today. At this point
>>>>>>>>>>>>>>>>>>>> my goal is to just gain clarity over the current reality and ideally drive
>>>>>>>>>>>>>>>>>>>> to a shared state of success. This message doesn't seem to be reflected in
>>>>>>>>>>>>>>>>>>>> the list yet. It could be because my membership hasn't been approved or
>>>>>>>>>>>>>>>>>>>> because of mail list delays (I miss Google groups). But I think these
>>>>>>>>>>>>>>>>>>>> questions will start the conversation.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> (This was just me asking questions as a curious Owasp
>>>>>>>>>>>>>>>>>>>> member, not any action on behalf of the board)
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Begin forwarded message:
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> *From:* Michael Coates <michael.coates at owasp.org>
>>>>>>>>>>>>>>>>>>>> *Date:* October 5, 2015 at 6:20:23 PM PDT
>>>>>>>>>>>>>>>>>>>> *To:* owasp-benchmark-project at lists.owasp.org
>>>>>>>>>>>>>>>>>>>> *Subject:* *Project Questions*
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> OWASP Benchmark List,
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> I've heard more about this project and am excited about
>>>>>>>>>>>>>>>>>>>> the idea of an independent perspective of tool performance. I'm trying to
>>>>>>>>>>>>>>>>>>>> understand a few things to better respond to questions from those in the
>>>>>>>>>>>>>>>>>>>> security & OWASP community.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> In my mind there are two big areas for consideration in
>>>>>>>>>>>>>>>>>>>> a benchmark process.
>>>>>>>>>>>>>>>>>>>> 1. Are the benchmarks testing the right areas?
>>>>>>>>>>>>>>>>>>>> 2. Is the process for creating the benchmark objective
>>>>>>>>>>>>>>>>>>>> & free from conflicts of interest.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> I think as a group OWASP is the right body to align on
>>>>>>>>>>>>>>>>>>>> #1.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> I'd like to ask for some clarifications on item #2. I
>>>>>>>>>>>>>>>>>>>> think it's important to avoid actual conflict of interest and also the
>>>>>>>>>>>>>>>>>>>> appearance of conflict of interest. The former is obvious why we mustn't
>>>>>>>>>>>>>>>>>>>> have that, the latter is critical so others have faith in the tool, process
>>>>>>>>>>>>>>>>>>>> and outputs of the process when viewing or hearing about the project.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> 1) Can we clarify whether other individuals have
>>>>>>>>>>>>>>>>>>>> submitted meaningful code to the project?
>>>>>>>>>>>>>>>>>>>> Observation:
>>>>>>>>>>>>>>>>>>>> Nearly all the code commits have come from 1 person
>>>>>>>>>>>>>>>>>>>> (project lead).
>>>>>>>>>>>>>>>>>>>> https://github.com/OWASP/Benchmark/graphs/contributors
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> 2) Can we clarify the contributions of others and their
>>>>>>>>>>>>>>>>>>>> represented organizations?
>>>>>>>>>>>>>>>>>>>> Observation:
>>>>>>>>>>>>>>>>>>>> The acknowledgements tab listed two developers (Juan
>>>>>>>>>>>>>>>>>>>> Gama & Nick Sanidas) both who work at the same company as the project lead.
>>>>>>>>>>>>>>>>>>>> It seems other people have submitted some small amounts of material, but
>>>>>>>>>>>>>>>>>>>> overall it seems all development has come from the same company.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> https://www.owasp.org/index.php/Benchmark#tab=Acknowledgements
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> 3) Can we clarify in what ways we've mitigated the
>>>>>>>>>>>>>>>>>>>> potential conflict of interest and also the appearance of a conflict of
>>>>>>>>>>>>>>>>>>>> interest? This seems like the largest blocker for wide spread acceptance of
>>>>>>>>>>>>>>>>>>>> this project and the biggest risk.
>>>>>>>>>>>>>>>>>>>> Observation:
>>>>>>>>>>>>>>>>>>>> The project lead and both of the project developers
>>>>>>>>>>>>>>>>>>>> works for a company with very close ties to one of the companies that is
>>>>>>>>>>>>>>>>>>>> evaluated by this project. Further, it appears the company is performing
>>>>>>>>>>>>>>>>>>>> very well on the project tests.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> 4) If we are going to list tool vendors then I'd
>>>>>>>>>>>>>>>>>>>> recommend listing multiple vendors for each category.
>>>>>>>>>>>>>>>>>>>> Observation:
>>>>>>>>>>>>>>>>>>>> The tools page only lists 1 IAST tool. Since this is
>>>>>>>>>>>>>>>>>>>> the point of the potential conflict of interest it is important to list
>>>>>>>>>>>>>>>>>>>> numerous IAST tools.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> https://www.owasp.org/index.php/Benchmark#tab=Tool_Support_2FResults
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> 5) Diverse body with multiple points of view
>>>>>>>>>>>>>>>>>>>> Observation:
>>>>>>>>>>>>>>>>>>>> There is no indication that multiple stakeholders are
>>>>>>>>>>>>>>>>>>>> present to review and decide on the future of this project. If they exist,
>>>>>>>>>>>>>>>>>>>> a new section should be added to the project page to raise awareness. If
>>>>>>>>>>>>>>>>>>>> they don't exist, we should reevaluate how we are obtaining an independent
>>>>>>>>>>>>>>>>>>>> view of the testing process.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Again, I think the idea of the project is great. From
>>>>>>>>>>>>>>>>>>>> my perspective clarifying these questions will help ensure the project is
>>>>>>>>>>>>>>>>>>>> not only objective, but also perceived as objective from someone reviewing
>>>>>>>>>>>>>>>>>>>> the material. Ultimately this will contribute to the success and growth of
>>>>>>>>>>>>>>>>>>>> the project.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Thanks!
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>> Michael Coates
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> On Oct 2, 2015, at 1:31 AM, psiinon <psiinon at gmail.com>
>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> OK, based on the concerns raised so far I think the
>>>>>>>>>>>>>>>>>>>> board should initiate a review of the OWASP Benchmark project.
>>>>>>>>>>>>>>>>>>>> I'm not raising a formal complaint against it, I'm just
>>>>>>>>>>>>>>>>>>>> requesting a review.
>>>>>>>>>>>>>>>>>>>> And I dont think it needs a 'standard' project review -
>>>>>>>>>>>>>>>>>>>> Johanna has already done a very good job of this.
>>>>>>>>>>>>>>>>>>>> Not sure what sort of review you'd call it, I'll leave
>>>>>>>>>>>>>>>>>>>> the naming to others :)
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> I'm concerned that we have an OWASP project lead by a
>>>>>>>>>>>>>>>>>>>> company who has a clear commercial stake in the results.
>>>>>>>>>>>>>>>>>>>> Bringing more companies on board will help, but I'm
>>>>>>>>>>>>>>>>>>>> still not sure that alone will make it independent enough.
>>>>>>>>>>>>>>>>>>>> Commercial companies can afford to dedicate staff to
>>>>>>>>>>>>>>>>>>>> improving Benchmark so that their products look better.
>>>>>>>>>>>>>>>>>>>> Open source projects just cant do that, so we are at a
>>>>>>>>>>>>>>>>>>>> distinct disadvantage.
>>>>>>>>>>>>>>>>>>>> Should we allow a commercially driven OWASP project
>>>>>>>>>>>>>>>>>>>> who's aim could be seen be to promote commercial software?
>>>>>>>>>>>>>>>>>>>> If so, what sort of checks and balances does it need?
>>>>>>>>>>>>>>>>>>>> Those are the sort of questions I'd like an independent
>>>>>>>>>>>>>>>>>>>> review to look at.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> I do think there are some immediate steps that could be
>>>>>>>>>>>>>>>>>>>> taken:
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>    - I'd like to see the Benchmark project page
>>>>>>>>>>>>>>>>>>>>    clearly state thats its at a very early stage and that the results are
>>>>>>>>>>>>>>>>>>>>    _not_ yet suitable for use in commercial literature.
>>>>>>>>>>>>>>>>>>>>    - I'd also like the main companies developing
>>>>>>>>>>>>>>>>>>>>    Benchmark to be clearly stated on the main page. If and when other
>>>>>>>>>>>>>>>>>>>>    companies get involved then this would actually help the project's claim of
>>>>>>>>>>>>>>>>>>>>    vendor independence.
>>>>>>>>>>>>>>>>>>>>    - And I'd love to see a respected co-leader added
>>>>>>>>>>>>>>>>>>>>    to the project who is not associated with any commercial or open source
>>>>>>>>>>>>>>>>>>>>    security tools:)
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> And we should carry on discussing the project on this
>>>>>>>>>>>>>>>>>>>> list - I think such discussions are very healthy, and I'd love to see this
>>>>>>>>>>>>>>>>>>>> project mature to a state where it can be a trusted, independent and valued
>>>>>>>>>>>>>>>>>>>> resource.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Cheers,
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Simon
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> On Thu, Oct 1, 2015 at 7:59 PM, Tobias <
>>>>>>>>>>>>>>>>>>>> tobias.gondrom at owasp.org> wrote:
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> @Simon:
>>>>>>>>>>>>>>>>>>>>> yes, the leaders list is the place for your
>>>>>>>>>>>>>>>>>>>>> discussions for project and chapter leaders
>>>>>>>>>>>>>>>>>>>>> @Timo: I like your framing of "Don't ask what OWASP
>>>>>>>>>>>>>>>>>>>>> can do for me, ask what I can do for OWASP."
>>>>>>>>>>>>>>>>>>>>> That should and is indeed the spirit of OWASP:-)
>>>>>>>>>>>>>>>>>>>>> Best regards, Tobias
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> On 30/09/15 09:42, Timo Goosen wrote:
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> ...
>>>
>>> [Message clipped]
>>> _______________________________________________
>>> Owasp-board mailing list
>>> Owasp-board at lists.owasp.org
>>> https://lists.owasp.org/mailman/listinfo/owasp-board
>>>
>>>
>>
>>
>> --
>> OWASP ZAP <https://www.owasp.org/index.php/ZAP> Project leader
>>
>> _______________________________________________
>> OWASP-Leaders mailing list
>> OWASP-Leaders at lists.owasp.org
>> https://lists.owasp.org/mailman/listinfo/owasp-leaders
>>
>>
>


-- 
OWASP ZAP <https://www.owasp.org/index.php/ZAP> Project leader
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.owasp.org/pipermail/owasp-board/attachments/20151130/897ef2ce/attachment-0001.html>


More information about the Owasp-board mailing list