[Owasp-board] [Owasp-leaders] OWASP Benchmark project - potential conflict of interest

Eoin Keary eoin.keary at owasp.org
Mon Nov 30 17:49:12 UTC 2015


So it's like...

49% OWASP
10% non OWASP.
40% commercial
1% trace elements.

That kind of thing?
What other "banners" do we have that we can use? Yes a banner is the solution!!!

Just a suggestion....with respect, could the board possibly take a lead on this issue and get off the fence?? That's what you were Voted in to do, popular or not the decision may be...please make one.







Eoin Keary
OWASP Volunteer
@eoinkeary



> On 30 Nov 2015, at 5:32 p.m., Claudia Casanovas <claudia.aviles-casanovas at owasp.org> wrote:
> 
> Hi All,
> 
> I can add the status as In Dispute Banner to the Benchmark Wiki Page immediately.
> 
> Claudia
> 
>> On Mon, Nov 30, 2015 at 9:27 AM, Tobias Glemser <tobias.glemser at owasp.org> wrote:
>> > At the _very_ least it should flag the project as being 'in dispute' (as Kevin
>> > suggested) while a more detailed evaluation is performed.
>> +1
>> 
>> The conflict is clear scrolling through all those E-Mails, Blogs, etc. Until it's sorted out we need a clarification visible to everyone, the Benchmark projects status is heavily discussed within the Community at the Moment.
>> 
>> > -----Ursprüngliche Nachricht-----
>> > Von: owasp-leaders-bounces at lists.owasp.org [mailto:owasp-leaders-
>> > bounces at lists.owasp.org] Im Auftrag von psiinon
>> > Gesendet: Montag, 30. November 2015 18:18
>> > An: Jim Manico
>> > Cc: OWASP Foundation Board List; owasp-leaders at lists.owasp.org
>> > Betreff: Re: [Owasp-leaders] [Owasp-board] OWASP Benchmark project -
>> > potential conflict of interest [ Z1 UNGESICHERT ]
>> >
>> > I'd like to start by saying that I actually _like_ the Benchmark project.
>> > Myself and other ZAP developers have made some contributions to it, and
>> > we have used (and will continue to use) it to make ZAP better.
>> > I think these sort of testing applications are very valuable to all security tools,
>> > and I'd like to thank Dave and his team for the significant amount of effort
>> > involved in developing and open sourcing it.
>> >
>> > But I dont think it should be an OWASP project.
>> > I do not think that a vendor led project can ever objectively evaluate
>> > competing commercial and open source projects.
>> > I do not think that just saying 'pull requests welcomed' makes a project
>> > vendor neutral.
>> > I do not think that a project as mired in controversy as the Benchmark project
>> > can ever recover to become truly independent.
>> >
>> > I am very disappointed in the Boards handling of this affair.
>> >
>> > Ideally I'd like Dave to understand how much damage this project has done
>> > and to withdraw it as an OWASP project, while still maintaining it as a very
>> > valuable vendor led open source resource.
>> >
>> > Failing that I really hope that the Board comes to its senses and ejects the
>> > Benchmark project before even more damage is done.
>> > At the _very_ least it should flag the project as being 'in dispute' (as Kevin
>> > suggested) while a more detailed evaluation is performed.
>> >
>> > However I'm rapidly loosing loosing faith that the Board will do the right thing
>> > and protect OWASP's image in the way that they should have already done.
>> > Members - please make your voices heard before more people and projects
>> > leave OWASP.
>> >
>> > Simon
>> >
>> >
>> > On Sat, Nov 28, 2015 at 5:14 AM, Jim Manico <jim.manico at owasp.org>
>> > wrote:
>> >
>> >
>> >       WAFEC does not "do vendor assessment"; they define a
>> > comprehensive standard built by many vendors and let the community use
>> > that standard to measure tools on their own. Just a FYI, I was involved in the
>> > early version of this project. (Things may have changed since my
>> > involvement, I'm sure Tony has more details here)
>> >
>> >       Johanna's comments on this issue lead me to believe that the
>> > damage done to both OWASP and DHS is even more destructive that I
>> > thought. It saddens me to see this level of abuse just to sell product.
>> >
>> >
>> >       --
>> >       Jim Manico
>> >       Global Board Member
>> >       OWASP Foundation
>> >       https://www.owasp.org <https://www.owasp.org/>
>> >       Join me in Rome for AppSecEU 2016!
>> >
>> >       On Nov 28, 2015, at 2:40 AM, Josh Sokol <josh.sokol at owasp.org>
>> > wrote:
>> >
>> >
>> >
>> >               One of the ideas that Andrew proposed was actually
>> > approaching WAFEC to learn more about how they do vendor assessment in
>> > a neutral way.  It's great to hear that we have a resource here already that
>> > we can leverage.  I wasn't aware of your affiliation.
>> >
>> >               ~josh
>> >
>> >               On Nov 27, 2015 2:47 PM, "Tony Turner"
>> > <tony.turner at owasp.org> wrote:
>> >
>> >
>> >                       I sincerely hope so. That's not the impression I got
>> > from others comments. Personally I haven't used the tool at all, but as I'm
>> > the project lead for another product evaluation project (WAFEC) I'm very
>> > sensitive to the need of collaboration with many different vendors. There
>> > really has to be a very high level (almost paranoid level) transparency with
>> > how vendors are approached, worked with, how requirements for
>> > evaluation are defined, and how metrics are derived.
>> >
>> >                       It appears the project team is attempting to address
>> > these last 2 somewhat but I'd like to see more specifics, and the lack of
>> > information on how they are addressing vendor communication,
>> > participation and transparency seems a bit concerning. Lastly, it is my opinion
>> > that project leadership should not belong to anyone working for or with a
>> > partnership/ownership stake for any vendor being evaluated. I think this is a
>> > flawed model and should transition to a vendor neutral party.
>> >
>> >                       On Nov 27, 2015 3:16 PM, "Josh Sokol"
>> > <josh.sokol at owasp.org> wrote:
>> >
>> >
>> >                               I don't know what qualifies as "significant" in
>> > your mind, but my understanding is that there have been contributions from
>> > other vendors:
>> >
>> >
>> >       https://www.owasp.org/index.php/Benchmark#tab=Acknowledgem
>> > ents
>> >
>> >
>> >                               Still, Dave would like more, but he can't force
>> > them to help.
>> >
>> >
>> >                               ~josh
>> >
>> >
>> >                               On Fri, Nov 27, 2015 at 1:45 PM, Tony Turner
>> > <tony.turner at owasp.org> wrote:
>> >
>> >
>> >                                       While I can appreciate that they
>> > started with Contrast, if there hasn't been significant effort to include other
>> > vendors it's a worthless benchmark. It's easy to state you haven't gotten
>> > support from other vendors and that's fine, but until you do there's really
>> > nothing to release. Why was it ever upgraded? Talking about the results
>> > without an accurate comparative analysis is akin to snake oil.
>> >
>> >                                       On Nov 27, 2015 1:49 PM, "Josh Sokol"
>> > <josh.sokol at owasp.org> wrote:
>> >
>> >
>> >                                               Thank you for the links to
>> > those articles.  The first one discusses the strengths and weaknesses of the
>> > different methods of evaluating for application vulnerabilities.  The section
>> > on the Benchmark seems wholly appropriate to me.  That seems like an
>> > excellent description of what the project is designed to do.  I see some
>> > metrics in there about which tools are more effective on which types of
>> > vulnerabilities, but I don't see him straight up saying "The OWASP Benchmark
>> > proves that Contrast is better".  This seems like statements made based on
>> > some level of testing and research.  Honestly, I don't see any OWASP brand
>> > abuse in that article.  Whether it's in good taste or not at this stage in the
>> > project is certainly debatable, but if you look at the brand usage guidelines
>> > (https://www.owasp.org/index.php/Marketing/Resources#tab=BRAND_GU
>> > IDELINES), I don't see any violations.  We need to govern to policy here which
>> > is why Paul and Noreen are evaluating changes to the guidelines and our
>> > enforcement policies to make abuse more difficult.
>> >
>> >
>> >                                               The second article is a
>> > competing vendor's reaction to the first.  He makes some good points about
>> > the issues with Benchmark, but he also says that he hopes that it will be
>> > improved over time, and Dave has committed to that.  What I don't see is the
>> > vendor saying "...and Veracode has committed resources to help make the
>> > Benchmark more accurate across all tool sets".  The Benchmark page is pretty
>> > clear that it does it's best to provide a benchmark without working exactly
>> > like a real-world application.  Maybe some more disclaimer text about where
>> > the project is at today would be in order to validate some of Chris' concerns,
>> > but I hardly see this as "brand abuse" or a reason to demote the project.
>> >
>> >
>> >                                               Please consider that I have
>> > spoken with both Dave and Jeff on this topic and read much of the
>> > discussions around it before formulating my opinion.  I doubt that you have
>> > done the same so I'm not sure how you can claim that you have researched
>> > the issues and all parties involved when you haven't even spoken with the
>> > two people whom you are accusing of impropriety.  I have no bias here.  I am
>> > simply speaking with the individuals involved, looking at the currently OWASP
>> > policies and guidelines, and helping to determine our next steps.
>> >
>> >
>> >                                               ~josh
>> >
>> >
>> >                                               On Fri, Nov 27, 2015 at 12:22
>> > PM, johanna curiel curiel <johanna.curiel at owasp.org> wrote:
>> >
>> >
>> >                                                       >>While I agree with
>> > you that there has been some brand abuse, it was abuse by Contrast
>> > (specifically their marketing department), and not by "these gentlemen" as
>> > you state.
>> >
>> >                                                       Really? ..'some brand
>> > abuse'..this is more than brand abuse
>> >
>> >
>> >
>> >                                                       Josh , please read also
>> > the article written by Jeff
>> >
>> >       http://www.darkreading.com/vulnerabilities---threats/why-its-
>> > insane-to-trust-static-analysis/a/d-id/1322274?
>> >
>> >
>> >
>> >                                                       And Veracode's
>> > reaction including others in Twitter
>> >
>> >       https://www.veracode.com/blog/2015/09/no-one-technology-
>> > silver-bullet
>> >
>> >
>> >                                                       My strong advice is to
>> > research the issues and all the parties involved before making statements
>> >
>> >
>> >
>> >
>> >                                                       On Fri, Nov 27, 2015 at
>> > 2:07 PM, Josh Sokol <josh.sokol at owasp.org> wrote:
>> >
>> >
>> >                                                               Jim,
>> >
>> >
>> >                                                               A concern was
>> > expressed to the Board and, frankly, I am insulted by you saying that this was
>> > "brushed under the rug".  The Board delegated Matt to talk with Dave and
>> > they had a lengthy conversation on the subject.  The Board delegated me to
>> > talk with Jeff and we had a lengthy conversation on the subject.  If you do
>> > not trust in our abilities to read people, ask the right questions, and provide
>> > honest feedback about our conversations, then that's a bigger issue that we
>> > should take offline.  After our conversations, we took the time to call a
>> > special two-hour session of the Board in order to discuss this subject (and
>> > only this subject).  We spoke about all facets of the issue at hand, about the
>> > challenges and possible solutions, and concluded on some very concrete
>> > next steps.
>> >
>> >                                                               While I agree
>> > with you that there has been some brand abuse, it was abuse by Contrast
>> > (specifically their marketing department), and not by "these gentlemen" as
>> > you state.  Unless you can point to some sort of evidence showing that Jeff
>> > and/or Dave first-hand abused the brand, then I believe that you are
>> > speaking with your heart instead of with your head.  I appreciate your
>> > passion, but I label this as conspiracy theory because without evidence to
>> > support your claims, I cannot accept it as anything other.
>> >
>> >
>> >                                                               ~josh
>> >
>> >
>> >                                                               On Fri, Nov 27,
>> > 2015 at 11:39 AM, Jim Manico <jim.manico at owasp.org> wrote:
>> >
>> >
>> >                                                                       Josh,
>> >
>> >                                                                       I stand
>> > by my comments and perspective, but I'm disheartened that you consider
>> > my presentation of facts (and the concerns of many active members of our
>> > community) as a "conspiracy theory".
>> >
>> >                                                                       In my
>> > experience, these kind of comments border on insults and only cause folks
>> > to harden their opinions.
>> >
>> >                                                                       Once
>> > again I feel these gentlemen got away with a kind of brand abuse that is very
>> > hurtful to the OWASP community but I am at a loss as to how handle or
>> > prevent these kinds of mishaps - especially when board members like
>> > yourself seem willing to - from what I see - brush it under the rug.
>> >
>> >
>> >                                                                       --
>> >                                                                       Jim
>> > Manico
>> >                                                                       Global
>> > Board Member
>> >                                                                       OWASP
>> > Foundation
>> >
>> >       https://www.owasp.org <https://www.owasp.org/>
>> >                                                                       Join me
>> > in Rome for AppSecEU 2016!
>> >
>> >                                                                       On Nov
>> > 27, 2015, at 7:23 PM, Josh Sokol <josh.sokol at owasp.org> wrote:
>> >
>> >
>> >
>> >
>> >       Admittedly, this was my gut reaction at first as well.  I began linking all
>> > of these companies, people, and projects together in my mind (there are
>> > some loose links there) and painted a big conspiracy picture similar to what
>> > Jim and Dinis have stated.  But, after speaking directly with Jeff, and hearing
>> > about the conversation that Dave and Matt had, I've changed my mind.
>> >
>> >
>> >
>> >       I think it begins with the project itself.  If you aren't sold on the idea
>> > of the Benchmark, then you'll never be able to get to the same place.  My
>> > original line of thinking was that it was just a bar for vendors to compare their
>> > tools against eachother, but that's a bit myopic.  We are in an industry where
>> > things evolve very quickly.  As a customer of these tools, I know firsthand
>> > that something that a tool does today may not be the case a week from now.
>> > Likewise, new features are being added daily and I need a point-in-time
>> > metric to be able to gauge continual effectiveness.  Cool, right?  But not a
>> > game changer.  The game changer part comes when you realize that by
>> > developing and evolving the tests that go into the Benchmark, we are
>> > moving the bar higher and higher.  We (OWASP) are effectively setting the
>> > standard by which these tools will be compared.  A tool that receives a lower
>> > score on the Benchmark today knows exactly what they need to work on in
>> > order to pass that test tomorrow and we already have examples of tools that
>> > have made improvements because of their Benchmark score (Ask Simon
>> > about ZAP's experience with the Benchmark).  I don't think that anyone can
>> > argue that the Benchmark project isn't being effective when OWASP's own
>> > tools are being driven forward as a result of using it.
>> >
>> >
>> >
>> >       But, but, but, Dave and Jeff own Aspect and have stock in Contrast
>> > and Jeff is the Contrast CTO and Contrast got good scores so it's a conspiracy
>> > right?  Is there some code that allows Contrast to use the Benchmark?
>> > Absolutely.  Can you really blame Dave for starting his testing on the
>> > effectiveness of the Benchmark with a tool that he owned and is familiar
>> > with?  If I were going to start a similar project, there's no question in my mind
>> > that I would begin my testing with the tools that I have available to me.  That
>> > said, is there code that allows other tools to use the Benchmark?  Absolutely.
>> >
>> >
>> >
>> >       Regarding "Dave has a history of breaching his duty to be vendor
>> > neutral", while I cannot comment on his past actions, I can judge what we've
>> > seen recently.  Matt saw a presentation from Dave on the Benchmark at a
>> > conference in Chicago.  He said that he felt that the message was appropriate
>> > and while IAST tools were mentioned as receiving higher scores, it wasn't a
>> > "Contrast is the best" type of message, more of a generality.  I saw a very
>> > similar (if not the same) talk by Jeff at LASCON 2015 and the message was
>> > exactly the same.  I watched the talk expecting some sort of impropriety, but
>> > found none.  So, perhaps Dave has abused some privilege granted to him in
>> > the past, but what I've seen from him at this point, with respect to the
>> > Benchmark, has been appropriate.
>> >
>> >
>> >
>> >       You have a very good point with respect to the Contrast marketing
>> > message around the Benchmark.  It's been completely absurd, over the top,
>> > and, in my personal opinion, intolerable.  In fact, I experienced the same
>> > thing that you talked about with them at LASCON 2015 where they stood in
>> > front of the door of the room Jeff was speaking in and scanned attendees as
>> > they went into the talk.  I agree that these types of aggressive marketing
>> > tactics cannot be tolerated at OWASP.  In addition, we have seen several
>> > marketing messages from them effectively implying that OWASP endorses
>> > Contrast.  Clearly this is not OK.  I've spoken with Jeff about it and we agreed
>> > that it is not in the Benchmark's best interest to have this aggressive Contrast
>> > marketing around it at such an early stage.  He has said that he is not
>> > responsible for Contrast's marketing team, but that he would speak with the
>> > people who are.  I haven't seen a single message from them since so I'm
>> > guessing that he's made good on this promise.  While that's an excellent
>> > start, OWASP's takeaway here should be that we need to do a better job
>> > with our brand usage guidelines both in terms of the wording and
>> > enforcement.  There are many other companies out there that use the
>> > OWASP brand and I think that we agree that selective enforcement against
>> > Contrast is not the right answer.  Paul and Noreen are actively working on
>> > this.  Either way, I think that implying that activities from a vendor's
>> > marketing department means that the project is not objective is not
>> > inappropriate.  If we feel that the project is not objective, then separate
>> > measures need to be taken to drive contribution diversity into it.  That I
>> > absolutely agree with and the message from Dave was that he would love to
>> > have more contributors to his project.  But, seeing as we cannot force people
>> > to work on it, this becomes a matter of "put up or shut up".  The same goes
>> > for the experts that you said reviewed the code.  If they feel that it is
>> > somehow skewed towards Contrast, they have the power to change that.
>> > Now, if someone tries to participate and Dave tells them "No thanks", then I
>> > agree we have a problem, but I don't hear anyone inferring that happened.
>> >
>> >
>> >
>> >       Please, let's drop the conspiracy theories and focus on the tangible
>> > things that we can do to help an OWASP project to be more successful.  Help
>> > find more participants to drive diversity, update our brand usage guidelines
>> > to prevent abuse, enforce them widely, etc.  Thank you.
>> >
>> >
>> >
>> >       ~josh
>> >
>> >
>> >
>> >       On Thu, Nov 26, 2015 at 4:24 PM, Jim Manico
>> > <jim.manico at owasp.org> wrote:
>> >
>> >
>> >
>> >       Dinis,
>> >
>> >
>> >       Like a rare celestial moment when all the planets plus Pluto are
>> > aligned, I just read your email on the future of OWASP projects thinking,
>> > "Dinis is spot on".
>> >
>> >
>> >       Reflecting on projects I manage or work on...
>> >
>> >
>> >       The Java Encoder and HTML Sanitizer are likely best moved to Apache
>> > now that they have reached a measure of adoption and maturity. Apache
>> > would be a much better long term custodian. Perhaps the same for
>> > AppSensor, but not my project - just thinking out loud.
>> >
>> >
>> >       Other similar defensive projects are still being noodled on, so OWASP
>> > is a decent home for these research efforts.
>> >
>> >
>> >       The whole tools category is also something to consider. Dependency
>> > Check and of course ZAP are some of the best projects that OWASP offers,
>> > are they best served where they are today? Both have rich communities of
>> > developers but I don't see the foundation doing much to support these
>> > efforts.
>> >
>> >
>> >       ASVS has the opportunity to effect massive change, I would to love
>> > to see major investment and volunteer activity here. Pro tech writer,
>> > detailed discourses on each individual requirement, etc. If I was king (and I
>> > am not, at all) I would invest in ASVS on a 6 figure scale. (And who started
>> > ASVS? Jeff, Dave and Boberski, hat tip to such a marvelous idea). Or maybe
>> > moving ASVS to the W3C or IETF would help it grow?
>> >
>> >
>> >       The Proactive Controls was a pet project but as we approach 2.0 we
>> > have several active/awesome volunteers working on it. We will be making
>> > the doc "world editable" to make contributions easy. OWASP seems like a
>> > good home for such an awareness doc. Same with T10, especially if
>> > community edits are welcome.
>> >
>> >
>> >       Anyhow, I'm with you on this Dinis. Once a project starts to reach
>> > production quality, spinning off the project as an external project or moving
>> > it to a different foundation where managing production software or formal
>> > standards is their thing seems realistic.
>> >
>> >
>> >       I don't have all the answers here, but your email certainly resonated
>> > with me.
>> >
>> >
>> >       Aloha,
>> >
>> >       --
>> >
>> >       Jim Manico
>> >
>> >       Global Board Member
>> >
>> >       OWASP Foundation
>> >
>> >       https://www.owasp.org <https://www.owasp.org/>
>> >
>> >       Join me in Rome for AppSecEU 2016!
>> >
>> >
>> >       On Nov 26, 2015, at 11:26 PM, Dinis Cruz <dinis.cruz at owasp.org>
>> > wrote:
>> >
>> >
>> >
>> >
>> >
>> >
>> >               Jim's reading of this situation is exactly my view on the value
>> > of the Contrast tool and how it has been 'pushing' the rules of engagement
>> > to an very 'fuzzy' moral/ethical/commercial limit :)
>> >
>> >
>> >               As per my last email, a key problem here is the 'perceived
>> > expectation' of what is an OWASP project, and how it should be consumed.
>> >
>> >
>> >               If you look at the OWASP benchmark as a research project,
>> > then the only way it could be making the kind of claims it makes (and have
>> > credibility) is if it had evolved from OWASP, with its own (diverse) community
>> >
>> >
>> >               On 26 November 2015 at 21:01, Jim Manico
>> > <jim.manico at owasp.org> wrote:
>> >
>> >
>> >
>> >
>> >                       I have a different take on this situation but my
>> > opinion is the "minority opinion". I will respect the rest of the boards take on
>> > this, but here is how I see it.
>> >
>> >
>> >                       First of all, Jeff has stated that he feels I am attacking
>> > him personally from a past personal grudge, and frankly I do not fault him for
>> > that perspective since we definitely have history with conflict. So it's fair to
>> > take my opinion on this with a grain of salt.
>> >
>> >
>> >                       I look at this situation from the perspective of a
>> > forensic investigator.
>> >
>> >
>> >                       1) The Benchmark project had Contrast hooks and
>> > only Contrast hooks in it when I reviewed it so this leads me to believe that
>> > the project was clearly built with Contrast in mind from the ground up, at
>> > least in some way.
>> >
>> >                       3) Dave has a history of breaching his duty to be
>> > vendor neutral. He was gifted with a keynote in South Korea a few years ago,
>> > and used that opportunity to discuss and pitch Contrast, on stage, during a
>> > keynote - with Contrast specific slides. This is just supporting evidence of his
>> > intention at OWASP to push Contrast in ways that I think are against the
>> > intentions and goals of our foundation.
>> >
>> >                       3) Other experts have reviewed the project and felt
>> > that many of the tests were very slanted and almost contrived to support
>> > Contrast. I can drag those folks into this conversation, but I do not think that
>> > would help in any way. So it's fair to call this point heresy.
>> >
>> >                       4) I do not see this project as revolutionary, at all.
>> > Every vendor has their own test suite tuned for their tool. As the benchmark
>> > stands today, I see it as just another vendors product-specific benchmark.
>> > Mass collaboration from many vendors is not just a "nice to have" but a base
>> > requirement to get even close to useful for objective tool measurement.
>> >
>> >                       5) Jeff stating that his Marketing people went over
>> > the line is also an admission that - well, they went over the line. By the same
>> > token Jeff was in his booth at AppSec USA surrounded by benchmark
>> > marketing material, discussing this to prospects and he even asked me and
>> > Mr Coates to wade into this debate and support Dave. So to say he was not
>> > involved and it was only his marketing people seems a stretch at best.
>> >
>> >                       6) The Contrast marketing team was wandering
>> > around the conference zapping folks to get leads, and I asked them to stay in
>> > their booth, which is standard conference policy. These folks know better
>> > but are again going over the line to sell product at OWASP. There is a better
>> > way (like focusing on product capability and language support, have
>> > consistent + stellar customer service, have a humble and gracious attitude to
>> > all prospects and customers, actively participate in OWASP in a vendor
>> > neutral and community supportive way, etc).
>> >
>> >
>> >                       Please note, I think Contrast is a decent tool, I've
>> > offered to resell in the past, and I have recommended it in certain situations -
>> > even after this situation arose. I'm stating this out of honestly and desire to
>> > put my cards on the table. I truly want Jeff and Dave to be successful. They
>> > have dedicated their lives to AppSec and if anyone should win big-time, I
>> > hope it's them. I even told Jeff I hope he hits the mother load and donates a
>> > little back to OWASP.
>> >
>> >
>> >                       However, my instinct and evidence tell me that they
>> > both went over the line in the use of the OWASP brand to sell product.
>> >
>> >
>> >                       Now, Jeff makes a good point. We as a board and
>> > staff are very poor at enforcing brand management policy and it's not fair to
>> > single out Contrast, when many other vendors violate the brand, IMO. Just
>> > google OWASP and watch the ads fly that use the OWASP name to sell
>> > product.
>> >
>> >
>> >                       Also, any and every request that was made of Dave to
>> > adjust the project for the sake of vendor neutrality was taken very seriously.
>> > Regardless of Daves past intentions, he is clearly trying to do the right thing
>> > moving forward.
>> >
>> >
>> >                       I look to "postels principle" in this situation (this is
>> > otherwise known as the "robustness principle" and dates back to the
>> > creation of TCP) . This is paraphrased as, "Be liberal in what you take from
>> > others but be conservative in what you dish out". So I think it's critical that
>> > OWASP and any OWASP resource present itself in a strict vendor neutral
>> > way. But unless OWASP wants to be much more "even" in the enforcement
>> > of brand policy across the board to all violators, we should be fairly lax in the
>> > enforcement of these issues from the outside world.
>> >
>> >
>> >                       I am trying to be objective here. My trigonometry
>> > teacher once told me "I'd fail my mother" when I asked him if he would ever
>> > fail me (I was an A student). If my mother owned a security company and
>> > tried the same stunt, I'd have the same opinions about her actions as well.
>> >
>> >
>> >                       So what next? Well hello from the other side. I'm
>> > going back to listening to Adele's new album where I can sit in my deep
>> > feelings and reflect upon what the OWASP foundation has done to enrich my
>> > life. I would much rather keep out of this (and any other conflict laden
>> > situation at OWASP), but I feel it's my responsibility to speak up.
>> >
>> >
>> >                       Aloha,
>> >
>> >
>> >
>> >                       --
>> >
>> >                       Jim Manico
>> >
>> >                       Global Board Member
>> >
>> >                       OWASP Foundation
>> >
>> >                       https://www.owasp.org <https://www.owasp.org/>
>> >
>> >                       Join me in Rome for AppSecEU 2016!
>> >
>> >
>> >                       On Nov 26, 2015, at 9:09 PM, Josh Sokol
>> > <josh.sokol at owasp.org> wrote:
>> >
>> >
>> >
>> >
>> >
>> >
>> >                               I would be happy to provide an update.
>> >
>> >
>> >
>> >
>> >                               *       Matt Konda and Dave Wichers, the
>> > Benchmark Project Leader, had a conversation a few weeks back.  To
>> > summarize their conversation, Dave acknowledges the currently lack of
>> > diversity in his project and it is his sincere desire to drive more people to it to
>> > help.  He also acknowledges the issues with Contrast's extreme marketing
>> > around the project and feels that it is in everyone's best interests for them to
>> > curb it back.  While he does have an ownership stake in Contrast, he works at
>> > Aspect and has no control over the marketing messages that they are putting
>> > out there.  From the Board perspective, there has been no evidence of any
>> > impropriety on Dave's part and it should be our goal to drive more diversity
>> > into the project to support Dave.  Dave appears to be sincere in his desires to
>> > create a tool where OWASP can tell vendors what we expect from their
>> > tools.  If the main issue is that only members of Aspect are working on it,
>> > then the best thing that we can do is try to get him some outside assistance.
>> > We are also asking that the project be opened up to commits via Git so that
>> > outsiders can push commits to it.
>> >
>> >
>> >
>> >                               *       Josh Sokol and Jeff Williams, the CTO
>> > of Contrast, had a conversation a few weeks back.  To summarize their
>> > conversation, Jeff believes that the work that Dave is doing on the
>> > Benchmark is a game changer in that it gives OWASP the power in dictating
>> > what these tools need to be finding.  He wants the Benchmark to be
>> > successful and understands that it needs to be diverse in order to be trusted.
>> > He recognizes that Dave is trying to do that and does not want the marketing
>> > message from Contrast to interfere with his efforts.  Jeff felt that the "Lab"
>> > status granted to Benchmark meant that it was ready for mainstream
>> > adoption, that it had 21k tests, and was almost a year old, and didn't see
>> > anything wrong with marketing their results, but has agreed to talk to their
>> > marketing team to get them to lay off that message for now.  From the Board
>> > perspective, we have come to the realization that our brand usage guidelines
>> > need an overhaul to clarify what is and is not allowed.  We have made a few
>> > proposals and have reached out to Mozilla to gain more insight on their
>> > guidelines and even ask for assistance.  Noreen and Paul are taking lead on
>> > these efforts.
>> >
>> >                               *       There is a note in the notes that the
>> > Board was supposed to follow up with an open letter to the community and
>> > companies involved describing our review and actions.  I don't think that has
>> > happened so I will remind the person who took on that action item.
>> >
>> >
>> >                               I'm happy to answer any questions that you
>> > may have.
>> >
>> >
>> >
>> >
>> >
>> >                               ~josh
>> >
>> >
>> >
>> >
>> >                               On Thu, Nov 26, 2015 at 11:55 AM, Tobias
>> > <tobias.gondrom at owasp.org> wrote:
>> >
>> >
>> >
>> >
>> >                                       There have been several
>> > conversations on that matter and a dedicated call. Unfortunately for personal
>> > reasons I could not attend the last call as it was at 04:00am my local time, but
>> > all other board members did participate.
>> >
>> >
>> >
>> >                                       Could please one of my fellow board
>> > members give an update.
>> >
>> >
>> >
>> >                                       Best, Tobias
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >                                       On 26/11/15 18:04, Timo Goosen
>> > wrote:
>> >
>> >
>> >
>> >
>> >                                               I would also like to know the
>> > answer to Simon's question. We need to get rid of bad apples in OWASP in
>> > my opinion, there are too many people just using the OWASP "name" or
>> > "brand" to improve their own financial situation or career.
>> >
>> >
>> >                                               Regards.
>> >
>> >                                               Timo
>> >
>> >
>> >                                               On Thu, Nov 26, 2015 at 1:13
>> > PM, psiinon <psiinon at gmail.com> wrote:
>> >
>> >
>> >
>> >
>> >                                                       Paul, and the rest of
>> > the board,
>> >
>> >
>> >
>> >
>> >
>> >                                                       Its been over 2 months
>> > since I raised this issue.
>> >
>> >
>> >
>> >                                                       Whats happening?
>> >
>> >
>> >
>> >                                                       Has the board even
>> > discussed it?
>> >
>> >
>> >
>> >
>> >
>> >                                                       Cheers,
>> >
>> >
>> >
>> >
>> >
>> >                                                       Simon
>> >
>> >
>> >
>> >
>> >
>> >                                                       On Tue, Oct 20, 2015 at
>> > 10:00 PM, Paul Ritchie <paul.ritchie at owasp.org> wrote:
>> >
>> >
>> >
>> >
>> >                                                               Eoin, Johanna,
>> > All:
>> >
>> >
>> >                                                               In an earlier
>> > email, Josh Sokol mentioned that he will be speaking in the next day or 2 to
>> > their CTO, while at LASCON, as a representative of the OWASP Board.
>> > Following that feedback, the Board has action to take the next steps.
>> >
>> >
>> >                                                               Just an FYI that
>> > all comments are recognized and action is being taken.
>> >
>> >
>> >                                                               Paul
>> >
>> >
>> >
>> >
>> >                                                               Best Regards,
>> > Paul Ritchie
>> >
>> >                                                               OWASP
>> > Executive Director
>> >
>> >
>> >       paul.ritchie at owasp.org
>> >
>> >
>> >
>> >                                                               On Tue, Oct 20,
>> > 2015 at 1:54 PM, johanna curiel curiel <johanna.curiel at owasp.org> wrote:
>> >
>> >
>> >
>> >
>> >                                                                       Time
>> > for owasp to do a public statement and put a clear story regarding this
>> > abusive behavior of Owasp brand
>> >
>> >
>> >
>> >                                                                       On
>> > Tuesday, October 20, 2015, Eoin Keary <eoin.keary at owasp.org> wrote:
>> >
>> >
>> >
>> >
>> >
>> >       Folks,
>> >
>> >
>> >
>> >       The project should be immediately shelved it's simply bad form.
>> >
>> >
>> >
>> >       This is damaging to OWASP, the industry and exactly what OWASP is
>> > not about.
>> >
>> >
>> >
>> >       There is a clear conflict of interest and distinct lack of science behind
>> > the claims made by Contrast.
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >       Eoin Keary
>> >
>> >
>> >       OWASP Volunteer
>> >
>> >
>> >       @eoinkeary
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >       On 7 Oct 2015, at 3:53 p.m., johanna curiel curiel
>> > <johanna.curiel at owasp.org> wrote:
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >       At the moment we did the project review, we observed that the
>> > project did not have enough testing to be considered in any form as 'ready'
>> > for benchmarking, neither that it had yet the community adoption, however
>> > technically speaking as it has been classified by the leaders, the project is at
>> > the beta stage.
>> >
>> >
>> >
>> >       Indeed , Dave had the push to have the project reviewed but it was
>> > never clear that later on the project was going to be advertisied this way.
>> > That all happend after the presentation at Appsec.
>> >
>> >
>> >
>> >       I had my concerns regarding how sensitive is the subject of the
>> > project ,but I think we should allow project leaders to develop their
>> > communication strategy even if this has conflict of interest. It all depends
>> > how they behave and how they manage this.
>> >
>> >
>> >
>> >
>> >       On Tuesday, October 6, 2015, Michael Coates
>> > <michael.coates at owasp.org> wrote:
>> >
>> >
>> >
>> >
>> >
>> >
>> >               It's not really that formal to add to the agenda, just a wiki that
>> > we add in the text.
>> >
>> >
>> >
>> >               I think you can safely assume it will get the appropriate
>> > discussion.
>> >
>> >
>> >
>> >               On Oct 6, 2015, at 7:16 AM, psiinon <psiinon at gmail.com>
>> > wrote:
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >                       Really?? Its not on the agenda yet for the next
>> > meeting??
>> >
>> >
>> >                       How does it get added to the agenda?
>> >
>> >
>> >
>> >
>> >
>> >                       And that was a formal request if that makes any
>> > difference :)
>> >
>> >
>> >
>> >
>> >
>> >                       I'm all in favour of getting the facts straight before any
>> > actions are taken, hence my request for an 'ethical review' or whatever it
>> > should be called.
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >                       Cheers,
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >                       Simon
>> >
>> >
>> >
>> >
>> >
>> >
>> >                       On Tue, Oct 6, 2015 at 3:07 PM, Michael Coates
>> > <michael.coates at owasp.org> wrote:
>> >
>> >
>> >
>> >
>> >
>> >
>> >                               First step is to get all of our information
>> > straight so we're clear on where things are at.
>> >
>> >
>> >
>> >                               This was not on the board agenda last
>> > meeting and is also not on the next agenda as of yet (of course it could
>> > always be added if needed).
>> >
>> >
>> >
>> >                               We are aware that people have raised
>> > questions though.   I'm hoping we can get a clear understanding of all the
>> > facts and then discuss if changes are needed.
>> >
>> >
>> >
>> >
>> >
>> >                               On Oct 6, 2015, at 1:52 AM, psiinon
>> > <psiinon at gmail.com> wrote:
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >                                       Hey Michael,
>> >
>> >
>> >
>> >
>> >
>> >                                       Is the board going to take any action?
>> >
>> >
>> >
>> >
>> >
>> >                                       Were there any discussions about this
>> > controversy in the board meeting at AppSec USA?
>> >
>> >
>> >
>> >
>> >
>> >                                       If not will it be on the agenda for the
>> > meeting on October 14th?
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >                                       Cheers,
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >                                       Simon
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >                                       On Tue, Oct 6, 2015 at 8:25 AM,
>> > Michael Coates <michael.coates at owasp.org> wrote:
>> >
>> >
>> >
>> >
>> >
>> >
>> >                                               Simon
>> >
>> >
>> >
>> >                                               I posted the below message
>> > earlier today. At this point my goal is to just gain clarity over the current
>> > reality and ideally drive to a shared state of success. This message doesn't
>> > seem to be reflected in the list yet. It could be because my membership
>> > hasn't been approved or because of mail list delays (I miss Google groups).
>> > But I think these questions will start the conversation.
>> >
>> >
>> >
>> >                                               (This was just me asking
>> > questions as a curious Owasp member, not any action on behalf of the
>> > board)
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >                                               Begin forwarded message:
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >                                                       From: Michael Coates
>> > <michael.coates at owasp.org>
>> >
>> >
>> >                                                       Date: October 5, 2015
>> > at 6:20:23 PM PDT
>> >
>> >
>> >                                                       To: owasp-benchmark-
>> > project at lists.owasp.org
>> >
>> >
>> >                                                       Subject: Project
>> > Questions
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >                                                       OWASP Benchmark
>> > List,
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >                                                       I've heard more about
>> > this project and am excited about the idea of an independent perspective of
>> > tool performance. I'm trying to understand a few things to better respond to
>> > questions from those in the security & OWASP community.
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >                                                       In my mind there are
>> > two big areas for consideration in a benchmark process.
>> >
>> >
>> >                                                       1. Are the benchmarks
>> > testing the right areas?
>> >
>> >
>> >                                                       2. Is the process for
>> > creating the benchmark objective & free from conflicts of interest.
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >                                                       I think as a group
>> > OWASP is the right body to align on #1.
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >                                                       I'd like to ask for some
>> > clarifications on item #2. I think it's important to avoid actual conflict of
>> > interest and also the appearance of conflict of interest. The former is obvious
>> > why we mustn't have that, the latter is critical so others have faith in the tool,
>> > process and outputs of the process when viewing or hearing about the
>> > project.
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >                                                       1) Can we clarify
>> > whether other individuals have submitted meaningful code to the project?
>> >
>> >
>> >                                                       Observation:
>> >
>> >
>> >                                                       Nearly all the code
>> > commits have come from 1 person (project lead).
>> >
>> >
>> >
>> >       https://github.com/OWASP/Benchmark/graphs/contributors
>> > <https://github.com/OWASP/Benchmark/graphs/contributors>
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >                                                       2) Can we clarify the
>> > contributions of others and their represented organizations?
>> >
>> >
>> >                                                       Observation:
>> >
>> >
>> >                                                       The
>> > acknowledgements tab listed two developers (Juan Gama & Nick Sanidas)
>> > both who work at the same company as the project lead. It seems other
>> > people have submitted some small amounts of material, but overall it seems
>> > all development has come from the same company.
>> >
>> >
>> >
>> >       https://www.owasp.org/index.php/Benchmark#tab=Acknowledgem
>> > ents
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >                                                       3) Can we clarify in
>> > what ways we've mitigated the potential conflict of interest and also the
>> > appearance of a conflict of interest? This seems like the largest blocker for
>> > wide spread acceptance of this project and the biggest risk.
>> >
>> >
>> >                                                       Observation:
>> >
>> >
>> >                                                       The project lead and
>> > both of the project developers works for a company with very close ties to
>> > one of the companies that is evaluated by this project. Further, it appears
>> > the company is performing very well on the project tests.
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >                                                       4) If we are going to list
>> > tool vendors then I'd recommend listing multiple vendors for each category.
>> >
>> >
>> >                                                       Observation:
>> >
>> >
>> >                                                       The tools page only
>> > lists 1 IAST tool. Since this is the point of the potential conflict of interest it is
>> > important to list numerous IAST tools.
>> >
>> >
>> >
>> >       https://www.owasp.org/index.php/Benchmark#tab=Tool_Support_
>> > 2FResults
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >                                                       5) Diverse body with
>> > multiple points of view
>> >
>> >
>> >                                                       Observation:
>> >
>> >
>> >                                                       There is no indication
>> > that multiple stakeholders are present to review and decide on the future of
>> > this project. If they exist, a new section should be added to the project page
>> > to raise awareness. If they don't exist, we should reevaluate how we are
>> > obtaining an independent view of the testing process.
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >                                                       Again, I think the idea
>> > of the project is great. From my perspective clarifying these questions will
>> > help ensure the project is not only objective, but also perceived as objective
>> > from someone reviewing the material. Ultimately this will contribute to the
>> > success and growth of the project.
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >                                                       Thanks!
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >                                                       --
>> >
>> >
>> >                                                       Michael Coates
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >                                               On Oct 2, 2015, at 1:31 AM,
>> > psiinon <psiinon at gmail.com> wrote:
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >                                                       OK, based on the
>> > concerns raised so far I think the board should initiate a review of the OWASP
>> > Benchmark project.
>> >
>> >
>> >
>> >
>> >
>> >                                                       I'm not raising a formal
>> > complaint against it, I'm just requesting a review.
>> >
>> >
>> >
>> >
>> >
>> >                                                       And I dont think it
>> > needs a 'standard' project review - Johanna has already done a very good job
>> > of this.
>> >
>> >
>> >
>> >
>> >
>> >                                                       Not sure what sort of
>> > review you'd call it, I'll leave the naming to others :)
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >                                                       I'm concerned that we
>> > have an OWASP project lead by a company who has a clear commercial stake
>> > in the results.
>> >
>> >
>> >
>> >
>> >
>> >                                                       Bringing more
>> > companies on board will help, but I'm still not sure that alone will make it
>> > independent enough.
>> >
>> >
>> >
>> >
>> >
>> >                                                       Commercial companies
>> > can afford to dedicate staff to improving Benchmark so that their products
>> > look better.
>> >
>> >
>> >
>> >
>> >
>> >                                                       Open source projects
>> > just cant do that, so we are at a distinct disadvantage.
>> >
>> >
>> >
>> >
>> >
>> >                                                       Should we allow a
>> > commercially driven OWASP project who's aim could be seen be to promote
>> > commercial software?
>> >
>> >
>> >
>> >
>> >
>> >                                                       If so, what sort of
>> > checks and balances does it need?
>> >
>> >
>> >
>> >
>> >
>> >                                                       Those are the sort of
>> > questions I'd like an independent review to look at.
>> >
>> >
>> >
>> >
>> >
>> >
>> >                                                       I do think there are
>> > some immediate steps that could be taken:
>> >
>> >
>> >
>> >
>> >
>> >
>> >                                                       *       I'd like to see
>> > the Benchmark project page clearly state thats its at a very early stage and
>> > that the results are _not_ yet suitable for use in commercial literature.
>> >
>> >
>> >                                                       *       I'd also like the
>> > main companies developing Benchmark to be clearly stated on the main
>> > page. If and when other companies get involved then this would actually
>> > help the project's claim of vendor independence.
>> >
>> >
>> >                                                       *       And I'd love to
>> > see a respected co-leader added to the project who is not associated with
>> > any commercial or open source security tools:)
>> >
>> >
>> >
>> >                                                       And we should carry
>> > on discussing the project on this list - I think such discussions are very
>> > healthy, and I'd love to see this project mature to a state where it can be a
>> > trusted, independent and valued resource.
>> >
>> >
>> >
>> >
>> >
>> >
>> >                                                       Cheers,
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >                                                       Simon
>> >
>> >
>> >
>> >
>> >
>> >
>> >                                                       On Thu, Oct 1, 2015 at
>> > 7:59 PM, Tobias <tobias.gondrom at owasp.org> wrote:
>> >
>> >
>> >
>> >
>> >
>> >
>> >                                                               @Simon:
>> >
>> >
>> >                                                               yes, the
>> > leaders list is the place for your discussions for project and chapter leaders
>> >
>> >
>> >                                                               @Timo: I like
>> > your framing of "Don't ask what OWASP can do for me, ask what I can do for
>> > OWASP."
>> >
>> >
>> >                                                               That should
>> > and is indeed the spirit of OWASP:-)
>> >
>> >
>> >                                                               Best regards,
>> > Tobias
>> >
>> >
>> >
>> >
>> >
>> >
>> >                                                               On 30/09/15
>> > 09:42, Timo Goosen wrote:
>> >
>> >
>> >
>> >
>> >       ...
>> >
>> >       [Message clipped]
>> >       _______________________________________________
>> >       Owasp-board mailing list
>> >       Owasp-board at lists.owasp.org
>> >       https://lists.owasp.org/mailman/listinfo/owasp-board
>> >
>> >
>> >
>> >
>> >
>> >
>> > --
>> >
>> > OWASP ZAP <https://www.owasp.org/index.php/ZAP>  Project leader
>> 
>> 
>> 
>> 
>> _______________________________________________
>> OWASP-Leaders mailing list
>> OWASP-Leaders at lists.owasp.org
>> https://lists.owasp.org/mailman/listinfo/owasp-leaders
> 
> 
> 
> -- 
> 
> 
> Claudia Aviles-Casanovas
> Project Coordinator
> Phone:973-288-1697
> _______________________________________________
> OWASP-Leaders mailing list
> OWASP-Leaders at lists.owasp.org
> https://lists.owasp.org/mailman/listinfo/owasp-leaders
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.owasp.org/pipermail/owasp-board/attachments/20151130/1d969803/attachment-0001.html>


More information about the Owasp-board mailing list