[Owasp-leaders] [Owasp-board] OWASP Benchmark project - potential conflict of interest

psiinon psiinon at gmail.com
Tue Dec 1 09:20:57 UTC 2015


Yes, exactly that.

On Tue, Dec 1, 2015 at 9:16 AM, Eoin Keary <eoin.keary at owasp.org> wrote:

> Hi Jeff,
>
> Being with OWASP for 11 years, being a previous board vice chair and
> global board member for 5 years and having lead many OWASP projects such as
> the testing guide, code review guide etc I'd like to share how this is
> perceived....
>
> My interpretation is the problem is that many folks see the OWASP
> benchmark run by Aspect/Contrast company staff and it happens the Contrast
> is top scoring. The second it was released as a project it was used in
> advertising and awareness campaigns.
>
> Second issue is that is its way to immature and does not have the sample
> data to be scientific and is simply used for commercial gain/product
> promotion.
>
> It's not as if the project has more than 50% of the tool vendors /open
> source tools taking part such that the sample space stands up
> statistically.
>
> There is a perceived independence issue given the project was conceived
> and is lead by a tool vendor.
>
> Many feel the project was conceived for the benefit of Contrast and it has
> lost credibility for that reason.
>
> The idea of the bench mark was/is good but it was rushed and taken
> advantage of without it being mature or proven.
>
> It does not have the data to back up its claims and many OWASP people
> believe it was simply an advertising vehicle regardless of its future use,
> merit and adoption.
>
> I believe this clarifies many peoples point of view.
>
> Best regards,
>
> Eoin Keary
> OWASP Volunteer
> @eoinkeary
>
>
>
> On 1 Dec 2015, at 5:46 a.m., Jeff Williams <jeff.williams at owasp.org>
> wrote:
>
> Hi Leaders,
>
>
>
> I've refrained from commenting on the OWASP Benchmark project because I
> work for Contrast and used to work with Dave.  But before you judge, please
> hear my thoughts on this project, vendors, and OWASP in general.
>
>
>
> As many of you know, I was the global chair of OWASP for over eight years.
> I set up the 501c3, created the chapters program, created the wiki, and led
> many successful projects. And I spent countless hours working with project
> leads, organizations, and vendors to teach them how OWASP works.
>
>
>
> Contrast has heard your concerns and is already changing its marketing
> materials to emphasize that neither OWASP nor DHS endorses commercial
> products.  Both Contrast and I are huge OWASP supporters and we will do
> whatever we can to help achieve the mission. We want to make sure we are
> following the OWASP brand usage guidelines and will work with you to make
> sure we are. I wrote the original rules years ago and if it’s time to
> re-examine them then let’s change them in an open process.
>
>
>
> The OWASP Benchmark is a real opportunity for OWASP to move the needle.
> I've spent considerable time with NSA’s Juliet and other similar test
> suites, and none are even close to the quality and ease of use of the OWASP
> Benchmark. I'd go so far as to say that if you haven't used the Benchmark,
> you really have no idea what the strengths and weaknesses of your tools
> are. The Benchmark isn't perfect. It could cover more vulns, but the ones
> in there are pretty damn important and the project is working on adding
> more vulnerability categories and more real world complexity. And the
> test cases aren't as complex as real code, but testing fundamental tool
> capabilities can tell us a lot.
>
>
>
> The force-multiplier here is that by influencing tools and their vendors,
> OWASP can reach far more developers than it could with any number of
> awesome lists, cheat sheets, or standards. Do you wish tools were better at
> XXE, for example? If the OWASP community puts it in the Benchmark, it
> starts influencing buying behavior and the tools themselves.  The Benchmark
> puts OWASP in the driver's seat and it's already started to work.
>
>
>
> On the leaders list, people have been making all kinds of assertions about
> marketing claims. But without a ruler, everyone is just guessing. This is
> exactly why we need the OWASP Benchmark. Without one, there’s no way to
> establish the truth of either the claims OR the counterclaims.  This
> ruler-free environment encourages vendors to make outrageous claims, like
> “we address the entire OWASP Top Ten” for example.  So, ironically, the
> Benchmark is the path towards more realistic marketing.
>
>
>
> With any benchmark, some vendors aren't going to do well, and they will
> argue the ruler is broken.  Other vendors will do well and will promote
> their results.  As the ruler changes over time, vendor positions will
> change.  All of this is good for the community and for OWASP.  The best
> imaginable outcome would be to touch off an "arms-race" for accuracy in the
> appsec tools space.  The OWASP Benchmark will evolve and get better, the
> tools will get dragged along, and consumers will benefit through increased
> visibility.  This is exactly OWASP's mission, making appsec visible so that
> market forces can work.
>
>
> --Jeff
>
>
>
>
> On Mon, Nov 30, 2015 at 10:37 PM, Dave Wichers <dave.wichers at owasp.org>
> wrote:
>
>> Justin.
>>
>> Thanks so much for your post. For clarification, Justin refers to Coverity
>> numerous times throughout this post, but really means Contrast.
>>
>> -Dave
>>
>> On 11/30/15, 4:55 PM, "Justin Searle" <justin at meeas.com> wrote:
>>
>> >Psiinon, out of curiosity, from a purely project/code perspective,
>> >what do you feel would make the Benchmark tool more "independent"?
>> >From digging around in the source code, it seems they already have a
>> >fair number of report parsers for several OSS and commercial tools in
>> >their tree.  To keep this thread a bit more clean, perhaps post your
>> >response in the Benchmark project mailling list or in a GitHub issue
>> >then reply to this thread with link.  That will allow the Benchmark
>> >team to directly receive that feedback and allow those of use
>> >interested to follow up there to have the needed discussions to
>> >improve the tool.
>> >
>> >Eoin, Dinis, and Jim, it has been a while.  Sorry I've been so removed
>> >from this community the last couple of years.  From reading through
>> >the thread and articles, I don't think it is fair to say the board has
>> >been sitting on the issue.  From Josh's comments, it seems like the
>> >board did take action on this, however I understand if you disagreed
>> >with their decision.  You have always been some of the most vocal
>> >leaders here, and I applaud that.  Your contributions have been great
>> >to OWASP over the years, so please keep it up!
>> >
>> >Most of the concerns I've seen stated are around use of the OWASP
>> >brand in marketing.  I personally do not see merit in many of the
>> >concerns, however my personal greatest concerns are around statements
>> >like this:
>> >
>> >    "OWASP reports that the best static analysis tools score in the
>> >low 30¹s (out of 100) against this benchmark."
>> >
>> >That was in the Dark Reading article, and other statements were made
>> >like that in the Twitter video posted above.  That I feel is the most
>> >egregious of the brand misuse as it implies that OWASP as an
>> >organization has formally made statement.  Keep the full project name
>> >in tact such as "The OWASP Benchmark tool reports..." would be a much
>> >more accurate and less brand abusive way to make that statement, and
>> >based on current OWASP policies, much more inline with what is
>> >permitted.  While OWASP members feel even that is going too far,
>> >please remember that OWASP does not own the vast majority of the OWASP
>> >projects.  Each OWASP project is usually owned by the author and in
>> >many cases, any contributor to that project.  OWASP as an legal entity
>> >owns a very small percentage (none that I can even name off the top of
>> >my head).
>> >
>> >A few years ago when we were initially working on the new project
>> >leaders handbook and project roadmap (before OWASP disbanded all the
>> >global committees in 2013, including the Global Projects Committee),
>> >we discussed chaining the official verbiage to and OWASP "sponsored"
>> >project.  I think it is unfortunate we didn't codify that terminology,
>> >but regardless, I think that is the always a good way to think of
>> >OWASP projects: projects owned and run by individuals of the
>> >community.  However, not everyone in the OWASP community understand
>> >this.  This is easy to see in such statements as "Allowing this
>> >project to exist without ..." and forcing a "project be opened up to
>> >commits via Git so that outsiders can push commits to it" and OWASP
>> >should "decide on the future of this project".  Personally, I think
>> >the most of the drama around OWASP projects comes from this
>> >misunderstanding and OWASP community members trying to manage an OWASP
>> >project that OWASP doesn't own.
>> >
>> >However one of the most difficult issues that perpetuates this problem
>> >and in many cases conflicts concerning brand abuse is project naming.
>> >Since OWASP currently allows projects to use the OWASP name in their
>> >project name (which I think is a mistake), it is hard to refer to a
>> >project without in some way evoking the OWASP brand.  There is very
>> >little legal recourse in most countries to state a fact that "tool X
>> >generated score Y for product Z" in their marketing literature.  That
>> >does not imply that tool X promotes product Z.  And if tool X happens
>> >to be named OWASP Benchmark, then that is not brand infringement in
>> >most countries.  If this is a concern to the OWASP community, then the
>> >better recourse would be to reconsider OWASP's permission to allow
>> >projects to use OWASP in their project name.
>> >
>> >Preventing people from making such statements is usually handled in a
>> >EULA saying how you can and can't use the tool and the tool output,
>> >which in most cases including this would be contrary from the official
>> >OSS definition and most OSS licenses.  So the best distance I think we
>> >could hope to obtain is to disallow the use of OWASP in any project
>> >names.
>> >
>> >And by the way, why would we ever want to stop ANY company out there
>> >from using OWASP tools and documentation?  Why would we ever want ANY
>> >company NOT to advertise that they use OWASP tools and documentation?
>> >Why would we NOT want a company to state they they use OWASP tools in
>> >their marketing literature?  As long as it is clear that OWASP does
>> >not endorse that company, we should encourage the spread and use of
>> >OWASP tools.  Does anyone have a problem with saying website Z has
>> >been tested for all OWASP Top 10 risks in their marketing literature?
>> >What about saying that all vulnerabilities identified by OWASP Zed
>> >Attack Proxy Project have been remediated in website Z literature?
>> >What about all the current DAST/SAST tools that have an "OWASP Top 10"
>> >testing mode?  I don't think any of these imply that a project is
>> >endorsed by OWASP, but if this is a concern for people, would ..."all
>> >vulnerabilities identified by Zed Attack Proxy Project have been
>> >remediated" be better?
>> >
>> >As for a Jeff (or his company) using the benchmark scores from his own
>> >OWASP sponsored project in marketing literature to help customers
>> >understand their commercial offering, I have no qualms with that.  I
>> >don't find that a breach of trust or brand abuse.  I only see brand
>> >abuse in statements mentioned above that stated "Owasp found..." and
>> >such where the tool name was not used, which is explicitly stating a
>> >false OWASP perspective.  Jeff and Coverity in benchmarking their tool
>> >against their own opensource project simply ties the two together in
>> >such a way that can be tested.  Based on statements made by Psiinon
>> >and others, including Coverity's competition, the tool works and does
>> >not seem to be skewed towards Coverity's tool, even though they score
>> >the highest.  The tool is opensource.  If anyone believes the tool
>> >unfairly scores Coverity's tool, or doesn't not provide benefit to
>> >other assessment tools who want to improve their scanning engines,
>> >please dig through the code and identify how it does that.  All I've
>> >seen so far is people disagreeing with how the metric is generated and
>> >the number of tests involved, which in itself doesn't seem to portray
>> >bias for one tool over another.  If the tool is found to favor
>> >Coverity's scanning tool, then that will be shown by someone with time
>> >and interest, and if that is the case, the brand loss will by
>> >Coverity's, not OWASP's.
>> >
>> >As for actions, I agree with the actions the board seems to have made
>> >so far.  I do not think any penalties be levied against the Benchmark
>> >project.  I do not think that they should be downgrade back to
>> >incubator, which seems a petty and meaningless action to me.  The
>> >maturity seems to say Lab quality more than many other existing Lab
>> >projects.  I do not think OWASP has any right or reason to force the
>> >Benchmark project to allow commits from additional persons.  Having a
>> >single person do actual commits to main trunks while other offer pull
>> >requests is common and very standard in OSS, and in now way portrays
>> >how "open" the project community is.  And banning companies from any
>> >mention OWASP projects in marketing efforts, wether project leaders
>> >are associate with said companies or not, would be foolish in our
>> >efforts to growing OWASP brand, as long as such marketing efforts do
>> >not implicitly or explicitly imply OWASP endorsement of a company, its
>> >tools, or its services.
>> >
>> >As for my suggestions to the OWASP board, I'd recommend the following:
>> > - An official statement on the Benchmark project page at owasp.org
>> >stating as Johanna suggested, that OWASP does not endorse any company,
>> >commercial tool, or commercial service
>> > - A request to Coverty to make a similar statement on their website
>> >and future marketing efforts just to clarify this misunderstanding
>> > - A formal cease and desist letter to Coverty to stop making explicit
>> >claims in OWASP's behalf such as "OWASP found ..." and to restrict all
>> >use of the term "OWASP" as part of the "OWASP Benchmark" project's
>> >formal name.
>> >
>> >As for the Benchmark project, I'd recommend the following:
>> > - If the tool doesn't already do so, I'd recommend a simply statement
>> >in the Benchmark reports saying that scores from the tool does not
>> >imply any endorsement for or against any tool tested.
>> > - Also, digging through your sourcecode tree, I noticed there doesn't
>> >seem to be any copyright notices.  I'd recommend you adding those
>> >copyright notices to whoever owns the code, otherwise it is hard to
>> >enforce any copyright license restrictions.
>> >
>> >And finally, as for the OWASP community, I'd encourage you to decide
>> >if it makes sense to remove the ability of projects to formally use
>> >OWASP as part of their project name.  If we don't do this then
>> >individual project brand and OWASP brand becomes commingled, and
>> >ownership becomes less clear.  Project leaders, for your current or
>> >future projects, I'd personally recommend you don't use OWASP in the
>> >title so you can build project brand recognition independent of OWASP,
>> >and instead do something like "Project X, an OWASP project" in your
>> >project marketing.
>> >
>> >Justin Searle
>> >Managing Partner - UtiliSec
>> >+1 801-784-2052
>> >justin at utilisec.com
>> >justin at meeas.com
>> >
>> >
>> >On Mon, Nov 30, 2015 at 10:40 AM, psiinon <psiinon at gmail.com> wrote:
>> >>>
>> >>> In the short term, It is better to remove it from OWASP, leaving the
>> >>>door
>> >>> open for its return (in a future when some of the independence and
>> >>>quality
>> >>> issues have been solved)
>> >>>
>> >>
>> >> I would be delighted to welcome an independent, high quality Benchmark
>> >> project back into OWASP :)
>> >>
>> >>
>> >>
>> >>>
>> >>> Specially when recently we made David Rook remove this much more
>> benign
>> >>> 'commercial content' from OWASP
>> >>>
>> >>> Dinis
>> >>>
>> >>> On 30 November 2015 at 17:17, psiinon <psiinon at gmail.com> wrote:
>> >>>>
>> >>>> I'd like to start by saying that I actually _like_ the Benchmark
>> >>>>project.
>> >>>> Myself and other ZAP developers have made some contributions to it,
>> >>>>and
>> >>>> we have used (and will continue to use) it to make ZAP better.
>> >>>> I think these sort of testing applications are very valuable to all
>> >>>> security tools, and I'd like to thank Dave and his team for the
>> >>>>significant
>> >>>> amount of effort involved in developing and open sourcing it.
>> >>>>
>> >>>> But I dont think it should be an OWASP project.
>> >>>> I do not think that a vendor led project can ever objectively
>> evaluate
>> >>>> competing commercial and open source projects.
>> >>>> I do not think that just saying 'pull requests welcomed' makes a
>> >>>>project
>> >>>> vendor neutral.
>> >>>> I do not think that a project as mired in controversy as the
>> Benchmark
>> >>>> project can ever recover to become truly independent.
>> >>>>
>> >>>> I am very disappointed in the Boards handling of this affair.
>> >>>>
>> >>>> Ideally I'd like Dave to understand how much damage this project has
>> >>>>done
>> >>>> and to withdraw it as an OWASP project, while still maintaining it as
>> >>>>a very
>> >>>> valuable vendor led open source resource.
>> >>>>
>> >>>> Failing that I really hope that the Board comes to its senses and
>> >>>>ejects
>> >>>> the Benchmark project before even more damage is done.
>> >>>> At the _very_ least it should flag the project as being 'in dispute'
>> >>>>(as
>> >>>> Kevin suggested) while a more detailed evaluation is performed.
>> >>>>
>> >>>> However I'm rapidly loosing loosing faith that the Board will do the
>> >>>> right thing and protect OWASP's image in the way that they should
>> have
>> >>>> already done.
>> >>>> Members - please make your voices heard before more people and
>> >>>>projects
>> >>>> leave OWASP.
>> >>>>
>> >>>> Simon
>> >>>>
>> >>>> On Sat, Nov 28, 2015 at 5:14 AM, Jim Manico <jim.manico at owasp.org>
>> >>>>wrote:
>> >>>>>
>> >>>>> WAFEC does not "do vendor assessment"; they define a comprehensive
>> >>>>> standard built by many vendors and let the community use that
>> >>>>>standard to
>> >>>>> measure tools on their own. Just a FYI, I was involved in the early
>> >>>>>version
>> >>>>> of this project. (Things may have changed since my involvement, I'm
>> >>>>>sure
>> >>>>> Tony has more details here)
>> >>>>>
>> >>>>> Johanna's comments on this issue lead me to believe that the damage
>> >>>>>done
>> >>>>> to both OWASP and DHS is even more destructive that I thought. It
>> >>>>>saddens me
>> >>>>> to see this level of abuse just to sell product.
>> >>>>>
>> >>>>> --
>> >>>>> Jim Manico
>> >>>>> Global Board Member
>> >>>>> OWASP Foundation
>> >>>>> https://www.owasp.org
>> >>>>> Join me in Rome for AppSecEU 2016!
>> >>>>>
>> >>>>> On Nov 28, 2015, at 2:40 AM, Josh Sokol <josh.sokol at owasp.org>
>> wrote:
>> >>>>>
>> >>>>> One of the ideas that Andrew proposed was actually approaching WAFEC
>> >>>>>to
>> >>>>> learn more about how they do vendor assessment in a neutral way.
>> >>>>>It's great
>> >>>>> to hear that we have a resource here already that we can leverage.
>> >>>>>I wasn't
>> >>>>> aware of your affiliation.
>> >>>>>
>> >>>>> ~josh
>> >>>>>
>> >>>>> On Nov 27, 2015 2:47 PM, "Tony Turner" <tony.turner at owasp.org>
>> wrote:
>> >>>>>>
>> >>>>>> I sincerely hope so. That's not the impression I got from others
>> >>>>>> comments. Personally I haven't used the tool at all, but as I'm the
>> >>>>>>project
>> >>>>>> lead for another product evaluation project (WAFEC) I'm very
>> >>>>>>sensitive to
>> >>>>>> the need of collaboration with many different vendors. There really
>> >>>>>>has to
>> >>>>>> be a very high level (almost paranoid level) transparency with how
>> >>>>>>vendors
>> >>>>>> are approached, worked with, how requirements for evaluation are
>> >>>>>>defined,
>> >>>>>> and how metrics are derived.
>> >>>>>>
>> >>>>>> It appears the project team is attempting to address these last 2
>> >>>>>> somewhat but I'd like to see more specifics, and the lack of
>> >>>>>>information on
>> >>>>>> how they are addressing vendor communication, participation and
>> >>>>>>transparency
>> >>>>>> seems a bit concerning. Lastly, it is my opinion that project
>> >>>>>>leadership
>> >>>>>> should not belong to anyone working for or with a
>> >>>>>>partnership/ownership
>> >>>>>> stake for any vendor being evaluated. I think this is a flawed
>> >>>>>>model and
>> >>>>>> should transition to a vendor neutral party.
>> >>>>>>
>> >>>>>> On Nov 27, 2015 3:16 PM, "Josh Sokol" <josh.sokol at owasp.org>
>> wrote:
>> >>>>>>>
>> >>>>>>> I don't know what qualifies as "significant" in your mind, but my
>> >>>>>>> understanding is that there have been contributions from other
>> >>>>>>>vendors:
>> >>>>>>>
>> >>>>>>> https://www.owasp.org/index.php/Benchmark#tab=Acknowledgements
>> >>>>>>>
>> >>>>>>> Still, Dave would like more, but he can't force them to help.
>> >>>>>>>
>> >>>>>>> ~josh
>> >>>>>>>
>> >>>>>>> On Fri, Nov 27, 2015 at 1:45 PM, Tony Turner
>> >>>>>>><tony.turner at owasp.org>
>> >>>>>>> wrote:
>> >>>>>>>>
>> >>>>>>>> While I can appreciate that they started with Contrast, if there
>> >>>>>>>> hasn't been significant effort to include other vendors it's a
>> >>>>>>>>worthless
>> >>>>>>>> benchmark. It's easy to state you haven't gotten support from
>> >>>>>>>>other vendors
>> >>>>>>>> and that's fine, but until you do there's really nothing to
>> >>>>>>>>release. Why was
>> >>>>>>>> it ever upgraded? Talking about the results without an accurate
>> >>>>>>>>comparative
>> >>>>>>>> analysis is akin to snake oil.
>> >>>>>>>>
>> >>>>>>>> On Nov 27, 2015 1:49 PM, "Josh Sokol" <josh.sokol at owasp.org>
>> >>>>>>>>wrote:
>> >>>>>>>>>
>> >>>>>>>>> Thank you for the links to those articles.  The first one
>> >>>>>>>>>discusses
>> >>>>>>>>> the strengths and weaknesses of the different methods of
>> >>>>>>>>>evaluating for
>> >>>>>>>>> application vulnerabilities.  The section on the Benchmark seems
>> >>>>>>>>>wholly
>> >>>>>>>>> appropriate to me.  That seems like an excellent description of
>> >>>>>>>>>what the
>> >>>>>>>>> project is designed to do.  I see some metrics in there about
>> >>>>>>>>>which tools
>> >>>>>>>>> are more effective on which types of vulnerabilities, but I
>> >>>>>>>>>don't see him
>> >>>>>>>>> straight up saying "The OWASP Benchmark proves that Contrast is
>> >>>>>>>>>better".
>> >>>>>>>>> This seems like statements made based on some level of testing
>> >>>>>>>>>and research.
>> >>>>>>>>> Honestly, I don't see any OWASP brand abuse in that article.
>> >>>>>>>>>Whether it's
>> >>>>>>>>> in good taste or not at this stage in the project is certainly
>> >>>>>>>>>debatable,
>> >>>>>>>>> but if you look at the brand usage guidelines
>> >>>>>>>>>
>> >>>>>>>>>(
>> https://www.owasp.org/index.php/Marketing/Resources#tab=BRAND_GUI
>> >>>>>>>>>DELINES),
>> >>>>>>>>> I don't see any violations.  We need to govern to policy here
>> >>>>>>>>>which is why
>> >>>>>>>>> Paul and Noreen are evaluating changes to the guidelines and our
>> >>>>>>>>>enforcement
>> >>>>>>>>> policies to make abuse more difficult.
>> >>>>>>>>>
>> >>>>>>>>> The second article is a competing vendor's reaction to the
>> first.
>> >>>>>>>>> He makes some good points about the issues with Benchmark, but
>> >>>>>>>>>he also says
>> >>>>>>>>> that he hopes that it will be improved over time, and Dave has
>> >>>>>>>>>committed to
>> >>>>>>>>> that.  What I don't see is the vendor saying "...and Veracode
>> >>>>>>>>>has committed
>> >>>>>>>>> resources to help make the Benchmark more accurate across all
>> >>>>>>>>>tool sets".
>> >>>>>>>>> The Benchmark page is pretty clear that it does it's best to
>> >>>>>>>>>provide a
>> >>>>>>>>> benchmark without working exactly like a real-world application.
>> >>>>>>>>> Maybe some
>> >>>>>>>>> more disclaimer text about where the project is at today would
>> >>>>>>>>>be in order
>> >>>>>>>>> to validate some of Chris' concerns, but I hardly see this as
>> >>>>>>>>>"brand abuse"
>> >>>>>>>>> or a reason to demote the project.
>> >>>>>>>>>
>> >>>>>>>>> Please consider that I have spoken with both Dave and Jeff on
>> >>>>>>>>>this
>> >>>>>>>>> topic and read much of the discussions around it before
>> >>>>>>>>>formulating my
>> >>>>>>>>> opinion.  I doubt that you have done the same so I'm not sure
>> >>>>>>>>>how you can
>> >>>>>>>>> claim that you have researched the issues and all parties
>> >>>>>>>>>involved when you
>> >>>>>>>>> haven't even spoken with the two people whom you are accusing of
>> >>>>>>>>> impropriety.  I have no bias here.  I am simply speaking with
>> the
>> >>>>>>>>> individuals involved, looking at the currently OWASP policies
>> and
>> >>>>>>>>> guidelines, and helping to determine our next steps.
>> >>>>>>>>>
>> >>>>>>>>> ~josh
>> >>>>>>>>>
>> >>>>>>>>> On Fri, Nov 27, 2015 at 12:22 PM, johanna curiel curiel
>> >>>>>>>>> <johanna.curiel at owasp.org> wrote:
>> >>>>>>>>>>
>> >>>>>>>>>> >>While I agree with you that there has been some brand abuse,
>> >>>>>>>>>>it
>> >>>>>>>>>> >> was abuse by Contrast (specifically their marketing
>> >>>>>>>>>>department), and not by
>> >>>>>>>>>> >> "these gentlemen" as  you state.
>> >>>>>>>>>>
>> >>>>>>>>>> Really? ..'some brand abuse'..this is more than brand abuse
>> >>>>>>>>>>
>> >>>>>>>>>> Josh , please read also the article written by Jeff
>> >>>>>>>>>>
>> >>>>>>>>>>
>> >>>>>>>>>>
>> http://www.darkreading.com/vulnerabilities---threats/why-its-insa
>> >>>>>>>>>>ne-to-trust-static-analysis/a/d-id/1322274?
>> >>>>>>>>>>
>> >>>>>>>>>> And Veracode's reaction including others in Twitter
>> >>>>>>>>>>
>> >>>>>>>>>>
>> >>>>>>>>>>
>> https://www.veracode.com/blog/2015/09/no-one-technology-silver-bu
>> >>>>>>>>>>llet
>> >>>>>>>>>>
>> >>>>>>>>>> My strong advice is to research the issues and all the parties
>> >>>>>>>>>> involved before making statements
>> >>>>>>>>>>
>> >>>>>>>>>>
>> >>>>>>>>>>
>> >>>>>>>>>>
>> >>>>>>>>>> On Fri, Nov 27, 2015 at 2:07 PM, Josh Sokol
>> >>>>>>>>>><josh.sokol at owasp.org>
>> >>>>>>>>>> wrote:
>> >>>>>>>>>>>
>> >>>>>>>>>>> Jim,
>> >>>>>>>>>>>
>> >>>>>>>>>>> A concern was expressed to the Board and, frankly, I am
>> >>>>>>>>>>>insulted
>> >>>>>>>>>>> by you saying that this was "brushed under the rug".  The
>> >>>>>>>>>>>Board delegated
>> >>>>>>>>>>> Matt to talk with Dave and they had a lengthy conversation on
>> >>>>>>>>>>>the subject.
>> >>>>>>>>>>> The Board delegated me to talk with Jeff and we had a lengthy
>> >>>>>>>>>>>conversation
>> >>>>>>>>>>> on the subject.  If you do not trust in our abilities to read
>> >>>>>>>>>>>people, ask
>> >>>>>>>>>>> the right questions, and provide honest feedback about our
>> >>>>>>>>>>>conversations,
>> >>>>>>>>>>> then that's a bigger issue that we should take offline.  After
>> >>>>>>>>>>>our
>> >>>>>>>>>>> conversations, we took the time to call a special two-hour
>> >>>>>>>>>>>session of the
>> >>>>>>>>>>> Board in order to discuss this subject (and only this
>> >>>>>>>>>>>subject).  We spoke
>> >>>>>>>>>>> about all facets of the issue at hand, about the challenges
>> >>>>>>>>>>>and possible
>> >>>>>>>>>>> solutions, and concluded on some very concrete next steps.
>> >>>>>>>>>>>
>> >>>>>>>>>>> While I agree with you that there has been some brand abuse,
>> it
>> >>>>>>>>>>> was abuse by Contrast (specifically their marketing
>> >>>>>>>>>>>department), and not by
>> >>>>>>>>>>> "these gentlemen" as  you state.  Unless you can point to some
>> >>>>>>>>>>>sort of
>> >>>>>>>>>>> evidence showing that Jeff and/or Dave first-hand abused the
>> >>>>>>>>>>>brand, then I
>> >>>>>>>>>>> believe that you are speaking with your heart instead of with
>> >>>>>>>>>>>your head.  I
>> >>>>>>>>>>> appreciate your passion, but I label this as conspiracy theory
>> >>>>>>>>>>>because
>> >>>>>>>>>>> without evidence to support your claims, I cannot accept it as
>> >>>>>>>>>>>anything
>> >>>>>>>>>>> other.
>> >>>>>>>>>>>
>> >>>>>>>>>>> ~josh
>> >>>>>>>>>>>
>> >>>>>>>>>>> On Fri, Nov 27, 2015 at 11:39 AM, Jim Manico
>> >>>>>>>>>>> <jim.manico at owasp.org> wrote:
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> Josh,
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> I stand by my comments and perspective, but I'm disheartened
>> >>>>>>>>>>>>that
>> >>>>>>>>>>>> you consider my presentation of facts (and the concerns of
>> >>>>>>>>>>>>many active
>> >>>>>>>>>>>> members of our community) as a "conspiracy theory".
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> In my experience, these kind of comments border on insults
>> and
>> >>>>>>>>>>>> only cause folks to harden their opinions.
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> Once again I feel these gentlemen got away with a kind of
>> >>>>>>>>>>>>brand
>> >>>>>>>>>>>> abuse that is very hurtful to the OWASP community but I am at
>> >>>>>>>>>>>>a loss as to
>> >>>>>>>>>>>> how handle or prevent these kinds of mishaps - especially
>> >>>>>>>>>>>>when board members
>> >>>>>>>>>>>> like yourself seem willing to - from what I see - brush it
>> >>>>>>>>>>>>under the rug.
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> --
>> >>>>>>>>>>>> Jim Manico
>> >>>>>>>>>>>> Global Board Member
>> >>>>>>>>>>>> OWASP Foundation
>> >>>>>>>>>>>> https://www.owasp.org
>> >>>>>>>>>>>> Join me in Rome for AppSecEU 2016!
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> On Nov 27, 2015, at 7:23 PM, Josh Sokol <
>> josh.sokol at owasp.org>
>> >>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> Admittedly, this was my gut reaction at first as well.  I
>> >>>>>>>>>>>>began
>> >>>>>>>>>>>> linking all of these companies, people, and projects together
>> >>>>>>>>>>>>in my mind
>> >>>>>>>>>>>> (there are some loose links there) and painted a big
>> >>>>>>>>>>>>conspiracy picture
>> >>>>>>>>>>>> similar to what Jim and Dinis have stated.  But, after
>> >>>>>>>>>>>>speaking directly
>> >>>>>>>>>>>> with Jeff, and hearing about the conversation that Dave and
>> >>>>>>>>>>>>Matt had, I've
>> >>>>>>>>>>>> changed my mind.
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> I think it begins with the project itself.  If you aren't
>> >>>>>>>>>>>>sold on
>> >>>>>>>>>>>> the idea of the Benchmark, then you'll never be able to get
>> >>>>>>>>>>>>to the same
>> >>>>>>>>>>>> place.  My original line of thinking was that it was just a
>> >>>>>>>>>>>>bar for vendors
>> >>>>>>>>>>>> to compare their tools against eachother, but that's a bit
>> >>>>>>>>>>>>myopic.  We are
>> >>>>>>>>>>>> in an industry where things evolve very quickly.  As a
>> >>>>>>>>>>>>customer of these
>> >>>>>>>>>>>> tools, I know firsthand that something that a tool does today
>> >>>>>>>>>>>>may not be the
>> >>>>>>>>>>>> case a week from now.  Likewise, new features are being added
>> >>>>>>>>>>>>daily and I
>> >>>>>>>>>>>> need a point-in-time metric to be able to gauge continual
>> >>>>>>>>>>>>effectiveness.
>> >>>>>>>>>>>> Cool, right?  But not a game changer.  The game changer part
>> >>>>>>>>>>>>comes when you
>> >>>>>>>>>>>> realize that by developing and evolving the tests that go
>> >>>>>>>>>>>>into the
>> >>>>>>>>>>>> Benchmark, we are moving the bar higher and higher.  We
>> >>>>>>>>>>>>(OWASP) are
>> >>>>>>>>>>>> effectively setting the standard by which these tools will be
>> >>>>>>>>>>>>compared.  A
>> >>>>>>>>>>>> tool that receives a lower score on the Benchmark today knows
>> >>>>>>>>>>>>exactly what
>> >>>>>>>>>>>> they need to work on in order to pass that test tomorrow and
>> >>>>>>>>>>>>we already have
>> >>>>>>>>>>>> examples of tools that have made improvements because of
>> >>>>>>>>>>>>their Benchmark
>> >>>>>>>>>>>> score (Ask Simon about ZAP's experience with the Benchmark).
>> >>>>>>>>>>>>I don't think
>> >>>>>>>>>>>> that anyone can argue that the Benchmark project isn't being
>> >>>>>>>>>>>>effective when
>> >>>>>>>>>>>> OWASP's own tools are being driven forward as a result of
>> >>>>>>>>>>>>using it.
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> But, but, but, Dave and Jeff own Aspect and have stock in
>> >>>>>>>>>>>> Contrast and Jeff is the Contrast CTO and Contrast got good
>> >>>>>>>>>>>>scores so it's a
>> >>>>>>>>>>>> conspiracy right?  Is there some code that allows Contrast to
>> >>>>>>>>>>>>use the
>> >>>>>>>>>>>> Benchmark?  Absolutely.  Can you really blame Dave for
>> >>>>>>>>>>>>starting his testing
>> >>>>>>>>>>>> on the effectiveness of the Benchmark with a tool that he
>> >>>>>>>>>>>>owned and is
>> >>>>>>>>>>>> familiar with?  If I were going to start a similar project,
>> >>>>>>>>>>>>there's no
>> >>>>>>>>>>>> question in my mind that I would begin my testing with the
>> >>>>>>>>>>>>tools that I have
>> >>>>>>>>>>>> available to me.  That said, is there code that allows other
>> >>>>>>>>>>>>tools to use
>> >>>>>>>>>>>> the Benchmark?  Absolutely.
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> Regarding "Dave has a history of breaching his duty to be
>> >>>>>>>>>>>>vendor
>> >>>>>>>>>>>> neutral", while I cannot comment on his past actions, I can
>> >>>>>>>>>>>>judge what we've
>> >>>>>>>>>>>> seen recently.  Matt saw a presentation from Dave on the
>> >>>>>>>>>>>>Benchmark at a
>> >>>>>>>>>>>> conference in Chicago.  He said that he felt that the message
>> >>>>>>>>>>>>was
>> >>>>>>>>>>>> appropriate and while IAST tools were mentioned as receiving
>> >>>>>>>>>>>>higher scores,
>> >>>>>>>>>>>> it wasn't a "Contrast is the best" type of message, more of a
>> >>>>>>>>>>>>generality.  I
>> >>>>>>>>>>>> saw a very similar (if not the same) talk by Jeff at LASCON
>> >>>>>>>>>>>>2015 and the
>> >>>>>>>>>>>> message was exactly the same.  I watched the talk expecting
>> >>>>>>>>>>>>some sort of
>> >>>>>>>>>>>> impropriety, but found none.  So, perhaps Dave has abused
>> >>>>>>>>>>>>some privilege
>> >>>>>>>>>>>> granted to him in the past, but what I've seen from him at
>> >>>>>>>>>>>>this point, with
>> >>>>>>>>>>>> respect to the Benchmark, has been appropriate.
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> You have a very good point with respect to the Contrast
>> >>>>>>>>>>>>marketing
>> >>>>>>>>>>>> message around the Benchmark.  It's been completely absurd,
>> >>>>>>>>>>>>over the top,
>> >>>>>>>>>>>> and, in my personal opinion, intolerable.  In fact, I
>> >>>>>>>>>>>>experienced the same
>> >>>>>>>>>>>> thing that you talked about with them at LASCON 2015 where
>> >>>>>>>>>>>>they stood in
>> >>>>>>>>>>>> front of the door of the room Jeff was speaking in and
>> >>>>>>>>>>>>scanned attendees as
>> >>>>>>>>>>>> they went into the talk.  I agree that these types of
>> >>>>>>>>>>>>aggressive marketing
>> >>>>>>>>>>>> tactics cannot be tolerated at OWASP.  In addition, we have
>> >>>>>>>>>>>>seen several
>> >>>>>>>>>>>> marketing messages from them effectively implying that OWASP
>> >>>>>>>>>>>>endorses
>> >>>>>>>>>>>> Contrast.  Clearly this is not OK.  I've spoken with Jeff
>> >>>>>>>>>>>>about it and we
>> >>>>>>>>>>>> agreed that it is not in the Benchmark's best interest to
>> >>>>>>>>>>>>have this
>> >>>>>>>>>>>> aggressive Contrast marketing around it at such an early
>> >>>>>>>>>>>>stage.  He has said
>> >>>>>>>>>>>> that he is not responsible for Contrast's marketing team, but
>> >>>>>>>>>>>>that he would
>> >>>>>>>>>>>> speak with the people who are.  I haven't seen a single
>> >>>>>>>>>>>>message from them
>> >>>>>>>>>>>> since so I'm guessing that he's made good on this promise.
>> >>>>>>>>>>>>While that's an
>> >>>>>>>>>>>> excellent start, OWASP's takeaway here should be that we need
>> >>>>>>>>>>>>to do a better
>> >>>>>>>>>>>> job with our brand usage guidelines both in terms of the
>> >>>>>>>>>>>>wording and
>> >>>>>>>>>>>> enforcement.  There are many other companies out there that
>> >>>>>>>>>>>>use the OWASP
>> >>>>>>>>>>>> brand and I think that we agree that selective enforcement
>> >>>>>>>>>>>>against Contrast
>> >>>>>>>>>>>> is not the right answer.  Paul and Noreen are actively
>> >>>>>>>>>>>>working on this.
>> >>>>>>>>>>>> Either way, I think that implying that activities from a
>> >>>>>>>>>>>>vendor's marketing
>> >>>>>>>>>>>> department means that the project is not objective is not
>> >>>>>>>>>>>>inappropriate.  If
>> >>>>>>>>>>>> we feel that the project is not objective, then separate
>> >>>>>>>>>>>>measures need to be
>> >>>>>>>>>>>> taken to drive contribution diversity into it.  That I
>> >>>>>>>>>>>>absolutely agree with
>> >>>>>>>>>>>> and the message from Dave was that he would love to have more
>> >>>>>>>>>>>>contributors
>> >>>>>>>>>>>> to his project.  But, seeing as we cannot force people to
>> >>>>>>>>>>>>work on it, this
>> >>>>>>>>>>>> becomes a matter of "put up or shut up".  The same goes for
>> >>>>>>>>>>>>the experts that
>> >>>>>>>>>>>> you said reviewed the code.  If they feel that it is somehow
>> >>>>>>>>>>>>skewed towards
>> >>>>>>>>>>>> Contrast, they have the power to change that.  Now, if
>> >>>>>>>>>>>>someone tries to
>> >>>>>>>>>>>> participate and Dave tells them "No thanks", then I agree we
>> >>>>>>>>>>>>have a problem,
>> >>>>>>>>>>>> but I don't hear anyone inferring that happened.
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> Please, let's drop the conspiracy theories and focus on the
>> >>>>>>>>>>>> tangible things that we can do to help an OWASP project to be
>> >>>>>>>>>>>>more
>> >>>>>>>>>>>> successful.  Help find more participants to drive diversity,
>> >>>>>>>>>>>>update our
>> >>>>>>>>>>>> brand usage guidelines to prevent abuse, enforce them widely,
>> >>>>>>>>>>>>etc.  Thank
>> >>>>>>>>>>>> you.
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> ~josh
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> On Thu, Nov 26, 2015 at 4:24 PM, Jim Manico
>> >>>>>>>>>>>> <jim.manico at owasp.org> wrote:
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> Dinis,
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> Like a rare celestial moment when all the planets plus Pluto
>> >>>>>>>>>>>>>are
>> >>>>>>>>>>>>> aligned, I just read your email on the future of OWASP
>> >>>>>>>>>>>>>projects thinking,
>> >>>>>>>>>>>>> "Dinis is spot on".
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> Reflecting on projects I manage or work on...
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> The Java Encoder and HTML Sanitizer are likely best moved to
>> >>>>>>>>>>>>> Apache now that they have reached a measure of adoption and
>> >>>>>>>>>>>>>maturity. Apache
>> >>>>>>>>>>>>> would be a much better long term custodian. Perhaps the same
>> >>>>>>>>>>>>>for AppSensor,
>> >>>>>>>>>>>>> but not my project - just thinking out loud.
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> Other similar defensive projects are still being noodled on,
>> >>>>>>>>>>>>>so
>> >>>>>>>>>>>>> OWASP is a decent home for these research efforts.
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> The whole tools category is also something to consider.
>> >>>>>>>>>>>>> Dependency Check and of course ZAP are some of the best
>> >>>>>>>>>>>>>projects that OWASP
>> >>>>>>>>>>>>> offers, are they best served where they are today? Both have
>> >>>>>>>>>>>>>rich
>> >>>>>>>>>>>>> communities of developers but I don't see the foundation
>> >>>>>>>>>>>>>doing much to
>> >>>>>>>>>>>>> support these efforts.
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> ASVS has the opportunity to effect massive change, I would
>> to
>> >>>>>>>>>>>>> love to see major investment and volunteer activity here.
>> >>>>>>>>>>>>>Pro tech writer,
>> >>>>>>>>>>>>> detailed discourses on each individual requirement, etc. If
>> >>>>>>>>>>>>>I was king (and
>> >>>>>>>>>>>>> I am not, at all) I would invest in ASVS on a 6 figure
>> >>>>>>>>>>>>>scale. (And who
>> >>>>>>>>>>>>> started ASVS? Jeff, Dave and Boberski, hat tip to such a
>> >>>>>>>>>>>>>marvelous idea). Or
>> >>>>>>>>>>>>> maybe moving ASVS to the W3C or IETF would help it grow?
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> The Proactive Controls was a pet project but as we approach
>> >>>>>>>>>>>>>2.0
>> >>>>>>>>>>>>> we have several active/awesome volunteers working on it. We
>> >>>>>>>>>>>>>will be making
>> >>>>>>>>>>>>> the doc "world editable" to make contributions easy. OWASP
>> >>>>>>>>>>>>>seems like a good
>> >>>>>>>>>>>>> home for such an awareness doc. Same with T10, especially if
>> >>>>>>>>>>>>>community edits
>> >>>>>>>>>>>>> are welcome.
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> Anyhow, I'm with you on this Dinis. Once a project starts to
>> >>>>>>>>>>>>> reach production quality, spinning off the project as an
>> >>>>>>>>>>>>>external project or
>> >>>>>>>>>>>>> moving it to a different foundation where managing
>> >>>>>>>>>>>>>production software or
>> >>>>>>>>>>>>> formal standards is their thing seems realistic.
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> I don't have all the answers here, but your email certainly
>> >>>>>>>>>>>>> resonated with me.
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> Aloha,
>> >>>>>>>>>>>>> --
>> >>>>>>>>>>>>> Jim Manico
>> >>>>>>>>>>>>> Global Board Member
>> >>>>>>>>>>>>> OWASP Foundation
>> >>>>>>>>>>>>> https://www.owasp.org
>> >>>>>>>>>>>>> Join me in Rome for AppSecEU 2016!
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> On Nov 26, 2015, at 11:26 PM, Dinis Cruz
>> >>>>>>>>>>>>><dinis.cruz at owasp.org>
>> >>>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> Jim's reading of this situation is exactly my view on the
>> >>>>>>>>>>>>>value
>> >>>>>>>>>>>>> of the Contrast tool and how it has been 'pushing' the rules
>> >>>>>>>>>>>>>of engagement
>> >>>>>>>>>>>>> to an very 'fuzzy' moral/ethical/commercial limit :)
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> As per my last email, a key problem here is the 'perceived
>> >>>>>>>>>>>>> expectation' of what is an OWASP project, and how it should
>> >>>>>>>>>>>>>be consumed.
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> If you look at the OWASP benchmark as a research project,
>> >>>>>>>>>>>>>then
>> >>>>>>>>>>>>> the only way it could be making the kind of claims it makes
>> >>>>>>>>>>>>>(and have
>> >>>>>>>>>>>>> credibility) is if it had evolved from OWASP, with its own
>> >>>>>>>>>>>>>(diverse)
>> >>>>>>>>>>>>> community
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> On 26 November 2015 at 21:01, Jim Manico
>> >>>>>>>>>>>>><jim.manico at owasp.org>
>> >>>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> I have a different take on this situation but my opinion is
>> >>>>>>>>>>>>>>the
>> >>>>>>>>>>>>>> "minority opinion". I will respect the rest of the boards
>> >>>>>>>>>>>>>>take on this, but
>> >>>>>>>>>>>>>> here is how I see it.
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> First of all, Jeff has stated that he feels I am attacking
>> >>>>>>>>>>>>>>him
>> >>>>>>>>>>>>>> personally from a past personal grudge, and frankly I do
>> >>>>>>>>>>>>>>not fault him for
>> >>>>>>>>>>>>>> that perspective since we definitely have history with
>> >>>>>>>>>>>>>>conflict. So it's
>> >>>>>>>>>>>>>> fair to take my opinion on this with a grain of salt.
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> I look at this situation from the perspective of a forensic
>> >>>>>>>>>>>>>> investigator.
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> 1) The Benchmark project had Contrast hooks and only
>> >>>>>>>>>>>>>>Contrast
>> >>>>>>>>>>>>>> hooks in it when I reviewed it so this leads me to believe
>> >>>>>>>>>>>>>>that the project
>> >>>>>>>>>>>>>> was clearly built with Contrast in mind from the ground up,
>> >>>>>>>>>>>>>>at least in some
>> >>>>>>>>>>>>>> way.
>> >>>>>>>>>>>>>> 3) Dave has a history of breaching his duty to be vendor
>> >>>>>>>>>>>>>> neutral. He was gifted with a keynote in South Korea a few
>> >>>>>>>>>>>>>>years ago, and
>> >>>>>>>>>>>>>> used that opportunity to discuss and pitch Contrast, on
>> >>>>>>>>>>>>>>stage, during a
>> >>>>>>>>>>>>>> keynote - with Contrast specific slides. This is just
>> >>>>>>>>>>>>>>supporting evidence of
>> >>>>>>>>>>>>>> his intention at OWASP to push Contrast in ways that I
>> >>>>>>>>>>>>>>think are against the
>> >>>>>>>>>>>>>> intentions and goals of our foundation.
>> >>>>>>>>>>>>>> 3) Other experts have reviewed the project and felt that
>> >>>>>>>>>>>>>>many
>> >>>>>>>>>>>>>> of the tests were very slanted and almost contrived to
>> >>>>>>>>>>>>>>support Contrast. I
>> >>>>>>>>>>>>>> can drag those folks into this conversation, but I do not
>> >>>>>>>>>>>>>>think that would
>> >>>>>>>>>>>>>> help in any way. So it's fair to call this point heresy.
>> >>>>>>>>>>>>>> 4) I do not see this project as revolutionary, at all.
>> Every
>> >>>>>>>>>>>>>> vendor has their own test suite tuned for their tool. As
>> >>>>>>>>>>>>>>the benchmark
>> >>>>>>>>>>>>>> stands today, I see it as just another vendors
>> >>>>>>>>>>>>>>product-specific benchmark.
>> >>>>>>>>>>>>>> Mass collaboration from many vendors is not just a "nice to
>> >>>>>>>>>>>>>>have" but a base
>> >>>>>>>>>>>>>> requirement to get even close to useful for objective tool
>> >>>>>>>>>>>>>>measurement.
>> >>>>>>>>>>>>>> 5) Jeff stating that his Marketing people went over the
>> >>>>>>>>>>>>>>line is
>> >>>>>>>>>>>>>> also an admission that - well, they went over the line. By
>> >>>>>>>>>>>>>>the same token
>> >>>>>>>>>>>>>> Jeff was in his booth at AppSec USA surrounded by benchmark
>> >>>>>>>>>>>>>>marketing
>> >>>>>>>>>>>>>> material, discussing this to prospects and he even asked me
>> >>>>>>>>>>>>>>and Mr Coates to
>> >>>>>>>>>>>>>> wade into this debate and support Dave. So to say he was
>> >>>>>>>>>>>>>>not involved and it
>> >>>>>>>>>>>>>> was only his marketing people seems a stretch at best.
>> >>>>>>>>>>>>>> 6) The Contrast marketing team was wandering around the
>> >>>>>>>>>>>>>> conference zapping folks to get leads, and I asked them to
>> >>>>>>>>>>>>>>stay in their
>> >>>>>>>>>>>>>> booth, which is standard conference policy. These folks
>> >>>>>>>>>>>>>>know better but are
>> >>>>>>>>>>>>>> again going over the line to sell product at OWASP. There
>> >>>>>>>>>>>>>>is a better way
>> >>>>>>>>>>>>>> (like focusing on product capability and language support,
>> >>>>>>>>>>>>>>have consistent +
>> >>>>>>>>>>>>>> stellar customer service, have a humble and gracious
>> >>>>>>>>>>>>>>attitude to all
>> >>>>>>>>>>>>>> prospects and customers, actively participate in OWASP in a
>> >>>>>>>>>>>>>>vendor neutral
>> >>>>>>>>>>>>>> and community supportive way, etc).
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> Please note, I think Contrast is a decent tool, I've
>> >>>>>>>>>>>>>>offered to
>> >>>>>>>>>>>>>> resell in the past, and I have recommended it in certain
>> >>>>>>>>>>>>>>situations - even
>> >>>>>>>>>>>>>> after this situation arose. I'm stating this out of
>> >>>>>>>>>>>>>>honestly and desire to
>> >>>>>>>>>>>>>> put my cards on the table. I truly want Jeff and Dave to be
>> >>>>>>>>>>>>>>successful. They
>> >>>>>>>>>>>>>> have dedicated their lives to AppSec and if anyone should
>> >>>>>>>>>>>>>>win big-time, I
>> >>>>>>>>>>>>>> hope it's them. I even told Jeff I hope he hits the mother
>> >>>>>>>>>>>>>>load and donates
>> >>>>>>>>>>>>>> a little back to OWASP.
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> However, my instinct and evidence tell me that they both
>> >>>>>>>>>>>>>>went
>> >>>>>>>>>>>>>> over the line in the use of the OWASP brand to sell
>> product.
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> Now, Jeff makes a good point. We as a board and staff are
>> >>>>>>>>>>>>>>very
>> >>>>>>>>>>>>>> poor at enforcing brand management policy and it's not fair
>> >>>>>>>>>>>>>>to single out
>> >>>>>>>>>>>>>> Contrast, when many other vendors violate the brand, IMO.
>> >>>>>>>>>>>>>>Just google OWASP
>> >>>>>>>>>>>>>> and watch the ads fly that use the OWASP name to sell
>> >>>>>>>>>>>>>>product.
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> Also, any and every request that was made of Dave to adjust
>> >>>>>>>>>>>>>>the
>> >>>>>>>>>>>>>> project for the sake of vendor neutrality was taken very
>> >>>>>>>>>>>>>>seriously.
>> >>>>>>>>>>>>>> Regardless of Daves past intentions, he is clearly trying
>> >>>>>>>>>>>>>>to do the right
>> >>>>>>>>>>>>>> thing moving forward.
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> I look to "postels principle" in this situation (this is
>> >>>>>>>>>>>>>> otherwise known as the "robustness principle" and dates
>> >>>>>>>>>>>>>>back to the creation
>> >>>>>>>>>>>>>> of TCP) . This is paraphrased as, "Be liberal in what you
>> >>>>>>>>>>>>>>take from others
>> >>>>>>>>>>>>>> but be conservative in what you dish out". So I think it's
>> >>>>>>>>>>>>>>critical that
>> >>>>>>>>>>>>>> OWASP and any OWASP resource present itself in a strict
>> >>>>>>>>>>>>>>vendor neutral way.
>> >>>>>>>>>>>>>> But unless OWASP wants to be much more "even" in the
>> >>>>>>>>>>>>>>enforcement of brand
>> >>>>>>>>>>>>>> policy across the board to all violators, we should be
>> >>>>>>>>>>>>>>fairly lax in the
>> >>>>>>>>>>>>>> enforcement of these issues from the outside world.
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> I am trying to be objective here. My trigonometry teacher
>> >>>>>>>>>>>>>>once
>> >>>>>>>>>>>>>> told me "I'd fail my mother" when I asked him if he would
>> >>>>>>>>>>>>>>ever fail me (I
>> >>>>>>>>>>>>>> was an A student). If my mother owned a security company
>> >>>>>>>>>>>>>>and tried the same
>> >>>>>>>>>>>>>> stunt, I'd have the same opinions about her actions as
>> well.
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> So what next? Well hello from the other side. I'm going
>> >>>>>>>>>>>>>>back to
>> >>>>>>>>>>>>>> listening to Adele's new album where I can sit in my deep
>> >>>>>>>>>>>>>>feelings and
>> >>>>>>>>>>>>>> reflect upon what the OWASP foundation has done to enrich
>> >>>>>>>>>>>>>>my life. I would
>> >>>>>>>>>>>>>> much rather keep out of this (and any other conflict laden
>> >>>>>>>>>>>>>>situation at
>> >>>>>>>>>>>>>> OWASP), but I feel it's my responsibility to speak up.
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> Aloha,
>> >>>>>>>>>>>>>> --
>> >>>>>>>>>>>>>> Jim Manico
>> >>>>>>>>>>>>>> Global Board Member
>> >>>>>>>>>>>>>> OWASP Foundation
>> >>>>>>>>>>>>>> https://www.owasp.org
>> >>>>>>>>>>>>>> Join me in Rome for AppSecEU 2016!
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> On Nov 26, 2015, at 9:09 PM, Josh Sokol
>> >>>>>>>>>>>>>><josh.sokol at owasp.org>
>> >>>>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> I would be happy to provide an update.
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> Matt Konda and Dave Wichers, the Benchmark Project Leader,
>> >>>>>>>>>>>>>>had
>> >>>>>>>>>>>>>> a conversation a few weeks back.  To summarize their
>> >>>>>>>>>>>>>>conversation, Dave
>> >>>>>>>>>>>>>> acknowledges the currently lack of diversity in his project
>> >>>>>>>>>>>>>>and it is his
>> >>>>>>>>>>>>>> sincere desire to drive more people to it to help.  He also
>> >>>>>>>>>>>>>>acknowledges the
>> >>>>>>>>>>>>>> issues with Contrast's extreme marketing around the project
>> >>>>>>>>>>>>>>and feels that
>> >>>>>>>>>>>>>> it is in everyone's best interests for them to curb it
>> >>>>>>>>>>>>>>back.  While he does
>> >>>>>>>>>>>>>> have an ownership stake in Contrast, he works at Aspect and
>> >>>>>>>>>>>>>>has no control
>> >>>>>>>>>>>>>> over the marketing messages that they are putting out
>> >>>>>>>>>>>>>>there.  From the Board
>> >>>>>>>>>>>>>> perspective, there has been no evidence of any impropriety
>> >>>>>>>>>>>>>>on Dave's part
>> >>>>>>>>>>>>>> and it should be our goal to drive more diversity into the
>> >>>>>>>>>>>>>>project to
>> >>>>>>>>>>>>>> support Dave.  Dave appears to be sincere in his desires to
>> >>>>>>>>>>>>>>create a tool
>> >>>>>>>>>>>>>> where OWASP can tell vendors what we expect from their
>> >>>>>>>>>>>>>>tools.  If the main
>> >>>>>>>>>>>>>> issue is that only members of Aspect are working on it,
>> >>>>>>>>>>>>>>then the best thing
>> >>>>>>>>>>>>>> that we can do is try to get him some outside assistance.
>> >>>>>>>>>>>>>>We are also
>> >>>>>>>>>>>>>> asking that the project be opened up to commits via Git so
>> >>>>>>>>>>>>>>that outsiders
>> >>>>>>>>>>>>>> can push commits to it.
>> >>>>>>>>>>>>>> Josh Sokol and Jeff Williams, the CTO of Contrast, had a
>> >>>>>>>>>>>>>> conversation a few weeks back.  To summarize their
>> >>>>>>>>>>>>>>conversation, Jeff
>> >>>>>>>>>>>>>> believes that the work that Dave is doing on the Benchmark
>> >>>>>>>>>>>>>>is a game changer
>> >>>>>>>>>>>>>> in that it gives OWASP the power in dictating what these
>> >>>>>>>>>>>>>>tools need to be
>> >>>>>>>>>>>>>> finding.  He wants the Benchmark to be successful and
>> >>>>>>>>>>>>>>understands that it
>> >>>>>>>>>>>>>> needs to be diverse in order to be trusted.  He recognizes
>> >>>>>>>>>>>>>>that Dave is
>> >>>>>>>>>>>>>> trying to do that and does not want the marketing message
>> >>>>>>>>>>>>>>from Contrast to
>> >>>>>>>>>>>>>> interfere with his efforts.  Jeff felt that the "Lab"
>> >>>>>>>>>>>>>>status granted to
>> >>>>>>>>>>>>>> Benchmark meant that it was ready for mainstream adoption,
>> >>>>>>>>>>>>>>that it had 21k
>> >>>>>>>>>>>>>> tests, and was almost a year old, and didn't see anything
>> >>>>>>>>>>>>>>wrong with
>> >>>>>>>>>>>>>> marketing their results, but has agreed to talk to their
>> >>>>>>>>>>>>>>marketing team to
>> >>>>>>>>>>>>>> get them to lay off that message for now.  From the Board
>> >>>>>>>>>>>>>>perspective, we
>> >>>>>>>>>>>>>> have come to the realization that our brand usage
>> >>>>>>>>>>>>>>guidelines need an
>> >>>>>>>>>>>>>> overhaul to clarify what is and is not allowed.  We have
>> >>>>>>>>>>>>>>made a few
>> >>>>>>>>>>>>>> proposals and have reached out to Mozilla to gain more
>> >>>>>>>>>>>>>>insight on their
>> >>>>>>>>>>>>>> guidelines and even ask for assistance.  Noreen and Paul
>> >>>>>>>>>>>>>>are taking lead on
>> >>>>>>>>>>>>>> these efforts.
>> >>>>>>>>>>>>>> There is a note in the notes that the Board was supposed to
>> >>>>>>>>>>>>>> follow up with an open letter to the community and
>> >>>>>>>>>>>>>>companies involved
>> >>>>>>>>>>>>>> describing our review and actions.  I don't think that has
>> >>>>>>>>>>>>>>happened so I
>> >>>>>>>>>>>>>> will remind the person who took on that action item.
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> I'm happy to answer any questions that you may have.
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> ~josh
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> On Thu, Nov 26, 2015 at 11:55 AM, Tobias
>> >>>>>>>>>>>>>> <tobias.gondrom at owasp.org> wrote:
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> There have been several conversations on that matter and a
>> >>>>>>>>>>>>>>> dedicated call. Unfortunately for personal reasons I could
>> >>>>>>>>>>>>>>>not attend the
>> >>>>>>>>>>>>>>> last call as it was at 04:00am my local time, but all
>> >>>>>>>>>>>>>>>other board members
>> >>>>>>>>>>>>>>> did participate.
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> Could please one of my fellow board members give an
>> update.
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> Best, Tobias
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> On 26/11/15 18:04, Timo Goosen wrote:
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> I would also like to know the answer to Simon's question.
>> >>>>>>>>>>>>>>>We
>> >>>>>>>>>>>>>>> need to get rid of bad apples in OWASP in my opinion,
>> >>>>>>>>>>>>>>>there are too many
>> >>>>>>>>>>>>>>> people just using the OWASP "name" or "brand" to improve
>> >>>>>>>>>>>>>>>their own financial
>> >>>>>>>>>>>>>>> situation or career.
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> Regards.
>> >>>>>>>>>>>>>>> Timo
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> On Thu, Nov 26, 2015 at 1:13 PM, psiinon
>> >>>>>>>>>>>>>>><psiinon at gmail.com>
>> >>>>>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> Paul, and the rest of the board,
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> Its been over 2 months since I raised this issue.
>> >>>>>>>>>>>>>>>> Whats happening?
>> >>>>>>>>>>>>>>>> Has the board even discussed it?
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> Cheers,
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> Simon
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> On Tue, Oct 20, 2015 at 10:00 PM, Paul Ritchie
>> >>>>>>>>>>>>>>>> <paul.ritchie at owasp.org> wrote:
>> >>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>> Eoin, Johanna, All:
>> >>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>> In an earlier email, Josh Sokol mentioned that he will
>> be
>> >>>>>>>>>>>>>>>>> speaking in the next day or 2 to their CTO, while at
>> >>>>>>>>>>>>>>>>>LASCON, as a
>> >>>>>>>>>>>>>>>>> representative of the OWASP Board.  Following that
>> >>>>>>>>>>>>>>>>>feedback, the Board has
>> >>>>>>>>>>>>>>>>> action to take the next steps.
>> >>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>> Just an FYI that all comments are recognized and action
>> >>>>>>>>>>>>>>>>>is
>> >>>>>>>>>>>>>>>>> being taken.
>> >>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>> Paul
>> >>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>> Best Regards, Paul Ritchie
>> >>>>>>>>>>>>>>>>> OWASP Executive Director
>> >>>>>>>>>>>>>>>>> paul.ritchie at owasp.org
>> >>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>> On Tue, Oct 20, 2015 at 1:54 PM, johanna curiel curiel
>> >>>>>>>>>>>>>>>>> <johanna.curiel at owasp.org> wrote:
>> >>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>> Time for owasp to do a public statement and put a clear
>> >>>>>>>>>>>>>>>>>> story regarding this abusive behavior of Owasp brand
>> >>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>> On Tuesday, October 20, 2015, Eoin Keary
>> >>>>>>>>>>>>>>>>>> <eoin.keary at owasp.org> wrote:
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> Folks,
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> The project should be immediately shelved it's simply
>> >>>>>>>>>>>>>>>>>>>bad
>> >>>>>>>>>>>>>>>>>>> form.
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> This is damaging to OWASP, the industry and exactly
>> >>>>>>>>>>>>>>>>>>>what
>> >>>>>>>>>>>>>>>>>>> OWASP is not about.
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> There is a clear conflict of interest and distinct
>> >>>>>>>>>>>>>>>>>>>lack of
>> >>>>>>>>>>>>>>>>>>> science behind the claims made by Contrast.
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> Eoin Keary
>> >>>>>>>>>>>>>>>>>>> OWASP Volunteer
>> >>>>>>>>>>>>>>>>>>> @eoinkeary
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> On 7 Oct 2015, at 3:53 p.m., johanna curiel curiel
>> >>>>>>>>>>>>>>>>>>> <johanna.curiel at owasp.org> wrote:
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> At the moment we did the project review, we observed
>> >>>>>>>>>>>>>>>>>>>that
>> >>>>>>>>>>>>>>>>>>> the project did not have enough testing to be
>> >>>>>>>>>>>>>>>>>>>considered in any form as
>> >>>>>>>>>>>>>>>>>>> 'ready'  for benchmarking, neither that it had yet the
>> >>>>>>>>>>>>>>>>>>>community adoption,
>> >>>>>>>>>>>>>>>>>>> however technically speaking as it has been classified
>> >>>>>>>>>>>>>>>>>>>by the leaders, the
>> >>>>>>>>>>>>>>>>>>> project is at the beta stage.
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> Indeed , Dave had the push to have the project
>> reviewed
>> >>>>>>>>>>>>>>>>>>> but it was never clear that later on the project was
>> >>>>>>>>>>>>>>>>>>>going to be advertisied
>> >>>>>>>>>>>>>>>>>>> this way. That all happend after the presentation at
>> >>>>>>>>>>>>>>>>>>>Appsec.
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> I had my concerns regarding how sensitive is the
>> >>>>>>>>>>>>>>>>>>>subject
>> >>>>>>>>>>>>>>>>>>> of the project ,but I think we should allow project
>> >>>>>>>>>>>>>>>>>>>leaders to develop their
>> >>>>>>>>>>>>>>>>>>> communication strategy even if this has conflict of
>> >>>>>>>>>>>>>>>>>>>interest. It all depends
>> >>>>>>>>>>>>>>>>>>> how they behave and how they manage this.
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> On Tuesday, October 6, 2015, Michael Coates
>> >>>>>>>>>>>>>>>>>>> <michael.coates at owasp.org> wrote:
>> >>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>> It's not really that formal to add to the agenda,
>> >>>>>>>>>>>>>>>>>>>>just a
>> >>>>>>>>>>>>>>>>>>>> wiki that we add in the text.
>> >>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>> I think you can safely assume it will get the
>> >>>>>>>>>>>>>>>>>>>>appropriate
>> >>>>>>>>>>>>>>>>>>>> discussion.
>> >>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>> On Oct 6, 2015, at 7:16 AM, psiinon
>> >>>>>>>>>>>>>>>>>>>><psiinon at gmail.com>
>> >>>>>>>>>>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>> Really?? Its not on the agenda yet for the next
>> >>>>>>>>>>>>>>>>>>>>meeting??
>> >>>>>>>>>>>>>>>>>>>> How does it get added to the agenda?
>> >>>>>>>>>>>>>>>>>>>> And that was a formal request if that makes any
>> >>>>>>>>>>>>>>>>>>>> difference :)
>> >>>>>>>>>>>>>>>>>>>> I'm all in favour of getting the facts straight
>> before
>> >>>>>>>>>>>>>>>>>>>> any actions are taken, hence my request for an
>> >>>>>>>>>>>>>>>>>>>>'ethical review' or whatever
>> >>>>>>>>>>>>>>>>>>>> it should be called.
>> >>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>> Cheers,
>> >>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>> Simon
>> >>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>> On Tue, Oct 6, 2015 at 3:07 PM, Michael Coates
>> >>>>>>>>>>>>>>>>>>>> <michael.coates at owasp.org> wrote:
>> >>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>> First step is to get all of our information
>> straight so
>> >>>>>>>>>>>>>>>>>>>>> we're clear on where things are at.
>> >>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>> This was not on the board agenda last meeting and is
>> >>>>>>>>>>>>>>>>>>>>> also not on the next agenda as of yet (of course it
>> could
>> always be added if
>> >>>>>>>>>>>>>>>>>>>>> needed).
>> >>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>> We are aware that people have raised questions
>> though.
>> >>>>>>>>>>>>>>>>>>>>> I'm hoping we can get a clear understanding of all
>> the
>> facts and then
>> >>>>>>>>>>>>>>>>>>>>> discuss if changes are needed.
>> >>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>> On Oct 6, 2015, at 1:52 AM, psiinon <
>> psiinon at gmail.com>
>> >>>>>>>>>>>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>> Hey Michael,
>> >>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>> Is the board going to take any action?
>> >>>>>>>>>>>>>>>>>>>>> Were there any discussions about this controversy
>> in the
>> >>>>>>>>>>>>>>>>>>>>> board meeting at AppSec USA?
>> >>>>>>>>>>>>>>>>>>>>> If not will it be on the agenda for the meeting on
>> >>>>>>>>>>>>>>>>>>>>> October 14th?
>> >>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>> Cheers,
>> >>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>> Simon
>> >>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>> On Tue, Oct 6, 2015 at 8:25 AM, Michael Coates
>> >>>>>>>>>>>>>>>>>>>>> <michael.coates at owasp.org> wrote:
>> >>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>> Simon
>> >>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>> I posted the below message earlier today. At this
>> point
>> >>>>>>>>>>>>>>>>>>>>>> my goal is to just gain clarity over the current
>> reality
>> and ideally drive
>> >>>>>>>>>>>>>>>>>>>>>> to a shared state of success. This message doesn't
>> seem
>> to be reflected in
>> >>>>>>>>>>>>>>>>>>>>>> the list yet. It could be because my membership
>> hasn't
>> been approved or
>> >>>>>>>>>>>>>>>>>>>>>> because of mail list delays (I miss Google
>> groups). But I
>> think these
>> >>>>>>>>>>>>>>>>>>>>>> questions will start the conversation.
>> >>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>> (This was just me asking questions as a curious
>> Owasp
>> >>>>>>>>>>>>>>>>>>>>>> member, not any action on behalf of the board)
>> >>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>> Begin forwarded message:
>> >>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>> From: Michael Coates <michael.coates at owasp.org>
>> >>>>>>>>>>>>>>>>>>>>>> Date: October 5, 2015 at 6:20:23 PM PDT
>> >>>>>>>>>>>>>>>>>>>>>> To: owasp-benchmark-project at lists.owasp.org
>> >>>>>>>>>>>>>>>>>>>>>> Subject: Project Questions
>> >>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>> OWASP Benchmark List,
>> >>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>> I've heard more about this project and am excited
>> about
>> >>>>>>>>>>>>>>>>>>>>>> the idea of an independent perspective of tool
>> performance. I'm trying to
>> >>>>>>>>>>>>>>>>>>>>>> understand a few things to better respond to
>> questions
>> from those in the
>> >>>>>>>>>>>>>>>>>>>>>> security & OWASP community.
>> >>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>> In my mind there are two big areas for
>> consideration in
>> >>>>>>>>>>>>>>>>>>>>>> a benchmark process.
>> >>>>>>>>>>>>>>>>>>>>>> 1. Are the benchmarks testing the right areas?
>> >>>>>>>>>>>>>>>>>>>>>> 2. Is the process for creating the benchmark
>> objective
>> >>>>>>>>>>>>>>>>>>>>>> & free from conflicts of interest.
>> >>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>> I think as a group OWASP is the right body to
>> align on
>> >>>>>>>>>>>>>>>>>>>>>> #1.
>> >>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>> I'd like to ask for some clarifications on item
>> #2. I
>> >>>>>>>>>>>>>>>>>>>>>> think it's important to avoid actual conflict of
>> interest
>> and also the
>> >>>>>>>>>>>>>>>>>>>>>> appearance of conflict of interest. The former is
>> obvious
>> why we mustn't
>> >>>>>>>>>>>>>>>>>>>>>> have that, the latter is critical so others have
>> faith in
>> the tool, process
>> >>>>>>>>>>>>>>>>>>>>>> and outputs of the process when viewing or hearing
>> about
>> the project.
>> >>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>> 1) Can we clarify whether other individuals have
>> >>>>>>>>>>>>>>>>>>>>>> submitted meaningful code to the project?
>> >>>>>>>>>>>>>>>>>>>>>> Observation:
>> >>>>>>>>>>>>>>>>>>>>>> Nearly all the code commits have come from 1 person
>> >>>>>>>>>>>>>>>>>>>>>> (project lead).
>> >>>>>>>>>>>>>>>>>>>>>>
>> https://github.com/OWASP/Benchmark/graphs/contributors
>> >>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>> 2) Can we clarify the contributions of others and
>> their
>> >>>>>>>>>>>>>>>>>>>>>> represented organizations?
>> >>>>>>>>>>>>>>>>>>>>>> Observation:
>> >>>>>>>>>>>>>>>>>>>>>> The acknowledgements tab listed two developers
>> (Juan
>> >>>>>>>>>>>>>>>>>>>>>> Gama & Nick Sanidas) both who work at the same
>> company as
>> the project lead.
>> >>>>>>>>>>>>>>>>>>>>>> It seems other people have submitted some small
>> amounts
>> of material, but
>> >>>>>>>>>>>>>>>>>>>>>> overall it seems all development has come from the
>> same
>> company.
>> >>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>
>> https://www.owasp.org/index.php/Benchmark#tab=Acknowledgements
>> >>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>> 3) Can we clarify in what ways we've mitigated the
>> >>>>>>>>>>>>>>>>>>>>>> potential conflict of interest and also the
>> appearance of
>> a conflict of
>> >>>>>>>>>>>>>>>>>>>>>> interest? This seems like the largest blocker for
>> wide
>> spread acceptance of
>> >>>>>>>>>>>>>>>>>>>>>> this project and the biggest risk.
>> >>>>>>>>>>>>>>>>>>>>>> Observation:
>> >>>>>>>>>>>>>>>>>>>>>> The project lead and both of the project developers
>> >>>>>>>>>>>>>>>>>>>>>> works for a company with very close ties to one of
>> the
>> companies that is
>> >>>>>>>>>>>>>>>>>>>>>> evaluated by this project. Further, it appears the
>> company is performing
>> >>>>>>>>>>>>>>>>>>>>>> very well on the project tests.
>> >>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>> 4) If we are going to list tool vendors then I'd
>> >>>>>>>>>>>>>>>>>>>>>> recommend listing multiple vendors for each
>> category.
>> >>>>>>>>>>>>>>>>>>>>>> Observation:
>> >>>>>>>>>>>>>>>>>>>>>> The tools page only lists 1 IAST tool. Since this
>> is
>> >>>>>>>>>>>>>>>>>>>>>> the point of the potential conflict of interest it
>> is
>> important to list
>> >>>>>>>>>>>>>>>>>>>>>> numerous IAST tools.
>> >>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>
>> https://www.owasp.org/index.php/Benchmark#tab=Tool_Support_2FResults
>> >>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>> 5) Diverse body with multiple points of view
>> >>>>>>>>>>>>>>>>>>>>>> Observation:
>> >>>>>>>>>>>>>>>>>>>>>> There is no indication that multiple stakeholders
>> are
>> >>>>>>>>>>>>>>>>>>>>>> present to review and decide on the future of this
>> project. If they exist, a
>> >>>>>>>>>>>>>>>>>>>>>> new section should be added to the project page to
>> raise
>> awareness. If they
>> >>>>>>>>>>>>>>>>>>>>>> don't exist, we should reevaluate how we are
>> obtaining an
>> independent view
>> >>>>>>>>>>>>>>>>>>>>>> of the testing process.
>> >>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>> Again, I think the idea of the project is great.
>> From
>> >>>>>>>>>>>>>>>>>>>>>> my perspective clarifying these questions will help
>> ensure the project is
>> >>>>>>>>>>>>>>>>>>>>>> not only objective, but also perceived as
>> objective from
>> someone reviewing
>> >>>>>>>>>>>>>>>>>>>>>> the material. Ultimately this will contribute to
>> the
>> success and growth of
>> >>>>>>>>>>>>>>>>>>>>>> the project.
>> >>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>> Thanks!
>> >>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>> --
>> >>>>>>>>>>>>>>>>>>>>>> Michael Coates
>> >>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>> On Oct 2, 2015, at 1:31 AM, psiinon <
>> psiinon at gmail.com>
>> >>>>>>>>>>>>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>> OK, based on the concerns raised so far I think the
>> >>>>>>>>>>>>>>>>>>>>>> board should initiate a review of the OWASP
>> Benchmark
>> project.
>> >>>>>>>>>>>>>>>>>>>>>> I'm not raising a formal complaint against it, I'm
>> just
>> >>>>>>>>>>>>>>>>>>>>>> requesting a review.
>> >>>>>>>>>>>>>>>>>>>>>> And I dont think it needs a 'standard' project
>> review -
>> >>>>>>>>>>>>>>>>>>>>>> Johanna has already done a very good job of this.
>> >>>>>>>>>>>>>>>>>>>>>> Not sure what sort of review you'd call it, I'll
>> leave
>> >>>>>>>>>>>>>>>>>>>>>> the naming to others :)
>> >>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>> I'm concerned that we have an OWASP project lead
>> by a
>> >>>>>>>>>>>>>>>>>>>>>> company who has a clear commercial stake in the
>> results.
>> >>>>>>>>>>>>>>>>>>>>>> Bringing more companies on board will help, but I'm
>> >>>>>>>>>>>>>>>>>>>>>> still not sure that alone will make it independent
>> enough.
>> >>>>>>>>>>>>>>>>>>>>>> Commercial companies can afford to dedicate staff
>> to
>> >>>>>>>>>>>>>>>>>>>>>> improving Benchmark so that their products look
>> better.
>> >>>>>>>>>>>>>>>>>>>>>> Open source projects just cant do that, so we are
>> at a
>> >>>>>>>>>>>>>>>>>>>>>> distinct disadvantage.
>> >>>>>>>>>>>>>>>>>>>>>> Should we allow a commercially driven OWASP project
>> >>>>>>>>>>>>>>>>>>>>>> who's aim could be seen be to promote commercial
>> software?
>> >>>>>>>>>>>>>>>>>>>>>> If so, what sort of checks and balances does it
>> need?
>> >>>>>>>>>>>>>>>>>>>>>> Those are the sort of questions I'd like an
>> independent
>> >>>>>>>>>>>>>>>>>>>>>> review to look at.
>> >>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>> I do think there are some immediate steps that
>> could be
>> >>>>>>>>>>>>>>>>>>>>>> taken:
>> >>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>> I'd like to see the Benchmark project page clearly
>> >>>>>>>>>>>>>>>>>>>>>> state thats its at a very early stage and that the
>> results are _not_ yet
>> >>>>>>>>>>>>>>>>>>>>>> suitable for use in commercial literature.
>> >>>>>>>>>>>>>>>>>>>>>> I'd also like the main companies developing
>> Benchmark
>> >>>>>>>>>>>>>>>>>>>>>> to be clearly stated on the main page. If and when
>> other
>> companies get
>> >>>>>>>>>>>>>>>>>>>>>> involved then this would actually help the
>> project's
>> claim of vendor
>> >>>>>>>>>>>>>>>>>>>>>> independence.
>> >>>>>>>>>>>>>>>>>>>>>> And I'd love to see a respected co-leader added to
>> the
>> >>>>>>>>>>>>>>>>>>>>>> project who is not associated with any commercial
>> or open
>> source security
>> >>>>>>>>>>>>>>>>>>>>>> tools:)
>> >>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>> And we should carry on discussing the project on
>> this
>> >>>>>>>>>>>>>>>>>>>>>> list - I think such discussions are very healthy,
>> and I'd
>> love to see this
>> >>>>>>>>>>>>>>>>>>>>>> project mature to a state where it can be a
>> trusted,
>> independent and valued
>> >>>>>>>>>>>>>>>>>>>>>> resource.
>> >>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>> Cheers,
>> >>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>> Simon
>> >>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>> On Thu, Oct 1, 2015 at 7:59 PM, Tobias
>> >>>>>>>>>>>>>>>>>>>>>> <tobias.gondrom at owasp.org> wrote:
>> >>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>> @Simon:
>> >>>>>>>>>>>>>>>>>>>>>>> yes, the leaders list is the place for your
>> >>>>>>>>>>>>>>>>>>>>>>> discussions for project and chapter leaders
>> >>>>>>>>>>>>>>>>>>>>>>> @Timo: I like your framing of "Don't ask what
>> OWASP
>> >>>>>>>>>>>>>>>>>>>>>>> can do for me, ask what I can do for OWASP."
>> >>>>>>>>>>>>>>>>>>>>>>> That should and is indeed the spirit of OWASP:-)
>> >>>>>>>>>>>>>>>>>>>>>>> Best regards, Tobias
>> >>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>> On 30/09/15 09:42, Timo Goosen wrote:
>> >>>>>
>> >>>>> ...
>> >>>>>
>> >>>>> [Message clipped]
>> >>>>> _______________________________________________
>> >>>>> Owasp-board mailing list
>> >>>>> Owasp-board at lists.owasp.org
>> >>>>> https://lists.owasp.org/mailman/listinfo/owasp-board
>> >>>>>
>> >>>>
>> >>>>
>> >>>>
>> >>>> --
>> >>>> OWASP ZAP Project leader
>> >>>>
>> >>>> _______________________________________________
>> >>>> OWASP-Leaders mailing list
>> >>>> OWASP-Leaders at lists.owasp.org
>> >>>> https://lists.owasp.org/mailman/listinfo/owasp-leaders
>> >>>>
>> >>>
>> >>
>> >>
>> >>
>> >> --
>> >> OWASP ZAP Project leader
>> >>
>> >> _______________________________________________
>> >> OWASP-Leaders mailing list
>> >> OWASP-Leaders at lists.owasp.org
>> >> https://lists.owasp.org/mailman/listinfo/owasp-leaders
>> >>
>> >_______________________________________________
>> >OWASP-Leaders mailing list
>> >OWASP-Leaders at lists.owasp.org
>> >https://lists.owasp.org/mailman/listinfo/owasp-leaders
>>
>>
>> _______________________________________________
>> OWASP-Leaders mailing list
>> OWASP-Leaders at lists.owasp.org
>> https://lists.owasp.org/mailman/listinfo/owasp-leaders
>>
>
> _______________________________________________
> OWASP-Leaders mailing list
> OWASP-Leaders at lists.owasp.org
> https://lists.owasp.org/mailman/listinfo/owasp-leaders
>
>
> _______________________________________________
> OWASP-Leaders mailing list
> OWASP-Leaders at lists.owasp.org
> https://lists.owasp.org/mailman/listinfo/owasp-leaders
>
>


-- 
OWASP ZAP <https://www.owasp.org/index.php/ZAP> Project leader
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.owasp.org/pipermail/owasp-leaders/attachments/20151201/7a4ee2bd/attachment-0001.html>


More information about the OWASP-Leaders mailing list