[Owasp-leaders] [Owasp-board] OWASP Benchmark project - potential conflict of interest

Gary Robinson gary.robinson at owasp.org
Tue Dec 1 09:29:13 UTC 2015


Gotta +1 that.

Gary

On Tue, Dec 1, 2015 at 9:20 AM, psiinon <psiinon at gmail.com> wrote:

> Yes, exactly that.
>
> On Tue, Dec 1, 2015 at 9:16 AM, Eoin Keary <eoin.keary at owasp.org> wrote:
>
>> Hi Jeff,
>>
>> Being with OWASP for 11 years, being a previous board vice chair and
>> global board member for 5 years and having lead many OWASP projects such as
>> the testing guide, code review guide etc I'd like to share how this is
>> perceived....
>>
>> My interpretation is the problem is that many folks see the OWASP
>> benchmark run by Aspect/Contrast company staff and it happens the Contrast
>> is top scoring. The second it was released as a project it was used in
>> advertising and awareness campaigns.
>>
>> Second issue is that is its way to immature and does not have the sample
>> data to be scientific and is simply used for commercial gain/product
>> promotion.
>>
>> It's not as if the project has more than 50% of the tool vendors /open
>> source tools taking part such that the sample space stands up
>> statistically.
>>
>> There is a perceived independence issue given the project was conceived
>> and is lead by a tool vendor.
>>
>> Many feel the project was conceived for the benefit of Contrast and it
>> has lost credibility for that reason.
>>
>> The idea of the bench mark was/is good but it was rushed and taken
>> advantage of without it being mature or proven.
>>
>> It does not have the data to back up its claims and many OWASP people
>> believe it was simply an advertising vehicle regardless of its future use,
>> merit and adoption.
>>
>> I believe this clarifies many peoples point of view.
>>
>> Best regards,
>>
>> Eoin Keary
>> OWASP Volunteer
>> @eoinkeary
>>
>>
>>
>> On 1 Dec 2015, at 5:46 a.m., Jeff Williams <jeff.williams at owasp.org>
>> wrote:
>>
>> Hi Leaders,
>>
>>
>>
>> I've refrained from commenting on the OWASP Benchmark project because I
>> work for Contrast and used to work with Dave.  But before you judge, please
>> hear my thoughts on this project, vendors, and OWASP in general.
>>
>>
>>
>> As many of you know, I was the global chair of OWASP for over eight
>> years. I set up the 501c3, created the chapters program, created the wiki,
>> and led many successful projects. And I spent countless hours working with
>> project leads, organizations, and vendors to teach them how OWASP works.
>>
>>
>>
>> Contrast has heard your concerns and is already changing its marketing
>> materials to emphasize that neither OWASP nor DHS endorses commercial
>> products.  Both Contrast and I are huge OWASP supporters and we will do
>> whatever we can to help achieve the mission. We want to make sure we are
>> following the OWASP brand usage guidelines and will work with you to make
>> sure we are. I wrote the original rules years ago and if it’s time to
>> re-examine them then let’s change them in an open process.
>>
>>
>>
>> The OWASP Benchmark is a real opportunity for OWASP to move the needle.
>> I've spent considerable time with NSA’s Juliet and other similar test
>> suites, and none are even close to the quality and ease of use of the OWASP
>> Benchmark. I'd go so far as to say that if you haven't used the Benchmark,
>> you really have no idea what the strengths and weaknesses of your tools
>> are. The Benchmark isn't perfect. It could cover more vulns, but the ones
>> in there are pretty damn important and the project is working on adding
>> more vulnerability categories and more real world complexity. And the
>> test cases aren't as complex as real code, but testing fundamental tool
>> capabilities can tell us a lot.
>>
>>
>>
>> The force-multiplier here is that by influencing tools and their vendors,
>> OWASP can reach far more developers than it could with any number of
>> awesome lists, cheat sheets, or standards. Do you wish tools were better at
>> XXE, for example? If the OWASP community puts it in the Benchmark, it
>> starts influencing buying behavior and the tools themselves.  The Benchmark
>> puts OWASP in the driver's seat and it's already started to work.
>>
>>
>>
>> On the leaders list, people have been making all kinds of assertions
>> about marketing claims. But without a ruler, everyone is just guessing.
>> This is exactly why we need the OWASP Benchmark. Without one, there’s no
>> way to establish the truth of either the claims OR the counterclaims.  This
>> ruler-free environment encourages vendors to make outrageous claims, like
>> “we address the entire OWASP Top Ten” for example.  So, ironically, the
>> Benchmark is the path towards more realistic marketing.
>>
>>
>>
>> With any benchmark, some vendors aren't going to do well, and they will
>> argue the ruler is broken.  Other vendors will do well and will promote
>> their results.  As the ruler changes over time, vendor positions will
>> change.  All of this is good for the community and for OWASP.  The best
>> imaginable outcome would be to touch off an "arms-race" for accuracy in the
>> appsec tools space.  The OWASP Benchmark will evolve and get better, the
>> tools will get dragged along, and consumers will benefit through increased
>> visibility.  This is exactly OWASP's mission, making appsec visible so that
>> market forces can work.
>>
>>
>> --Jeff
>>
>>
>>
>>
>> On Mon, Nov 30, 2015 at 10:37 PM, Dave Wichers <dave.wichers at owasp.org>
>> wrote:
>>
>>> Justin.
>>>
>>> Thanks so much for your post. For clarification, Justin refers to
>>> Coverity
>>> numerous times throughout this post, but really means Contrast.
>>>
>>> -Dave
>>>
>>> On 11/30/15, 4:55 PM, "Justin Searle" <justin at meeas.com> wrote:
>>>
>>> >Psiinon, out of curiosity, from a purely project/code perspective,
>>> >what do you feel would make the Benchmark tool more "independent"?
>>> >From digging around in the source code, it seems they already have a
>>> >fair number of report parsers for several OSS and commercial tools in
>>> >their tree.  To keep this thread a bit more clean, perhaps post your
>>> >response in the Benchmark project mailling list or in a GitHub issue
>>> >then reply to this thread with link.  That will allow the Benchmark
>>> >team to directly receive that feedback and allow those of use
>>> >interested to follow up there to have the needed discussions to
>>> >improve the tool.
>>> >
>>> >Eoin, Dinis, and Jim, it has been a while.  Sorry I've been so removed
>>> >from this community the last couple of years.  From reading through
>>> >the thread and articles, I don't think it is fair to say the board has
>>> >been sitting on the issue.  From Josh's comments, it seems like the
>>> >board did take action on this, however I understand if you disagreed
>>> >with their decision.  You have always been some of the most vocal
>>> >leaders here, and I applaud that.  Your contributions have been great
>>> >to OWASP over the years, so please keep it up!
>>> >
>>> >Most of the concerns I've seen stated are around use of the OWASP
>>> >brand in marketing.  I personally do not see merit in many of the
>>> >concerns, however my personal greatest concerns are around statements
>>> >like this:
>>> >
>>> >    "OWASP reports that the best static analysis tools score in the
>>> >low 30¹s (out of 100) against this benchmark."
>>> >
>>> >That was in the Dark Reading article, and other statements were made
>>> >like that in the Twitter video posted above.  That I feel is the most
>>> >egregious of the brand misuse as it implies that OWASP as an
>>> >organization has formally made statement.  Keep the full project name
>>> >in tact such as "The OWASP Benchmark tool reports..." would be a much
>>> >more accurate and less brand abusive way to make that statement, and
>>> >based on current OWASP policies, much more inline with what is
>>> >permitted.  While OWASP members feel even that is going too far,
>>> >please remember that OWASP does not own the vast majority of the OWASP
>>> >projects.  Each OWASP project is usually owned by the author and in
>>> >many cases, any contributor to that project.  OWASP as an legal entity
>>> >owns a very small percentage (none that I can even name off the top of
>>> >my head).
>>> >
>>> >A few years ago when we were initially working on the new project
>>> >leaders handbook and project roadmap (before OWASP disbanded all the
>>> >global committees in 2013, including the Global Projects Committee),
>>> >we discussed chaining the official verbiage to and OWASP "sponsored"
>>> >project.  I think it is unfortunate we didn't codify that terminology,
>>> >but regardless, I think that is the always a good way to think of
>>> >OWASP projects: projects owned and run by individuals of the
>>> >community.  However, not everyone in the OWASP community understand
>>> >this.  This is easy to see in such statements as "Allowing this
>>> >project to exist without ..." and forcing a "project be opened up to
>>> >commits via Git so that outsiders can push commits to it" and OWASP
>>> >should "decide on the future of this project".  Personally, I think
>>> >the most of the drama around OWASP projects comes from this
>>> >misunderstanding and OWASP community members trying to manage an OWASP
>>> >project that OWASP doesn't own.
>>> >
>>> >However one of the most difficult issues that perpetuates this problem
>>> >and in many cases conflicts concerning brand abuse is project naming.
>>> >Since OWASP currently allows projects to use the OWASP name in their
>>> >project name (which I think is a mistake), it is hard to refer to a
>>> >project without in some way evoking the OWASP brand.  There is very
>>> >little legal recourse in most countries to state a fact that "tool X
>>> >generated score Y for product Z" in their marketing literature.  That
>>> >does not imply that tool X promotes product Z.  And if tool X happens
>>> >to be named OWASP Benchmark, then that is not brand infringement in
>>> >most countries.  If this is a concern to the OWASP community, then the
>>> >better recourse would be to reconsider OWASP's permission to allow
>>> >projects to use OWASP in their project name.
>>> >
>>> >Preventing people from making such statements is usually handled in a
>>> >EULA saying how you can and can't use the tool and the tool output,
>>> >which in most cases including this would be contrary from the official
>>> >OSS definition and most OSS licenses.  So the best distance I think we
>>> >could hope to obtain is to disallow the use of OWASP in any project
>>> >names.
>>> >
>>> >And by the way, why would we ever want to stop ANY company out there
>>> >from using OWASP tools and documentation?  Why would we ever want ANY
>>> >company NOT to advertise that they use OWASP tools and documentation?
>>> >Why would we NOT want a company to state they they use OWASP tools in
>>> >their marketing literature?  As long as it is clear that OWASP does
>>> >not endorse that company, we should encourage the spread and use of
>>> >OWASP tools.  Does anyone have a problem with saying website Z has
>>> >been tested for all OWASP Top 10 risks in their marketing literature?
>>> >What about saying that all vulnerabilities identified by OWASP Zed
>>> >Attack Proxy Project have been remediated in website Z literature?
>>> >What about all the current DAST/SAST tools that have an "OWASP Top 10"
>>> >testing mode?  I don't think any of these imply that a project is
>>> >endorsed by OWASP, but if this is a concern for people, would ..."all
>>> >vulnerabilities identified by Zed Attack Proxy Project have been
>>> >remediated" be better?
>>> >
>>> >As for a Jeff (or his company) using the benchmark scores from his own
>>> >OWASP sponsored project in marketing literature to help customers
>>> >understand their commercial offering, I have no qualms with that.  I
>>> >don't find that a breach of trust or brand abuse.  I only see brand
>>> >abuse in statements mentioned above that stated "Owasp found..." and
>>> >such where the tool name was not used, which is explicitly stating a
>>> >false OWASP perspective.  Jeff and Coverity in benchmarking their tool
>>> >against their own opensource project simply ties the two together in
>>> >such a way that can be tested.  Based on statements made by Psiinon
>>> >and others, including Coverity's competition, the tool works and does
>>> >not seem to be skewed towards Coverity's tool, even though they score
>>> >the highest.  The tool is opensource.  If anyone believes the tool
>>> >unfairly scores Coverity's tool, or doesn't not provide benefit to
>>> >other assessment tools who want to improve their scanning engines,
>>> >please dig through the code and identify how it does that.  All I've
>>> >seen so far is people disagreeing with how the metric is generated and
>>> >the number of tests involved, which in itself doesn't seem to portray
>>> >bias for one tool over another.  If the tool is found to favor
>>> >Coverity's scanning tool, then that will be shown by someone with time
>>> >and interest, and if that is the case, the brand loss will by
>>> >Coverity's, not OWASP's.
>>> >
>>> >As for actions, I agree with the actions the board seems to have made
>>> >so far.  I do not think any penalties be levied against the Benchmark
>>> >project.  I do not think that they should be downgrade back to
>>> >incubator, which seems a petty and meaningless action to me.  The
>>> >maturity seems to say Lab quality more than many other existing Lab
>>> >projects.  I do not think OWASP has any right or reason to force the
>>> >Benchmark project to allow commits from additional persons.  Having a
>>> >single person do actual commits to main trunks while other offer pull
>>> >requests is common and very standard in OSS, and in now way portrays
>>> >how "open" the project community is.  And banning companies from any
>>> >mention OWASP projects in marketing efforts, wether project leaders
>>> >are associate with said companies or not, would be foolish in our
>>> >efforts to growing OWASP brand, as long as such marketing efforts do
>>> >not implicitly or explicitly imply OWASP endorsement of a company, its
>>> >tools, or its services.
>>> >
>>> >As for my suggestions to the OWASP board, I'd recommend the following:
>>> > - An official statement on the Benchmark project page at owasp.org
>>> >stating as Johanna suggested, that OWASP does not endorse any company,
>>> >commercial tool, or commercial service
>>> > - A request to Coverty to make a similar statement on their website
>>> >and future marketing efforts just to clarify this misunderstanding
>>> > - A formal cease and desist letter to Coverty to stop making explicit
>>> >claims in OWASP's behalf such as "OWASP found ..." and to restrict all
>>> >use of the term "OWASP" as part of the "OWASP Benchmark" project's
>>> >formal name.
>>> >
>>> >As for the Benchmark project, I'd recommend the following:
>>> > - If the tool doesn't already do so, I'd recommend a simply statement
>>> >in the Benchmark reports saying that scores from the tool does not
>>> >imply any endorsement for or against any tool tested.
>>> > - Also, digging through your sourcecode tree, I noticed there doesn't
>>> >seem to be any copyright notices.  I'd recommend you adding those
>>> >copyright notices to whoever owns the code, otherwise it is hard to
>>> >enforce any copyright license restrictions.
>>> >
>>> >And finally, as for the OWASP community, I'd encourage you to decide
>>> >if it makes sense to remove the ability of projects to formally use
>>> >OWASP as part of their project name.  If we don't do this then
>>> >individual project brand and OWASP brand becomes commingled, and
>>> >ownership becomes less clear.  Project leaders, for your current or
>>> >future projects, I'd personally recommend you don't use OWASP in the
>>> >title so you can build project brand recognition independent of OWASP,
>>> >and instead do something like "Project X, an OWASP project" in your
>>> >project marketing.
>>> >
>>> >Justin Searle
>>> >Managing Partner - UtiliSec
>>> >+1 801-784-2052
>>> >justin at utilisec.com
>>> >justin at meeas.com
>>> >
>>> >
>>> >On Mon, Nov 30, 2015 at 10:40 AM, psiinon <psiinon at gmail.com> wrote:
>>> >>>
>>> >>> In the short term, It is better to remove it from OWASP, leaving the
>>> >>>door
>>> >>> open for its return (in a future when some of the independence and
>>> >>>quality
>>> >>> issues have been solved)
>>> >>>
>>> >>
>>> >> I would be delighted to welcome an independent, high quality Benchmark
>>> >> project back into OWASP :)
>>> >>
>>> >>
>>> >>
>>> >>>
>>> >>> Specially when recently we made David Rook remove this much more
>>> benign
>>> >>> 'commercial content' from OWASP
>>> >>>
>>> >>> Dinis
>>> >>>
>>> >>> On 30 November 2015 at 17:17, psiinon <psiinon at gmail.com> wrote:
>>> >>>>
>>> >>>> I'd like to start by saying that I actually _like_ the Benchmark
>>> >>>>project.
>>> >>>> Myself and other ZAP developers have made some contributions to it,
>>> >>>>and
>>> >>>> we have used (and will continue to use) it to make ZAP better.
>>> >>>> I think these sort of testing applications are very valuable to all
>>> >>>> security tools, and I'd like to thank Dave and his team for the
>>> >>>>significant
>>> >>>> amount of effort involved in developing and open sourcing it.
>>> >>>>
>>> >>>> But I dont think it should be an OWASP project.
>>> >>>> I do not think that a vendor led project can ever objectively
>>> evaluate
>>> >>>> competing commercial and open source projects.
>>> >>>> I do not think that just saying 'pull requests welcomed' makes a
>>> >>>>project
>>> >>>> vendor neutral.
>>> >>>> I do not think that a project as mired in controversy as the
>>> Benchmark
>>> >>>> project can ever recover to become truly independent.
>>> >>>>
>>> >>>> I am very disappointed in the Boards handling of this affair.
>>> >>>>
>>> >>>> Ideally I'd like Dave to understand how much damage this project has
>>> >>>>done
>>> >>>> and to withdraw it as an OWASP project, while still maintaining it
>>> as
>>> >>>>a very
>>> >>>> valuable vendor led open source resource.
>>> >>>>
>>> >>>> Failing that I really hope that the Board comes to its senses and
>>> >>>>ejects
>>> >>>> the Benchmark project before even more damage is done.
>>> >>>> At the _very_ least it should flag the project as being 'in dispute'
>>> >>>>(as
>>> >>>> Kevin suggested) while a more detailed evaluation is performed.
>>> >>>>
>>> >>>> However I'm rapidly loosing loosing faith that the Board will do the
>>> >>>> right thing and protect OWASP's image in the way that they should
>>> have
>>> >>>> already done.
>>> >>>> Members - please make your voices heard before more people and
>>> >>>>projects
>>> >>>> leave OWASP.
>>> >>>>
>>> >>>> Simon
>>> >>>>
>>> >>>> On Sat, Nov 28, 2015 at 5:14 AM, Jim Manico <jim.manico at owasp.org>
>>> >>>>wrote:
>>> >>>>>
>>> >>>>> WAFEC does not "do vendor assessment"; they define a comprehensive
>>> >>>>> standard built by many vendors and let the community use that
>>> >>>>>standard to
>>> >>>>> measure tools on their own. Just a FYI, I was involved in the early
>>> >>>>>version
>>> >>>>> of this project. (Things may have changed since my involvement, I'm
>>> >>>>>sure
>>> >>>>> Tony has more details here)
>>> >>>>>
>>> >>>>> Johanna's comments on this issue lead me to believe that the damage
>>> >>>>>done
>>> >>>>> to both OWASP and DHS is even more destructive that I thought. It
>>> >>>>>saddens me
>>> >>>>> to see this level of abuse just to sell product.
>>> >>>>>
>>> >>>>> --
>>> >>>>> Jim Manico
>>> >>>>> Global Board Member
>>> >>>>> OWASP Foundation
>>> >>>>> https://www.owasp.org
>>> >>>>> Join me in Rome for AppSecEU 2016!
>>> >>>>>
>>> >>>>> On Nov 28, 2015, at 2:40 AM, Josh Sokol <josh.sokol at owasp.org>
>>> wrote:
>>> >>>>>
>>> >>>>> One of the ideas that Andrew proposed was actually approaching
>>> WAFEC
>>> >>>>>to
>>> >>>>> learn more about how they do vendor assessment in a neutral way.
>>> >>>>>It's great
>>> >>>>> to hear that we have a resource here already that we can leverage.
>>> >>>>>I wasn't
>>> >>>>> aware of your affiliation.
>>> >>>>>
>>> >>>>> ~josh
>>> >>>>>
>>> >>>>> On Nov 27, 2015 2:47 PM, "Tony Turner" <tony.turner at owasp.org>
>>> wrote:
>>> >>>>>>
>>> >>>>>> I sincerely hope so. That's not the impression I got from others
>>> >>>>>> comments. Personally I haven't used the tool at all, but as I'm
>>> the
>>> >>>>>>project
>>> >>>>>> lead for another product evaluation project (WAFEC) I'm very
>>> >>>>>>sensitive to
>>> >>>>>> the need of collaboration with many different vendors. There
>>> really
>>> >>>>>>has to
>>> >>>>>> be a very high level (almost paranoid level) transparency with how
>>> >>>>>>vendors
>>> >>>>>> are approached, worked with, how requirements for evaluation are
>>> >>>>>>defined,
>>> >>>>>> and how metrics are derived.
>>> >>>>>>
>>> >>>>>> It appears the project team is attempting to address these last 2
>>> >>>>>> somewhat but I'd like to see more specifics, and the lack of
>>> >>>>>>information on
>>> >>>>>> how they are addressing vendor communication, participation and
>>> >>>>>>transparency
>>> >>>>>> seems a bit concerning. Lastly, it is my opinion that project
>>> >>>>>>leadership
>>> >>>>>> should not belong to anyone working for or with a
>>> >>>>>>partnership/ownership
>>> >>>>>> stake for any vendor being evaluated. I think this is a flawed
>>> >>>>>>model and
>>> >>>>>> should transition to a vendor neutral party.
>>> >>>>>>
>>> >>>>>> On Nov 27, 2015 3:16 PM, "Josh Sokol" <josh.sokol at owasp.org>
>>> wrote:
>>> >>>>>>>
>>> >>>>>>> I don't know what qualifies as "significant" in your mind, but my
>>> >>>>>>> understanding is that there have been contributions from other
>>> >>>>>>>vendors:
>>> >>>>>>>
>>> >>>>>>> https://www.owasp.org/index.php/Benchmark#tab=Acknowledgements
>>> >>>>>>>
>>> >>>>>>> Still, Dave would like more, but he can't force them to help.
>>> >>>>>>>
>>> >>>>>>> ~josh
>>> >>>>>>>
>>> >>>>>>> On Fri, Nov 27, 2015 at 1:45 PM, Tony Turner
>>> >>>>>>><tony.turner at owasp.org>
>>> >>>>>>> wrote:
>>> >>>>>>>>
>>> >>>>>>>> While I can appreciate that they started with Contrast, if there
>>> >>>>>>>> hasn't been significant effort to include other vendors it's a
>>> >>>>>>>>worthless
>>> >>>>>>>> benchmark. It's easy to state you haven't gotten support from
>>> >>>>>>>>other vendors
>>> >>>>>>>> and that's fine, but until you do there's really nothing to
>>> >>>>>>>>release. Why was
>>> >>>>>>>> it ever upgraded? Talking about the results without an accurate
>>> >>>>>>>>comparative
>>> >>>>>>>> analysis is akin to snake oil.
>>> >>>>>>>>
>>> >>>>>>>> On Nov 27, 2015 1:49 PM, "Josh Sokol" <josh.sokol at owasp.org>
>>> >>>>>>>>wrote:
>>> >>>>>>>>>
>>> >>>>>>>>> Thank you for the links to those articles.  The first one
>>> >>>>>>>>>discusses
>>> >>>>>>>>> the strengths and weaknesses of the different methods of
>>> >>>>>>>>>evaluating for
>>> >>>>>>>>> application vulnerabilities.  The section on the Benchmark
>>> seems
>>> >>>>>>>>>wholly
>>> >>>>>>>>> appropriate to me.  That seems like an excellent description of
>>> >>>>>>>>>what the
>>> >>>>>>>>> project is designed to do.  I see some metrics in there about
>>> >>>>>>>>>which tools
>>> >>>>>>>>> are more effective on which types of vulnerabilities, but I
>>> >>>>>>>>>don't see him
>>> >>>>>>>>> straight up saying "The OWASP Benchmark proves that Contrast is
>>> >>>>>>>>>better".
>>> >>>>>>>>> This seems like statements made based on some level of testing
>>> >>>>>>>>>and research.
>>> >>>>>>>>> Honestly, I don't see any OWASP brand abuse in that article.
>>> >>>>>>>>>Whether it's
>>> >>>>>>>>> in good taste or not at this stage in the project is certainly
>>> >>>>>>>>>debatable,
>>> >>>>>>>>> but if you look at the brand usage guidelines
>>> >>>>>>>>>
>>> >>>>>>>>>(
>>> https://www.owasp.org/index.php/Marketing/Resources#tab=BRAND_GUI
>>> >>>>>>>>>DELINES),
>>> >>>>>>>>> I don't see any violations.  We need to govern to policy here
>>> >>>>>>>>>which is why
>>> >>>>>>>>> Paul and Noreen are evaluating changes to the guidelines and
>>> our
>>> >>>>>>>>>enforcement
>>> >>>>>>>>> policies to make abuse more difficult.
>>> >>>>>>>>>
>>> >>>>>>>>> The second article is a competing vendor's reaction to the
>>> first.
>>> >>>>>>>>> He makes some good points about the issues with Benchmark, but
>>> >>>>>>>>>he also says
>>> >>>>>>>>> that he hopes that it will be improved over time, and Dave has
>>> >>>>>>>>>committed to
>>> >>>>>>>>> that.  What I don't see is the vendor saying "...and Veracode
>>> >>>>>>>>>has committed
>>> >>>>>>>>> resources to help make the Benchmark more accurate across all
>>> >>>>>>>>>tool sets".
>>> >>>>>>>>> The Benchmark page is pretty clear that it does it's best to
>>> >>>>>>>>>provide a
>>> >>>>>>>>> benchmark without working exactly like a real-world
>>> application.
>>> >>>>>>>>> Maybe some
>>> >>>>>>>>> more disclaimer text about where the project is at today would
>>> >>>>>>>>>be in order
>>> >>>>>>>>> to validate some of Chris' concerns, but I hardly see this as
>>> >>>>>>>>>"brand abuse"
>>> >>>>>>>>> or a reason to demote the project.
>>> >>>>>>>>>
>>> >>>>>>>>> Please consider that I have spoken with both Dave and Jeff on
>>> >>>>>>>>>this
>>> >>>>>>>>> topic and read much of the discussions around it before
>>> >>>>>>>>>formulating my
>>> >>>>>>>>> opinion.  I doubt that you have done the same so I'm not sure
>>> >>>>>>>>>how you can
>>> >>>>>>>>> claim that you have researched the issues and all parties
>>> >>>>>>>>>involved when you
>>> >>>>>>>>> haven't even spoken with the two people whom you are accusing
>>> of
>>> >>>>>>>>> impropriety.  I have no bias here.  I am simply speaking with
>>> the
>>> >>>>>>>>> individuals involved, looking at the currently OWASP policies
>>> and
>>> >>>>>>>>> guidelines, and helping to determine our next steps.
>>> >>>>>>>>>
>>> >>>>>>>>> ~josh
>>> >>>>>>>>>
>>> >>>>>>>>> On Fri, Nov 27, 2015 at 12:22 PM, johanna curiel curiel
>>> >>>>>>>>> <johanna.curiel at owasp.org> wrote:
>>> >>>>>>>>>>
>>> >>>>>>>>>> >>While I agree with you that there has been some brand abuse,
>>> >>>>>>>>>>it
>>> >>>>>>>>>> >> was abuse by Contrast (specifically their marketing
>>> >>>>>>>>>>department), and not by
>>> >>>>>>>>>> >> "these gentlemen" as  you state.
>>> >>>>>>>>>>
>>> >>>>>>>>>> Really? ..'some brand abuse'..this is more than brand abuse
>>> >>>>>>>>>>
>>> >>>>>>>>>> Josh , please read also the article written by Jeff
>>> >>>>>>>>>>
>>> >>>>>>>>>>
>>> >>>>>>>>>>
>>> http://www.darkreading.com/vulnerabilities---threats/why-its-insa
>>> >>>>>>>>>>ne-to-trust-static-analysis/a/d-id/1322274?
>>> >>>>>>>>>>
>>> >>>>>>>>>> And Veracode's reaction including others in Twitter
>>> >>>>>>>>>>
>>> >>>>>>>>>>
>>> >>>>>>>>>>
>>> https://www.veracode.com/blog/2015/09/no-one-technology-silver-bu
>>> >>>>>>>>>>llet
>>> >>>>>>>>>>
>>> >>>>>>>>>> My strong advice is to research the issues and all the parties
>>> >>>>>>>>>> involved before making statements
>>> >>>>>>>>>>
>>> >>>>>>>>>>
>>> >>>>>>>>>>
>>> >>>>>>>>>>
>>> >>>>>>>>>> On Fri, Nov 27, 2015 at 2:07 PM, Josh Sokol
>>> >>>>>>>>>><josh.sokol at owasp.org>
>>> >>>>>>>>>> wrote:
>>> >>>>>>>>>>>
>>> >>>>>>>>>>> Jim,
>>> >>>>>>>>>>>
>>> >>>>>>>>>>> A concern was expressed to the Board and, frankly, I am
>>> >>>>>>>>>>>insulted
>>> >>>>>>>>>>> by you saying that this was "brushed under the rug".  The
>>> >>>>>>>>>>>Board delegated
>>> >>>>>>>>>>> Matt to talk with Dave and they had a lengthy conversation on
>>> >>>>>>>>>>>the subject.
>>> >>>>>>>>>>> The Board delegated me to talk with Jeff and we had a lengthy
>>> >>>>>>>>>>>conversation
>>> >>>>>>>>>>> on the subject.  If you do not trust in our abilities to read
>>> >>>>>>>>>>>people, ask
>>> >>>>>>>>>>> the right questions, and provide honest feedback about our
>>> >>>>>>>>>>>conversations,
>>> >>>>>>>>>>> then that's a bigger issue that we should take offline.
>>> After
>>> >>>>>>>>>>>our
>>> >>>>>>>>>>> conversations, we took the time to call a special two-hour
>>> >>>>>>>>>>>session of the
>>> >>>>>>>>>>> Board in order to discuss this subject (and only this
>>> >>>>>>>>>>>subject).  We spoke
>>> >>>>>>>>>>> about all facets of the issue at hand, about the challenges
>>> >>>>>>>>>>>and possible
>>> >>>>>>>>>>> solutions, and concluded on some very concrete next steps.
>>> >>>>>>>>>>>
>>> >>>>>>>>>>> While I agree with you that there has been some brand abuse,
>>> it
>>> >>>>>>>>>>> was abuse by Contrast (specifically their marketing
>>> >>>>>>>>>>>department), and not by
>>> >>>>>>>>>>> "these gentlemen" as  you state.  Unless you can point to
>>> some
>>> >>>>>>>>>>>sort of
>>> >>>>>>>>>>> evidence showing that Jeff and/or Dave first-hand abused the
>>> >>>>>>>>>>>brand, then I
>>> >>>>>>>>>>> believe that you are speaking with your heart instead of with
>>> >>>>>>>>>>>your head.  I
>>> >>>>>>>>>>> appreciate your passion, but I label this as conspiracy
>>> theory
>>> >>>>>>>>>>>because
>>> >>>>>>>>>>> without evidence to support your claims, I cannot accept it
>>> as
>>> >>>>>>>>>>>anything
>>> >>>>>>>>>>> other.
>>> >>>>>>>>>>>
>>> >>>>>>>>>>> ~josh
>>> >>>>>>>>>>>
>>> >>>>>>>>>>> On Fri, Nov 27, 2015 at 11:39 AM, Jim Manico
>>> >>>>>>>>>>> <jim.manico at owasp.org> wrote:
>>> >>>>>>>>>>>>
>>> >>>>>>>>>>>> Josh,
>>> >>>>>>>>>>>>
>>> >>>>>>>>>>>> I stand by my comments and perspective, but I'm disheartened
>>> >>>>>>>>>>>>that
>>> >>>>>>>>>>>> you consider my presentation of facts (and the concerns of
>>> >>>>>>>>>>>>many active
>>> >>>>>>>>>>>> members of our community) as a "conspiracy theory".
>>> >>>>>>>>>>>>
>>> >>>>>>>>>>>> In my experience, these kind of comments border on insults
>>> and
>>> >>>>>>>>>>>> only cause folks to harden their opinions.
>>> >>>>>>>>>>>>
>>> >>>>>>>>>>>> Once again I feel these gentlemen got away with a kind of
>>> >>>>>>>>>>>>brand
>>> >>>>>>>>>>>> abuse that is very hurtful to the OWASP community but I am
>>> at
>>> >>>>>>>>>>>>a loss as to
>>> >>>>>>>>>>>> how handle or prevent these kinds of mishaps - especially
>>> >>>>>>>>>>>>when board members
>>> >>>>>>>>>>>> like yourself seem willing to - from what I see - brush it
>>> >>>>>>>>>>>>under the rug.
>>> >>>>>>>>>>>>
>>> >>>>>>>>>>>> --
>>> >>>>>>>>>>>> Jim Manico
>>> >>>>>>>>>>>> Global Board Member
>>> >>>>>>>>>>>> OWASP Foundation
>>> >>>>>>>>>>>> https://www.owasp.org
>>> >>>>>>>>>>>> Join me in Rome for AppSecEU 2016!
>>> >>>>>>>>>>>>
>>> >>>>>>>>>>>> On Nov 27, 2015, at 7:23 PM, Josh Sokol <
>>> josh.sokol at owasp.org>
>>> >>>>>>>>>>>> wrote:
>>> >>>>>>>>>>>>
>>> >>>>>>>>>>>> Admittedly, this was my gut reaction at first as well.  I
>>> >>>>>>>>>>>>began
>>> >>>>>>>>>>>> linking all of these companies, people, and projects
>>> together
>>> >>>>>>>>>>>>in my mind
>>> >>>>>>>>>>>> (there are some loose links there) and painted a big
>>> >>>>>>>>>>>>conspiracy picture
>>> >>>>>>>>>>>> similar to what Jim and Dinis have stated.  But, after
>>> >>>>>>>>>>>>speaking directly
>>> >>>>>>>>>>>> with Jeff, and hearing about the conversation that Dave and
>>> >>>>>>>>>>>>Matt had, I've
>>> >>>>>>>>>>>> changed my mind.
>>> >>>>>>>>>>>>
>>> >>>>>>>>>>>> I think it begins with the project itself.  If you aren't
>>> >>>>>>>>>>>>sold on
>>> >>>>>>>>>>>> the idea of the Benchmark, then you'll never be able to get
>>> >>>>>>>>>>>>to the same
>>> >>>>>>>>>>>> place.  My original line of thinking was that it was just a
>>> >>>>>>>>>>>>bar for vendors
>>> >>>>>>>>>>>> to compare their tools against eachother, but that's a bit
>>> >>>>>>>>>>>>myopic.  We are
>>> >>>>>>>>>>>> in an industry where things evolve very quickly.  As a
>>> >>>>>>>>>>>>customer of these
>>> >>>>>>>>>>>> tools, I know firsthand that something that a tool does
>>> today
>>> >>>>>>>>>>>>may not be the
>>> >>>>>>>>>>>> case a week from now.  Likewise, new features are being
>>> added
>>> >>>>>>>>>>>>daily and I
>>> >>>>>>>>>>>> need a point-in-time metric to be able to gauge continual
>>> >>>>>>>>>>>>effectiveness.
>>> >>>>>>>>>>>> Cool, right?  But not a game changer.  The game changer part
>>> >>>>>>>>>>>>comes when you
>>> >>>>>>>>>>>> realize that by developing and evolving the tests that go
>>> >>>>>>>>>>>>into the
>>> >>>>>>>>>>>> Benchmark, we are moving the bar higher and higher.  We
>>> >>>>>>>>>>>>(OWASP) are
>>> >>>>>>>>>>>> effectively setting the standard by which these tools will
>>> be
>>> >>>>>>>>>>>>compared.  A
>>> >>>>>>>>>>>> tool that receives a lower score on the Benchmark today
>>> knows
>>> >>>>>>>>>>>>exactly what
>>> >>>>>>>>>>>> they need to work on in order to pass that test tomorrow and
>>> >>>>>>>>>>>>we already have
>>> >>>>>>>>>>>> examples of tools that have made improvements because of
>>> >>>>>>>>>>>>their Benchmark
>>> >>>>>>>>>>>> score (Ask Simon about ZAP's experience with the Benchmark).
>>> >>>>>>>>>>>>I don't think
>>> >>>>>>>>>>>> that anyone can argue that the Benchmark project isn't being
>>> >>>>>>>>>>>>effective when
>>> >>>>>>>>>>>> OWASP's own tools are being driven forward as a result of
>>> >>>>>>>>>>>>using it.
>>> >>>>>>>>>>>>
>>> >>>>>>>>>>>> But, but, but, Dave and Jeff own Aspect and have stock in
>>> >>>>>>>>>>>> Contrast and Jeff is the Contrast CTO and Contrast got good
>>> >>>>>>>>>>>>scores so it's a
>>> >>>>>>>>>>>> conspiracy right?  Is there some code that allows Contrast
>>> to
>>> >>>>>>>>>>>>use the
>>> >>>>>>>>>>>> Benchmark?  Absolutely.  Can you really blame Dave for
>>> >>>>>>>>>>>>starting his testing
>>> >>>>>>>>>>>> on the effectiveness of the Benchmark with a tool that he
>>> >>>>>>>>>>>>owned and is
>>> >>>>>>>>>>>> familiar with?  If I were going to start a similar project,
>>> >>>>>>>>>>>>there's no
>>> >>>>>>>>>>>> question in my mind that I would begin my testing with the
>>> >>>>>>>>>>>>tools that I have
>>> >>>>>>>>>>>> available to me.  That said, is there code that allows other
>>> >>>>>>>>>>>>tools to use
>>> >>>>>>>>>>>> the Benchmark?  Absolutely.
>>> >>>>>>>>>>>>
>>> >>>>>>>>>>>> Regarding "Dave has a history of breaching his duty to be
>>> >>>>>>>>>>>>vendor
>>> >>>>>>>>>>>> neutral", while I cannot comment on his past actions, I can
>>> >>>>>>>>>>>>judge what we've
>>> >>>>>>>>>>>> seen recently.  Matt saw a presentation from Dave on the
>>> >>>>>>>>>>>>Benchmark at a
>>> >>>>>>>>>>>> conference in Chicago.  He said that he felt that the
>>> message
>>> >>>>>>>>>>>>was
>>> >>>>>>>>>>>> appropriate and while IAST tools were mentioned as receiving
>>> >>>>>>>>>>>>higher scores,
>>> >>>>>>>>>>>> it wasn't a "Contrast is the best" type of message, more of
>>> a
>>> >>>>>>>>>>>>generality.  I
>>> >>>>>>>>>>>> saw a very similar (if not the same) talk by Jeff at LASCON
>>> >>>>>>>>>>>>2015 and the
>>> >>>>>>>>>>>> message was exactly the same.  I watched the talk expecting
>>> >>>>>>>>>>>>some sort of
>>> >>>>>>>>>>>> impropriety, but found none.  So, perhaps Dave has abused
>>> >>>>>>>>>>>>some privilege
>>> >>>>>>>>>>>> granted to him in the past, but what I've seen from him at
>>> >>>>>>>>>>>>this point, with
>>> >>>>>>>>>>>> respect to the Benchmark, has been appropriate.
>>> >>>>>>>>>>>>
>>> >>>>>>>>>>>> You have a very good point with respect to the Contrast
>>> >>>>>>>>>>>>marketing
>>> >>>>>>>>>>>> message around the Benchmark.  It's been completely absurd,
>>> >>>>>>>>>>>>over the top,
>>> >>>>>>>>>>>> and, in my personal opinion, intolerable.  In fact, I
>>> >>>>>>>>>>>>experienced the same
>>> >>>>>>>>>>>> thing that you talked about with them at LASCON 2015 where
>>> >>>>>>>>>>>>they stood in
>>> >>>>>>>>>>>> front of the door of the room Jeff was speaking in and
>>> >>>>>>>>>>>>scanned attendees as
>>> >>>>>>>>>>>> they went into the talk.  I agree that these types of
>>> >>>>>>>>>>>>aggressive marketing
>>> >>>>>>>>>>>> tactics cannot be tolerated at OWASP.  In addition, we have
>>> >>>>>>>>>>>>seen several
>>> >>>>>>>>>>>> marketing messages from them effectively implying that OWASP
>>> >>>>>>>>>>>>endorses
>>> >>>>>>>>>>>> Contrast.  Clearly this is not OK.  I've spoken with Jeff
>>> >>>>>>>>>>>>about it and we
>>> >>>>>>>>>>>> agreed that it is not in the Benchmark's best interest to
>>> >>>>>>>>>>>>have this
>>> >>>>>>>>>>>> aggressive Contrast marketing around it at such an early
>>> >>>>>>>>>>>>stage.  He has said
>>> >>>>>>>>>>>> that he is not responsible for Contrast's marketing team,
>>> but
>>> >>>>>>>>>>>>that he would
>>> >>>>>>>>>>>> speak with the people who are.  I haven't seen a single
>>> >>>>>>>>>>>>message from them
>>> >>>>>>>>>>>> since so I'm guessing that he's made good on this promise.
>>> >>>>>>>>>>>>While that's an
>>> >>>>>>>>>>>> excellent start, OWASP's takeaway here should be that we
>>> need
>>> >>>>>>>>>>>>to do a better
>>> >>>>>>>>>>>> job with our brand usage guidelines both in terms of the
>>> >>>>>>>>>>>>wording and
>>> >>>>>>>>>>>> enforcement.  There are many other companies out there that
>>> >>>>>>>>>>>>use the OWASP
>>> >>>>>>>>>>>> brand and I think that we agree that selective enforcement
>>> >>>>>>>>>>>>against Contrast
>>> >>>>>>>>>>>> is not the right answer.  Paul and Noreen are actively
>>> >>>>>>>>>>>>working on this.
>>> >>>>>>>>>>>> Either way, I think that implying that activities from a
>>> >>>>>>>>>>>>vendor's marketing
>>> >>>>>>>>>>>> department means that the project is not objective is not
>>> >>>>>>>>>>>>inappropriate.  If
>>> >>>>>>>>>>>> we feel that the project is not objective, then separate
>>> >>>>>>>>>>>>measures need to be
>>> >>>>>>>>>>>> taken to drive contribution diversity into it.  That I
>>> >>>>>>>>>>>>absolutely agree with
>>> >>>>>>>>>>>> and the message from Dave was that he would love to have
>>> more
>>> >>>>>>>>>>>>contributors
>>> >>>>>>>>>>>> to his project.  But, seeing as we cannot force people to
>>> >>>>>>>>>>>>work on it, this
>>> >>>>>>>>>>>> becomes a matter of "put up or shut up".  The same goes for
>>> >>>>>>>>>>>>the experts that
>>> >>>>>>>>>>>> you said reviewed the code.  If they feel that it is somehow
>>> >>>>>>>>>>>>skewed towards
>>> >>>>>>>>>>>> Contrast, they have the power to change that.  Now, if
>>> >>>>>>>>>>>>someone tries to
>>> >>>>>>>>>>>> participate and Dave tells them "No thanks", then I agree we
>>> >>>>>>>>>>>>have a problem,
>>> >>>>>>>>>>>> but I don't hear anyone inferring that happened.
>>> >>>>>>>>>>>>
>>> >>>>>>>>>>>> Please, let's drop the conspiracy theories and focus on the
>>> >>>>>>>>>>>> tangible things that we can do to help an OWASP project to
>>> be
>>> >>>>>>>>>>>>more
>>> >>>>>>>>>>>> successful.  Help find more participants to drive diversity,
>>> >>>>>>>>>>>>update our
>>> >>>>>>>>>>>> brand usage guidelines to prevent abuse, enforce them
>>> widely,
>>> >>>>>>>>>>>>etc.  Thank
>>> >>>>>>>>>>>> you.
>>> >>>>>>>>>>>>
>>> >>>>>>>>>>>> ~josh
>>> >>>>>>>>>>>>
>>> >>>>>>>>>>>> On Thu, Nov 26, 2015 at 4:24 PM, Jim Manico
>>> >>>>>>>>>>>> <jim.manico at owasp.org> wrote:
>>> >>>>>>>>>>>>>
>>> >>>>>>>>>>>>> Dinis,
>>> >>>>>>>>>>>>>
>>> >>>>>>>>>>>>> Like a rare celestial moment when all the planets plus
>>> Pluto
>>> >>>>>>>>>>>>>are
>>> >>>>>>>>>>>>> aligned, I just read your email on the future of OWASP
>>> >>>>>>>>>>>>>projects thinking,
>>> >>>>>>>>>>>>> "Dinis is spot on".
>>> >>>>>>>>>>>>>
>>> >>>>>>>>>>>>> Reflecting on projects I manage or work on...
>>> >>>>>>>>>>>>>
>>> >>>>>>>>>>>>> The Java Encoder and HTML Sanitizer are likely best moved
>>> to
>>> >>>>>>>>>>>>> Apache now that they have reached a measure of adoption and
>>> >>>>>>>>>>>>>maturity. Apache
>>> >>>>>>>>>>>>> would be a much better long term custodian. Perhaps the
>>> same
>>> >>>>>>>>>>>>>for AppSensor,
>>> >>>>>>>>>>>>> but not my project - just thinking out loud.
>>> >>>>>>>>>>>>>
>>> >>>>>>>>>>>>> Other similar defensive projects are still being noodled
>>> on,
>>> >>>>>>>>>>>>>so
>>> >>>>>>>>>>>>> OWASP is a decent home for these research efforts.
>>> >>>>>>>>>>>>>
>>> >>>>>>>>>>>>> The whole tools category is also something to consider.
>>> >>>>>>>>>>>>> Dependency Check and of course ZAP are some of the best
>>> >>>>>>>>>>>>>projects that OWASP
>>> >>>>>>>>>>>>> offers, are they best served where they are today? Both
>>> have
>>> >>>>>>>>>>>>>rich
>>> >>>>>>>>>>>>> communities of developers but I don't see the foundation
>>> >>>>>>>>>>>>>doing much to
>>> >>>>>>>>>>>>> support these efforts.
>>> >>>>>>>>>>>>>
>>> >>>>>>>>>>>>> ASVS has the opportunity to effect massive change, I would
>>> to
>>> >>>>>>>>>>>>> love to see major investment and volunteer activity here.
>>> >>>>>>>>>>>>>Pro tech writer,
>>> >>>>>>>>>>>>> detailed discourses on each individual requirement, etc. If
>>> >>>>>>>>>>>>>I was king (and
>>> >>>>>>>>>>>>> I am not, at all) I would invest in ASVS on a 6 figure
>>> >>>>>>>>>>>>>scale. (And who
>>> >>>>>>>>>>>>> started ASVS? Jeff, Dave and Boberski, hat tip to such a
>>> >>>>>>>>>>>>>marvelous idea). Or
>>> >>>>>>>>>>>>> maybe moving ASVS to the W3C or IETF would help it grow?
>>> >>>>>>>>>>>>>
>>> >>>>>>>>>>>>> The Proactive Controls was a pet project but as we approach
>>> >>>>>>>>>>>>>2.0
>>> >>>>>>>>>>>>> we have several active/awesome volunteers working on it. We
>>> >>>>>>>>>>>>>will be making
>>> >>>>>>>>>>>>> the doc "world editable" to make contributions easy. OWASP
>>> >>>>>>>>>>>>>seems like a good
>>> >>>>>>>>>>>>> home for such an awareness doc. Same with T10, especially
>>> if
>>> >>>>>>>>>>>>>community edits
>>> >>>>>>>>>>>>> are welcome.
>>> >>>>>>>>>>>>>
>>> >>>>>>>>>>>>> Anyhow, I'm with you on this Dinis. Once a project starts
>>> to
>>> >>>>>>>>>>>>> reach production quality, spinning off the project as an
>>> >>>>>>>>>>>>>external project or
>>> >>>>>>>>>>>>> moving it to a different foundation where managing
>>> >>>>>>>>>>>>>production software or
>>> >>>>>>>>>>>>> formal standards is their thing seems realistic.
>>> >>>>>>>>>>>>>
>>> >>>>>>>>>>>>> I don't have all the answers here, but your email certainly
>>> >>>>>>>>>>>>> resonated with me.
>>> >>>>>>>>>>>>>
>>> >>>>>>>>>>>>> Aloha,
>>> >>>>>>>>>>>>> --
>>> >>>>>>>>>>>>> Jim Manico
>>> >>>>>>>>>>>>> Global Board Member
>>> >>>>>>>>>>>>> OWASP Foundation
>>> >>>>>>>>>>>>> https://www.owasp.org
>>> >>>>>>>>>>>>> Join me in Rome for AppSecEU 2016!
>>> >>>>>>>>>>>>>
>>> >>>>>>>>>>>>> On Nov 26, 2015, at 11:26 PM, Dinis Cruz
>>> >>>>>>>>>>>>><dinis.cruz at owasp.org>
>>> >>>>>>>>>>>>> wrote:
>>> >>>>>>>>>>>>>
>>> >>>>>>>>>>>>> Jim's reading of this situation is exactly my view on the
>>> >>>>>>>>>>>>>value
>>> >>>>>>>>>>>>> of the Contrast tool and how it has been 'pushing' the
>>> rules
>>> >>>>>>>>>>>>>of engagement
>>> >>>>>>>>>>>>> to an very 'fuzzy' moral/ethical/commercial limit :)
>>> >>>>>>>>>>>>>
>>> >>>>>>>>>>>>> As per my last email, a key problem here is the 'perceived
>>> >>>>>>>>>>>>> expectation' of what is an OWASP project, and how it should
>>> >>>>>>>>>>>>>be consumed.
>>> >>>>>>>>>>>>>
>>> >>>>>>>>>>>>> If you look at the OWASP benchmark as a research project,
>>> >>>>>>>>>>>>>then
>>> >>>>>>>>>>>>> the only way it could be making the kind of claims it makes
>>> >>>>>>>>>>>>>(and have
>>> >>>>>>>>>>>>> credibility) is if it had evolved from OWASP, with its own
>>> >>>>>>>>>>>>>(diverse)
>>> >>>>>>>>>>>>> community
>>> >>>>>>>>>>>>>
>>> >>>>>>>>>>>>> On 26 November 2015 at 21:01, Jim Manico
>>> >>>>>>>>>>>>><jim.manico at owasp.org>
>>> >>>>>>>>>>>>> wrote:
>>> >>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>> I have a different take on this situation but my opinion
>>> is
>>> >>>>>>>>>>>>>>the
>>> >>>>>>>>>>>>>> "minority opinion". I will respect the rest of the boards
>>> >>>>>>>>>>>>>>take on this, but
>>> >>>>>>>>>>>>>> here is how I see it.
>>> >>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>> First of all, Jeff has stated that he feels I am attacking
>>> >>>>>>>>>>>>>>him
>>> >>>>>>>>>>>>>> personally from a past personal grudge, and frankly I do
>>> >>>>>>>>>>>>>>not fault him for
>>> >>>>>>>>>>>>>> that perspective since we definitely have history with
>>> >>>>>>>>>>>>>>conflict. So it's
>>> >>>>>>>>>>>>>> fair to take my opinion on this with a grain of salt.
>>> >>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>> I look at this situation from the perspective of a
>>> forensic
>>> >>>>>>>>>>>>>> investigator.
>>> >>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>> 1) The Benchmark project had Contrast hooks and only
>>> >>>>>>>>>>>>>>Contrast
>>> >>>>>>>>>>>>>> hooks in it when I reviewed it so this leads me to believe
>>> >>>>>>>>>>>>>>that the project
>>> >>>>>>>>>>>>>> was clearly built with Contrast in mind from the ground
>>> up,
>>> >>>>>>>>>>>>>>at least in some
>>> >>>>>>>>>>>>>> way.
>>> >>>>>>>>>>>>>> 3) Dave has a history of breaching his duty to be vendor
>>> >>>>>>>>>>>>>> neutral. He was gifted with a keynote in South Korea a few
>>> >>>>>>>>>>>>>>years ago, and
>>> >>>>>>>>>>>>>> used that opportunity to discuss and pitch Contrast, on
>>> >>>>>>>>>>>>>>stage, during a
>>> >>>>>>>>>>>>>> keynote - with Contrast specific slides. This is just
>>> >>>>>>>>>>>>>>supporting evidence of
>>> >>>>>>>>>>>>>> his intention at OWASP to push Contrast in ways that I
>>> >>>>>>>>>>>>>>think are against the
>>> >>>>>>>>>>>>>> intentions and goals of our foundation.
>>> >>>>>>>>>>>>>> 3) Other experts have reviewed the project and felt that
>>> >>>>>>>>>>>>>>many
>>> >>>>>>>>>>>>>> of the tests were very slanted and almost contrived to
>>> >>>>>>>>>>>>>>support Contrast. I
>>> >>>>>>>>>>>>>> can drag those folks into this conversation, but I do not
>>> >>>>>>>>>>>>>>think that would
>>> >>>>>>>>>>>>>> help in any way. So it's fair to call this point heresy.
>>> >>>>>>>>>>>>>> 4) I do not see this project as revolutionary, at all.
>>> Every
>>> >>>>>>>>>>>>>> vendor has their own test suite tuned for their tool. As
>>> >>>>>>>>>>>>>>the benchmark
>>> >>>>>>>>>>>>>> stands today, I see it as just another vendors
>>> >>>>>>>>>>>>>>product-specific benchmark.
>>> >>>>>>>>>>>>>> Mass collaboration from many vendors is not just a "nice
>>> to
>>> >>>>>>>>>>>>>>have" but a base
>>> >>>>>>>>>>>>>> requirement to get even close to useful for objective tool
>>> >>>>>>>>>>>>>>measurement.
>>> >>>>>>>>>>>>>> 5) Jeff stating that his Marketing people went over the
>>> >>>>>>>>>>>>>>line is
>>> >>>>>>>>>>>>>> also an admission that - well, they went over the line. By
>>> >>>>>>>>>>>>>>the same token
>>> >>>>>>>>>>>>>> Jeff was in his booth at AppSec USA surrounded by
>>> benchmark
>>> >>>>>>>>>>>>>>marketing
>>> >>>>>>>>>>>>>> material, discussing this to prospects and he even asked
>>> me
>>> >>>>>>>>>>>>>>and Mr Coates to
>>> >>>>>>>>>>>>>> wade into this debate and support Dave. So to say he was
>>> >>>>>>>>>>>>>>not involved and it
>>> >>>>>>>>>>>>>> was only his marketing people seems a stretch at best.
>>> >>>>>>>>>>>>>> 6) The Contrast marketing team was wandering around the
>>> >>>>>>>>>>>>>> conference zapping folks to get leads, and I asked them to
>>> >>>>>>>>>>>>>>stay in their
>>> >>>>>>>>>>>>>> booth, which is standard conference policy. These folks
>>> >>>>>>>>>>>>>>know better but are
>>> >>>>>>>>>>>>>> again going over the line to sell product at OWASP. There
>>> >>>>>>>>>>>>>>is a better way
>>> >>>>>>>>>>>>>> (like focusing on product capability and language support,
>>> >>>>>>>>>>>>>>have consistent +
>>> >>>>>>>>>>>>>> stellar customer service, have a humble and gracious
>>> >>>>>>>>>>>>>>attitude to all
>>> >>>>>>>>>>>>>> prospects and customers, actively participate in OWASP in
>>> a
>>> >>>>>>>>>>>>>>vendor neutral
>>> >>>>>>>>>>>>>> and community supportive way, etc).
>>> >>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>> Please note, I think Contrast is a decent tool, I've
>>> >>>>>>>>>>>>>>offered to
>>> >>>>>>>>>>>>>> resell in the past, and I have recommended it in certain
>>> >>>>>>>>>>>>>>situations - even
>>> >>>>>>>>>>>>>> after this situation arose. I'm stating this out of
>>> >>>>>>>>>>>>>>honestly and desire to
>>> >>>>>>>>>>>>>> put my cards on the table. I truly want Jeff and Dave to
>>> be
>>> >>>>>>>>>>>>>>successful. They
>>> >>>>>>>>>>>>>> have dedicated their lives to AppSec and if anyone should
>>> >>>>>>>>>>>>>>win big-time, I
>>> >>>>>>>>>>>>>> hope it's them. I even told Jeff I hope he hits the mother
>>> >>>>>>>>>>>>>>load and donates
>>> >>>>>>>>>>>>>> a little back to OWASP.
>>> >>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>> However, my instinct and evidence tell me that they both
>>> >>>>>>>>>>>>>>went
>>> >>>>>>>>>>>>>> over the line in the use of the OWASP brand to sell
>>> product.
>>> >>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>> Now, Jeff makes a good point. We as a board and staff are
>>> >>>>>>>>>>>>>>very
>>> >>>>>>>>>>>>>> poor at enforcing brand management policy and it's not
>>> fair
>>> >>>>>>>>>>>>>>to single out
>>> >>>>>>>>>>>>>> Contrast, when many other vendors violate the brand, IMO.
>>> >>>>>>>>>>>>>>Just google OWASP
>>> >>>>>>>>>>>>>> and watch the ads fly that use the OWASP name to sell
>>> >>>>>>>>>>>>>>product.
>>> >>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>> Also, any and every request that was made of Dave to
>>> adjust
>>> >>>>>>>>>>>>>>the
>>> >>>>>>>>>>>>>> project for the sake of vendor neutrality was taken very
>>> >>>>>>>>>>>>>>seriously.
>>> >>>>>>>>>>>>>> Regardless of Daves past intentions, he is clearly trying
>>> >>>>>>>>>>>>>>to do the right
>>> >>>>>>>>>>>>>> thing moving forward.
>>> >>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>> I look to "postels principle" in this situation (this is
>>> >>>>>>>>>>>>>> otherwise known as the "robustness principle" and dates
>>> >>>>>>>>>>>>>>back to the creation
>>> >>>>>>>>>>>>>> of TCP) . This is paraphrased as, "Be liberal in what you
>>> >>>>>>>>>>>>>>take from others
>>> >>>>>>>>>>>>>> but be conservative in what you dish out". So I think it's
>>> >>>>>>>>>>>>>>critical that
>>> >>>>>>>>>>>>>> OWASP and any OWASP resource present itself in a strict
>>> >>>>>>>>>>>>>>vendor neutral way.
>>> >>>>>>>>>>>>>> But unless OWASP wants to be much more "even" in the
>>> >>>>>>>>>>>>>>enforcement of brand
>>> >>>>>>>>>>>>>> policy across the board to all violators, we should be
>>> >>>>>>>>>>>>>>fairly lax in the
>>> >>>>>>>>>>>>>> enforcement of these issues from the outside world.
>>> >>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>> I am trying to be objective here. My trigonometry teacher
>>> >>>>>>>>>>>>>>once
>>> >>>>>>>>>>>>>> told me "I'd fail my mother" when I asked him if he would
>>> >>>>>>>>>>>>>>ever fail me (I
>>> >>>>>>>>>>>>>> was an A student). If my mother owned a security company
>>> >>>>>>>>>>>>>>and tried the same
>>> >>>>>>>>>>>>>> stunt, I'd have the same opinions about her actions as
>>> well.
>>> >>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>> So what next? Well hello from the other side. I'm going
>>> >>>>>>>>>>>>>>back to
>>> >>>>>>>>>>>>>> listening to Adele's new album where I can sit in my deep
>>> >>>>>>>>>>>>>>feelings and
>>> >>>>>>>>>>>>>> reflect upon what the OWASP foundation has done to enrich
>>> >>>>>>>>>>>>>>my life. I would
>>> >>>>>>>>>>>>>> much rather keep out of this (and any other conflict laden
>>> >>>>>>>>>>>>>>situation at
>>> >>>>>>>>>>>>>> OWASP), but I feel it's my responsibility to speak up.
>>> >>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>> Aloha,
>>> >>>>>>>>>>>>>> --
>>> >>>>>>>>>>>>>> Jim Manico
>>> >>>>>>>>>>>>>> Global Board Member
>>> >>>>>>>>>>>>>> OWASP Foundation
>>> >>>>>>>>>>>>>> https://www.owasp.org
>>> >>>>>>>>>>>>>> Join me in Rome for AppSecEU 2016!
>>> >>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>> On Nov 26, 2015, at 9:09 PM, Josh Sokol
>>> >>>>>>>>>>>>>><josh.sokol at owasp.org>
>>> >>>>>>>>>>>>>> wrote:
>>> >>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>> I would be happy to provide an update.
>>> >>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>> Matt Konda and Dave Wichers, the Benchmark Project Leader,
>>> >>>>>>>>>>>>>>had
>>> >>>>>>>>>>>>>> a conversation a few weeks back.  To summarize their
>>> >>>>>>>>>>>>>>conversation, Dave
>>> >>>>>>>>>>>>>> acknowledges the currently lack of diversity in his
>>> project
>>> >>>>>>>>>>>>>>and it is his
>>> >>>>>>>>>>>>>> sincere desire to drive more people to it to help.  He
>>> also
>>> >>>>>>>>>>>>>>acknowledges the
>>> >>>>>>>>>>>>>> issues with Contrast's extreme marketing around the
>>> project
>>> >>>>>>>>>>>>>>and feels that
>>> >>>>>>>>>>>>>> it is in everyone's best interests for them to curb it
>>> >>>>>>>>>>>>>>back.  While he does
>>> >>>>>>>>>>>>>> have an ownership stake in Contrast, he works at Aspect
>>> and
>>> >>>>>>>>>>>>>>has no control
>>> >>>>>>>>>>>>>> over the marketing messages that they are putting out
>>> >>>>>>>>>>>>>>there.  From the Board
>>> >>>>>>>>>>>>>> perspective, there has been no evidence of any impropriety
>>> >>>>>>>>>>>>>>on Dave's part
>>> >>>>>>>>>>>>>> and it should be our goal to drive more diversity into the
>>> >>>>>>>>>>>>>>project to
>>> >>>>>>>>>>>>>> support Dave.  Dave appears to be sincere in his desires
>>> to
>>> >>>>>>>>>>>>>>create a tool
>>> >>>>>>>>>>>>>> where OWASP can tell vendors what we expect from their
>>> >>>>>>>>>>>>>>tools.  If the main
>>> >>>>>>>>>>>>>> issue is that only members of Aspect are working on it,
>>> >>>>>>>>>>>>>>then the best thing
>>> >>>>>>>>>>>>>> that we can do is try to get him some outside assistance.
>>> >>>>>>>>>>>>>>We are also
>>> >>>>>>>>>>>>>> asking that the project be opened up to commits via Git so
>>> >>>>>>>>>>>>>>that outsiders
>>> >>>>>>>>>>>>>> can push commits to it.
>>> >>>>>>>>>>>>>> Josh Sokol and Jeff Williams, the CTO of Contrast, had a
>>> >>>>>>>>>>>>>> conversation a few weeks back.  To summarize their
>>> >>>>>>>>>>>>>>conversation, Jeff
>>> >>>>>>>>>>>>>> believes that the work that Dave is doing on the Benchmark
>>> >>>>>>>>>>>>>>is a game changer
>>> >>>>>>>>>>>>>> in that it gives OWASP the power in dictating what these
>>> >>>>>>>>>>>>>>tools need to be
>>> >>>>>>>>>>>>>> finding.  He wants the Benchmark to be successful and
>>> >>>>>>>>>>>>>>understands that it
>>> >>>>>>>>>>>>>> needs to be diverse in order to be trusted.  He recognizes
>>> >>>>>>>>>>>>>>that Dave is
>>> >>>>>>>>>>>>>> trying to do that and does not want the marketing message
>>> >>>>>>>>>>>>>>from Contrast to
>>> >>>>>>>>>>>>>> interfere with his efforts.  Jeff felt that the "Lab"
>>> >>>>>>>>>>>>>>status granted to
>>> >>>>>>>>>>>>>> Benchmark meant that it was ready for mainstream adoption,
>>> >>>>>>>>>>>>>>that it had 21k
>>> >>>>>>>>>>>>>> tests, and was almost a year old, and didn't see anything
>>> >>>>>>>>>>>>>>wrong with
>>> >>>>>>>>>>>>>> marketing their results, but has agreed to talk to their
>>> >>>>>>>>>>>>>>marketing team to
>>> >>>>>>>>>>>>>> get them to lay off that message for now.  From the Board
>>> >>>>>>>>>>>>>>perspective, we
>>> >>>>>>>>>>>>>> have come to the realization that our brand usage
>>> >>>>>>>>>>>>>>guidelines need an
>>> >>>>>>>>>>>>>> overhaul to clarify what is and is not allowed.  We have
>>> >>>>>>>>>>>>>>made a few
>>> >>>>>>>>>>>>>> proposals and have reached out to Mozilla to gain more
>>> >>>>>>>>>>>>>>insight on their
>>> >>>>>>>>>>>>>> guidelines and even ask for assistance.  Noreen and Paul
>>> >>>>>>>>>>>>>>are taking lead on
>>> >>>>>>>>>>>>>> these efforts.
>>> >>>>>>>>>>>>>> There is a note in the notes that the Board was supposed
>>> to
>>> >>>>>>>>>>>>>> follow up with an open letter to the community and
>>> >>>>>>>>>>>>>>companies involved
>>> >>>>>>>>>>>>>> describing our review and actions.  I don't think that has
>>> >>>>>>>>>>>>>>happened so I
>>> >>>>>>>>>>>>>> will remind the person who took on that action item.
>>> >>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>> I'm happy to answer any questions that you may have.
>>> >>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>> ~josh
>>> >>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>> On Thu, Nov 26, 2015 at 11:55 AM, Tobias
>>> >>>>>>>>>>>>>> <tobias.gondrom at owasp.org> wrote:
>>> >>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>> There have been several conversations on that matter and
>>> a
>>> >>>>>>>>>>>>>>> dedicated call. Unfortunately for personal reasons I
>>> could
>>> >>>>>>>>>>>>>>>not attend the
>>> >>>>>>>>>>>>>>> last call as it was at 04:00am my local time, but all
>>> >>>>>>>>>>>>>>>other board members
>>> >>>>>>>>>>>>>>> did participate.
>>> >>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>> Could please one of my fellow board members give an
>>> update.
>>> >>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>> Best, Tobias
>>> >>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>> On 26/11/15 18:04, Timo Goosen wrote:
>>> >>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>> I would also like to know the answer to Simon's question.
>>> >>>>>>>>>>>>>>>We
>>> >>>>>>>>>>>>>>> need to get rid of bad apples in OWASP in my opinion,
>>> >>>>>>>>>>>>>>>there are too many
>>> >>>>>>>>>>>>>>> people just using the OWASP "name" or "brand" to improve
>>> >>>>>>>>>>>>>>>their own financial
>>> >>>>>>>>>>>>>>> situation or career.
>>> >>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>> Regards.
>>> >>>>>>>>>>>>>>> Timo
>>> >>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>> On Thu, Nov 26, 2015 at 1:13 PM, psiinon
>>> >>>>>>>>>>>>>>><psiinon at gmail.com>
>>> >>>>>>>>>>>>>>> wrote:
>>> >>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>> Paul, and the rest of the board,
>>> >>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>> Its been over 2 months since I raised this issue.
>>> >>>>>>>>>>>>>>>> Whats happening?
>>> >>>>>>>>>>>>>>>> Has the board even discussed it?
>>> >>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>> Cheers,
>>> >>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>> Simon
>>> >>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>> On Tue, Oct 20, 2015 at 10:00 PM, Paul Ritchie
>>> >>>>>>>>>>>>>>>> <paul.ritchie at owasp.org> wrote:
>>> >>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>> Eoin, Johanna, All:
>>> >>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>> In an earlier email, Josh Sokol mentioned that he will
>>> be
>>> >>>>>>>>>>>>>>>>> speaking in the next day or 2 to their CTO, while at
>>> >>>>>>>>>>>>>>>>>LASCON, as a
>>> >>>>>>>>>>>>>>>>> representative of the OWASP Board.  Following that
>>> >>>>>>>>>>>>>>>>>feedback, the Board has
>>> >>>>>>>>>>>>>>>>> action to take the next steps.
>>> >>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>> Just an FYI that all comments are recognized and action
>>> >>>>>>>>>>>>>>>>>is
>>> >>>>>>>>>>>>>>>>> being taken.
>>> >>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>> Paul
>>> >>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>> Best Regards, Paul Ritchie
>>> >>>>>>>>>>>>>>>>> OWASP Executive Director
>>> >>>>>>>>>>>>>>>>> paul.ritchie at owasp.org
>>> >>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>> On Tue, Oct 20, 2015 at 1:54 PM, johanna curiel curiel
>>> >>>>>>>>>>>>>>>>> <johanna.curiel at owasp.org> wrote:
>>> >>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>> Time for owasp to do a public statement and put a
>>> clear
>>> >>>>>>>>>>>>>>>>>> story regarding this abusive behavior of Owasp brand
>>> >>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>> On Tuesday, October 20, 2015, Eoin Keary
>>> >>>>>>>>>>>>>>>>>> <eoin.keary at owasp.org> wrote:
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> Folks,
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> The project should be immediately shelved it's simply
>>> >>>>>>>>>>>>>>>>>>>bad
>>> >>>>>>>>>>>>>>>>>>> form.
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> This is damaging to OWASP, the industry and exactly
>>> >>>>>>>>>>>>>>>>>>>what
>>> >>>>>>>>>>>>>>>>>>> OWASP is not about.
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> There is a clear conflict of interest and distinct
>>> >>>>>>>>>>>>>>>>>>>lack of
>>> >>>>>>>>>>>>>>>>>>> science behind the claims made by Contrast.
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> Eoin Keary
>>> >>>>>>>>>>>>>>>>>>> OWASP Volunteer
>>> >>>>>>>>>>>>>>>>>>> @eoinkeary
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> On 7 Oct 2015, at 3:53 p.m., johanna curiel curiel
>>> >>>>>>>>>>>>>>>>>>> <johanna.curiel at owasp.org> wrote:
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> At the moment we did the project review, we observed
>>> >>>>>>>>>>>>>>>>>>>that
>>> >>>>>>>>>>>>>>>>>>> the project did not have enough testing to be
>>> >>>>>>>>>>>>>>>>>>>considered in any form as
>>> >>>>>>>>>>>>>>>>>>> 'ready'  for benchmarking, neither that it had yet
>>> the
>>> >>>>>>>>>>>>>>>>>>>community adoption,
>>> >>>>>>>>>>>>>>>>>>> however technically speaking as it has been
>>> classified
>>> >>>>>>>>>>>>>>>>>>>by the leaders, the
>>> >>>>>>>>>>>>>>>>>>> project is at the beta stage.
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> Indeed , Dave had the push to have the project
>>> reviewed
>>> >>>>>>>>>>>>>>>>>>> but it was never clear that later on the project was
>>> >>>>>>>>>>>>>>>>>>>going to be advertisied
>>> >>>>>>>>>>>>>>>>>>> this way. That all happend after the presentation at
>>> >>>>>>>>>>>>>>>>>>>Appsec.
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> I had my concerns regarding how sensitive is the
>>> >>>>>>>>>>>>>>>>>>>subject
>>> >>>>>>>>>>>>>>>>>>> of the project ,but I think we should allow project
>>> >>>>>>>>>>>>>>>>>>>leaders to develop their
>>> >>>>>>>>>>>>>>>>>>> communication strategy even if this has conflict of
>>> >>>>>>>>>>>>>>>>>>>interest. It all depends
>>> >>>>>>>>>>>>>>>>>>> how they behave and how they manage this.
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> On Tuesday, October 6, 2015, Michael Coates
>>> >>>>>>>>>>>>>>>>>>> <michael.coates at owasp.org> wrote:
>>> >>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>> It's not really that formal to add to the agenda,
>>> >>>>>>>>>>>>>>>>>>>>just a
>>> >>>>>>>>>>>>>>>>>>>> wiki that we add in the text.
>>> >>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>> I think you can safely assume it will get the
>>> >>>>>>>>>>>>>>>>>>>>appropriate
>>> >>>>>>>>>>>>>>>>>>>> discussion.
>>> >>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>> On Oct 6, 2015, at 7:16 AM, psiinon
>>> >>>>>>>>>>>>>>>>>>>><psiinon at gmail.com>
>>> >>>>>>>>>>>>>>>>>>>> wrote:
>>> >>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>> Really?? Its not on the agenda yet for the next
>>> >>>>>>>>>>>>>>>>>>>>meeting??
>>> >>>>>>>>>>>>>>>>>>>> How does it get added to the agenda?
>>> >>>>>>>>>>>>>>>>>>>> And that was a formal request if that makes any
>>> >>>>>>>>>>>>>>>>>>>> difference :)
>>> >>>>>>>>>>>>>>>>>>>> I'm all in favour of getting the facts straight
>>> before
>>> >>>>>>>>>>>>>>>>>>>> any actions are taken, hence my request for an
>>> >>>>>>>>>>>>>>>>>>>>'ethical review' or whatever
>>> >>>>>>>>>>>>>>>>>>>> it should be called.
>>> >>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>> Cheers,
>>> >>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>> Simon
>>> >>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>> On Tue, Oct 6, 2015 at 3:07 PM, Michael Coates
>>> >>>>>>>>>>>>>>>>>>>> <michael.coates at owasp.org> wrote:
>>> >>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>> First step is to get all of our information
>>> straight so
>>> >>>>>>>>>>>>>>>>>>>>> we're clear on where things are at.
>>> >>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>> This was not on the board agenda last meeting and
>>> is
>>> >>>>>>>>>>>>>>>>>>>>> also not on the next agenda as of yet (of course
>>> it could
>>> always be added if
>>> >>>>>>>>>>>>>>>>>>>>> needed).
>>> >>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>> We are aware that people have raised questions
>>> though.
>>> >>>>>>>>>>>>>>>>>>>>> I'm hoping we can get a clear understanding of all
>>> the
>>> facts and then
>>> >>>>>>>>>>>>>>>>>>>>> discuss if changes are needed.
>>> >>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>> On Oct 6, 2015, at 1:52 AM, psiinon <
>>> psiinon at gmail.com>
>>> >>>>>>>>>>>>>>>>>>>>> wrote:
>>> >>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>> Hey Michael,
>>> >>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>> Is the board going to take any action?
>>> >>>>>>>>>>>>>>>>>>>>> Were there any discussions about this controversy
>>> in the
>>> >>>>>>>>>>>>>>>>>>>>> board meeting at AppSec USA?
>>> >>>>>>>>>>>>>>>>>>>>> If not will it be on the agenda for the meeting on
>>> >>>>>>>>>>>>>>>>>>>>> October 14th?
>>> >>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>> Cheers,
>>> >>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>> Simon
>>> >>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>> On Tue, Oct 6, 2015 at 8:25 AM, Michael Coates
>>> >>>>>>>>>>>>>>>>>>>>> <michael.coates at owasp.org> wrote:
>>> >>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>> Simon
>>> >>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>> I posted the below message earlier today. At this
>>> point
>>> >>>>>>>>>>>>>>>>>>>>>> my goal is to just gain clarity over the current
>>> reality
>>> and ideally drive
>>> >>>>>>>>>>>>>>>>>>>>>> to a shared state of success. This message
>>> doesn't seem
>>> to be reflected in
>>> >>>>>>>>>>>>>>>>>>>>>> the list yet. It could be because my membership
>>> hasn't
>>> been approved or
>>> >>>>>>>>>>>>>>>>>>>>>> because of mail list delays (I miss Google
>>> groups). But I
>>> think these
>>> >>>>>>>>>>>>>>>>>>>>>> questions will start the conversation.
>>> >>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>> (This was just me asking questions as a curious
>>> Owasp
>>> >>>>>>>>>>>>>>>>>>>>>> member, not any action on behalf of the board)
>>> >>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>> Begin forwarded message:
>>> >>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>> From: Michael Coates <michael.coates at owasp.org>
>>> >>>>>>>>>>>>>>>>>>>>>> Date: October 5, 2015 at 6:20:23 PM PDT
>>> >>>>>>>>>>>>>>>>>>>>>> To: owasp-benchmark-project at lists.owasp.org
>>> >>>>>>>>>>>>>>>>>>>>>> Subject: Project Questions
>>> >>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>> OWASP Benchmark List,
>>> >>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>> I've heard more about this project and am excited
>>> about
>>> >>>>>>>>>>>>>>>>>>>>>> the idea of an independent perspective of tool
>>> performance. I'm trying to
>>> >>>>>>>>>>>>>>>>>>>>>> understand a few things to better respond to
>>> questions
>>> from those in the
>>> >>>>>>>>>>>>>>>>>>>>>> security & OWASP community.
>>> >>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>> In my mind there are two big areas for
>>> consideration in
>>> >>>>>>>>>>>>>>>>>>>>>> a benchmark process.
>>> >>>>>>>>>>>>>>>>>>>>>> 1. Are the benchmarks testing the right areas?
>>> >>>>>>>>>>>>>>>>>>>>>> 2. Is the process for creating the benchmark
>>> objective
>>> >>>>>>>>>>>>>>>>>>>>>> & free from conflicts of interest.
>>> >>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>> I think as a group OWASP is the right body to
>>> align on
>>> >>>>>>>>>>>>>>>>>>>>>> #1.
>>> >>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>> I'd like to ask for some clarifications on item
>>> #2. I
>>> >>>>>>>>>>>>>>>>>>>>>> think it's important to avoid actual conflict of
>>> interest
>>> and also the
>>> >>>>>>>>>>>>>>>>>>>>>> appearance of conflict of interest. The former is
>>> obvious
>>> why we mustn't
>>> >>>>>>>>>>>>>>>>>>>>>> have that, the latter is critical so others have
>>> faith in
>>> the tool, process
>>> >>>>>>>>>>>>>>>>>>>>>> and outputs of the process when viewing or
>>> hearing about
>>> the project.
>>> >>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>> 1) Can we clarify whether other individuals have
>>> >>>>>>>>>>>>>>>>>>>>>> submitted meaningful code to the project?
>>> >>>>>>>>>>>>>>>>>>>>>> Observation:
>>> >>>>>>>>>>>>>>>>>>>>>> Nearly all the code commits have come from 1
>>> person
>>> >>>>>>>>>>>>>>>>>>>>>> (project lead).
>>> >>>>>>>>>>>>>>>>>>>>>>
>>> https://github.com/OWASP/Benchmark/graphs/contributors
>>> >>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>> 2) Can we clarify the contributions of others and
>>> their
>>> >>>>>>>>>>>>>>>>>>>>>> represented organizations?
>>> >>>>>>>>>>>>>>>>>>>>>> Observation:
>>> >>>>>>>>>>>>>>>>>>>>>> The acknowledgements tab listed two developers
>>> (Juan
>>> >>>>>>>>>>>>>>>>>>>>>> Gama & Nick Sanidas) both who work at the same
>>> company as
>>> the project lead.
>>> >>>>>>>>>>>>>>>>>>>>>> It seems other people have submitted some small
>>> amounts
>>> of material, but
>>> >>>>>>>>>>>>>>>>>>>>>> overall it seems all development has come from
>>> the same
>>> company.
>>> >>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>
>>> https://www.owasp.org/index.php/Benchmark#tab=Acknowledgements
>>> >>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>> 3) Can we clarify in what ways we've mitigated the
>>> >>>>>>>>>>>>>>>>>>>>>> potential conflict of interest and also the
>>> appearance of
>>> a conflict of
>>> >>>>>>>>>>>>>>>>>>>>>> interest? This seems like the largest blocker for
>>> wide
>>> spread acceptance of
>>> >>>>>>>>>>>>>>>>>>>>>> this project and the biggest risk.
>>> >>>>>>>>>>>>>>>>>>>>>> Observation:
>>> >>>>>>>>>>>>>>>>>>>>>> The project lead and both of the project
>>> developers
>>> >>>>>>>>>>>>>>>>>>>>>> works for a company with very close ties to one
>>> of the
>>> companies that is
>>> >>>>>>>>>>>>>>>>>>>>>> evaluated by this project. Further, it appears the
>>> company is performing
>>> >>>>>>>>>>>>>>>>>>>>>> very well on the project tests.
>>> >>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>> 4) If we are going to list tool vendors then I'd
>>> >>>>>>>>>>>>>>>>>>>>>> recommend listing multiple vendors for each
>>> category.
>>> >>>>>>>>>>>>>>>>>>>>>> Observation:
>>> >>>>>>>>>>>>>>>>>>>>>> The tools page only lists 1 IAST tool. Since this
>>> is
>>> >>>>>>>>>>>>>>>>>>>>>> the point of the potential conflict of interest
>>> it is
>>> important to list
>>> >>>>>>>>>>>>>>>>>>>>>> numerous IAST tools.
>>> >>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>
>>> https://www.owasp.org/index.php/Benchmark#tab=Tool_Support_2FResults
>>> >>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>> 5) Diverse body with multiple points of view
>>> >>>>>>>>>>>>>>>>>>>>>> Observation:
>>> >>>>>>>>>>>>>>>>>>>>>> There is no indication that multiple stakeholders
>>> are
>>> >>>>>>>>>>>>>>>>>>>>>> present to review and decide on the future of this
>>> project. If they exist, a
>>> >>>>>>>>>>>>>>>>>>>>>> new section should be added to the project page
>>> to raise
>>> awareness. If they
>>> >>>>>>>>>>>>>>>>>>>>>> don't exist, we should reevaluate how we are
>>> obtaining an
>>> independent view
>>> >>>>>>>>>>>>>>>>>>>>>> of the testing process.
>>> >>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>> Again, I think the idea of the project is great.
>>> From
>>> >>>>>>>>>>>>>>>>>>>>>> my perspective clarifying these questions will
>>> help
>>> ensure the project is
>>> >>>>>>>>>>>>>>>>>>>>>> not only objective, but also perceived as
>>> objective from
>>> someone reviewing
>>> >>>>>>>>>>>>>>>>>>>>>> the material. Ultimately this will contribute to
>>> the
>>> success and growth of
>>> >>>>>>>>>>>>>>>>>>>>>> the project.
>>> >>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>> Thanks!
>>> >>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>> --
>>> >>>>>>>>>>>>>>>>>>>>>> Michael Coates
>>> >>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>> On Oct 2, 2015, at 1:31 AM, psiinon <
>>> psiinon at gmail.com>
>>> >>>>>>>>>>>>>>>>>>>>>> wrote:
>>> >>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>> OK, based on the concerns raised so far I think
>>> the
>>> >>>>>>>>>>>>>>>>>>>>>> board should initiate a review of the OWASP
>>> Benchmark
>>> project.
>>> >>>>>>>>>>>>>>>>>>>>>> I'm not raising a formal complaint against it,
>>> I'm just
>>> >>>>>>>>>>>>>>>>>>>>>> requesting a review.
>>> >>>>>>>>>>>>>>>>>>>>>> And I dont think it needs a 'standard' project
>>> review -
>>> >>>>>>>>>>>>>>>>>>>>>> Johanna has already done a very good job of this.
>>> >>>>>>>>>>>>>>>>>>>>>> Not sure what sort of review you'd call it, I'll
>>> leave
>>> >>>>>>>>>>>>>>>>>>>>>> the naming to others :)
>>> >>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>> I'm concerned that we have an OWASP project lead
>>> by a
>>> >>>>>>>>>>>>>>>>>>>>>> company who has a clear commercial stake in the
>>> results.
>>> >>>>>>>>>>>>>>>>>>>>>> Bringing more companies on board will help, but
>>> I'm
>>> >>>>>>>>>>>>>>>>>>>>>> still not sure that alone will make it independent
>>> enough.
>>> >>>>>>>>>>>>>>>>>>>>>> Commercial companies can afford to dedicate staff
>>> to
>>> >>>>>>>>>>>>>>>>>>>>>> improving Benchmark so that their products look
>>> better.
>>> >>>>>>>>>>>>>>>>>>>>>> Open source projects just cant do that, so we are
>>> at a
>>> >>>>>>>>>>>>>>>>>>>>>> distinct disadvantage.
>>> >>>>>>>>>>>>>>>>>>>>>> Should we allow a commercially driven OWASP
>>> project
>>> >>>>>>>>>>>>>>>>>>>>>> who's aim could be seen be to promote commercial
>>> software?
>>> >>>>>>>>>>>>>>>>>>>>>> If so, what sort of checks and balances does it
>>> need?
>>> >>>>>>>>>>>>>>>>>>>>>> Those are the sort of questions I'd like an
>>> independent
>>> >>>>>>>>>>>>>>>>>>>>>> review to look at.
>>> >>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>> I do think there are some immediate steps that
>>> could be
>>> >>>>>>>>>>>>>>>>>>>>>> taken:
>>> >>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>> I'd like to see the Benchmark project page clearly
>>> >>>>>>>>>>>>>>>>>>>>>> state thats its at a very early stage and that the
>>> results are _not_ yet
>>> >>>>>>>>>>>>>>>>>>>>>> suitable for use in commercial literature.
>>> >>>>>>>>>>>>>>>>>>>>>> I'd also like the main companies developing
>>> Benchmark
>>> >>>>>>>>>>>>>>>>>>>>>> to be clearly stated on the main page. If and
>>> when other
>>> companies get
>>> >>>>>>>>>>>>>>>>>>>>>> involved then this would actually help the
>>> project's
>>> claim of vendor
>>> >>>>>>>>>>>>>>>>>>>>>> independence.
>>> >>>>>>>>>>>>>>>>>>>>>> And I'd love to see a respected co-leader added
>>> to the
>>> >>>>>>>>>>>>>>>>>>>>>> project who is not associated with any commercial
>>> or open
>>> source security
>>> >>>>>>>>>>>>>>>>>>>>>> tools:)
>>> >>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>> And we should carry on discussing the project on
>>> this
>>> >>>>>>>>>>>>>>>>>>>>>> list - I think such discussions are very healthy,
>>> and I'd
>>> love to see this
>>> >>>>>>>>>>>>>>>>>>>>>> project mature to a state where it can be a
>>> trusted,
>>> independent and valued
>>> >>>>>>>>>>>>>>>>>>>>>> resource.
>>> >>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>> Cheers,
>>> >>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>> Simon
>>> >>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>> On Thu, Oct 1, 2015 at 7:59 PM, Tobias
>>> >>>>>>>>>>>>>>>>>>>>>> <tobias.gondrom at owasp.org> wrote:
>>> >>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>> @Simon:
>>> >>>>>>>>>>>>>>>>>>>>>>> yes, the leaders list is the place for your
>>> >>>>>>>>>>>>>>>>>>>>>>> discussions for project and chapter leaders
>>> >>>>>>>>>>>>>>>>>>>>>>> @Timo: I like your framing of "Don't ask what
>>> OWASP
>>> >>>>>>>>>>>>>>>>>>>>>>> can do for me, ask what I can do for OWASP."
>>> >>>>>>>>>>>>>>>>>>>>>>> That should and is indeed the spirit of OWASP:-)
>>> >>>>>>>>>>>>>>>>>>>>>>> Best regards, Tobias
>>> >>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>> On 30/09/15 09:42, Timo Goosen wrote:
>>> >>>>>
>>> >>>>> ...
>>> >>>>>
>>> >>>>> [Message clipped]
>>> >>>>> _______________________________________________
>>> >>>>> Owasp-board mailing list
>>> >>>>> Owasp-board at lists.owasp.org
>>> >>>>> https://lists.owasp.org/mailman/listinfo/owasp-board
>>> >>>>>
>>> >>>>
>>> >>>>
>>> >>>>
>>> >>>> --
>>> >>>> OWASP ZAP Project leader
>>> >>>>
>>> >>>> _______________________________________________
>>> >>>> OWASP-Leaders mailing list
>>> >>>> OWASP-Leaders at lists.owasp.org
>>> >>>> https://lists.owasp.org/mailman/listinfo/owasp-leaders
>>> >>>>
>>> >>>
>>> >>
>>> >>
>>> >>
>>> >> --
>>> >> OWASP ZAP Project leader
>>> >>
>>> >> _______________________________________________
>>> >> OWASP-Leaders mailing list
>>> >> OWASP-Leaders at lists.owasp.org
>>> >> https://lists.owasp.org/mailman/listinfo/owasp-leaders
>>> >>
>>> >_______________________________________________
>>> >OWASP-Leaders mailing list
>>> >OWASP-Leaders at lists.owasp.org
>>> >https://lists.owasp.org/mailman/listinfo/owasp-leaders
>>>
>>>
>>> _______________________________________________
>>> OWASP-Leaders mailing list
>>> OWASP-Leaders at lists.owasp.org
>>> https://lists.owasp.org/mailman/listinfo/owasp-leaders
>>>
>>
>> _______________________________________________
>> OWASP-Leaders mailing list
>> OWASP-Leaders at lists.owasp.org
>> https://lists.owasp.org/mailman/listinfo/owasp-leaders
>>
>>
>> _______________________________________________
>> OWASP-Leaders mailing list
>> OWASP-Leaders at lists.owasp.org
>> https://lists.owasp.org/mailman/listinfo/owasp-leaders
>>
>>
>
>
> --
> OWASP ZAP <https://www.owasp.org/index.php/ZAP> Project leader
>
> _______________________________________________
> OWASP-Leaders mailing list
> OWASP-Leaders at lists.owasp.org
> https://lists.owasp.org/mailman/listinfo/owasp-leaders
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.owasp.org/pipermail/owasp-leaders/attachments/20151201/7d77468f/attachment-0001.html>


More information about the OWASP-Leaders mailing list