[Owasp-leaders] [Owasp-board] OWASP Benchmark project - potential conflict of interest
Tobias
tobias.gondrom at owasp.org
Thu Nov 26 17:55:50 UTC 2015
There have been several conversations on that matter and a dedicated
call. Unfortunately for personal reasons I could not attend the last
call as it was at 04:00am my local time, but all other board members did
participate.
Could please one of my fellow board members give an update.
Best, Tobias
On 26/11/15 18:04, Timo Goosen wrote:
> I would also like to know the answer to Simon's question. We need to
> get rid of bad apples in OWASP in my opinion, there are too many
> people just using the OWASP "name" or "brand" to improve their own
> financial situation or career.
>
> Regards.
> Timo
>
> On Thu, Nov 26, 2015 at 1:13 PM, psiinon <psiinon at gmail.com
> <mailto:psiinon at gmail.com>> wrote:
>
> Paul, and the rest of the board,
>
> Its been over 2 months since I raised this issue.
> Whats happening?
> Has the board even discussed it?
>
> Cheers,
>
> Simon
>
>
> On Tue, Oct 20, 2015 at 10:00 PM, Paul Ritchie
> <paul.ritchie at owasp.org <mailto:paul.ritchie at owasp.org>> wrote:
>
> Eoin, Johanna, All:
>
> In an earlier email, Josh Sokol mentioned that he will be
> speaking in the next day or 2 to their CTO, while at LASCON,
> as a representative of the OWASP Board. Following that
> feedback, the Board has action to take the next steps.
>
> Just an FYI that all comments are recognized and action is
> being taken.
>
> Paul
>
>
>
> Best Regards, Paul Ritchie
> OWASP Executive Director
> paul.ritchie at owasp.org <mailto:paul.ritchie at owasp.org>
>
>
> On Tue, Oct 20, 2015 at 1:54 PM, johanna curiel curiel
> <johanna.curiel at owasp.org <mailto:johanna.curiel at owasp.org>>
> wrote:
>
> Time for owasp to do a public statement and put a clear
> story regarding this abusive behavior of Owasp brand
>
>
> On Tuesday, October 20, 2015, Eoin Keary
> <eoin.keary at owasp.org <mailto:eoin.keary at owasp.org>> wrote:
>
> Folks,
>
> The project should be immediately shelved it's simply
> bad form.
>
> This is damaging to OWASP, the industry and exactly
> what OWASP is not about.
>
> There is a clear conflict of interest and distinct
> lack of science behind the claims made by Contrast.
>
>
>
>
>
>
> Eoin Keary
> OWASP Volunteer
> @eoinkeary
>
>
>
> On 7 Oct 2015, at 3:53 p.m., johanna curiel curiel
> <johanna.curiel at owasp.org> wrote:
>
>> At the moment we did the project review, we observed
>> that the project did not have enough testing to be
>> considered in any form as 'ready' for benchmarking,
>> neither that it had yet the community adoption,
>> however technically speaking as it has been
>> classified by the leaders, the project is at the beta
>> stage.
>>
>> Indeed , Dave had the push to have the project
>> reviewed but it was never clear that later on the
>> project was going to be advertisied this way. That
>> all happend after the presentation at Appsec.
>>
>> I had my concerns regarding how sensitive is the
>> subject of the project ,but I think we should allow
>> project leaders to develop their communication
>> strategy even if this has conflict of interest. It
>> all depends how they behave and how they manage this.
>>
>>
>> On Tuesday, October 6, 2015, Michael Coates
>> <michael.coates at owasp.org> wrote:
>>
>> It's not really that formal to add to the agenda,
>> just a wiki that we add in the text.
>>
>> I think you can safely assume it will get the
>> appropriate discussion.
>>
>> On Oct 6, 2015, at 7:16 AM, psiinon
>> <psiinon at gmail.com> wrote:
>>
>>> Really?? Its not on the agenda yet for the next
>>> meeting??
>>> How does it get added to the agenda?
>>> And that was a formal request if that makes any
>>> difference :)
>>> I'm all in favour of getting the facts straight
>>> before any actions are taken, hence my request
>>> for an 'ethical review' or whatever it should be
>>> called.
>>>
>>> Cheers,
>>>
>>> Simon
>>>
>>> On Tue, Oct 6, 2015 at 3:07 PM, Michael Coates
>>> <michael.coates at owasp.org> wrote:
>>>
>>> First step is to get all of our information
>>> straight so we're clear on where things are at.
>>>
>>> This was not on the board agenda last
>>> meeting and is also not on the next agenda
>>> as of yet (of course it could always be
>>> added if needed).
>>>
>>> We are aware that people have raised
>>> questions though. I'm hoping we can get a
>>> clear understanding of all the facts and
>>> then discuss if changes are needed.
>>>
>>>
>>>
>>> On Oct 6, 2015, at 1:52 AM, psiinon
>>> <psiinon at gmail.com> wrote:
>>>
>>>> Hey Michael,
>>>>
>>>> Is the board going to take any action?
>>>> Were there any discussions about this
>>>> controversy in the board meeting at AppSec USA?
>>>> If not will it be on the agenda for the
>>>> meeting on October 14th?
>>>>
>>>> Cheers,
>>>>
>>>> Simon
>>>>
>>>>
>>>> On Tue, Oct 6, 2015 at 8:25 AM, Michael
>>>> Coates <michael.coates at owasp.org> wrote:
>>>>
>>>> Simon
>>>>
>>>> I posted the below message earlier
>>>> today. At this point my goal is to just
>>>> gain clarity over the current reality
>>>> and ideally drive to a shared state of
>>>> success. This message doesn't seem to
>>>> be reflected in the list yet. It could
>>>> be because my membership hasn't been
>>>> approved or because of mail list delays
>>>> (I miss Google groups). But I think
>>>> these questions will start the
>>>> conversation.
>>>>
>>>> (This was just me asking questions as a
>>>> curious Owasp member, not any action on
>>>> behalf of the board)
>>>>
>>>>
>>>>
>>>>
>>>> Begin forwarded message:
>>>>
>>>>> *From:* Michael Coates
>>>>> <michael.coates at owasp.org>
>>>>> *Date:* October 5, 2015 at 6:20:23 PM PDT
>>>>> *To:*
>>>>> owasp-benchmark-project at lists.owasp.org
>>>>> *Subject:* *Project Questions*
>>>>>
>>>>> OWASP Benchmark List,
>>>>>
>>>>> I've heard more about this project and
>>>>> am excited about the idea of an
>>>>> independent perspective of tool
>>>>> performance. I'm trying to understand
>>>>> a few things to better respond to
>>>>> questions from those in the security &
>>>>> OWASP community.
>>>>>
>>>>> In my mind there are two big areas for
>>>>> consideration in a benchmark process.
>>>>> 1. Are the benchmarks testing the
>>>>> right areas?
>>>>> 2. Is the process for creating the
>>>>> benchmark objective & free from
>>>>> conflicts of interest.
>>>>>
>>>>> I think as a group OWASP is the right
>>>>> body to align on #1.
>>>>>
>>>>> I'd like to ask for some
>>>>> clarifications on item #2. I think
>>>>> it's important to avoid actual
>>>>> conflict of interest and also the
>>>>> appearance of conflict of interest.
>>>>> The former is obvious why we mustn't
>>>>> have that, the latter is critical so
>>>>> others have faith in the tool, process
>>>>> and outputs of the process when
>>>>> viewing or hearing about the project.
>>>>>
>>>>>
>>>>> 1) Can we clarify whether other
>>>>> individuals have submitted meaningful
>>>>> code to the project?
>>>>> Observation:
>>>>> Nearly all the code commits have come
>>>>> from 1 person (project lead).
>>>>> https://github.com/OWASP/Benchmark/graphs/contributors
>>>>>
>>>>> 2) Can we clarify the contributions of
>>>>> others and their represented
>>>>> organizations?
>>>>> Observation:
>>>>> The acknowledgements tab listed two
>>>>> developers (Juan Gama & Nick Sanidas)
>>>>> both who work at the same company as
>>>>> the project lead. It seems other
>>>>> people have submitted some small
>>>>> amounts of material, but overall it
>>>>> seems all development has come from
>>>>> the same company.
>>>>> https://www.owasp.org/index.php/Benchmark#tab=Acknowledgements
>>>>>
>>>>> 3) Can we clarify in what ways we've
>>>>> mitigated the potential conflict of
>>>>> interest and also the appearance of a
>>>>> conflict of interest? This seems like
>>>>> the largest blocker for wide spread
>>>>> acceptance of this project and the
>>>>> biggest risk.
>>>>> Observation:
>>>>> The project lead and both of the
>>>>> project developers works for a company
>>>>> with very close ties to one of the
>>>>> companies that is evaluated by this
>>>>> project. Further, it appears the
>>>>> company is performing very well on the
>>>>> project tests.
>>>>>
>>>>> 4) If we are going to list tool
>>>>> vendors then I'd recommend listing
>>>>> multiple vendors for each category.
>>>>> Observation:
>>>>> The tools page only lists 1 IAST tool.
>>>>> Since this is the point of the
>>>>> potential conflict of interest it is
>>>>> important to list numerous IAST tools.
>>>>> https://www.owasp.org/index.php/Benchmark#tab=Tool_Support_2FResults
>>>>>
>>>>> 5) Diverse body with multiple points
>>>>> of view
>>>>> Observation:
>>>>> There is no indication that multiple
>>>>> stakeholders are present to review and
>>>>> decide on the future of this project.
>>>>> If they exist, a new section should be
>>>>> added to the project page to raise
>>>>> awareness. If they don't exist, we
>>>>> should reevaluate how we are obtaining
>>>>> an independent view of the testing
>>>>> process.
>>>>>
>>>>>
>>>>> Again, I think the idea of the project
>>>>> is great. From my perspective
>>>>> clarifying these questions will help
>>>>> ensure the project is not only
>>>>> objective, but also perceived as
>>>>> objective from someone reviewing the
>>>>> material. Ultimately this will
>>>>> contribute to the success and growth
>>>>> of the project.
>>>>>
>>>>> Thanks!
>>>>>
>>>>>
>>>>> --
>>>>> Michael Coates
>>>>>
>>>>>
>>>>>
>>>>
>>>> On Oct 2, 2015, at 1:31 AM, psiinon
>>>> <psiinon at gmail.com> wrote:
>>>>
>>>>> OK, based on the concerns raised so
>>>>> far I think the board should initiate
>>>>> a review of the OWASP Benchmark project.
>>>>> I'm not raising a formal complaint
>>>>> against it, I'm just requesting a review.
>>>>> And I dont think it needs a 'standard'
>>>>> project review - Johanna has already
>>>>> done a very good job of this.
>>>>> Not sure what sort of review you'd
>>>>> call it, I'll leave the naming to
>>>>> others :)
>>>>>
>>>>> I'm concerned that we have an OWASP
>>>>> project lead by a company who has a
>>>>> clear commercial stake in the results.
>>>>> Bringing more companies on board will
>>>>> help, but I'm still not sure that
>>>>> alone will make it independent enough.
>>>>> Commercial companies can afford to
>>>>> dedicate staff to improving Benchmark
>>>>> so that their products look better.
>>>>> Open source projects just cant do
>>>>> that, so we are at a distinct
>>>>> disadvantage.
>>>>> Should we allow a commercially driven
>>>>> OWASP project who's aim could be seen
>>>>> be to promote commercial software?
>>>>> If so, what sort of checks and
>>>>> balances does it need?
>>>>> Those are the sort of questions I'd
>>>>> like an independent review to look at.
>>>>>
>>>>> I do think there are some immediate
>>>>> steps that could be taken:
>>>>>
>>>>> * I'd like to see the Benchmark
>>>>> project page clearly state thats
>>>>> its at a very early stage and that
>>>>> the results are _not_ yet suitable
>>>>> for use in commercial literature.
>>>>> * I'd also like the main companies
>>>>> developing Benchmark to be clearly
>>>>> stated on the main page. If and
>>>>> when other companies get involved
>>>>> then this would actually help the
>>>>> project's claim of vendor
>>>>> independence.
>>>>> * And I'd love to see a respected
>>>>> co-leader added to the project who
>>>>> is not associated with any
>>>>> commercial or open source security
>>>>> tools:)
>>>>>
>>>>> And we should carry on discussing the
>>>>> project on this list - I think such
>>>>> discussions are very healthy, and I'd
>>>>> love to see this project mature to a
>>>>> state where it can be a trusted,
>>>>> independent and valued resource.
>>>>>
>>>>> Cheers,
>>>>>
>>>>> Simon
>>>>>
>>>>> On Thu, Oct 1, 2015 at 7:59 PM, Tobias
>>>>> <tobias.gondrom at owasp.org> wrote:
>>>>>
>>>>> @Simon:
>>>>> yes, the leaders list is the place
>>>>> for your discussions for project
>>>>> and chapter leaders
>>>>> @Timo: I like your framing of
>>>>> "Don't ask what OWASP can do for
>>>>> me, ask what I can do for OWASP."
>>>>> That should and is indeed the
>>>>> spirit of OWASP:-)
>>>>> Best regards, Tobias
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On 30/09/15 09:42, Timo Goosen wrote:
>>>>>> I don't know enough about the
>>>>>> matter to comment on this case,
>>>>>> but I feel that any situation
>>>>>> where an OWASP project or any
>>>>>> OWASP initiative for that matter,
>>>>>> is using OWASP to promote its own
>>>>>> business interests should be
>>>>>> stopped. We need to get rid of
>>>>>> bad apples in OWASP.
>>>>>>
>>>>>> OWASP is becoming a brand if you
>>>>>> would like to think of it that
>>>>>> way and we are going to see many
>>>>>> more cases of people trying to
>>>>>> use OWASP to spread their
>>>>>> business interests. At the end of
>>>>>> the day everyone should be acting
>>>>>> with an attitude of:"Don't ask
>>>>>> what OWASP can do for me, ask
>>>>>> what I can do for OWASP?"
>>>>>>
>>>>>>
>>>>>>
>>>>>> Regards.
>>>>>> Timo
>>>>>>
>>>>>> On Wed, Sep 30, 2015 at 11:48 AM,
>>>>>> psiinon <psiinon at gmail.com> wrote:
>>>>>>
>>>>>> So, a load of controversy
>>>>>> about OWASP Benchmark on
>>>>>> twitter, but no discussion on
>>>>>> the leaders list :(
>>>>>> Is this now the wrong place
>>>>>> to discuss OWASP projects??
>>>>>>
>>>>>> Simon
>>>>>>
>>>>>>
>>>>>> On Thu, Sep 24, 2015 at 10:36
>>>>>> AM, psiinon
>>>>>> <psiinon at gmail.com> wrote:
>>>>>>
>>>>>> Hi folks,
>>>>>>
>>>>>> I've got some concerns
>>>>>> about the OWASP Benchmark
>>>>>> project.
>>>>>>
>>>>>> I _like_ benchmarks, and
>>>>>> I'm very pleased to see
>>>>>> an active OWASP project
>>>>>> focused on delivering one.
>>>>>> I think the project has
>>>>>> some technical
>>>>>> limitations, but thats
>>>>>> fine given the stage the
>>>>>> project is at, ie _very_
>>>>>> early.
>>>>>> I dont think that any
>>>>>> firm conclusions should
>>>>>> be drawn from it until
>>>>>> its been significantly
>>>>>> enhanced.
>>>>>>
>>>>>> My concerns are around
>>>>>> the marketing that one of
>>>>>> the companies sponsoring
>>>>>> the Benchmark project has
>>>>>> started using.
>>>>>>
>>>>>> Here we have a company
>>>>>> that leads an OWASP
>>>>>> project that just happens
>>>>>> to show that their
>>>>>> offering in this area
>>>>>> appears to be
>>>>>> _significantly_ better
>>>>>> than any of the competition.
>>>>>> Their recent press
>>>>>> release stresses that its
>>>>>> an OWASP project, make
>>>>>> the most of the fact that
>>>>>> the US DHS helped fund it
>>>>>> but make no mention of
>>>>>> their role in developing it.
>>>>>>
>>>>>> Regardless of the
>>>>>> accuracy of the results,
>>>>>> it seems like a huge
>>>>>> conflict of interest :(
>>>>>>
>>>>>> It appears that I'm not
>>>>>> the only one with
>>>>>> concerns related to the
>>>>>> project:
>>>>>>
>>>>>> https://www.veracode.com/blog/2015/09/no-one-technology-silver-bullet
>>>>>>
>>>>>> What do other people think?
>>>>>>
>>>>>> Cheers,
>>>>>>
>>>>>> Simon
>>>>>>
>>>>>> --
>>>>>> OWASP ZAP
>>>>>> <https://www.owasp.org/index.php/ZAP>
>>>>>> Project leader
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> OWASP ZAP
>>>>>> <https://www.owasp.org/index.php/ZAP>
>>>>>> Project leader
>>>>>>
>>>>>> _______________________________________________
>>>>>> OWASP-Leaders mailing list
>>>>>> OWASP-Leaders at lists.owasp.org
>>>>>> https://lists.owasp.org/mailman/listinfo/owasp-leaders
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> OWASP-Leaders mailing list
>>>>>> OWASP-Leaders at lists.owasp.org
>>>>>> https://lists.owasp.org/mailman/listinfo/owasp-leaders
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> OWASP ZAP
>>>>> <https://www.owasp.org/index.php/ZAP>
>>>>> Project leader
>>>>> _______________________________________________
>>>>> OWASP-Leaders mailing list
>>>>> OWASP-Leaders at lists.owasp.org
>>>>> https://lists.owasp.org/mailman/listinfo/owasp-leaders
>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> OWASP ZAP
>>>> <https://www.owasp.org/index.php/ZAP>
>>>> Project leader
>>>
>>>
>>>
>>>
>>> --
>>> OWASP ZAP <https://www.owasp.org/index.php/ZAP>
>>> Project leader
>>
>> _______________________________________________
>> Owasp-board mailing list
>> Owasp-board at lists.owasp.org
>> https://lists.owasp.org/mailman/listinfo/owasp-board
>
>
> _______________________________________________
> Owasp-board mailing list
> Owasp-board at lists.owasp.org
> <mailto:Owasp-board at lists.owasp.org>
> https://lists.owasp.org/mailman/listinfo/owasp-board
>
>
>
> _______________________________________________
> Owasp-board mailing list
> Owasp-board at lists.owasp.org <mailto:Owasp-board at lists.owasp.org>
> https://lists.owasp.org/mailman/listinfo/owasp-board
>
>
>
>
> --
> OWASP ZAP <https://www.owasp.org/index.php/ZAP> Project leader
>
> _______________________________________________
> Owasp-board mailing list
> Owasp-board at lists.owasp.org <mailto:Owasp-board at lists.owasp.org>
> https://lists.owasp.org/mailman/listinfo/owasp-board
>
>
>
>
> _______________________________________________
> Owasp-board mailing list
> Owasp-board at lists.owasp.org
> https://lists.owasp.org/mailman/listinfo/owasp-board
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.owasp.org/pipermail/owasp-leaders/attachments/20151126/df792ad9/attachment-0001.html>
More information about the OWASP-Leaders
mailing list