[Owasp-leaders] The Only Thing That Is Constant Is Change

Kevin W. Wall kevin.w.wall at gmail.com
Sun May 8 05:05:50 UTC 2016

I shall try to reply to both of you at once rather than writing two
separate TL;DR responses. :)

On Sat, May 7, 2016 at 12:07 PM, johanna curiel curiel
<johanna.curiel at owasp.org> wrote:
> On Sat, May 7, 2016 at 10:16 AM, Tom Brennan - OWASP <tomb at owasp.org> wrote:
>> "The stakes have never been higher for open-source software security. With
>> millions of people around the world relying on open source software — and
>> vulnerabilities like Heartbleed putting everyone at risk — it's time to
>> change the way we support, protect, and fortify open software."
>> Interesting article and project(s) now available
>> http://www.linuxinsider.com/story/83463.html
>> Why should OWASP code based projects not work together on this initiative
>> in raising visibility for Software security, improving our project quality
>> and management.
>> Discussion, Debate, Agreement where do you stand on it OWASP Leaders?
>> https://www.coreinfrastructure.org/programs
>> Tom Brennan
> Tom,
> CII Badge criteria is a heavy set of checklist to control that an open
> source project complies with certain norms in different fields such as
> proper development and security

And I think the debate part is whether compliance to some "CII Badge criteria"
is going to actually change things. Case in point, OpenSSL; not to pick on
them for any reason than I already pointed them out to 200+ developers at
Stir Trek yesterday as a case of where we are failing despite trying to
do all the right things. OpenSSL is already at 100% compliance with the
CII badge criteria (see
https://bestpractices.coreinfrastructure.org/projects/54). This is in comparison
to their 64% compliance *before* Heartbleed (see
https://bestpractices.coreinfrastructure.org/projects/87). Heartbleed was
CVE-2014-0160; as per cve.mitre.org, the CVE was created 12/03/2013.
(The article that Tom referenced says Heartbleed was discovered in April 2014,
but I think that was when it was publicly disclosed, not discovered.)

Okay; now let's look at the CVE data for OpenSSL, pre and post Heartbleed:

    From 1999-2013 (pre-Heartbleed): 73 CVEs
    From 2014-current (post-Heartbleed): 76 CVEs

Well, there is a lot of things that we could attribute this to, most likely
more people hunting for vulnerabilities in OpenSSL since Heartbleed, but I
certainly fail to see that their process changes have had much of an impact.
(Of course, I suppose one could argue that had they continued processes as
usual at OpenSSL post-Heartbleed, the # of CVEs since 2014 may have been
considerably more than 76. Or it could be argued that most of those were
discovered by the OpenSSL team themselves, although a few weeks ago I had
read something that showed otherwise.)

Anyhow, my point is that if OpenSSL has 18 CVEs already in on 5 months into
2016, what possible meaning does 100% compliance with CII Badging mean?

> Being someone who has looked closely most projects code and development
> process, I can tell with confidence , most, including those labeled as
> flagship , won't be able to comply with these norms

That is likely true. I read through some of the questions that ZAP wasn't
compliant with. (See https://bestpractices.coreinfrastructure.org/projects/24
and then click on "Hide met or N/A criteria".) The only thing that shows up
is 'What is the Common Platform Enumeration (CPE) name for the project (if it
has one)?" The "(if it has one)" part implies that Simon could have left it
blank or answered as N/A and ended up with 100%.  I doubt that any of our
OWASP projects have a CPE. (And if it's as much of a pain in the ass to get
a CPE as it is for Mitre and NVD to acknowledge a CVE, I doubt there will be
many who will try to get one either, but I digress.)

Also, if you look at some of the other questions that CII Badging asks,
you can tell this seems very subjective in places. E.g., questions like
"The project MUST provide basic documentation for the software." is pretty
open-ended. It certainly concerns me that we would even allow a developer
to answer this about their own software projects. Is the CII Badging criteria
going to do follow-up on these questions to confirm they were answered
accurately and without bias? If not, then how can we trust any meaning behind
these badges in the first place. A lot of developers that I've worked with
have disdained writing documentation and they think that whatever miniscule
amount they have written is sufficient or that surely someone else will
follow up with YouTube tutorials, answering Stack Overflow questions, etc.
and that they don't really need to document stuff beyond the very basics.

There are also questions like "The project MUST acknowledge a majority of bug
reports submitted in the last 2-12 months (inclusive); the response need not
include a fix." What does that really mean unless there is someone regularly
confirming that this is still the case every few months?

There is also this question related to security:
    "A cryptographic hash (e.g., a sha1sum) MUST NOT be retrieved over http and
    used without checking for a cryptographic signature."

This question is mostly pointless because someone can claim to be mirroring
your site, but change your software and modify the hashes. A cryptographic
hash is NOT equivalent to a cryptographic signature. Given all the clueful
people on their advisory board who know about cryptography, I'm not sure
how this question even got through.

Also, it would be interesting to know if all questions are scored equally.
If the main purpose of CII Badging is to prevent another Heartbleed like
vulnerability from being made public, one would think that questions such as
    "There MUST be no unpatched vulnerabilities of medium or high severity that
    have been publicly known for more than 60 days."
ought to carry a lot more weight than one about (say) documentation. And
one really has to question the question
    "Projects SHOULD fix all critical vulnerabilities rapidly after they are
I SHOULD think that MUST is the more appropriate wording here! :)

There there is this question, under "Future":
    "(Future criterion) The project SHOULD provide a way to easily install and
    uninstall the software using a commonly-used convention."

It is not clear what is expected here, but one big concern that I have is that
over vendors (some FLOSS projects, some not) have repackaged OWASP ESAPI without
our being aware. E.g., ESAPI was retrieval as an RPM under Fedora 20 (and might
still be) and it is is also delivered as part of Oracle WebLogic Server 12.3.
The BSD license of course allows this, but if when I look at these things from
a security perspective, it raises concerns.  For instance, when there are
fixes we post them to the 2 ESAPI related mailing lists and for the 2
vulnerabilities, I was able to convince Mitre and NVD to have CVEs created.
There are other security-related issues similar to the Java deserialization
issue that Apache Commons Collections had with InvokerTransformer (see
COLLECTIONS-580) that Mitre did not grant a CVE for so it was unlikely
we would have received one for the Java deserialization issue in the method
Base64.decodeToObject() (see
either. (Not that it seems to matter; Oracle WebLogic Server 12.3 is still
using ESAPI 2.0GA from May 2011!)

Give that (for example) Red Hat provided ESAPI as an RPM so it could be
installed via 'yum', is Linux Foundation expecting that we make things
retrieval via Linux repositories via things like yum and apt-get? What
exactly are their expectations here? Of course, if no one is looking at
the answers, I guess it really doesn't matter.

> Right now I think OWASP needs to set focus on developing a better platform
> to attract developers, volunteers and project leaders, motivating them to
> produce quality projects.

I agree, but I don't think that the problem is primarily the "platform",
but rather one of motivation. I'll use myself as an example. I used
open source ever since the early USENET Newsgroups days, but until ESAPI
all I ever contributed was an occasional bug fix for a bug that irritated
me so much and no one else would seem to fix that I was motived to fix it
myself. Unfortunately, I think most of the development community was like
that. I started contributing because I thought it would be a good way to
keep my skills sharp (since as a tech lead, ironically, my management no
longer wanted me to write code).

> A volunteer program and platform that can help match volunteers with
> initiatives and projects.

Yes; and one thing that I would like to see are people in OWASP who might
be willing to sign up for short stints as "subject matter experts" on
subjects like Maven, git, JavaScript, JUnit, cryptography, C#, Java, C++,
using various frameworks like Spring and Struts, etc. I have mentioned this
before, but I will mention it again. The last ESAPI release in February 2016
could have been done maybe 6 months earlier had I known how to manually
resolve merge conflicts on pull requests on GitHub. (This was before we
had a .gitattributes file so the EOL issues between Windows and Linux caused
literally hundreds of merge conflicts.) I appealed several times to the
community until I finally found someone who explained it to a git newb like
me. If there were an SME list at OWASP and someone listed as a git / GitHub
SME, I could have just contacted her and had this resolved months sooner.
Such SME volunteers would not have to commit to any specific project on a
long term basis. I think that concern over long term commitments might be
one thing that is keeping people from stepping up and volunteering.

> Producing a quality project like ZAP needs dedication and resources
> including a deep commitment to make it work. ZAP project leader and
> volunteers work 100% on ZAP, this is by no means a 'hobby' or side
> project.Even so ZAP is right now 92% compliant with the CII criteria and
> still needs to work on it.
> Most project leaders are doing this as side-hobby projects and in this way ,
> we will never be able to pull off projects compliant with CII criteria.Most
> are lonely leaders building their projects when they have time and once in a
> while they have the collaboration of contributors.


> So we need to be realistic and be careful not to impose projects a criteria
> or process they will never be able to fulfill without the right platform and
> incentives.
> As I mentioned before I strongly recommend to focus on creating and building
> a volunteer program and really think through how to attract and retain
> volunteers, create initiatives that can help produce quality projects and
> work with those project leaders looking for help.

I'm open to suggestions of how we can do that.

> Collaboration and support is the key for creating meaningful and lasting
> open source projects.
> Regards
> Johanna

Thanks for taking the time to read my rants. As far as the CII Badges go,
I think it is well-intentioned, but unlikely to improve either security
or quality of participating projects, but I'm willing to give it a go.

Blog: http://off-the-wall-security.blogspot.com/    | Twitter: @KevinWWall
NSA: All your crypto bit are belong to us.

More information about the OWASP-Leaders mailing list