[Owasp-topten] Top 10 certification discussion with Raoul today

Andrew van der Stock vanderaj at owasp.org
Wed Jul 5 10:39:54 EDT 2006


Hi there,

I had lunch with Raoul today, and we had an excellent discussion.  
Basically, we came up with a way to manage the Top 10 certification  
quite well.

I suggest we:

* write the Top 10 in three sections (attacks, prevent root causes,  
what you need in your software security program), so it's really the  
Top 30. Keep each domain to 10 pages, so it's still going to be a  
roughly 40-50 page document. Any longer and we've failed as an  
education piece.

* Write certification overlays for each of the Top 10's three domains.

This allows real people and vendors to be sifted out of the dross of  
snake oil merchants. In the overlays *we* dictate what it actually  
means to comply with each point, rather than letting snake oil  
marketing people make outrageous claims. As some things are harder  
than others for some types of device or service, it makes sense to  
have an increasing scale of trust from self-assured to fully assured.

Once bootstrapped, we can create income from certifying products and  
ensuring that only certified products  and services can carry our  
marks. Initially, this would be direct income, but later, it would  
have to come from the membership fees of the certification providers.

* Certification

Certification prevents the OWASP brand being diminished and weeds out  
the snake oil merchants. However, the existing OWASP logo has  
probably been diluted and we've done only a little to ensure  
compliance and correct usage. So let's make it much more obvious  
about logo usage and possibly increase the trust of the Top 10 by  
creating *specific* assessment logos and phrases.

Self-assured (Bronze Logo)
- self assessment (for free) which can be randomly audited by OWASP  
at any time. This is the ideal choice for open source software and  
has a low barrier of entry.

Assured (Silver Logo)
- assured (for a fee) by OWASP appointed resources for partial  
coverage of a particular domain

Fully Assured (Gold Logo)
- assured (for a fee) by OWASP appointed resources for full coverage  
of all relevant Top 10 domains

OWASP should insist on the right of review to all three levels, and  
revoke them if it turns out (particularly in self-assessment) that  
they lied to us.

We would have to create new logos and trademark the logo and phrase  
used.

* Train the Trainer

We need to provide training materials to bootstrap this process. I'm  
presenting a three+ hour tutorial at OSCON, which is so close to what  
is required I'm going to turn it into Top 10 training materials and  
donate it to the OWASP Project.

We have a pool of acknowledged people here (and on the certification  
project). Initially, let's bootstrap it by creating a pool of say 5  
peers. They will become the initial seed volunteers. We will need  
more. Once it gets going, it will be possible to earn a living doing  
this stuff as a certification provider (see below).


*Companies* who wish to be "certified" Silver or Gold OWASP compliant  
devices or services will need to:

a) maintain current OWASP membership as per their status

b) provide proof that their services or equipment satisfies the Top  
10 by describing in a paragraph or two about how it did that, such as  
with screen shots and repeatable steps. If we're convinced, we go to  
the next stage

c) send sample equipment to the nominated testers, or fly the  
nominated testers (obviously will need to organize a bit more here)  
to the testing site

d) allow the tester to perform any arbitrary (non-destructive) tests  
against the unit or device or software as to suffice to the tester  
that their claims are true.

e) Like Thawte's web of trust, we need to audit results. We make it a  
condition of usage of the OWASP Certified mark / title that we can  
ask their customers / users how they're going, or if they have no  
customers/users, ask them to re-certify.

f) Re-certification. Every twelve months or if the service or product  
changes in a significant fashion, we need to insist on re-certification.

Companies which pass will be granted the time limited right to carry  
the Silver or Gold certified logo and trademarked certification phrase.


*People* who wish to be "certified" OWASP web app sec reviewers will  
need to:

a) maintain current OWASP membership as per their status

b) find an open source product of their choosing and go through each  
of the Top 10 attacks and write a paragraph on what they found and  
how they would fix it, referencing the Top 10 remediations / other  
OWASP materials, and so on. They would provide a copy of this report  
to a private mail list we manage and to the open source project  
they've reviewed. We could talk to Sourceforge about creating this as  
a service to the open source community so that projects could  
nominate themselves for help.

c) At least three reviewing peers go over the supplied submission and  
if they all agree with the submission (Pass / Fail). With 100%  
agreement, the submitter is given the ability to use a new OWASP mark  
and the title "OWASP Certified Reviewer(tm)". If 2 out of 3 agree,  
the three agree to a list of questions to fire back to the submitter  
and get them to answer. If the answers are inadequate, the submitter  
is failed, otherwise a pass.

d) Like Thawte's web of trust, we need to audit results. We make it a  
condition of usage of the OWASP Certified Reviewer mark / title that  
we can ask their customers / users how they're going, or if they have  
no customers/users, ask them to re-certify.

e) Re-certification. Every three years, we need to insist on re- 
certification. I know that I knew a lot more .NET three years ago  
than I know now. Let's ensure that we don't let people become stale.

Individuals who pass will be granted the time limited right to carry  
the Bronze, Silver or Gold certified logo and trademarked  
certification phrase.

Global spread

We need this to be available in every major market. So what we need  
is to have is a competitive market for certification services, whilst  
maintaining the quality of the certification.

Therefore, let's figure out a way of getting keen web app sec  
security firms (and new opportunities for OWASP members) to provide  
the certification services - as long as they are suitable. As many of  
them will also have certified reviewers, we need to create a rule  
that prevents any firm from certifying itself, and ensure that  
certification providers keep to our standards.

OWASP (sets the standards, audits results, and initially will be the  
sole certification provider)
|
---    Certified OWASP Reviewers (provides reliable reviewing services)
|
---    Approved OWASP Certification Providers (provides certifications)
|
---    Certified OWASP Products (provides compliant products which  
are certified by the previous two)

In this model, initially OWASP would have to bootstrap the process.  
If we set the standards and audit compliance with our standards, I  
think from a basic trust point of view, we will need to step out of  
the certification market once established, unless there are no local  
certification providers.

Once the market has been established, about the only form of  
certification left with us is the self-assured category as it will be  
provided for free using volunteers. I see this only being useful for  
open source projects / open source authors.

Thoughts?

thanks,
Andrew 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 2458 bytes
Desc: not available
Url : http://lists.owasp.org/pipermail/owasp-topten/attachments/20060706/d292e3fd/attachment-0003.bin 


More information about the Owasp-topten mailing list