[Owasp-webcert] Assurance Levels

Mark Curphey mark.curphey at sourceclear.com
Mon Jun 11 04:26:47 EDT 2007

I propose to make assurance levels an integral part of the OWASP Web Certification Criteria and want your feedback on the concept.

In many ways it's one of those things that's so damn obvious when you see it described with clarity. Enlightenment came for me when Chris Wysopal <http://www.veracode.com/management-team.php#Wysopal>  sent me the fantastic graphic atttached describing Veracodes <http://www.veracode.com/>  view of assurance levels.  Of course it is nothing new, the basic concept of assurance (confidence) is as follows;

	Different testing techniques provide different levels of assurance (confidence) on claims about the security of a web site. 

An automated static analysis tool will provide a lower level of assurance than an automated dynamic analysis tool which will in-turn provide a lower level of assurance than a comprehensive manual code review.  It also follows that an automated web application penetration test will provide a lower level of assurance than a manual penetration test. Both types of penetration testing will provide lower levels of assurance than code reviews. It also makes sense that if a company has demonstrated that security is an integral part of the security DNA of their SDLC (define, design, develop, deploy and maintain) then there is a higher level of assurance that any test results will be consistent in the future. 

So why wouldn't everyone just go for the approach that provides the highest level of assurance? It's very simple, cost. The appropriate level of assurance should be based on risk. 

Of course all of this things have a butterfly effect, no two tools are the same and no two testers are the same. Imagine a control panel with multiple dials but where people want a single output display (not necessarily a single reading).  I expect lots of people arguing that a specific tool or firm is as good as the next level up on the assurance level but well deal with that as well.

This also enables us to define what a web app firewall is good for, what it isn't and place it into an assurance level bucket. More on that in a while. 

By incorporating assurance levels into the criteria, industry sectors, business partners or regulators can require a level of security with an assurance level based on risk. This would be a significant step forward from where we are today with broken schemes like PCI DSS <http://securitybuddha.com/2007/03/23/the-problems-with-the-pci-data-security-standard-part-1/> .

So what do y'all think?

-------------- next part --------------
An HTML attachment was scrubbed...
URL: https://lists.owasp.org/pipermail/owasp-webcert/attachments/20070611/184e9b1d/attachment-0001.html 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: mapping-assurance-to-testing.pdf
Type: application/pdf
Size: 66303 bytes
Desc: mapping-assurance-to-testing.pdf
Url : https://lists.owasp.org/pipermail/owasp-webcert/attachments/20070611/184e9b1d/attachment-0001.pdf 

More information about the Owasp-webcert mailing list