[Owasp-leaders] Metrics

Tony UV tonyuv at owasp.org
Tue Nov 30 12:46:54 EST 2010


Noodle away. I think it’s a superb idea and one far overdue.  In terms of knowing InfoSec pros that have operational BI/ EDW experience, I actually know several.  As much as a unicorn as they seem to be, there are several that originated in business ops or where immersed in IT ops that were closely aligned to supporting biz ops so they have that background to their security service delivery.  In fact our entire team is pretty much that life story.  The hybrid approach (meshing meaningful ops data with supportive InfoSec data and initiatives) is still very much in its infancy in most places not even on the horizon.  

 

>From a Word to product standpoint, I heard that Modulo has upped their product to include a multi-dimension of frameworks, but I don’t know if the usual suspects are present in terms of IT controls (CoBIT, ISO 20000 (service delivery), ISO 27001-2 (InfoSec), NIST 800-53, etc) and beyond (COSO-financial) and/ or if they have baked in things from OWASP, WASC, etc and even other sources related to maturity modeling in security like SAMM or BSIMM.

 

Let me know if and how I can help.

 

Tony UcedaVelez, CISM, CISA, GSEC

Chapter Lead

OWASP Atlanta

http://www.owasp.org/index.php/Atlanta_Georgia

Twitter: @versprite

 

From: owasp-leaders-bounces at lists.owasp.org [mailto:owasp-leaders-bounces at lists.owasp.org] On Behalf Of James McGovern
Sent: Tuesday, November 30, 2010 12:21 PM
To: owasp-leaders at lists.owasp.org
Subject: Re: [Owasp-leaders] Metrics

 

In my day job, I am currently working on a compliance initiative (EU Solvency II) where concepts such as operational risk are starting to become of interest to regulators. Sadly, no one has provided the “data model” for how the amalgamation of infosec stuff can provide a business view. At best, we have different taxonomies provided by different organizations with little ability for an end-user to stitch together. If you were to look at the world of Static Analysis, at best they provide basic reporting (counts, groups, sums, etc) but don’t allow you to ask more interesting information such as what a data warehouse/business intelligence(BI) discipline would provide. I am noodling whether the creation of a common and open OLAP model in this space ala data.gov would help end users do their own reporting.

 

Only a BI approach will allow us to analyze from threat to vulnerability to event to finding to risk, etc. Anyone know of infosec pros that understand EDW/BI deeply (better than my three months worth of experience?)

 

Tony, I absolutely agree that we should understand what other measurement criteria exists. This would be the first step. The second step would be to harness all the work that currently lives in Word documents where people have attempted to map A to B. Examples include CWE to Top Ten, SAMM to BSIMM, etc. All of this mapping needs to be leveraged in more than just word docs and I believe that BI/OLAP could unleash additional value.

 

James McGovern
http://twitter.com/McGovernTheory

 

From: Tony UV [mailto:tonyuv at owasp.org] 
Sent: Monday, November 29, 2010 4:56 PM
To: owasp-leaders at lists.owasp.org
Cc: James McGovern
Subject: RE: [Owasp-leaders] Metrics

 

James,

 

I think this is a great idea and I would love to help.  Long overdue.  I do think that a common baseline is needed or simply an agreement on what existing baseline of x, y, z consists of is needed.  Whether x = CVSS for vuln rating, and y = FIPS 199 (HIGH, MOD, LOW) for impact analysis, and so on, I think we need to see what existing baseline of measurement criteria is out there for respective security disciplines (security incident monitoring, pen testing, web application testing (dyn/ static analysis), etc, before creating an amalgamation of source information.  

 

Jeff,

 

I looked at your presentation and saw your email.  I couldn’t marry what your response on data labeling how to do with security metrics outside of the fact that in order for metrics to be applied, proper labels or categories of infosec info needs to be defined, but I couldn’t tell if your research spoke against or for this in general.

 

Best,

 

Tony UcedaVelez, CISM, CISA, GSEC

Chapter Lead

OWASP Atlanta

http://www.owasp.org/index.php/Atlanta_Georgia

Twitter: @versprite

 

From: owasp-leaders-bounces at lists.owasp.org [mailto:owasp-leaders-bounces at lists.owasp.org] On Behalf Of Jeff Williams
Sent: Monday, November 29, 2010 2:36 PM
To: owasp-leaders at lists.owasp.org
Cc: James McGovern
Subject: Re: [Owasp-leaders] Metrics

 

Hi James,

 

I studied a number of different labeling regimes and presented the results at OWASP AppSec DC 2010.  Here are the slides and what I intended to say is in the notes.

 

http://www.owasp.org/images/1/17/2010-11_OWASP_Software_Labels.pptx

 

The key finding from my research is that the content of the label doesn’t really matter that much. Even though it seems like the point is to inform the consumer, that doesn’t work very well.  Actually what you end up doing is affecting the producers.  Which is probably what we wanted to achieve in the first place.

 

You can try my “Security Facts” label generating software at:

 

https://www.aspectsecurity.com/SecurityFacts

 

Have fun!

 

--Jeff

 

Jeff Williams, Chair

The OWASP Foundation

work: 410-707-1487

main: 301-604-4882

 

From: owasp-leaders-bounces at lists.owasp.org [mailto:owasp-leaders-bounces at lists.owasp.org] On Behalf Of James McGovern
Sent: Monday, November 29, 2010 2:02 PM
To: owasp-leaders at lists.owasp.org
Cc: Rohit Sethi
Subject: [Owasp-leaders] Metrics

 

Monday Thoughts.  We are great at capturing metrics around activities (number of incidents), money (security budgets) and even innovative ways to count things (CWE, CVE, etc) yet security still isn’t “visible” to the masses.  Jeff Williams and the Rugged crowd, threw out the idea of software ingredients. Should we attempt to capture that notion as part of Rohit’s Web Application Security Framework Manifesto where the protections are more of the ingredients than things such as weaknesses?

 

If you are a baseball fan (I am not), there is a popular book named MoneyBall where they have figured out how to measure/model the performance of baseball players which aided in creating market efficiencies. This was huge for baseball and I believe that infosec needs the equivalent. Is there merit in the metrics project figuring out how to build a data warehouse (think data.gov) that everyone could query to understand their security posture? It could contain products, vulnerabilities, weaknesses, ingredients, etc. We could even figure out a novel way of including the “findings” format that Dinis Cruz often leverages to discover new insights.

 

James McGovern
Insurance SBU 

Virtusa Corporation

100 Northfield Drive, Suite 305 | Windsor, CT | 06095

Phone:  860 688 9900 Ext:  1037 | Facsimile:  860 688 2890  

 <http://www.virtusa.com/> cid:image011.jpg at 01CB08A4.F95CFA30  <http://www.virtusa.com/blog/> cid:image012.gif at 01CB08A4.F95CFA30  <https://twitter.com/VirtusaCorp> cid:image004.gif at 01CB08A4.F95CFA30  <http://www.linkedin.com/companies/virtusa> cid:image005.gif at 01CB08A4.F95CFA30  <http://www.facebook.com/VirtusaCorp> cid:image006.gif at 01CB08A4.F95CFA30

 


Virtusa was recently ranked and featured in 2010 Deloitte Technology Fast 500, 2010 Global Services 100, IAOP's 2010 Global Outsourcing 100 sub-list and 2010 FinTech 100 among others.
 
---------------------------------------------------------------------------------------------
 
This message, including any attachments, contains confidential information intended for a specific individual and purpose, and is intended for the addressee only. Any unauthorized disclosure, use, dissemination, copying, or distribution of this message or any of its attachments or the information contained in this e-mail, or the taking of any action based on it, is strictly prohibited. If you are not the intended recipient, please notify the sender immediately by return e-mail and delete this message.
 
---------------------------------------------------------------------------------------------

 


Virtusa was recently ranked and featured in 2010 Deloitte Technology Fast 500, 2010 Global Services 100, IAOP's 2010 Global Outsourcing 100 sub-list and 2010 FinTech 100 among others.
 
---------------------------------------------------------------------------------------------
 
This message, including any attachments, contains confidential information intended for a specific individual and purpose, and is intended for the addressee only. Any unauthorized disclosure, use, dissemination, copying, or distribution of this message or any of its attachments or the information contained in this e-mail, or the taking of any action based on it, is strictly prohibited. If you are not the intended recipient, please notify the sender immediately by return e-mail and delete this message.
 
---------------------------------------------------------------------------------------------

 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: https://lists.owasp.org/pipermail/owasp-leaders/attachments/20101130/4168bd82/attachment-0001.html 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: image/jpeg
Size: 1397 bytes
Desc: not available
Url : https://lists.owasp.org/pipermail/owasp-leaders/attachments/20101130/4168bd82/attachment-0001.jpe 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: image/gif
Size: 744 bytes
Desc: not available
Url : https://lists.owasp.org/pipermail/owasp-leaders/attachments/20101130/4168bd82/attachment-0004.gif 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: image/gif
Size: 1211 bytes
Desc: not available
Url : https://lists.owasp.org/pipermail/owasp-leaders/attachments/20101130/4168bd82/attachment-0005.gif 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: image/gif
Size: 789 bytes
Desc: not available
Url : https://lists.owasp.org/pipermail/owasp-leaders/attachments/20101130/4168bd82/attachment-0006.gif 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: image/gif
Size: 763 bytes
Desc: not available
Url : https://lists.owasp.org/pipermail/owasp-leaders/attachments/20101130/4168bd82/attachment-0007.gif 


More information about the OWASP-Leaders mailing list