[Owasp-leaders] [Owasp-topten] OWASP Top 10 Methodology

Dave Wichers dave.wichers at owasp.org
Fri Mar 15 14:03:21 UTC 2013

The only hard numbers currently used are vulnerability stats that help us
determine the typical prevalence of these kinds of vulnerabilities.


Ryan and others are trying to gather stats related to the prevalence of
actual attacks, which right now we haven't used, so we currently are using
our professional opinion. And what we actually need is the prevalence of
successful attacks, which is very hard to know/measure.


If you can contribute to their efforts to gather actual attack statistics in
some way, that would be great.




From: Tony UV [mailto:tonyuv at owasp.org] 
Sent: Thursday, March 14, 2013 11:53 PM
To: Dave Wichers; Rory McCune; Ryan Barnett
Cc: OWASP Leaders; OWASP TopTen
Subject: RE: [Owasp-leaders] [Owasp-topten] OWASP Top 10 Methodology


Dave/ Ryan:


I suppose I should read the methodology on the OWASP Top 10 thread but
wanted to inquire about whether what makes up the top 10 is driven on what
is actually captured by a range of honeynet projects, vendor products or
partnering global MNCs willing to share incident/ alert data in arrears?  If
this is the case, then all top 10 issues really reflect a research approach
versus a sentiment over what seems to be more prevalent.  I also agree that
DoS/ DDoS attacks, particularly across FIs, is extremely prevalent - however
this is based upon experience with companies in those fields and not based
upon hard numbers.  SO the question is, are hard numbers obtained to support
the top ten or are they driven by a conclave of security perceptions?  




Tony UV


Sent from tablet device - please excuse any typos


From: Ryan Barnett <ryan.barnett at owasp.org>
Sent: March 14, 2013 2:05 PM
To: Dave Wichers <dave.wichers at owasp.org>,Rory McCune
<rory.mccune at owasp.org>
CC: OWASP Leaders <owasp-leaders at lists.owasp.org>,OWASP TopTen
<owasp-topten at lists.owasp.org>
Subject: Re: [Owasp-leaders] [Owasp-topten] OWASP Top 10 Methodology



I agree with all of your points.  Determining the estimated Risk level for
DDoS is challenging, especially when you consider the org's vertical market.
As you referenced in your BankInfoSecurity story, Finance verticals are
being targeted as part of a multi-pronged attack where DDoS are used as a
smoke-screen for AHC transfers.  The attacker us a combination of Banking
Trojans client-side with C&C to IRC botnets which can launch DDoS floods.
So the impact of the website downtime will be removed once the attack stops,
however the end results is that funds were also stolen.


I will get with Pawel and see if we can put together a Risk rating
estimation for consideration.




From: Dave Wichers <dave.wichers at owasp.org>
Date: Thursday, March 14, 2013 1:52 PM
To: 'Rory McCune' <rory.mccune at owasp.org>
Cc: Ryan Barnett <ryan.barnett at owasp.org>, 'OWASP Leaders'
<owasp-leaders at lists.owasp.org>, 'OWASP TopTen'
<owasp-topten at lists.owasp.org>
Subject: RE: [Owasp-topten] [Owasp-leaders] OWASP Top 10 Methodology


I was thinking that was one option too. To add DDOS as a serious network
level threat to Apps, but have it not directly be in the Top 10.


But that is also a bit weird and opens up the question 'Well, what else
belongs in that other list?"


It appears that 20%-25% of the DOS attacks are against the app directly,
from the stats Ryan references in his email anyway. So that's not an
inconsequential #. And even if you ignore the network level DOS attacks,
that probably still ranks DOS in the Top 5 most common application attacks
anyway, possibly in the Top 2-3. Top 5 successful attacks? That's harder to
figure out. And DOS is not a black/white issue. No matter what, an attack
slows you down, and so in some sense is somewhat successful no matter what.
Does it have a significant impact on the apps users, that to me would be
considered a successful attack. Another thing weird about DOS is the benefit
of the attack is primarily only during the attack. Once it stops, it doesn't
have much long term impact, whereas stealing sensitive info/credentials has
a serious long term impact.


Ryan, do you and Pawel, and whoever else is interested want to try to take a
shot at calculating the risk of DOS to web apps using the OWASP Top 10 risk
rating methodology to see how you think it scores? In terms of prevalence
data, app level DOS vulnerabilities are rarely found/reported, but I suspect
that's primarily because people rarely look for them.


Given the prevalence data you've been looking at, it looks like DOS would
rank at least a Medium. Again, prevalence is likelihood of successful attack
in the Top 10 methodology, not just likelihood of attack. But as I just
said, the definition of success for a DOS attack is a bit fuzzy. If we said
success was the site was effectively unavailable during the attack, then
that would clarify the definition of success. Although how long it was
unavailable probably merits considering in the definition of success too. I
have no idea how many attacks result in the site essentially being
unavailable, nor for how long.




From: Rory McCune [mailto:rory.mccune at owasp.org] 
Sent: Thursday, March 14, 2013 1:34 PM
To: Dave Wichers
Cc: Ryan Barnett; OWASP Leaders; OWASP TopTen
Subject: Re: [Owasp-topten] [Owasp-leaders] OWASP Top 10 Methodology




I'd agree (although without strong data to back that up) that my perception
is that most DoS attacks are network level DDoS, primarily due to the ease
of putting together (or buying) and using a Botnet.


I suppose that the challenge is that business look at risks which impact
their assets and these days external facing systems tend to be web
applications so most threats target them, so they see more threats as being
relevant to web applications than are actually in the domain of the
application itself.


Is there scope to include an "others" appendix to the Top 10 to put context
round why other issues which are related to apps but not primarily protected
against at the app level (with this being an example) weren't included.
Having said that though it perhaps just moves the problem a little bit and
then we'd end up with a huge appendix of things which impact applications..






On Thu, Mar 14, 2013 at 4:41 PM, Dave Wichers <dave.wichers at owasp.org>

I definitely agree that there are things that APPS can do to help prevent
app level DOS attacks.


But here's the issue.  The OWASP Top 10 is about the current biggest risks.
And I suspect, but don't know for sure, that most real world DOS attacks
against apps are actually against the apps infrastructure, not the app
itself. We need to look into the details of whatever metrics we have access
to, to see if that is true.


So, if the real world attacks are against the infrastructure, then if we add
DOS to the Top 10, we should talk about infrastructure defenses, since
that's what is really happening.  We don't want to add DOS to the Top 10
because its common, and then present a bunch of app level defenses that
don't actually stop the kinds of attacks that are actually occurring.


And adding a bunch of network level defenses against DOS to the OWASP Top 10
feels a little weird. And to others, NOT having DOS in the Top 10 feels a
little weird too.


As I said, this is a tricky issue for the Top 10.




From: Rory McCune [mailto:rory.mccune at owasp.org] 
Sent: Thursday, March 14, 2013 12:36 PM
To: Dave Wichers
Cc: Ryan Barnett; OWASP Leaders; OWASP TopTen
Subject: Re: [Owasp-topten] [Owasp-leaders] OWASP Top 10 Methodology




I think that there are some things that app. developers /owners could do to
address app DoS (although as you say I think that network DDoS is more


The kind of App DoS I'm thinking of would be where a simple GET or POST
request could trigger a computationally expensive transaction on the
application database or server. So for example something like a large
database query that's triggered by a product search.  Presumably the
application has been tested for standard usage patterns but may not have
been tested for someone making very large numbers of searches quickly.


In terms of mitigation, I'd say that there would be a two phase approach.
First would be identification of what transactions/requests caused large
processing loads and secondly would be implementing some form of protection,
which could take the form of basic rate limiting for a given transaction or
perhaps at a more advanced level detecting an unusual usage pattern (i.e.
ordinary users browse from the login page through to the search page and
then search once, whereas these IPs are hitting the search page repeatedly
without any other page visit) and then blocking/limiting those IPs in
relation to those transactions.


The advantage of defending this at the application layer is that there's
likely to be more visibility/understanding of what constitutes unusual
behavior and also what the high processing requirement transactions are.






On Thu, Mar 14, 2013 at 4:26 PM, Dave Wichers <dave.wichers at owasp.org>

Hey everyone. Related to DDOS, (Today's latest event is:
http://www.bankinfosecurity.com/ddos-6-banks-hit-on-same-day-a-5607), do we
have any stats/metrics on how many DDOS attacks are at the application level
vs. the network level?


The OWASP Top 10 is about Web Apps, not network security. And I know if they
DDOS the server and take out the app, then it's an app problem, but is there
anything the APP itself can do about the most common DDOS attacks?


I'm trying to figure out, if we added DDOS to the Top 10, what advice we
could provide to developers/app owners on how to mitigate this risk? And if
all the advice is at the network level, because that's the best / easiest
place to defend against this, does that belong in a top 10 list for apps?
Maybe/Maybe not.


I'm trying to encourage discussion here. I'm not saying I don't think it
belongs in the Top 10. This is tricky/complex issue worth discussing.





From: Ryan Barnett [mailto:ryan.barnett at owasp.org] 
Sent: Wednesday, March 13, 2013 11:00 AM
To: Dave Wichers
Cc: Michael Coates; OWASP Leaders; OWASP TopTen

Subject: Re: [Owasp-leaders] OWASP Top 10 Methodology


FYI - I have added links to sample attack reports to this page -





On Tue, Mar 5, 2013 at 9:33 AM, Dave Wichers <dave.wichers at owasp.org> wrote:

Thanks Ryan for taking the lead on this step of the methodology. I'm very
interested in seeing what the various attack metric sources we can get our
hands on say about the prevalence of different kinds of attacks.


One comment about the prevalence factor in the Top 10 is that its definition


The likelihood that an attacker would successfully attack the application
given this vulnerability.  I could imagine some attack metrics only measure
attempts to attack (like random DOSing, or random attempts at SQL
injection/XSS) but don't or can't measure the number of actually successful


And I think the likelihood of success is pretty important. Take Reflected
XSS for example. It's pretty prevalent, it's pretty easy to find, but it can
be hard to successfully pull off.


Don't get me wrong, I think knowing what attack attempts are actually
occurring out there in the wild is great information to know. But I'm not
sure if that data is an exact match to what we consider the likelihood of
actual successful attack in the Top 10 as its defined today.




From: owasp-leaders-bounces at lists.owasp.org
[mailto:owasp-leaders-bounces at lists.owasp.org] On Behalf Of Ryan Barnett
Sent: Tuesday, March 05, 2013 9:25 AM
To: Michael Coates; OWASP Leaders; OWASP TopTen

Subject: Re: [Owasp-leaders] OWASP Top 10 Methodology


With regards to "Additional data sources to be considered" Enhancement item
- I am contacting various vendors that I listed to try and get access to web
attack metrics.  I have heard back from both Akamai and Incapsula and they
are willing to share so I will work with them.


I will update the group when I have more info.




From: Michael Coates <michael.coates at owasp.org>
Date: Saturday, March 2, 2013 7:15 PM
To: OWASP Leaders <owasp-leaders at lists.owasp.org>, OWASP TopTen
<owasp-topten at lists.owasp.org>
Subject: Re: [Owasp-leaders] OWASP Top 10 Methodology



The OWASP Top 10 Methodology wiki page (as described in the below email) is
now live - https://owasp.org/index.php/Top_10_2013/ProjectMethodology

As you'll see in the first line of the wiki - "The goal of this page is to
provide the baseline of knowledge to begin a thoughtful conversation of
enhancements and changes to continue growing the OWASP top 10."

Next Steps:

- Have ideas on how we can enhance the methodology? Please add it here

- We'll then begin making changes based on these ideas

Overall Goal:

Increase participation, enhance methodology, and continue to grow the
excellent OWASP top 10 resource 

Thanks for everyone's hard work so far on the Top 10 and all the good ideas
that have been floating around. I'm confident we can all work together as a
community to make this next top 10 awesome.  I look forward to continuing
this conversation with everyone.

Michael Coates | OWASP | @_mwc


On Tue, Feb 26, 2013 at 12:05 PM, Michael Coates <michael.coates at owasp.org>

Leaders & Top 10 Enthusiasts,

Dave and I had a great conversation today about the Top 10 and some of the
questions that have been posed by many in our owasp community.

We're going to build a wiki page that describes the overall project
methodology of the owasp top 10, what's currently happening, suggestions for
improvements, and an FAQ.

The project has continually grown over the various releases and has
successfully attracted more worldwide attention. As we've grown as an
organization we've seen many new ways to further open the top 10 and invite
greater participation.

This methodology wiki page will help clarify the activities to date and
provide a feedback channel to continue growing.

Please look for this page later this week. It would have been great for me
to include the completed page with this email, but it will take a day or two
and I wanted to send this info to the list now.


Michael Coates | OWASP | @_mwc


_______________________________________________ OWASP-Leaders mailing list
OWASP-Leaders at lists.owasp.org


Owasp-topten mailing list
Owasp-topten at lists.owasp.org



OWASP-Leaders mailing list
OWASP-Leaders at lists.owasp.org

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.owasp.org/pipermail/owasp-leaders/attachments/20130315/171a9c27/attachment-0001.html>

More information about the OWASP-Leaders mailing list