[Owasp-testing] Are the Risk Rating Wiki Pages Broken?

Pete Herzog lists at isecom.org
Tue Aug 27 15:30:49 UTC 2013

Hi Christian,

On 8/27/2013 2:09 AM, Christian Heinrich wrote:

> The link to the CVSS-SIG is prominently displayed on the left side of
> the http://www.first.org/ homepage

Can you provide me the link to the document as I still can't find it.
I found a presentation, a letter, and some stuff but not the CVSSv3
document itself.

> OWASP has also followed a similar "secret" process as
> perhttp://lists.owasp.org/pipermail/owasp-topten/2013-January/000831.html

How is it secret if anyone can join? If I hold a party in a room and
invite everyone as long as they bring something or help set the tables
is it a secret party because you didn't come?

> Yes, I have been familiar with OSSTMM since release v2 (maybe earlier)
> from memory.  The reason that I walked away from it was that it was
> full of self righteous statements related to ethics and "Hacking
> Exposed" has more technical content while being what is a
> sensationalised publication (i.e. "Hacking Exposed").

There'll always be haters. But what you point to is the Rules of
Engagement which were designed to protect the tester not force ethics
on them. You don't have to follow them to do an OSSTMM test but they
will help you as a security professional. Too often pen testers were
caught in Catch22 situations where they did their job but the
remediation wasn't done or some contract details were missing, or the
client made the scope too narrow and then there were problems, the
company gets hacked, and people point and say, XYZ did our security!
And XYZ signed an NDA that doesn't let them refute or explain that
they tested under restrictions or had nothing to do with remediation.

Anyway, we never wanted to compete with hacking exposed. We're a
research organization looking for the best way to test and analyze
operations to secure them. That's enough work without trying to detail
how to exploit each bug that might not even be a technology in use

> Neither would the markings of both "Alpha" and "Draft" convince me to
> read your webappsec methodology since it doesn't have the same peer
> reviewed contributors as the OWASP Testing Guide,
> http://mdsec.net/wahh/, etc.

Okay. Sorry we lost you then.

> Raoul Chiesa and I know each other from presenting at the same
> conference in Europe. Coincidently he want me to replace
> http://msmvps.com/blogs/spywaresucks/archive/2011/12/10/1803461.aspx
> as the Australian representative for  http://www.isecom.org/team.html
> because of their well known lack of skill.

I like Ty and I think he's a capable tester. You posting a link to a
blog where someone states their opinion doesn't prove a "well known
lack of skill". And Raoul has never said anything to me about this but
I'm happy to include him here (in CC). I can't imagine he said to
replace Ty as we don't "replace" people on the team, we grow it.
Because we can all learn things from each other.

The link you post about Ty's skill again goes back to what I said
about that Catch 22. I don't know details or have access to the
confidential security reports as you apparently do but when I see that
I see so many possibilities where someone got it wrong that I don't
immediately blame the tester. There's the scope of the test,
restrictions on types of tests, remediation, etc. I just hope you
never find yourself in that situation as it can happen to ANY tester.

> After watching the heckling from the audience at
> http://www.sector.ca/speakers2008.htm#Pete_Herzog I can't see the
> additional value in travelling to North Carolina.  I believe the

I'm just so glad they didn't do it out loud as that would have thrown
me off my presentation! I didn't even know about it. It was hard
enough since I was presenting on the New New Thieves, which was based
on work that Raoul Chiesa (you mentioned above as your endorsement)
pioneered with the United Nations for profiling hackers.

> positive reviews of the North Carolina event are attributed to Michael
> Menefee not disclosing his conflict of interest i.e.
> http://www.isecom.org/team.html.

Mike joined ISECOM about a month after that event. I don't see how
that's a conflict of interest to invite me as a speaker either though.
As an example of what might be a conflict of interest is someone
trying to shut down OWASP risk metrics because of their work CVSS v3.
But that's just an opinion too.

> This endorsement of the OWASP Risk Rating Methodology is the similar
> to that from people who believe vulnerability assessment is a
> different name of penetration testing and therefore consider OSSTMM as
> a creditable body of knowledge.

I really don't follow that line of thinking. But I do believe nobody
sits on all the great ideas- no person and no group. I think people
should work in the areas they see they can improve because of their
specialized knowledge and even if it's not a flawless body of work,
there are still likely to be some worthwhile ideas in it. Having
worked with attack surface metrics on web apps I know it is hard and
so I'm interested to see how another approaches it. And that's how
science works.

OSSTMM is neither vuln testing or pen testing. It's a methodology
which shows that no matter what you call what you do, you need to get
XYZ intelligence from it so you know where your problems are. If you
can do that with a vuln scanner or a red team or a magic wand, that's

> Due to criticisms of OSSTMM from multiple parties with no relationship
> to each other (hence no collusion) i.e.

Research is supposed to have critics. That's how progress happens. And
then whenever you put yourself out there, well, there'll always be
haters. That's life.

> http://infosecwarrior.wordpress.com/2011/05/25/ptes-penetration-testing-execution-standard/,
> https://twitter.com/mhackling/status/46376971336290304, etc OWASP has
> aligned itself with PTES too i.e.
> https://twitter.com/indi303/status/78476821305360385 and
> https://twitter.com/brennantom/status/78431000081858560.

So some blog that makes up a percentage in passing about the OSSTMM
and 3 twitter feeds is "multiple criticisms"? Your old drama teacher
must be proud.

Stop being so negative and so conflicted and just enjoy that there's
so many people trying to figure out how best to secure the world.
There's a lot of work to do so just do your thing and stop trying to
block others from doing theirs.


Pete Herzog - Managing Director - pete at isecom.org
ISECOM - Institute for Security and Open Methodologies
www.isecom.org - www.osstmm.org
www.hackerhighschool.org - www.badpeopleproject.org

Pete Herzog - Managing Director - pete at isecom.org
ISECOM - Institute for Security and Open Methodologies
www.isecom.org - www.osstmm.org
www.hackerhighschool.org - www.badpeopleproject.org

More information about the Owasp-testing mailing list