[Owasp-testing] Are the Risk Rating Wiki Pages Broken?

Christian Heinrich christian.heinrich at cmlh.id.au
Wed Aug 28 04:41:44 UTC 2013


Pete,

On Wed, Aug 28, 2013 at 1:13 AM, Pete Herzog <pete at isecom.org> wrote:
> Can you provide me the link to the document as I still can't find it.
> I found a presentation, a letter, and some stuff but not the CVSSv3
> document itself.

CVSSv3 is still under active development i.e. I believe this was
obvious since "development" is quoted within the URI i.e.
http://www.first.org/cvss/v3/development

On Wed, Aug 28, 2013 at 1:13 AM, Pete Herzog <pete at isecom.org> wrote:
> How is it secret if anyone can join? If I hold a party in a room and
> invite everyone as long as they bring something or help set the tables
> is it a secret party because you didn't come?

If you understood this thread on the Top Ten Mailing List then OWASP
tends to lean towards transparency (including the refusal of NDAs).
This policy tends to be enforced when it suits the ulterior motive of
the person i.e for an example of OWASP own hypocrisy related to the
thread on the Top Ten Mailing List then refer to later paragraphs from
Jeff within https://lists.owasp.org/pipermail/owasp-leaders/2011-August/006011.html

On Wed, Aug 28, 2013 at 1:13 AM, Pete Herzog <pete at isecom.org> wrote:
> There'll always be haters. But what you point to is the Rules of
> Engagement which were designed to protect the tester not force ethics
> on them. You don't have to follow them to do an OSSTMM test but they
> will help you as a security professional. Too often pen testers were
> caught in Catch22 situations where they did their job but the
> remediation wasn't done or some contract details were missing, or the
> client made the scope too narrow and then there were problems, the
> company gets hacked, and people point and say, XYZ did our security!
> And XYZ signed an NDA that doesn't let them refute or explain that
> they tested under restrictions or had nothing to do with remediation.

Well is appears "regular contributors the Open Source Security Testing
Methodology Manual (OSSTMM), certified Trainers of Penetration Testing
Professionals of OSSTMM" from Pure Hacking are "...regularly .."
caught out making inaccurate claims "... in Australia’s media." time
after time e.g.
http://www.zdnet.com/thomsons-phone-clone-claims-uncertain-1339338352/

>From memory of my reading at the time it appeared that OSSTMM hadn't
considered the ramification that an experienced auditor cannot provide
their referees to prior penetration test which therefore left the
person evaluating the proposal without an independent view and
therefore might result in the lesser experienced auditor being awarded
the opportunity.

On Wed, Aug 28, 2013 at 1:13 AM, Pete Herzog <pete at isecom.org> wrote:
> Anyway, we never wanted to compete with hacking exposed. We're a
> research organization looking for the best way to test and analyze
> operations to secure them. That's enough work without trying to detail
> how to exploit each bug that might not even be a technology in use
> tomorrow.

The above doesn't convince me that you intent is to otherwise extract
as much revenue out of the market while being in denial about high
level understanding required to maintain the state of the art.

On Wed, Aug 28, 2013 at 1:13 AM, Pete Herzog <pete at isecom.org> wrote:
> Okay. Sorry we lost you then.

OSSTMM has been in a steady decline which has upheld my first impression.

On Wed, Aug 28, 2013 at 1:13 AM, Pete Herzog <pete at isecom.org> wrote:
> I like Ty and I think he's a capable tester. You posting a link to a
> blog where someone states their opinion doesn't prove a "well known
> lack of skill". And Raoul has never said anything to me about this but
> I'm happy to include him here (in CC). I can't imagine he said to
> replace Ty as we don't "replace" people on the team, we grow it.
> Because we can all learn things from each other.

To select one of several stories that I have heard, it took a
webappsec subcontractor to "Pure Hacking" 1 hour to bypass a simple
authentication framework which Ty Miller had wasted two days
attempting to figure out without any tangible result.

I'll locate the e-mail(s) from Raoul and send them to you but I would
have preferred not to do this since my brief dealings with Raoul have
always been positive since we avoided discussing his association with
convicted criminal Kevin Mitnick e.g.
https://twitter.com/kevinmitnick/status/30784499897860097

On Wed, Aug 28, 2013 at 1:13 AM, Pete Herzog <pete at isecom.org> wrote:
> The link you post about Ty's skill again goes back to what I said
> about that Catch 22. I don't know details or have access to the
> confidential security reports as you apparently do but when I see that
> I see so many possibilities where someone got it wrong that I don't
> immediately blame the tester. There's the scope of the test,
> restrictions on types of tests, remediation, etc. I just hope you
> never find yourself in that situation as it can happen to ANY tester.

Since my other recent e-mail hasn't been reflected in the mailing list
archive where I addressed this I'll predict it will be found at
http://lists.owasp.org/pipermail/owasp-testing/2013-August/date.html
later today.

On Wed, Aug 28, 2013 at 1:13 AM, Pete Herzog <pete at isecom.org> wrote:
> I'm just so glad they didn't do it out loud as that would have thrown
> me off my presentation! I didn't even know about it. It was hard
> enough since I was presenting on the New New Thieves, which was based
> on work that Raoul Chiesa (you mentioned above as your endorsement)
> pioneered with the United Nations for profiling hackers.

I won't have interpreted that as an endorsement otherwise I would have
contributed to OSSTMM.

Coincidently what you may have heard about
http://cmlh.id.au/tagged/grubbgate wasn't an accurate version of
events of what I presented at Security B-Sides in 2011 and you may
also want to pay particular attention to
http://cmlh.id.au/post/57402913158/jodymelbourne-doxed

To link this back to "Pure Hacking", I was disappointed when Ty
attempt to exploit this unfortunate situation for "Pure Hacking" gain
i.e. http://www.theaustralian.com.au/australian-it/facebook-images-open-to-access/story-e6frgakx-1226059138255,
http://www.zdnet.com.au/stuxnet-routing-hacks-and-a-seized-ipad-339315484.htm,
etc in light of:
1.  "Pure Hacking" appearance on
http://www.sbs.com.au/insight/episode/transcript/30/Stolen-IDTV and
associated comments hyped with delusion of grandeurs.
2.  "Pure Hacking" reversal on seeking permission from the end user of
social network i.e. content on social networks is public domain i.e.
http://www.zdnet.com/penetration-testing-employees-social-media-to-improve-policy-7000017234/.
 Their claim about "no one else in the world is currently doing" is
pure fabrication too due to my prior work and that of Tom Ryan i.e.
"Robin Sage", IOActive i.e. "The Leaky Web: Owning Your Favorite
CEOs", etc

On Wed, Aug 28, 2013 at 1:13 AM, Pete Herzog <pete at isecom.org> wrote:
> Mike joined ISECOM about a month after that event. I don't see how
> that's a conflict of interest to invite me as a speaker either though.
> As an example of what might be a conflict of interest is someone
> trying to shut down OWASP risk metrics because of their work CVSS v3.
> But that's just an opinion too.

I have no association with the development of CVSSv3 aside from
http://cmlh.id.au/post/25150772855/cvssv3-call-subjects which was open
to the public.  I am *not* a member of the CVSS-SIG.

Rather, I recommend CVSS because it is assists in prioritising the
workaround and/or patches from multiple vendors.

On Wed, Aug 28, 2013 at 1:13 AM, Pete Herzog <pete at isecom.org> wrote:
> I really don't follow that line of thinking. But I do believe nobody
> sits on all the great ideas- no person and no group. I think people
> should work in the areas they see they can improve because of their
> specialized knowledge and even if it's not a flawless body of work,
> there are still likely to be some worthwhile ideas in it. Having
> worked with attack surface metrics on web apps I know it is hard and
> so I'm interested to see how another approaches it. And that's how
> science works.

The difference is that scientists publish in journals, of majority are
removing their paywall, that are available to the public

On Wed, Aug 28, 2013 at 1:13 AM, Pete Herzog <pete at isecom.org> wrote:
> OSSTMM is neither vuln testing or pen testing. It's a methodology
> which shows that no matter what you call what you do, you need to get
> XYZ intelligence from it so you know where your problems are. If you
> can do that with a vuln scanner or a red team or a magic wand, that's
> okay.

This wasn't the purpose of the 2.2 (or earlier) release of OSSTMM (I
remember it had alot of content about nmap results) but I can't
comment on the more recent releases since I haven't read them.

On Wed, Aug 28, 2013 at 1:13 AM, Pete Herzog <pete at isecom.org> wrote:
> Research is supposed to have critics. That's how progress happens. And
> then whenever you put yourself out there, well, there'll always be
> haters. That's life.

If ISECOM had been less concerned about creating profit and more
concerned with the implementation of transparency as part of the
development of OSSTMM then possibly the people saw the potential of
OSSTMM would be promoting on your behalf today.

On Wed, Aug 28, 2013 at 1:13 AM, Pete Herzog <pete at isecom.org> wrote:
> So some blog that makes up a percentage in passing about the OSSTMM
> and 3 twitter feeds is "multiple criticisms"? Your old drama teacher
> must be proud.

There is no collusion between these parties so each opinion can be
considered independent.

I will state again that one of these parties is an OWASP Board Member.

On Wed, Aug 28, 2013 at 1:13 AM, Pete Herzog <pete at isecom.org> wrote:
> Stop being so negative and so conflicted and just enjoy that there's
> so many people trying to figure out how best to secure the world.
> There's a lot of work to do so just do your thing and stop trying to
> block others from doing theirs.

I neither have the power or desire to politically block people within
OWASP.  Rather I am attempting to guide people to better alternatives,
such as CVSS and ISO 31000 based on my own experience with the OWASP
Risk Rating Methodology.

Marco, Josh and anyone else is more than welcome disregard my advice
and update the OWASP Risk Rating Methodology.


More information about the Owasp-testing mailing list