[Owasp-testing] Are the Risk Rating Wiki Pages Broken?

Pete Herzog lists at isecom.org
Wed Aug 28 10:11:02 UTC 2013


> CVSSv3 is still under active development i.e. I believe this was
> obvious since "development" is quoted within the URI i.e.
> http://www.first.org/cvss/v3/development

Snippy much? As a fellow researcher I am aware when something is under
development and if it's other research then these things are always in
some state of development.

> Well is appears "regular contributors the Open Source Security Testing
> Methodology Manual (OSSTMM), certified Trainers of Penetration Testing
> Professionals of OSSTMM" from Pure Hacking are "...regularly .."
> caught out making inaccurate claims "... in Australia’s media." time
> after time e.g.
> http://www.zdnet.com/thomsons-phone-clone-claims-uncertain-1339338352/

No matter who contributes, all research is vetted. We don't vet
people, just what they contribute. As Mike mentioned, there are some
things we've still debated on list for years.

>>From memory of my reading at the time it appeared that OSSTMM hadn't
> considered the ramification that an experienced auditor cannot provide
> their referees to prior penetration test which therefore left the
> person evaluating the proposal without an independent view and
> therefore might result in the lesser experienced auditor being awarded
> the opportunity.

No inaccurate name dropping. I remember. This was to prevent audit
companies who did non sec work, like an e-mail implementation at
well-known company XYZ, to say XYZ was a past client. There is a lot
of lies and fraud in the pen test industry claiming to have done
things they haven't. It also prevented breaks in NDA. What was allowed
was for the companies themselves to give recommendations. We were just
trying to get at more accurate referrals. Again, a recommendation and
not a requirement.

> On Wed, Aug 28, 2013 at 1:13 AM, Pete Herzog <pete at isecom.org> wrote:
>> Anyway, we never wanted to compete with hacking exposed. We're a
>> research organization looking for the best way to test and analyze
>> operations to secure them. That's enough work without trying to detail
>> how to exploit each bug that might not even be a technology in use
>> tomorrow.
> The above doesn't convince me that you intent is to otherwise extract
> as much revenue out of the market while being in denial about high
> level understanding required to maintain the state of the art.

It wasn't meant to. Of course we do need to earn money to continue our
research. We have some very large, costly projects that bring
absolutely zero return financially, like Hacker Highschool. But I
wouldn't say we're trying to "extract as much revenue" because there
are many things we don't do.

> OSSTMM has been in a steady decline which has upheld my first impression.

Based on what research? 4 tweets and a blog? I'd love to see your
research. Don't bother if it's informal. I have actual numbers that
show it's not. We shifted as we came to understanding about the
inadequacies of pen testing and vuln scanning as well as many of the
ways security has been "productized". That lost us people in those
areas, like you apparently, but it gained us much more in other areas.
It wasn't our intention. We just followed the facts of the research
and adjusted the methodology accordingly.

> To select one of several stories that I have heard, it took a
> webappsec subcontractor to "Pure Hacking" 1 hour to bypass a simple
> authentication framework which Ty Miller had wasted two days
> attempting to figure out without any tangible result.

Anecdotal evidence. Doesn't help.

> I'll locate the e-mail(s) from Raoul and send them to you but I would
> have preferred not to do this since my brief dealings with Raoul have
> always been positive since we avoided discussing his association with
> convicted criminal Kevin Mitnick e.g.
> https://twitter.com/kevinmitnick/status/30784499897860097

Many many people talk to Kevin. And as far as I can tell, he's no
longer a convicted criminal unless he just went back to jail....

> I won't have interpreted that as an endorsement otherwise I would have
> contributed to OSSTMM.

I'm sure. After how many years? I won't hold my breath.

> Coincidently what you may have heard about
> http://cmlh.id.au/tagged/grubbgate wasn't an accurate version of
> events of what I presented at Security B-Sides in 2011 and you may
> also want to pay particular attention to
> http://cmlh.id.au/post/57402913158/jodymelbourne-doxed

Again, there are many variables. And all you can provide is anecdotal
and indirect evidence.

> To link this back to "Pure Hacking", I was disappointed when Ty
> attempt to exploit this unfortunate situation for "Pure Hacking" gain
> i.e. http://www.theaustralian.com.au/australian-it/facebook-images-open-to-access/story-e6frgakx-1226059138255,
> http://www.zdnet.com.au/stuxnet-routing-hacks-and-a-seized-ipad-339315484.htm,
> etc in light of:
> 1.  "Pure Hacking" appearance on
> http://www.sbs.com.au/insight/episode/transcript/30/Stolen-IDTV and
> associated comments hyped with delusion of grandeurs.
> 2.  "Pure Hacking" reversal on seeking permission from the end user of
> social network i.e. content on social networks is public domain i.e.
> http://www.zdnet.com/penetration-testing-employees-social-media-to-improve-policy-7000017234/.
>  Their claim about "no one else in the world is currently doing" is
> pure fabrication too due to my prior work and that of Tom Ryan i.e.
> "Robin Sage", IOActive i.e. "The Leaky Web: Owning Your Favorite
> CEOs", etc

Again, what I'm interested in is ideas and contributions from Ty. We
get contributions from everywhere and I'm not going to discount good
work because other people don't like the way a certain person behaves
professionally. And all this coming from you who was against the
"ethics" in the OSSTMM!

> Rather, I recommend CVSS because it is assists in prioritising the
> workaround and/or patches from multiple vendors.

Okay, well, at ISECOM we don't advocate patching except for features
and under change control procedures. Therefore we don't see the need
for CVSS. We prioritize security remediation by operation and not risk.

> The difference is that scientists publish in journals, of majority are
> removing their paywall, that are available to the public

Do those scientists make open all their work while they're working on
it? No. Just their team sees it. We also publish for free.

> This wasn't the purpose of the 2.2 (or earlier) release of OSSTMM (I
> remember it had alot of content about nmap results) but I can't
> comment on the more recent releases since I haven't read them.

Yes, and we were wrong. We followed research from the industry and it
mislead us until we figured out that security was impossible that way.
So we took 7 years to restart our research from the ground up and came
out with OSSTMM 3. All stuff I've said publicly.

> If ISECOM had been less concerned about creating profit and more
> concerned with the implementation of transparency as part of the
> development of OSSTMM then possibly the people saw the potential of
> OSSTMM would be promoting on your behalf today.

Again, it's free for anyone to join and contribute to the team and you
have access.

> There is no collusion between these parties so each opinion can be
> considered independent.

But it's still just opinions. I can go online and find 3 twitter posts
and a blog post that says smoking is good but that doesn't make it true.

> I will state again that one of these parties is an OWASP Board Member.

Yes, and he didn't mention OSSTMM. Choosing something is a preference
and an opinion.

> I neither have the power or desire to politically block people within
> OWASP.  Rather I am attempting to guide people to better alternatives,
> such as CVSS and ISO 31000 based on my own experience with the OWASP
> Risk Rating Methodology.

I hope at least that you learn from our exchange is that things change
and evolve and you shouldn't be dismissive of things because you
didn't like the old version. I mean, you did support a new version of
CVSS despite version 1 being pretty bad.

Let's take this exchange offline from here on out.


Pete Herzog - Managing Director - pete at isecom.org
ISECOM - Institute for Security and Open Methodologies
www.isecom.org - www.osstmm.org
www.hackerhighschool.org - www.badpeopleproject.org

Pete Herzog - Managing Director - pete at isecom.org
ISECOM - Institute for Security and Open Methodologies
www.isecom.org - www.osstmm.org
www.hackerhighschool.org - www.badpeopleproject.org

More information about the Owasp-testing mailing list