[Owasp-leaders] 2012 Rugged Summit

Jeff Williams jeff.williams at owasp.org
Fri Aug 31 03:19:45 UTC 2012


Hi John,

Confusion here is my fault. Of course all those practices have been
around for a long time.  If we're going to get all historical, I would
direct your attention to the Systems Security Engineering CMM (ISO
21827) that I (and others) wrote back in the mid-to-late 90's.  We had
hundreds of large organizations implementing the model and it's now an
international standard.

I think you *massively* underestimate (insult actually) OWASP if you
think If we're all about "Top-10 lists, penetration testing, and
training." There is more information about implementing the practices
in the BSIMM at OWASP than in any other place. More presentations,
articles, talks, videos, and experts... including the guys who drafted
up the Rugged Handbook (Strawman).

There's an important lesson I learned creating the SSE-CMM: process
models don't change culture. There are plenty of good practices listed
in BSIMM and OpenSAMM, but I'm a little surprised that your "it's a
survey, it's a survey, it's a survey" mantra doesn't make the case for
itself that we need something more.

I know of many organizations that do the things in the BSIMM yet make
no progress because they're just going through the motions.  They
don't really believe. As I look forward, I don't think this approach
can possibly keep pace with advances that bring more complexity, more
communications, more critical information, and are increasingly
difficult to verify.

On the other hand, I've worked with a handful of organizations who I
DO consider Rugged.  But it's not clear how they got there or how it
works.  They don't do half of the BSIMM activities, but they get
consistently great results.  For example, I had a great conversation
today with the CEO of CabForward, a small ROR shop.  They build
software that's important and they committed to being Rugged because
they believe in it and want it. It's not being forced on them.

We're trying to figure it out.  We spent quite a lot of time studying
what might bootstrap this type of culture in organizations that are,
frankly, stuck in a negative, reactive hamster-wheel-of-pain with
regard to application security.  It's not about the list of
activities, but how you apply them to get to an organization that
WANTS to produce rugged code.

I think BSIMM and Rugged are perfectly suited to live side-by-side,
and I'm looking forward to our next get-together so we can argue about
it :-)

--Jeff



On Thu, Aug 30, 2012 at 10:11 PM, John Steven <John.Steven at owasp.org> wrote:
> Jeff,
>
> I think at best you've overstated.
>
> [Role Ideas]
> Cursory consideration shows Rugged's ideas for each role are covered
> by BSIMM3 activities and often overwhelming precedent exists. A
> notable exception I found is "Model your data instead of using
> strings" [*MS]. Those unfamiliar with BSIMM should take note: several
> BSIMM respondents implement the activities they're credited with using
> developers, as implied by the Rugged Handbook. When the study refers
> to QA it references the function not necessarily the staff-in-role.
>
> [Prairie Dogs]
> Ken Van Wyk (and others) have written extensively on the concept of
> integrating post-deployment personnel and monitoring in a feedback
> loop affecting policy, architecture, and development. Several
> organizations in multiple verticals do this effectively now. I'm glad
> this notion has gained buzz under the term DevOps. I, as well as many
> of the organizations I've had the pleasure to work with, believe that
> several advantages to both development and operations come from their
> interaction.
>
> [Ant Colonies]
> I covered, published, and presented the concept of creating a
> multi-stakeholder security initiative as early as 2003 at S3-con under
> the name of ESSF (*link no longer available). Look at material I
> published again in '06 [*EF], and McGraw's "Software Security" [*SS]
> and you'll find the five (5) roles described on page 11 of the Rugged
> Handbook [*RR]. Many organizations which have progressed past the
> "one-man-shop" model in multiple verticals integrate these
> stakeholders effectively now.
>
> [Musk Oxen]
> Again, easily tied to BSIMM activities and a key component of
> organizations' strategies. Though, this topic's coverage in the
> handbook is naive for organizations with more than one web app.
>
> [Beavers]
> Pravir Chandra fought vehemently (and in my opinion knowledgeably and
> presciently) for what became "Software Environment" in the BSIMM SSF
> and in a form I prefer: "Deployment" in OpenSAMM.
>
> [Honeybadgers, BigHorns, and Humans]
> Again, well-trodden territory.
>
> [Sum]
> I'm excited folk within the OWASP community have come to the
> conclusion that Application Security is more than just Top-10 lists,
> penetration testing, and training. I'm thrilled that a focus on
> developers has emerged and that this focus has not precluded
> architecture, operational, and executive stakeholder out-reach.
>
> For those who like the ideas prescribed to each role in the Rugged
> Handbook, I'd suggest consider all the activities in BSIMM (*CC). In
> its SSF you may find some other pursuits well-suited to your
> particular organization, maturity, and strengths. You won't find
> effective explanation of HOW to do these things though--not there. The
> good news is that each activity has been found in *at least* three (3)
> organizations. This means that someone has likely written something
> public you can find and consume to help you with more specifics "who",
> "what", and "how".
>
> The notion of a security story may be a solid organizing framework for
> those just beginning their journey from one-man-shop to three-man
> practice. I expect that if it's to gain traction, Rugged's notion of
> "proving it" is going to have to mature dramatically and explicitly
> weave itself amongst the "ideas", "budget", "measurement", and
> "outreach" dimensions. Personal experience has shown this is where
> many trip up. So, to me, fleshing these interactions out further is
> essential.
>
> [Assurance - Bonus]
> And this leads us back to the Handbook's largely implied notion of
> "assurance" that Jeff mentions specifically in his email. His quote
> from Training Day aptly hits why most "mature" and "expensive"
> security practices (and vendors, for that matter) have much room for
> improvement. Yet, despite a large body of existing software assurance
> work *[SA], I don't see much reference at all in the Handbook to what
> we can do to 1) specifically and concretely improve traditional
> activities or 2) augment/replace those activities with ones that will
> produce a more risk-driven and iron-clad assurance case. This, I
> believe is a "brass ring" worth reaching for. However,
> higher-assurance software typically comes with a level of formality
> beyond the appetite of commercial software.
>
> [insert Mars Lander analogy here].
>
> -jOHN
> --
> Phone: 703.727.4034
> Rss: http://feeds.feedburner.com/M1splacedOnTheWeb
>
> * [CC] - The documentation is open source and you can download it,
> edit it, and fork it for your own use as you see fit.
> * [EF] - https://buildsecurityin.us-cert.gov/bsi/568-BSI/version/2/part/4/data/Steven_IEEE_SP.pdf?branch=main&language=default
> * [MS] - Developer idea #2 - A Solid suggestion. One I've made and
> actually implemented in several Enterprises. This, however, is less of
> cultural change and more of a secure design pattern. I prefer a former
> name: "Maintain type-safety of data". Strings are, of course, only one
> of many offensive catch-all types that defeat such safety.
> * [RR] - technology execs, testers, developers, architects, and security folk
> * [SA] - Here I'm referring to the "Software Assurance" work of the
> '70's and 80's and even 90's, not the US Government's SwAF.
> * [SS] - Chapter 10, in particular, though throughout.
>
>
> On Thu, Aug 30, 2012 at 4:41 PM, Jeff Williams <jeff.williams at owasp.org> wrote:
>> Hi Jerry,
>>
>> I don't see Rugged as a reformulation of *anything* people are already doing.  Many traditional activities are valuable, but there are an awful lot that don't end up producing any value.  That is, they're disconnected from what the business needs.
>>
>> Rugged is focused on generating a software development culture that produces security in a tangible defensible way. As Denzel said in Training Day, "it's not what you know -- it's what you can prove"
>>
>> Think about the security story for any website on the planet.  I guarantee if you capture it, it will reveal gaps.  And over time it will drive the communication you need to improve your organization.  This is the "visible" that OWASP should champion.
>>
>> I hope you'll try it.  I'm happy to help anyone interested create a security story and start "Thinking in Rugged"
>>
>> --Jeff
>>
>>
>>
>> On Aug 30, 2012, at 4:18 PM, "Tom Brennan" <tomb at owasp.org> wrote:
>>
>>> Good feedback -- cc to the guys leading the effort.
>>>
>>> A suggestion was already floated on getting it up on a wiki hmmm I know a group that has a wiki *cough* https://www.owasp.org/index.php/Category:OWASP_RuggedSoftware  -- for community review and contribution to sections or to take inbound feedback for Version 5.0 or a another working group that is open to anyone who wants to attend and contribute.
>>>
>>> Will relay what we hear to the list; poke the badgers, bears and ice-T @ http://www.ruggedsoftware.org/about.html
>>>
>>>
>>>
>>> -----Original Message-----
>>> From: Jerry Hoff [mailto:jerry at owasp.org]
>>> Sent: Thursday, August 30, 2012 2:24 PM
>>> To: tomb at owasp.org
>>> Cc: <owasp-leaders at lists.owasp.org>
>>> Subject: Re: [Owasp-leaders] 2012 Rugged Summit
>>>
>>> Hello all,
>>>
>>> Nice work! Although my first reaction was: Honey badgers? Ice-T? Seriously?
>>>
>>> I think this is an interesting document - but I hope as an organization and as an industry we focus on reproducible best practices, quantitative metrics and real data behind works such as this one, rather than yet another reformulation / restatement of the same basic advice we as an industry have been preaching over the years.
>>>
>>> A guide such as this coming out of actual metrics and real-world best practices would be much more appealing. The blurbish case studies at the end should have driven the document, instead of the other way around.
>>>
>>> Not trying to be antagonistic - just food for thought.
>>>
>>> Jerry
>>>
>>>
>>>
>>>
>>>
>>> On Aug 30, 2012, at 1:49 PM, "Tom Brennan" <tomb at owasp.org> wrote:
>>>
>>>> A Software Security Philosophy *RELEASED* 2012-Aug is creating quite a buzz around in a very short time -- this was a HOT TOPIC at last week's DHS / US-CERT event in the USA.
>>>>
>>>> http://www.ruggedsoftware.org/docs/RuggedHandbookv4.pdf
>>>>
>>>> In summary a group of well-known participants spent a week together, developing the details; kudos to them for volunteering their time with attribution to OWASP
>>>>
>>>>  Justin Berman
>>>>  John Bernero
>>>>  Nick Coblentz
>>>>  Josh Corman
>>>>  Gene Kim
>>>>  Jason Li
>>>>  John Pavone
>>>>  Ken van Wyk
>>>>  John Wilander
>>>>  Jeff Williams
>>>>  Chris Wysopal
>>>>
>>>> If you would like to get involved see:  http://www.ruggedsoftware.org/about.html


More information about the OWASP-Leaders mailing list