[Owasp-leaders] SecDevOps Risk Workflow Book (please help with your feedback)

Jason Johnson jason.johnson at p7n.net
Sat Nov 5 16:04:10 UTC 2016

Yeah we mostly use the defaults and write some of our own. Would be nice to make a owasp sonar plugin with something like the dependency plugin. We dont pay for the vb.net stuff. Mainly the work is defining the poor descriptions of the default rules.

On November 5, 2016 10:59:48 AM CDT, Sherif Mansour <sherif.mansour at owasp.org> wrote:
>@Jason, do you have security rules for Sonar?
>On Sat, Nov 5, 2016 at 3:27 PM, Jason Johnson <jason.johnson at p7n.net>
>> We are currently working through this now. I am curious how the
>> is set up in jira. We use that also and I would like to integrate
>> results and track the changes. We use sonar and owasp dependency
>checker to
>> scan code. Sonar tracks all the vulnerable code crap. DevOps is a
>> on the security tracking for sure. I too am looking for input.
>> On November 5, 2016 10:08:08 AM CDT, Sherif Mansour <
>> sherif.mansour at owasp.org> wrote:
>>> +Francois
>>> Hey guys,
>>> So we are now hitting on some important and amazing points, and
>>> for sharing all this.
>>> In order to help, and in the spirit of sharing I have attached my
>>> "Security in a Continuous Delivery World" slides 20 & 21 of my
>>> slidedeck. This was from the first OWASP London chapter meeting this
>>> What I will do is respond to some great points made here, and also
>>> propose a few things we might work on.
>>> *So first Mario:*
>>> Yes and a thousand times yes we need fields like the ones you have
>>> in order to do metrics and provide enough information to the
>developer in
>>> order for it to be useful.
>>> For each ticket I wrote down 3 guiding principles to use as a "North
>>> Star":
>>>    - *Unique* - No duplicate tickets
>>>    - *Useful* - Improves the security and quality of the software
>>>    - *Actionable* - All necessary information is in the ticket
>>> So I had custom fields that looked like this:
>>> [image: Inline image 1]
>>> *In order to create metrics like this:*
>>> [image: Inline image 2]
>>> I did not realise you could add detailed fields like
>request/response and
>>> PoC fields which is perfect for this.
>>> Where possible wanted to add URL, Domain, Subdomain, impacted
>>> For Static Code analysis you need to know the App, File, Line number
>>> Alternate paths/flows for the same issue (i.e. the sources and sinks
>>> the vulnerability).
>>> @Mario, on the point that there should never be FPs raised as Jira
>>> tickets, I agree these should be vetted and tweaked to never do
>>> However there is no guarantee that mistakes will not be made, and in
>>> security more often than not mistakes are made so it would help to
>have a
>>> resolution state for false positives, is is also an acknowledgment
>>> cooperation between the devs and security team, and a commitment to
>>> improvement.
>>> I.e. we know crap happens, in security crap/mistakes will happen and
>>> need to improve on it.
>>> *Issue #1*
>>> @Dinis @Mario @Simon, the challenge is when you have say 334x XSS
>and you
>>> do not want to create hundreds of tickets and you want to
>consolidate them
>>> into one.
>>> On the other hand you need to have a way of tracking which issues
>>> already been raises as a unique ticket of or as part of a ticket so
>>> you do not constantly spam the developers.
>>> *Possible solution: *The tool found the results needs to be able to
>>> the option to "group" issues in a single ticket as an option, but
>also to
>>> track each issue over time so it can inform the bug tracker if the
>>> has been resolved or not.
>>> Additionally it needs to NOT raise an issue in the bug tracker if it
>>> already raised and the developer is working on it
>>> *Issue #2*
>>> @Mario, each org is a bit different so they might not score, or want
>>> same attributes so we might want to consider the lowest common
>>> of stuff that should be in there in order for the tickets to be
>>> useful, and actionable.
>>> *Possible solution: *Document a set of guiding principles and
>>> requirements. Publish an ideal/boiler plate jira project that meets
>>> requirements so 1) Tech Teams have something ready made to customize
>off of
>>> 2) Have a set of principles to know what to customize towards.
>>> *Issue #3*
>>> @Simon, I have been thinking about the false positive thing for
>about a
>>> year now. In order to get false positive data the tool (I am just
>going to
>>> zap in this example to make things easier) would either need to do
>>> things:
>>>    1. Have a facility for the user to input false positives from the
>>>    or..
>>>    2. The tool would need to be able to connect to the bug tracker
>>>    identify which issues zap raised are not marked as false
>positives there.
>>> *Now that you have the data, then what do you do with it?*
>>> To @Mario's point to I really want to ship my security issues data
>>> somewhere else? I this case there are a few things that can be done
>>>    1. Keep the data local to the org, and simply use the info to
>>>    leverage as rules to suppress future false positives
>>>       1. e.g. The following cookies do not need to be set to SECURE
>>>       2. e.g. The following pages/sub domain can be iframed
>>>       3. e.g. The following domain is a static domain and we can
>>>       CORS set to "*" wildcard
>>>       4. Ok I'll stop now :-)
>>>    2. Ask the user if its ok to get diagnostic data and make it
>>>    what we are asking for e.g.
>>>       1. we will only ask for how many times a specific rule
>triggered a
>>>       false positive (but not the actual content of the
>>>    3. Finally you can give the tech team to send more verbose
>>>    information, if they are happy to do so. Academics and open
>source tools
>>>    might be an example.
>>>       1. There has to be a very clear feature that carefully
>explains to
>>>       them what they are actually doing so they can't turn it on by
>>>    4. I have been thinking about Machine Learning and other AI
>>>    techniques in this use case to improve the quality of ZAP, there
>are two
>>>    areas it can work:
>>>       1. Filters false positives
>>>          1. Create a baseline model where ZAP takes all the data
>>>          contributed by the community to leverage a machine learning
>algorithm such
>>>          as logistic regression and user that to "auto filter" that
>it thinks are
>>>          false positives
>>>          2. Create a local model which takes the individual
>>>          organisation's data and does pretty much the same thing,
>only in this case
>>>          the data doesn't leave the organisation.
>>>          3. I think Spark can be useful for the baseline version,
>and I
>>>          have played around with it a little bit.
>>>       2. Improves the scanner's ability to find issues:
>>>          1. Ahhh.... this is going to be tough, my first thought is
>>>          leverage neural networks such as TensorFlows Deep learning
>but I have never
>>>          used it.
>>>          2. I can see it working for SQLi and a few others pretty
>>>          but this will require a lot of thought
>>> *Next steps?*
>>> *@Dinis*, I think you got quite a bit of info to think about and try
>>> incorporate into the draft, so you might want to take some time and
>>> out what you think about all this info
>>> *@all *do you think it makes sense to 1) set some guiding principles
>>> a jira project with all this info to leverage with the goal to have
>>> teal to be able to:
>>>    - Have something ready made to customize off of
>>>    - Have a set of principles to know what to customize towards.
>>> *@Simon *This might be a bit further in the future, but if there is
>>> way to configure zap to query a bugtracker for such information and
>use the
>>> info to improve either the local instance of zap or (with
>permission) take
>>> some statistics to help improve the overall quality of ZAP.
>>> -Sherif
>>> On Thu, Nov 3, 2016 at 1:59 PM, Mario Robles OWASP <
>>> mario.robles at owasp.org> wrote:
>>>> The workflow I use is very simple actually because need to be
>adapted to
>>>> different teams with different SDLC models on different Countries,
>>>> more generic I would say:
>>>> Fixing: The issue is assigned to someone working on fixing it (link
>>>> issue in their own Agile board), if they challenge the issue and
>risk is
>>>> accepted the issue is sent to Done using Risk Accepted or Not an
>issue as
>>>> resolution
>>>> Testing: When security test the issue as part of the QA process
>>>> Deploying: Security accept or reject the fix sending it back to
>>>> or providing approval moving it to the Deploying queue
>>>> Acceptance: Dev team move the issue to Acceptance when it’s ready
>on UAT
>>>> for final tests
>>>> Done: Security will send the issue back to fixing is something
>>>> happened, otherwise will provide sign off by moving it to Done
>>>> resolution Fixed
>>>> I use Jira dashboards but also some custom macro based metrics
>based on
>>>> Jira exports
>>>> I do really like your workflow, however in my experience Dev teams
>>>> getting hesitant to follow your process when more clicks from their
>end are
>>>> needed
>>>> btw, false positives are not included in my workflow because we
>>>> should have a FP included in a list of issues, everything should be
>>>> validated before including it as an issue, if I have to add it, I
>>>> that will be as a Resolution type
>>>> Mario
>>>> On Nov 3, 2016, at 06:42, Dinis Cruz <dinis.cruz at owasp.org> wrote:
>>>> Mario that is really nice, thanks for sharing
>>>> What workflow do you use to track the changes? Is it something like
>>>> (Kanban-like) right-hand side of :
>>>> <image.png>
>>>> What about reporting? How do you visualise the data and stats you
>>>> collect? (in Jira Dashboards or in confluence?)
>>>> Dinis
>>> ------------------------------
>>> OWASP-Leaders mailing list
>>> OWASP-Leaders at lists.owasp.org
>>> https://lists.owasp.org/mailman/listinfo/owasp-leaders
>> --
>> Sent from my phone. Please excuse my brevity.

Sent from my phone. Please excuse my brevity.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.owasp.org/pipermail/owasp-leaders/attachments/20161105/0350260b/attachment.html>

More information about the OWASP-Leaders mailing list