[Owasp-leaders] SecDevOps Risk Workflow Book (please help with your feedback)

Sherif Mansour sherif.mansour at owasp.org
Sat Nov 5 17:41:12 UTC 2016

I am, and I think I know a few others who might be interested.

Although this needs a bit more thought, as it has many moving parts like:

   - Looking for the same vulnerability types across different languages
   - Looking for language specific and framework specific vulnerabilities
   (MVCs like Spring, Django, RoR, and front end framworks like React.io or
   AngularJS etc).
   - Having the relevant automation functionality + rule suppression
   conditions in place for it to work for different teams.

One simple (and I do mean very simple) rule is to look for common strings
like "Begin Private Key" etc.. which show hardcoded keys/API strings. I
have seen many Static Code Analysers miss those and instead show the string
"password =" which flags a ridiculous amount of false positives.


On Sat, Nov 5, 2016 at 5:03 PM, Jason Johnson <jason.johnson at p7n.net> wrote:

> True, I'm trying to make it a standard at work. I would like to help if
> anyone is interested with the rules.
> On November 5, 2016 11:29:41 AM CDT, Sherif Mansour <
> sherif.mansour at owasp.org> wrote:
>> +1 an OWASP rule pack + plugin for Sonar feels like a worthwhike project
>> given its Open Source and scans many languages which is the issue with many
>> Static Code Analyzers i.e. only language specific.
>> On Saturday, 5 November 2016, Jason Johnson <jason.johnson at p7n.net>
>> wrote:
>>> Yeah we mostly use the defaults and write some of our own. Would be nice
>>> to make a owasp sonar plugin with something like the dependency plugin. We
>>> dont pay for the vb.net stuff. Mainly the work is defining the poor
>>> descriptions of the default rules.
>>> On November 5, 2016 10:59:48 AM CDT, Sherif Mansour <
>>> sherif.mansour at owasp.org> wrote:
>>>> @Jason, do you have security rules for Sonar?
>>>> On Sat, Nov 5, 2016 at 3:27 PM, Jason Johnson <jason.johnson at p7n.net>
>>>> wrote:
>>>>> We are currently working through this now. I am curious how the
>>>>> workflow is set up in jira. We use that also and I would like to integrate
>>>>> security results and track the changes. We use sonar and owasp dependency
>>>>> checker to scan code. Sonar tracks all the vulnerable code crap. DevOps is
>>>>> a struggle on the security tracking for sure. I too am looking for input.
>>>>> On November 5, 2016 10:08:08 AM CDT, Sherif Mansour <
>>>>> sherif.mansour at owasp.org> wrote:
>>>>>> +Francois
>>>>>> Hey guys,
>>>>>> So we are now hitting on some important and amazing points, and
>>>>>> thanks for sharing all this.
>>>>>> In order to help, and in the spirit of sharing I have attached my
>>>>>> deck "Security in a Continuous Delivery World" slides 20 & 21 of my
>>>>>> attached slidedeck. This was from the first OWASP London chapter meeting
>>>>>> this year.
>>>>>> What I will do is respond to some great points made here, and also
>>>>>> propose a few things we might work on.
>>>>>> *So first Mario:*
>>>>>> Yes and a thousand times yes we need fields like the ones you have
>>>>>> added in order to do metrics and provide enough information to the
>>>>>> developer in order for it to be useful.
>>>>>> For each ticket I wrote down 3 guiding principles to use as a "North
>>>>>> Star":
>>>>>>    - *Unique* - No duplicate tickets
>>>>>>    - *Useful* - Improves the security and quality of the software
>>>>>>    - *Actionable* - All necessary information is in the ticket
>>>>>> So I had custom fields that looked like this:
>>>>>> [image: Inline image 1]
>>>>>> *In order to create metrics like this:*
>>>>>> [image: Inline image 2]
>>>>>> I did not realise you could add detailed fields like request/response
>>>>>> and PoC fields which is perfect for this.
>>>>>> Where possible wanted to add URL, Domain, Subdomain, impacted
>>>>>> parameter(s)
>>>>>> For Static Code analysis you need to know the App, File, Line number
>>>>>> + Alternate paths/flows for the same issue (i.e. the sources and sinks for
>>>>>> the vulnerability).
>>>>>> @Mario, on the point that there should never be FPs raised as Jira
>>>>>> tickets, I agree these should be vetted and tweaked to never do that.
>>>>>> However there is no guarantee that mistakes will not be made, and in
>>>>>> security more often than not mistakes are made so it would help to have a
>>>>>> resolution state for false positives, is is also an acknowledgment of
>>>>>> cooperation between the devs and security team, and a commitment to
>>>>>> improvement.
>>>>>> I.e. we know crap happens, in security crap/mistakes will happen and
>>>>>> we need to improve on it.
>>>>>> *Issue #1*
>>>>>> @Dinis @Mario @Simon, the challenge is when you have say 334x XSS and
>>>>>> you do not want to create hundreds of tickets and you want to consolidate
>>>>>> them into one.
>>>>>> On the other hand you need to have a way of tracking which issues
>>>>>> have already been raises as a unique ticket of or as part of a ticket so
>>>>>> that you do not constantly spam the developers.
>>>>>> *Possible solution: *The tool found the results needs to be able to
>>>>>> have the option to "group" issues in a single ticket as an option, but also
>>>>>> to track each issue over time so it can inform the bug tracker if the issue
>>>>>> has been resolved or not.
>>>>>> Additionally it needs to NOT raise an issue in the bug tracker if it
>>>>>> is already raised and the developer is working on it
>>>>>> *Issue #2*
>>>>>> @Mario, each org is a bit different so they might not score, or want
>>>>>> the same attributes so we might want to consider the lowest common
>>>>>> denominator of stuff that should be in there in order for the tickets to be
>>>>>> unique, useful, and actionable.
>>>>>> *Possible solution: *Document a set of guiding principles and
>>>>>> requirements. Publish an ideal/boiler plate jira project that meets these
>>>>>> requirements so 1) Tech Teams have something ready made to customize off of
>>>>>> 2) Have a set of principles to know what to customize towards.
>>>>>> *Issue #3*
>>>>>> @Simon, I have been thinking about the false positive thing for about
>>>>>> a year now. In order to get false positive data the tool (I am just going
>>>>>> to zap in this example to make things easier) would either need to do two
>>>>>> things:
>>>>>>    1. Have a facility for the user to input false positives from the
>>>>>>    zap or..
>>>>>>    2. The tool would need to be able to connect to the bug tracker
>>>>>>    and identify which issues zap raised are not marked as false positives
>>>>>>    there.
>>>>>> *Now that you have the data, then what do you do with it?*
>>>>>> To @Mario's point to I really want to ship my security issues data to
>>>>>> somewhere else? I this case there are a few things that can be done
>>>>>>    1. Keep the data local to the org, and simply use the info to
>>>>>>    leverage as rules to suppress future false positives
>>>>>>       1. e.g. The following cookies do not need to be set to SECURE
>>>>>>       etc..
>>>>>>       2. e.g. The following pages/sub domain can be iframed
>>>>>>       3. e.g. The following domain is a static domain and we can
>>>>>>       have CORS set to "*" wildcard
>>>>>>       4. Ok I'll stop now :-)
>>>>>>    2. Ask the user if its ok to get diagnostic data and make it
>>>>>>    explicit what we are asking for e.g.
>>>>>>       1. we will only ask for how many times a specific rule
>>>>>>       triggered a false positive (but not the actual content of the
>>>>>>       request/response)
>>>>>>    3. Finally you can give the tech team to send more verbose
>>>>>>    information, if they are happy to do so. Academics and open source tools
>>>>>>    might be an example.
>>>>>>       1. There has to be a very clear feature that carefully
>>>>>>       explains to them what they are actually doing so they can't turn it on by
>>>>>>       accident.
>>>>>>    4. I have been thinking about Machine Learning and other AI
>>>>>>    techniques in this use case to improve the quality of ZAP, there are two
>>>>>>    areas it can work:
>>>>>>       1. Filters false positives
>>>>>>          1. Create a baseline model where ZAP takes all the data
>>>>>>          contributed by the community to leverage a machine learning algorithm such
>>>>>>          as logistic regression and user that to "auto filter" that it thinks are
>>>>>>          false positives
>>>>>>          2. Create a local model which takes the individual
>>>>>>          organisation's data and does pretty much the same thing, only in this case
>>>>>>          the data doesn't leave the organisation.
>>>>>>          3. I think Spark can be useful for the baseline version,
>>>>>>          and I have played around with it a little bit.
>>>>>>       2. Improves the scanner's ability to find issues:
>>>>>>          1. Ahhh.... this is going to be tough, my first thought is
>>>>>>          to leverage neural networks such as TensorFlows Deep learning but I have
>>>>>>          never used it.
>>>>>>          2. I can see it working for SQLi and a few others pretty
>>>>>>          well but this will require a lot of thought
>>>>>> *Next steps?*
>>>>>> *@Dinis*, I think you got quite a bit of info to think about and try
>>>>>> to incorporate into the draft, so you might want to take some time and find
>>>>>> out what you think about all this info
>>>>>> *@all *do you think it makes sense to 1) set some guiding principles
>>>>>> 2) a jira project with all this info to leverage with the goal to have tech
>>>>>> teal to be able to:
>>>>>>    - Have something ready made to customize off of
>>>>>>    - Have a set of principles to know what to customize towards.
>>>>>> *@Simon *This might be a bit further in the future, but if there is
>>>>>> a way to configure zap to query a bugtracker for such information and use
>>>>>> the info to improve either the local instance of zap or (with permission)
>>>>>> take some statistics to help improve the overall quality of ZAP.
>>>>>> -Sherif
>>>>>> On Thu, Nov 3, 2016 at 1:59 PM, Mario Robles OWASP <
>>>>>> mario.robles at owasp.org> wrote:
>>>>>>> The workflow I use is very simple actually because need to be
>>>>>>> adapted to different teams with different SDLC models on different
>>>>>>> Countries, it’s more generic I would say:
>>>>>>> Fixing: The issue is assigned to someone working on fixing it (link
>>>>>>> to issue in their own Agile board), if they challenge the issue and risk is
>>>>>>> accepted the issue is sent to Done using Risk Accepted or Not an issue as
>>>>>>> resolution
>>>>>>> Testing: When security test the issue as part of the QA process
>>>>>>> Deploying: Security accept or reject the fix sending it back to
>>>>>>> Fixing or providing approval moving it to the Deploying queue
>>>>>>> Acceptance: Dev team move the issue to Acceptance when it’s ready on
>>>>>>> UAT for final tests
>>>>>>> Done: Security will send the issue back to fixing is something wrong
>>>>>>> happened, otherwise will provide sign off by moving it to Done using
>>>>>>> resolution Fixed
>>>>>>> I use Jira dashboards but also some custom macro based metrics based
>>>>>>> on Jira exports
>>>>>>> I do really like your workflow, however in my experience Dev teams
>>>>>>> start getting hesitant to follow your process when more clicks from their
>>>>>>> end are needed
>>>>>>> btw, false positives are not included in my workflow because we
>>>>>>> never should have a FP included in a list of issues, everything should be
>>>>>>> validated before including it as an issue, if I have to add it, I think
>>>>>>> that will be as a Resolution type
>>>>>>> Mario
>>>>>>> On Nov 3, 2016, at 06:42, Dinis Cruz <dinis.cruz at owasp.org> wrote:
>>>>>>> Mario that is really nice, thanks for sharing
>>>>>>> What workflow do you use to track the changes? Is it something like
>>>>>>> the (Kanban-like) right-hand side of :
>>>>>>> <image.png>
>>>>>>> What about reporting? How do you visualise the data and stats you
>>>>>>> collect? (in Jira Dashboards or in confluence?)
>>>>>>> Dinis
>>>>>> ------------------------------
>>>>>> OWASP-Leaders mailing list
>>>>>> OWASP-Leaders at lists.owasp.org
>>>>>> https://lists.owasp.org/mailman/listinfo/owasp-leaders
>>>>> --
>>>>> Sent from my phone. Please excuse my brevity.
>>> --
>>> Sent from my phone. Please excuse my brevity.
> --
> Sent from my phone. Please excuse my brevity.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.owasp.org/pipermail/owasp-leaders/attachments/20161105/e7d8f93d/attachment-0001.html>

More information about the OWASP-Leaders mailing list