[Owasp-testing] OWASP Testing Guide v3: introduce section for Software System Upgrade and Migration testing

Thu May 22 09:06:45 EDT 2008

Eoin

> I sense scope creep here :)
> Patch management in an application security testing guide?
>

Not at all actually. The discussion is where application security stops and
what to do with known vulnerability risks beyond it. I think most of us
agree that these things do not belong in a application security
manual/framework, but I think it's also not a good idea to just completely
ignore them.
Like I said, the least we can do is make a (small) note (with reference)
when vulnerabilities are identified which are beyond the scope
of application security.
Ignorance should be avoided at all times.

> Also upgrade and migration testing, What are we actually going to test for
> here? If there is an application upgrade one does not simply test the
> upgraded piece [..]
>
 Migration testing would cover the same suite of tests as a regular
> application security test, but there may be some security review if the task
> was more than blackbox. SO we would look at config files etc. but testing
> specific to migration as opposed to testing to see if something is secure,
> the difference is minimal.
>
I agree. As a functional tester I have had these situations before and the
sensible way to scope your testplan is to do a risk analysis and decide
whether a regression test is enough or a complete functional test is
necessary. The same goes for security testing in my opinion.

Dave

>  On 21/05/2008, Dave van Stein <dvstein at gmail.com> wrote:
>>
>> Basim,
>>
>> I agree. In the mentioned case of upgrade en migration testing we could
>> add a note that at least patch management, authentication & access control
>> and configuration files should get extra attention to limit possible
>> vulnerabilities.
>>
>> regards, dave
>>
>>
>> 2008/5/21 Basim Baassiri <basim at baassiri.ca>:
>>
>>> I agree with your points Dave, there should be mention of this topic
>>> (upgrade and migration testing) when the scope of the manual has been
>>> reached.  Something along lines of "Although this topic is not in the scope
>>> of the manual, we feel that it should be mentioned so it is not an oversight
>>> when doing testing"
>>>
>>> Thanks again for all of your feedback
>>>
>>> Looking forward to helping out with the new guide
>>>
>>> Basim
>>>
>>>  On Tue, May 20, 2008 at 3:40 PM, Dave van Stein <dvstein at gmail.com>
>>> wrote:
>>>
>>>
>>>> All,
>>>>
>>>> I think that setting the scope and boundaries is one of the hardest
>>>> things concerning creating an application security testing manual. In the
>>>> case described above/below it handles the boundary between testing and SDLC,
>>>> but the same problem arises when defining the boundary between for example
>>>> application testing and ('classic') penetration testing. Clearly the goal of
>>>> OWASP is not testing things like the existence of open SSH sockets, but
>>>> things like command injection can be used to gain root access on a
>>>> webserver; where does application testing stop in that case ?
>>>>
>>>> On the other hand is limiting the scope to far not good either. In my
>>>> opinion the testing manual should at least give some information about risks
>>>> and things to improve when a 'boundary' is crossed.
>>>>
>>>> regards, Dave
>>>>
>>>> 2008/5/20 kevin horvath <kevin.horvath at gmail.com>:
>>>>
>>>> I agree that this is more on the SDLC side then the actual security
>>>>> testing of an application.  Secure SDLC testing would be a nice
>>>>> project to start for OWASP but I think it would be too much to
>>>>> incorporate into this project.
>>>>>
>>>>> On Tue, May 20, 2008 at 10:42 AM, Eoin <eoin.keary at owasp.org> wrote:
>>>>> > To Me this sounds more like Secure SDLC Testing and not pen testing,
>>>>> not
>>>>> > that the guide is strictly pen testing.
>>>>> >
>>>>> > Secure SDLC testing, test plans relating to functional requirements
>>>>> > etc could be a project on its own? Maintenance lifecycle, regression
>>>>> and
>>>>> > root cause analysis relating to the software development lifecycle
>>>>> can (and
>>>>> > should) involve an element of security. This area is very large and
>>>>> needs to
>>>>> > be comprehensively discussed and is a valid candidate for its own
>>>>> guide?
>>>>> >
>>>>> > I would be concerned that the guide may be stretching itself and
>>>>> covering
>>>>> > too many things rather than covering fewer things in a more complete
>>>>> manner.
>>>>> >
>>>>> > Eoin
>>>>> >
>>>>> > OWASP Code Review Guide Lead
>>>>> >
>>>>> >
>>>>> > On 20/05/2008, Basim Baassiri <basim at baassiri.ca> wrote:
>>>>> >>
>>>>> >> Has anyone had the chance to read below
>>>>> >>
>>>>> >> Basim
>>>>> >>
>>>>> >> On Mon, May 12, 2008 at 12:09 AM, Basim Baassiri <basim at baassiri.ca
>>>>> >
>>>>> >> wrote:
>>>>> >>>
>>>>> >>> Hello Matteo and all,
>>>>> >>>
>>>>> >>> When you say new concept, I'm assuming a new concept for the guide.
>>>>> As to
>>>>> >>> where it fits as part of the index, I can't clearly see it at this
>>>>> time.  To
>>>>> >>> be clearer on the term Software System upgrades and migrations,
>>>>> please see
>>>>> >>> my definition below.
>>>>> >>>
>>>>> >>> Software System Upgrade:
>>>>> >>> Basically, is testing the end result of the the data and
>>>>> application when
>>>>> >>> a patch/upgrade/hotfix has been applied to it (Upgrade path
>>>>> Testing)
>>>>> >>>
>>>>> >>> Typically, testing this process is as follows
>>>>> >>> 1. installation verification
>>>>> >>> Did the installation of the patches/upgrade executable/script to
>>>>> the
>>>>> >>> existing system cause an failures. In the context of a web
>>>>> application is
>>>>> >>> there exceptions/failures in the application log
>>>>> >>>
>>>>> >>> 2. Database verification (if required, but typically required in
>>>>> majority
>>>>> >>> of web applications)
>>>>> >>> This is a two step process where a tester would look to see (either
>>>>> the
>>>>> >>> output of the upgrade script or database logs) if any error
>>>>> occurred in the
>>>>> >>> logs
>>>>> >>> Second, a capture of a subset of a data from before and after would
>>>>> be
>>>>> >>> done to analyze transformations of data
>>>>> >>>
>>>>> >>> 3. Does the application behave the same way it did before
>>>>> (Functional
>>>>> >>> testing)
>>>>> >>>
>>>>> >>> As for migrations, this is defined by completely moving over the
>>>>> another
>>>>> >>> system.
>>>>> >>> Examples of migrations:
>>>>> >>> application A uses MySQL Database -> Apply migration -> application
>>>>> A now
>>>>> >>> uses PostGres database
>>>>> >>> application B uses PHP -> Apply migration -> application B now uses
>>>>> Java
>>>>> >>>
>>>>> >>> The reason I mention the above examples is because I have been
>>>>> exposed to
>>>>> >>> this scenario where it was expected that the application continues
>>>>> to behave
>>>>> >>> to same after the transformation
>>>>> >>>
>>>>> >>>
>>>>> >>> Thats the background of upgrade and migration testings
>>>>> >>>
>>>>> >>> I guess the next question is how does the above apply to security
>>>>> testing
>>>>> >>>
>>>>> >>> I've seen changes to table schemas where by the code is not
>>>>> reflective of
>>>>> >>> the change in the database and as a result "private" or sensitive
>>>>> data has
>>>>> >>> been exposed.
>>>>> >>>
>>>>> >>> Not sure where this falls under the current index.  This topic is
>>>>> not
>>>>> >>> commonly thought of in both the development and testing teams.
>>>>> >>>
>>>>> >>> Looking for feedback from anyone
>>>>> >>>
>>>>> >>> Cheers
>>>>> >>>
>>>>> >>> Basim
>>>>> >>>
>>>>> >>>
>>>>> >>> On Thu, May 8, 2008 at 12:00 PM, Basim Baassiri <basim at baassiri.ca
>>>>> >
>>>>> >>> wrote:
>>>>> >>>>
>>>>> >>>> Hello all
>>>>> >>>>
>>>>> >>>> I've been watching the list for about 6 months now
>>>>> >>>>
>>>>> >>>> Just wanted to introduce myself
>>>>> >>>>
>>>>> >>>> My Background
>>>>> >>>> I am Computer Science graduate since 2000 and been doing software
>>>>> >>>> testing since 2004.  I'm currently in a QA lead role in a startup,
>>>>> >>>> www.overlay.tv.  I recently completed certification from cstb.ca.
>>>>>  I've done
>>>>> >>>> upgrade and migration testing as well as web application testing
>>>>> >>>>
>>>>> >>>>
>>>>> >>>> I'm noticing that the heading for the new v3 guide contain a wide
>>>>> range
>>>>> >>>> of testing activities and areas.  I would like to propose adding
>>>>> the
>>>>> >>>> following section
>>>>> >>>>
>>>>> >>>> Software System Upgrade and Migration
>>>>> >>>>
>>>>> >>>> Looking forward to your thoughts
>>>>> >>>>
>>>>> >>>> Basim
>>>>> >>>
>>>>> >>>
>>>>> >>
>>>>> >> _______________________________________________
>>>>> >> Owasp-testing mailing list
>>>>> >> Owasp-testing at lists.owasp.org
>>>>> >> https://lists.owasp.org/mailman/listinfo/owasp-testing
>>>>> >>
>>>>> >
>>>>> >
>>>>> >
>>>>> > --
>>>>> > Eoin Keary OWASP - Ireland
>>>>> > http://www.owasp.org/local/ireland.html
>>>>> > http://www.owasp.org/index.php/OWASP_Code_Review_Project
>>>>> > _______________________________________________
>>>>> > Owasp-testing mailing list
>>>>> > Owasp-testing at lists.owasp.org
>>>>> > https://lists.owasp.org/mailman/listinfo/owasp-testing
>>>>> >
>>>>> >
>>>>> _______________________________________________
>>>>> Owasp-testing mailing list
>>>>> Owasp-testing at lists.owasp.org
>>>>> https://lists.owasp.org/mailman/listinfo/owasp-testing
>>>>>
>>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> Owasp-testing mailing list
>>>> Owasp-testing at lists.owasp.org
>>>> https://lists.owasp.org/mailman/listinfo/owasp-testing
>>>>
>>>>
>>>
>>
>> _______________________________________________
>> Owasp-testing mailing list
>> Owasp-testing at lists.owasp.org
>> https://lists.owasp.org/mailman/listinfo/owasp-testing
>>
>>
>
>
> --
> Eoin Keary OWASP - Ireland
> http://www.owasp.org/local/ireland.html
> http://www.owasp.org/index.php/OWASP_Code_Review_Project
>
> _______________________________________________
> Owasp-testing mailing list
> Owasp-testing at lists.owasp.org
> https://lists.owasp.org/mailman/listinfo/owasp-testing
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: https://lists.owasp.org/pipermail/owasp-testing/attachments/20080522/200d6ffd/attachment.html 