[Owasp-testing] OWASP Testing Guide v3: introduce section for Software System Upgrade and Migration testing

Daniel Cuthbert daniel.cuthbert at owasp.org
Thu May 22 10:01:02 EDT 2008 

Problem is going off scope as to what the Testing guide covers. I  
agree with you that we need to ensure that all risks have to be  
covered, the Testing guide was always a method of looking at an  
applications security from a Black/White hat perspective. There are a  
load of guides out there that explain how to look at the  
infrastructure side of testing, and whilst we can reference them,  
sticking to the application layer is more advisable, knowing how hard  
this guide is to manage and control.

It's an interesting area though


On 22 May 2008, at 3:06 PM, Dave van Stein wrote:

> Eoin
> I sense scope creep here :)
> Patch management in an application security testing guide?
>
> Not at all actually. The discussion is where application security  
> stops and what to do with known vulnerability risks beyond it. I  
> think most of us agree that these things do not belong in a  
> application security manual/framework, but I think it's also not a  
> good idea to just completely ignore them.
> Like I said, the least we can do is make a (small) note (with  
> reference) when vulnerabilities are identified which are beyond the  
> scope of application security.
> Ignorance should be avoided at all times.
>
> Also upgrade and migration testing, What are we actually going to  
> test for here? If there is an application upgrade one does not  
> simply test the upgraded piece [..]
> Migration testing would cover the same suite of tests as a regular  
> application security test, but there may be some security review if  
> the task was more than blackbox. SO we would look at config files  
> etc. but testing specific to migration as opposed to testing to see  
> if something is secure, the difference is minimal.
> I agree. As a functional tester I have had these situations before  
> and the sensible way to scope your testplan is to do a risk analysis  
> and decide whether a regression test is enough or a complete  
> functional test is necessary. The same goes for security testing in  
> my opinion.
>
> Dave
>
>
> On 21/05/2008, Dave van Stein <dvstein at gmail.com> wrote:
> Basim,
>
> I agree. In the mentioned case of upgrade en migration testing we  
> could add a note that at least patch management, authentication &  
> access control and configuration files should get extra attention to  
> limit possible vulnerabilities.
>
> regards, dave
>
>
> 2008/5/21 Basim Baassiri <basim at baassiri.ca>:
>
> I agree with your points Dave, there should be mention of this topic  
> (upgrade and migration testing) when the scope of the manual has  
> been reached.  Something along lines of "Although this topic is not  
> in the scope of the manual, we feel that it should be mentioned so  
> it is not an oversight when doing testing"
>
> Thanks again for all of your feedback
>
> Looking forward to helping out with the new guide
>
> Basim
>
> On Tue, May 20, 2008 at 3:40 PM, Dave van Stein <dvstein at gmail.com>  
> wrote:
>
> All,
>
> I think that setting the scope and boundaries is one of the hardest  
> things concerning creating an application security testing manual.  
> In the case described above/below it handles the boundary between  
> testing and SDLC, but the same problem arises when defining the  
> boundary between for example application testing and ('classic')  
> penetration testing. Clearly the goal of OWASP is not testing things  
> like the existence of open SSH sockets, but things like command  
> injection can be used to gain root access on a webserver; where does  
> application testing stop in that case ?
>
> On the other hand is limiting the scope to far not good either. In  
> my opinion the testing manual should at least give some information  
> about risks and things to improve when a 'boundary' is crossed.
>
> regards, Dave
>
> 2008/5/20 kevin horvath <kevin.horvath at gmail.com>:
>
> I agree that this is more on the SDLC side then the actual security
> testing of an application.  Secure SDLC testing would be a nice
> project to start for OWASP but I think it would be too much to
> incorporate into this project.
>
> On Tue, May 20, 2008 at 10:42 AM, Eoin <eoin.keary at owasp.org> wrote:
> > To Me this sounds more like Secure SDLC Testing and not pen  
> testing, not
> > that the guide is strictly pen testing.
> >
> > Secure SDLC testing, test plans relating to functional requirements
> > etc could be a project on its own? Maintenance lifecycle,  
> regression and
> > root cause analysis relating to the software development lifecycle  
> can (and
> > should) involve an element of security. This area is very large  
> and needs to
> > be comprehensively discussed and is a valid candidate for its own  
> guide?
> >
> > I would be concerned that the guide may be stretching itself and  
> covering
> > too many things rather than covering fewer things in a more  
> complete manner.
> >
> > Eoin
> >
> > OWASP Code Review Guide Lead
> >
> >
> > On 20/05/2008, Basim Baassiri <basim at baassiri.ca> wrote:
> >>
> >> Has anyone had the chance to read below
> >>
> >> Basim
> >>
> >> On Mon, May 12, 2008 at 12:09 AM, Basim Baassiri  
> <basim at baassiri.ca>
> >> wrote:
> >>>
> >>> Hello Matteo and all,
> >>>
> >>> When you say new concept, I'm assuming a new concept for the  
> guide. As to
> >>> where it fits as part of the index, I can't clearly see it at  
> this time.  To
> >>> be clearer on the term Software System upgrades and migrations,  
> please see
> >>> my definition below.
> >>>
> >>> Software System Upgrade:
> >>> Basically, is testing the end result of the the data and  
> application when
> >>> a patch/upgrade/hotfix has been applied to it (Upgrade path  
> Testing)
> >>>
> >>> Typically, testing this process is as follows
> >>> 1. installation verification
> >>> Did the installation of the patches/upgrade executable/script to  
> the
> >>> existing system cause an failures. In the context of a web  
> application is
> >>> there exceptions/failures in the application log
> >>>
> >>> 2. Database verification (if required, but typically required in  
> majority
> >>> of web applications)
> >>> This is a two step process where a tester would look to see  
> (either the
> >>> output of the upgrade script or database logs) if any error  
> occurred in the
> >>> logs
> >>> Second, a capture of a subset of a data from before and after  
> would be
> >>> done to analyze transformations of data
> >>>
> >>> 3. Does the application behave the same way it did before  
> (Functional
> >>> testing)
> >>>
> >>> As for migrations, this is defined by completely moving over the  
> another
> >>> system.
> >>> Examples of migrations:
> >>> application A uses MySQL Database -> Apply migration ->  
> application A now
> >>> uses PostGres database
> >>> application B uses PHP -> Apply migration -> application B now  
> uses Java
> >>>
> >>> The reason I mention the above examples is because I have been  
> exposed to
> >>> this scenario where it was expected that the application  
> continues to behave
> >>> to same after the transformation
> >>>
> >>>
> >>> Thats the background of upgrade and migration testings
> >>>
> >>> I guess the next question is how does the above apply to  
> security testing
> >>>
> >>> I've seen changes to table schemas where by the code is not  
> reflective of
> >>> the change in the database and as a result "private" or  
> sensitive data has
> >>> been exposed.
> >>>
> >>> Not sure where this falls under the current index.  This topic  
> is not
> >>> commonly thought of in both the development and testing teams.
> >>>
> >>> Looking for feedback from anyone
> >>>
> >>> Cheers
> >>>
> >>> Basim
> >>>
> >>>
> >>> On Thu, May 8, 2008 at 12:00 PM, Basim Baassiri  
> <basim at baassiri.ca>
> >>> wrote:
> >>>>
> >>>> Hello all
> >>>>
> >>>> I've been watching the list for about 6 months now
> >>>>
> >>>> Just wanted to introduce myself
> >>>>
> >>>> My Background
> >>>> I am Computer Science graduate since 2000 and been doing software
> >>>> testing since 2004.  I'm currently in a QA lead role in a  
> startup,
> >>>> www.overlay.tv.  I recently completed certification from  
> cstb.ca.  I've done
> >>>> upgrade and migration testing as well as web application testing
> >>>>
> >>>>
> >>>> I'm noticing that the heading for the new v3 guide contain a  
> wide range
> >>>> of testing activities and areas.  I would like to propose  
> adding the
> >>>> following section
> >>>>
> >>>> Software System Upgrade and Migration
> >>>>
> >>>> Looking forward to your thoughts
> >>>>
> >>>> Basim
> >>>
> >>>
> >>
> >> _______________________________________________
> >> Owasp-testing mailing list
> >> Owasp-testing at lists.owasp.org
> >> https://lists.owasp.org/mailman/listinfo/owasp-testing
> >>
> >
> >
> >
> > --
> > Eoin Keary OWASP - Ireland
> > http://www.owasp.org/local/ireland.html
> > http://www.owasp.org/index.php/OWASP_Code_Review_Project
> > _______________________________________________
> > Owasp-testing mailing list
> > Owasp-testing at lists.owasp.org
> > https://lists.owasp.org/mailman/listinfo/owasp-testing
> >
> >
> _______________________________________________
> Owasp-testing mailing list
> Owasp-testing at lists.owasp.org
> https://lists.owasp.org/mailman/listinfo/owasp-testing
>
>
>
> _______________________________________________
> Owasp-testing mailing list
> Owasp-testing at lists.owasp.org
> https://lists.owasp.org/mailman/listinfo/owasp-testing
>
>
>
>
> _______________________________________________
> Owasp-testing mailing list
> Owasp-testing at lists.owasp.org
> https://lists.owasp.org/mailman/listinfo/owasp-testing
>
>
>
>
> -- 
> Eoin Keary OWASP - Ireland
> http://www.owasp.org/local/ireland.html
> http://www.owasp.org/index.php/OWASP_Code_Review_Project
>
> _______________________________________________
> Owasp-testing mailing list
> Owasp-testing at lists.owasp.org
> https://lists.owasp.org/mailman/listinfo/owasp-testing
>
>
> _______________________________________________
> Owasp-testing mailing list
> Owasp-testing at lists.owasp.org
> https://lists.owasp.org/mailman/listinfo/owasp-testing

-------------- next part --------------
An HTML attachment was scrubbed...
URL: https://lists.owasp.org/pipermail/owasp-testing/attachments/20080522/66e18a79/attachment-0001.html

More information about the Owasp-testing mailing list