[OWASP-TESTING] final draft of the outline

Eoin Keary eoinkeary at hotmail.com
Fri May 13 10:40:26 EDT 2005


yep, I do somthing similar.


>From: Andrew van der Stock <vanderaj at greebo.net>
>To: Eoin Keary 
><eoinkeary at hotmail.com>,<mark.curphey at foundstone.com>,"daniel.cuthbert at owasp.org" 
><owasp-testing at lists.sourceforge.net>
>Subject: Re: [OWASP-TESTING] final draft of the outline
>Date: Sat, 14 May 2005 00:26:09 +1000
>
>I personally use when estimating:
>
>1 day per 1,000 lines of code for OWASP Top 10 reviews
>1 day per 100 lines of code for comprehensive OWASP reviews
>- and I add a bit for every trust boundary I identify in the component
>model.
>
>I also factor in what my client wants - an "absolute" guarantee or just a
>feeling of how their application is going.
>
>If it's a best effort basis, such as capped at X days or a certain fixed
>price, it's fun to read  the disclaimer to cover my large derriere as
>capping it like this is a sure fire way to miss something in the rush to 
>not
>lose money on the deal.
>
>I've done many reviews, and most have been a few weeks in length, producing
>reports in the 80+ page range. I do look for the bigger issues first, so I
>know the worst problems early on.
>
>However, I don't think the Testing guide should have day estimation
>information. Honestly, I'd prefer it included decent approaches to testing
>the application in various ways, and let individuals work out how to cost
>and price these activities.
>
>Andrew
>
>
>On 13/5/05 5:55 PM, "Eoin Keary" <eoinkeary at hotmail.com> wrote:
>
> > Hi Mark,
> > The original thread was discussing the idea of "How long would testing
> > take".
> > People suggested guessing. Putting this into a document that we hope is
> > going to be industry standard is a bit foolish if this document is to be
> > taken seriously.
> > Another idea (below) is to do some qualitative analysis on the 
>application
> > to be tested and to take into account some factors which may affect the
> > timeline. The solution below is better than guessing (don't you think?). 
>Do
> > you have any better ideas how to estimate work effort for a arbitrary 
>test?
> >
> > Eoin
> >
> >
> >
> >> From: "Curphey, Mark" <mark.curphey at foundstone.com>
> >> To: <owasp-testing at lists.sourceforge.net>
> >> Subject: RE: [OWASP-TESTING] final draft of the outline
> >> Date: Thu, 12 May 2005 14:03:28 -0700
> >>
> >> OK someone asked me to clarify. I am not against checklists. I am
> >> against someone proposing an industry accepted pricing model (times to
> >> test, same difference) that will lead to people working backwards from
> >> money and not forwards from technical needs.
> >>
> >> -----Original Message-----
> >> From: owasp-testing-admin at lists.sourceforge.net
> >> [mailto:owasp-testing-admin at lists.sourceforge.net] On Behalf Of 
>Curphey,
> >> Mark
> >> Sent: Thursday, May 12, 2005 10:16 AM
> >> To: Eoin Keary; a.revelli at reply.it; 
>owasp-testing at lists.sourceforge.net;
> >> daniel.cuthbert at owasp.org
> >> Subject: RE: [OWASP-TESTING] final draft of the outline
> >>
> >> 2 cents
> >>
> >> This is going to lead to an industry where joe blogs does the minimum
> >> possible to complete a checklist for the lowest cost possible. This is
> >> not condusive to promoting good quality testing and is 100% wrong IMHO.
> >>
> >> -----Original Message-----
> >> From: owasp-testing-admin at lists.sourceforge.net
> >> [mailto:owasp-testing-admin at lists.sourceforge.net] On Behalf Of Eoin
> >> Keary
> >> Sent: Thursday, May 12, 2005 9:00 AM
> >> To: a.revelli at reply.it; owasp-testing at lists.sourceforge.net;
> >> daniel.cuthbert at owasp.org
> >> Subject: RE: [OWASP-TESTING] final draft of the outline
> >>
> >> Hi Alberto,
> >> Good call, I think that is a much better solution.
> >> A qualitative instead of a quantitative approach. Similar to trying to
> >> nail unknowns in risk assessment.
> >> Better than a "guesstimate" anyways!
> >>
> >> So points to affect the critical path of the timeline:
> >>
> >> Complexity: (which is a subjective measure).
> >>
> >> Size (relative measure also) is it 10 "pages" or 1000?
> >>
> >> Technology - related to complexity. Pen testing web services may take
> >> longer than a "vanilla" application.
> >>
> >> Areas of focus, obviously confidential areas would take more time and
> >> one would take more effort to test these areas?
> >>
> >> Stability of the app is always an issue.
> >>
> >> Types of user: For every distinct user type the test shall be different
> >> because what is available to that user changes with role type. (if you
> >> know what I mean)?^&*##
> >>
> >> Any other suggestions to add to this list, anybody???
> >> Send them to Dan C (daniel.cuthbert at owasp.org), Not me, he's the
> >> leader!!
> >>
> >> Eoin
> >>
> >>
> >>
> >>
> >>
> >>> From: "Revelli Alberto" <a.revelli at reply.it>
> >>> To: <owasp-testing at lists.sourceforge.net>
> >>> Subject: RE: [OWASP-TESTING] final draft of the outline
> >>> Date: Thu, 12 May 2005 17:10:58 +0200
> >>>
> >>> very clear point, Eoin.
> >>>
> >>> What if instead of trying to answer a question like "how long does it
> >> take"
> >>> we limit ourselves to something like "what should you consider to
> >>> estimate the effort" ?
> >>>
> >>> I am thinking here about a short section that briefly mentions aspects
> >>> like:
> >>>
> >>> - dimension of the application (10 pages/views ? 100 ? 1,000 ?)
> >>> - complexity (static ? dynamic ? requires authentication ?)
> >>> - interconnections to other systems (is it stand-alone ? or fetches
> >>> data from a dozen of backend databases) ?
> >>>
> >>> No more than 1 or 2 pages, describing the variables that affect the
> >>> needed effort, but without giving a quantitative measure of the effort
> >> itself.
> >>> That would help managers to perform a reasonable analysis of the 
>needed
> >>
> >>> resources, and at the same there would not be any "guesstimate" in our
> >>> document...
> >>>
> >>> Alberto
> >>>
> >>> -----Original Message-----
> >>> From: owasp-testing-admin at lists.sourceforge.net on behalf of Eoin 
>Keary
> >>> Sent: Thu 5/12/2005 4:20 PM
> >>> To: owasp-testing at lists.sourceforge.net
> >>> Subject: RE: [OWASP-TESTING] final draft of the outline
> >>>
> >>> After talking to a number of people in the academic and industry I
> >>> still dont think we should use "guesstimates" in documents we would
> >>> like to become industry standards.
> >>>
> >>> The document is depicting fact, procedure and best practice, but under
> >>> the "How long shall this take/Cost" section we are going to answer:
> >> "Guess".
> >>> Sounds a bit foolish and also damages the integrity of the document. I
> >>> know ISO 17799 or COBIT docs dont have guessing games ;0)
> >>>
> >>> Just a thought.
> >>> Eoin
> >>>
> >>>
> >>>> From: "Shields, Larry" <Larry.Shields at FMR.COM>
> >>>> To: <owasp-testing at lists.sourceforge.net>
> >>>> Subject: RE: [OWASP-TESTING] final draft of the outline
> >>>> Date: Thu, 5 May 2005 08:52:33 -0400
> >>>>
> >>>>
> >>>> I agree.  Especially in black box testing, you can provide some rough
> >>
> >>>> rule of thumb stuff for a timebox for various applications.  It will
> >>>> at least provide a rough guesstimate for the poor project manager
> >>>> who's trying to make sure the application is secure and needs to put
> >>>> some number in the budget.
> >>>>
> >>>> -Larry
> >>>>
> >>>> -----Original Message-----
> >>>> From: Revelli Alberto [mailto:a.revelli at reply.it]
> >>>> Sent: Thursday, May 05, 2005 6:54 AM
> >>>> To: owasp-testing at lists.sourceforge.net
> >>>> Subject: RE: [OWASP-TESTING] final draft of the outline
> >>>>
> >>>>
> >>>>
> >>>>> True, but a guideline (with a massive caveat stating this isnt set
> >>>>> in concrete blah blah blah) would help companies who have zero
> >>>>> understanding of app testing to understand if they are being taken
> >>>>> for
> >>>> a ride.
> >>>>
> >>>> Exactly.
> >>>> I agree that needed resources heavily depend on the
> >>>> complexity/size/insert_your_favorite_variable_here of the
> >>>> application, but exactly for this reason it would be great to provide
> >>
> >>>> a few hints to help companies to have some clues about how much
> >>>> effort is needed to test their apps.
> >>>>
> >>>>> Ok, hows about i'll write up the section and everyone can review it
> >>
> >>>>> once the rest of the sections are finished?
> >>>>
> >>>> Sounds great :)
> >>>>
> >>>> Cheers
> >>>>
> >>>> Alberto
> >>>>
> >>>>
> >>>> -------------------------------------------------------
> >>>> This SF.Net email is sponsored by: NEC IT Guy Games.
> >>>> Get your fingers limbered up and give it your best shot. 4 great
> >>>> events,
> >>>> 4 opportunities to win big! Highest score wins.NEC IT Guy Games. Play
> >>
> >>>> to win an NEC 61 plasma display. Visit http://www.necitguy.com/?r
> >>>> _______________________________________________
> >>>> owasp-testing mailing list
> >>>> owasp-testing at lists.sourceforge.net
> >>>> https://lists.sourceforge.net/lists/listinfo/owasp-testing
> >>>>
> >>>>
> >>>>
> >>>> -------------------------------------------------------
> >>>> This SF.Net email is sponsored by: NEC IT Guy Games.
> >>>> Get your fingers limbered up and give it your best shot. 4 great
> >>>> events,
> >>> 4
> >>>> opportunities to win big! Highest score wins.NEC IT Guy Games. Play
> >>>> to win an NEC 61 plasma display. Visit http://www.necitguy.com/?r
> >>>> _______________________________________________
> >>>> owasp-testing mailing list
> >>>> owasp-testing at lists.sourceforge.net
> >>>> https://lists.sourceforge.net/lists/listinfo/owasp-testing
> >>>
> >>> _________________________________________________________________
> >>> Go where quality Irish singles meet - get FREE Match.com membership!
> >>> http://match.msn.ie
> >>>
> >>>
> >>>
> >>> -------------------------------------------------------
> >>> This SF.Net email is sponsored by Oracle Space Sweepstakes Want to be
> >>> the first software developer in space?
> >>> Enter now for the Oracle Space Sweepstakes!
> >>> http://ads.osdn.com/?ad_id=7393&alloc_id=16281&op=click
> >>> _______________________________________________
> >>> owasp-testing mailing list
> >>> owasp-testing at lists.sourceforge.net
> >>> https://lists.sourceforge.net/lists/listinfo/owasp-testing
> >>>
> >>>
> >>>
> >>>
> >>> -------------------------------------------------------
> >>> This SF.Net email is sponsored by Oracle Space Sweepstakes Want to be
> >>> the first software developer in space?
> >>> Enter now for the Oracle Space Sweepstakes!
> >>> http://ads.osdn.com/?ad_ids93&alloc_id281&op=click
> >>> _______________________________________________
> >>> owasp-testing mailing list
> >>> owasp-testing at lists.sourceforge.net
> >>> https://lists.sourceforge.net/lists/listinfo/owasp-testing
> >>
> >> _________________________________________________________________
> >> Upgrade to Messenger 7.0 - more fun features, still totally FREE!
> >> http://messenger.msn.co.uk
> >>
> >>
> >>
> >> -------------------------------------------------------
> >> This SF.Net email is sponsored by Oracle Space Sweepstakes Want to be
> >> the first software developer in space?
> >> Enter now for the Oracle Space Sweepstakes!
> >> http://ads.osdn.com/?ad_id=7393&alloc_id=16281&op=click
> >> _______________________________________________
> >> owasp-testing mailing list
> >> owasp-testing at lists.sourceforge.net
> >> https://lists.sourceforge.net/lists/listinfo/owasp-testing
> >>
> >>
> >>
> >> -------------------------------------------------------
> >> This SF.Net email is sponsored by Oracle Space Sweepstakes Want to be
> >> the first software developer in space?
> >> Enter now for the Oracle Space Sweepstakes!
> >> http://ads.osdn.com/?ad_ids93&alloc_id281&op=ick
> >> _______________________________________________
> >> owasp-testing mailing list
> >> owasp-testing at lists.sourceforge.net
> >> https://lists.sourceforge.net/lists/listinfo/owasp-testing
> >>
> >>
> >>
> >> -------------------------------------------------------
> >> This SF.Net email is sponsored by Oracle Space Sweepstakes
> >> Want to be the first software developer in space?
> >> Enter now for the Oracle Space Sweepstakes!
> >> http://ads.osdn.com/?ad_ids93&alloc_id281&op=click
> >> _______________________________________________
> >> owasp-testing mailing list
> >> owasp-testing at lists.sourceforge.net
> >> https://lists.sourceforge.net/lists/listinfo/owasp-testing
> >
> > _________________________________________________________________
> > Start dating right now with FREE Match.com membership! 
>http://match.msn.ie
> >
> >
> >
> > -------------------------------------------------------
> > This SF.Net email is sponsored by Oracle Space Sweepstakes
> > Want to be the first software developer in space?
> > Enter now for the Oracle Space Sweepstakes!
> > http://ads.osdn.com/?ad_id=7393&alloc_id=16281&op=click
> > _______________________________________________
> > owasp-testing mailing list
> > owasp-testing at lists.sourceforge.net
> > https://lists.sourceforge.net/lists/listinfo/owasp-testing
> >
>
>

_________________________________________________________________
Send a sexy animated wink with Messenger 7.0 - FREE download! 
http://messenger.msn.co.uk





More information about the Owasp-testing mailing list