[Owasp-leaders] [Global_tools_and_project_committee] [Owasp-board] FW: REQUESTFOR DECISION/CALL FOR CONTRIBUTIONS TO UPDATE THE ASSESSMENT CRITERIA

dinis cruz dinis.cruz at owasp.org
Mon Mar 9 20:09:02 EDT 2009


Mike, Larry and Alex, I know you are all motivated and excited about the new
tab system that you are implementing (which is great) and have started
playing with it on the main OWASP projects page, EASPI and ASVS
But, can you guys (and who is also involved in these WIKI experiments), make
sure that you pay attention to the small details related to the way OWASP
projects are mapped, classified and rated.

I know that in the past, the information about the OWASP projects has not
been very well organized, but over the past year (mainly via the SoCs and
Paulo Coimbra and Kate's work) there has an been an enormous amount of work
done on them, and we must make sure we don't lose it.

A good example are the standard project information tables we implemented
and that need to be taken into account in the new design (btw, Pravir just
added the previous template as a tab to the EASPI page to see how it works,
and it looked ok).

Another example of the type of detail that is important (today, in OWASP's
2009 world) are the places where the current assessment criteria has been
applied. For example following the idea suggested in this thread about
an  "(Assessment
Criteria v1.0)" , you guys added that tag to all projects currently listed
on the main OWASP project page:
http://www.owasp.org/index.php/Category:OWASP_Project

The problem is that NOT ALL projects listed there have gone thought that
assessment criteria (see here for a page containing the most up-to-date
information about the project criteria status of the SoC projects:
http://docs.google.com/Doc?id=dcn8962c_41hjg48kd4) and in fact a majority of
those projects have NOT been classified at all (they are Alpha, Beta or
Release for Historic reasons).

You also dropped the comment:

"...We have started the process of defining detailed guidelines which
indicate what will be required from an OWASP Project in order for it to be
classified an OWASP Release quality project (see Project Assessment
Criteria<http://www.owasp.org/index.php/Category:OWASP_Project_Assessment>).
Please note that not all the projects below have been evaluated under this
criteria and might be re-classified once that process is completed. ..."

that was present in the old version (
http://www.owasp.org/index.php?title=Category:OWASP_Project&oldid=55662) ,
and contained that important caveat (in bold).

So, keep up the great work, but please take into account these small details
that take time to get right, are a pain to implement, but are what is making
OWASP what it is today :)

The reason I wanted to raise these 'small' points in this discussion is that
we must remember that we are making (or proposing) changes that apply to ALL
OWASP projects, and there are a lot of 'unintended consequences' when we
make any type of global changes.

Please keep up with this thread since there are still quite a lot of items
that need to be clarified and debated.

Thanks

Dinis Cruz



2009/3/9 Paulo Coimbra <paulo.coimbra at owasp.org>

>  Leaders,
>
>
>
> As Matt has said, the discussion of downgrading to Alpha grew out of the
> committee trying to figure out *how to handle non-responsive project leads
> and/or dead projects which had never been assessed*.
>
>
>
> In fact we’ve never thought about downgrading projects already evaluated.
> We were simply thinking in proposing a division in two categories –projects
> with and without assessment.
>
>
>
> For what it's worth, I agree with Stephan when he says “Before a new
> assessment criteria version becomes official or released, there would be a 3
> or 6 month period allowed for project owners to "certify" their projects for
> the new version.”
>
>
>
> In addition, I believe the suggestion made by Stephan/Mike is also worth
> following – assessment references included in the OWASP Projects page should
> prominently inform about the followed assessment criteria version - i.e.
> “Assessment Criteria version 1.0” if the projects were assessed under the
> assessment criteria currently in use or “Assessment Criteria version 2.0”
> for those ones that are yet to be assessed accordingly with the, in time
> to come, updated assessment criteria version.
>
>
>
> If you allow me, I take the opportunity to challenge you all again to find
> the spare cycles to think about the assessment criteria itself -
> http://www.owasp.org/index.php/Category:OWASP_Project_Assessment_-_Update - until now just one change as been done.
>
>
>
> Thanks,
>
>
>
> Paulo Coimbra,
>
> OWASP Project Manager <https://www.owasp.org/index.php/Main_Page>
>
>
>
> *From:* Matt Tesauro [mailto:mtesauro at gmail.com]
> *Sent:* segunda-feira, 9 de Março de 2009 14:10
>
> *To:* Boberski, Michael [USA]
> *Cc:* Dave Wichers; paulo.coimbra at owasp.org;
> global_tools_and_project_committee at lists.owasp.org
> *Subject:* Re: [Global_tools_and_project_committee] [Owasp-board] FW:
> REQUESTFOR DECISION/CALL FOR CONTRIBUTIONS TO UPDATE THE ASSESSMENT CRITERIA
>
>
>
>
>
> Boberski, Michael [USA] wrote:
>
> > One last thought re "As for downgrading projects, I think it is a
>
> > necessary"
>
> >
>
> > Would you consider for example legal project now alpha status, since
>
> > the only update in years has been the one I did for ASVS?
>
> Definitely not.  Before any downgrade occurs, I'd look at several factors
> beyond just pre-dating the assessment criteria:
>
> * non-responsive project lead
>
> * long period of inactivity on the mail list
>
> * reduced relevance due to changing technology
>
> * out of date information
>
>
>
> Considering the fact that I recommended the Legal Annex where I work on
> Friday, I'd definitely not consider downgrading it.
>
>
>
> My hope is that the vast majority of projects won't be in this situation.
> All of last years SoC won't be.  From my perspective, the downgrading was
> only to address the situation of older, unmaintained projects of reduced
> relevance.  Hopefully these will be the exception rather then the rule - a
> few statistical outliers.
>
>
>
> Actually in preparing for the DHS event on Friday, I looked up the previous
> OWASP Live CD (2007) again.  OWASP.org has a page for it but when you follow
> the links to download it, the external linked site no longer hosts any
> ISOs.  This is a project that should be archived somehow but not prominently
> displayed as it would appear to only have historical value.
>
>
>
> http://www.owasp.org/index.php/Category:OWASP_Live_CD_2007_Project
>
>
>
> >
>
> > The answer I would argue is no. The quality of a tool or document
>
> > doesn't degrade over time.
>
> Absolutely true in this case.  If we had a hardening guide for Tomcat 3.x,
> it would be less relevant now then when created.  I'd consider moving
> something like that from Release to Alpha in hopes that a new project lead
> would grab it and get it up to date.  That's why I listed the factors above
> - downgrading is much more than assessed or not.
>
>
>
> Thanks for all the great back and forth on this - the process will be much
> better off for this dialog.
>
>
>
> -- Matt Tesauro
>
> OWASP Live CD Project Lead
>
> http://www.owasp.org/index.php/Category:OWASP_Live_CD_2008_Project
>
> http://mtesauro.com/livecd/ - Documentation Wiki
>
>
>
> >
>
> > I've gone and made some updates over the weekend to the projects page
>
> > and to ESAPI and ASVS pages, let's start perhaps from what is proposed
>
> > there, with the criteria versioning and such.
>
> >
>
> > Best,
>
> >
>
> > Mike B.
>
> >
>
> >
>
> > -----Original Message-----
>
> > From: Matt Tesauro [mailto:mtesauro at gmail.com]
>
> > Sent: Saturday, March 07, 2009 1:53 PM
>
> > To: Boberski, Michael [USA]
>
> > Cc: Dave Wichers; paulo.coimbra at owasp.org;
>
> > global_tools_and_project_committee at lists.owasp.org
>
> > Subject: Re: [Global_tools_and_project_committee] [Owasp-board] FW:
>
> > REQUESTFOR DECISION/CALL FOR CONTRIBUTIONS TO UPDATE THE
>
> > ASSESSMENTCRITERIA
>
> >
>
> > Responses inline below:
>
> >
>
> > Boberski, Michael [USA] wrote:
>
> >> Team, OWASP is getting overly bureaucratic, it seems to me.
>
> > Trust me I hate bureaucracy.  This is about providing a process to
>
> > gauge quality.  The proposal represents a large, one-time effort plus
>
> > small efforts going forward.  If OWASP wants quality, there is work to
>
> > be done and a time frame to do it in.  We're trying to get the bulk
>
> > done upfront and provide a means to never again accumulate a backlog of
> work.
>
> >
>
> >>
>
> >> I'd rather see people putting time/energy into tightening up their
>
> >> project pages, tools, and project presentations/datasheets. An
>
> >> example
>
> >
>
> >> are PHP and .NET ESAPI, there's no published mapping of Java ESAPI to
>
> >> PHP/ESAPI, that also should then identify which interfaces are being
>
> >> targeted for which releases. I'm going to try to work with Andrew to
>
> > fix
>
> >> that problem for PHP since I may have a need for a PHP ESAPI for a
>
> >> customer engagement, but it's still a good example.
>
> > Under the assessment criteria, there are incentives to do this work.
>
> > If
>
> >
>
> >   a project is Beta and wants to be Release, these are the types of
>
> > things that need to happen.  Release project = prominence,
>
> > particularly on the project page.  If you want these things done, how
>
> > better then provide a clear incentive for the project leads?
>
> >
>
> >>
>
> >> The more complete and professional a page/doc/tool looks, the easier
>
> > it
>
> >> is to identify the status and content of a doc/tool, the easier is to
>
> >> figure out its usefulness and to promote its adoption. That a
>
> >> doc/tool
>
> >
>
> >> has correct content or works is taken as a given, that is completely
>
> >> secondary to the initial figuring out if a doc/tool is a potential
>
> >> solution to one's problem of the day.
>
> > I'd suggest that the process we're proposing will get you there.  One
>
> > aspect of looking professional is consistency.  The criteria and
>
> > project
>
> >
>
> > page requirements are aimed an getting that consistent representation
>
> > while allowing a project to further explain its relevance to its
>
> > target audience.  That's why we're shooting for a standard frame +
>
> > whatever else the project wants/needs.
>
> >
>
> > As for "correct content or works is taken as a given", I'd ask for
>
> > what project and what categorization?  Can this be said for an Alpha
> project?
>
> >
>
> >   Beta or Release?  How about the many non-assessed projects?  Once we
>
> > have all projects assessed, statements like that can be made with
>
> > certainty.
>
> >
>
> > I know I actually liked the fact that I could read the assessment
>
> > criteria and know explicitly where my project would fall based on how
>
> > I fulfilled the listed criteria.  I have N things to do in Y time and
>
> > I can plan according to make the SoC deadline with the quality rating
>
> > I'm shooting for.  No guessing.
>
> >
>
> >>
>
> >> I would also caution against downgrading projects, which is what one
>
> > of
>
> >> the comments seems to imply could happen. If you must address some
>
> >> perceived contention over project assessment criteria, you should
>
> > simply
>
> >> put dates against ratings, and identify the criteria version that a
>
> >> project was assessed against, then leave that rating alone as the
>
> >> criteria continues to evolve over time. That is what more
>
> >> well-established and formal testing programs for instance like Common
>
> >> Criteria and FIPS 140 do. I hope I am misreading comments on this
>
> > point
>
> >> however.
>
> > I really like the idea of dating the assessment ratings.  Please join
>
> > the discussion and add those to the page Paulo has created.  That's
>
> > fantastic stuff and why we've posed this to the collective
>
> > intelligence of OWASP.
>
> > http://www.owasp.org/index.php/Category:OWASP_Project_Assessment_-_Upd
>
> > at
>
> > e
>
> >
>
> > As for downgrading projects, I think it is a necessary thing to
>
> > determine projects that are dormant, abandoned or otherwise moribund.
>
> > I
>
> >
>
> > personally would like to start an "archive" "deprecated" or similar
>
> > location for projects that aren't actively developed and/or no longer
>
> > timely (think PHP 3 coding guidelines) but may still have value.
>
> > Also, those abandoned but potentially useful projects could become SoC
>
> > projects so their quality will improve.
>
> >
>
> > Finally, by engaging the project leads, OWASP could also find projects
>
> > were the lead no longer wants or can be the maintainer.  Identifying
>
> > those projects would allow someone else to take up that project and
>
> > move
>
> >
>
> > it forward.
>
> >
>
> > -- Matt Tesauro
>
> > OWASP Live CD Project Lead
>
> > http://www.owasp.org/index.php/Category:OWASP_Live_CD_2008_Project
>
> > http://mtesauro.com/livecd/ - Documentation Wiki
>
> >
>
> >>
>
> >> Best,
>
> >>
>
> >> Mike B.
>
> >>
>
> >>
>
> >>
>
> >>
>
> > ----------------------------------------------------------------------
>
> > --
>
> >> *From:* global_tools_and_project_committee-bounces at lists.owasp.org
>
> >> [mailto:global_tools_and_project_committee-bounces at lists.owasp.org]
>
> > *On
>
> >> Behalf Of *Dave Wichers
>
> >> *Sent:* Wednesday, March 04, 2009 5:34 PM
>
> >> *To:* paulo.coimbra at owasp.org; 'OWASP Foundation Board List';
>
> >> global_tools_and_project_committee at lists.owasp.org
>
> >> *Subject:* Re: [Global_tools_and_project_committee] [Owasp-board] FW:
>
> >> REQUESTFOR DECISION/CALL FOR CONTRIBUTIONS TO UPDATE THE
>
> > ASSESSMENTCRITERIA
>
> >> I'm OK with this, although I'm not a big fan of many of the
>
> > suggestions
>
> >> below. But that's OK. Lets get the ideas out there and we can then
>
> > make
>
> >> some decisions.
>
> >>
>
> >>
>
> >>
>
> >> -Dave
>
> >>
>
> >>
>
> >>
>
> >> *From:* owasp-board-bounces at lists.owasp.org
>
> >> [mailto:owasp-board-bounces at lists.owasp.org] *On Behalf Of *Paulo
>
> > Coimbra
>
> >> *Sent:* Wednesday, March 04, 2009 11:25 AM
>
> >> *To:* 'OWASP Foundation Board List';
>
> >> global_tools_and_project_committee at lists.owasp.org
>
> >> *Subject:* [Owasp-board] FW: REQUEST FOR DECISION/CALL FOR
>
> > CONTRIBUTIONS
>
> >> TO UPDATE THE ASSESSMENT CRITERIA
>
> >> *Importance:* High
>
> >>
>
> >>
>
> >>
>
> >> Board, Project's Committee,
>
> >>
>
> >>
>
> >>
>
> >> In consequence of the comments received in the last Committee
>
> >> meeting,
>
> >
>
> >> I've introduced the changes yellow underlined.  Please let me know if
>
> >> this email can be sent off.*/ /*
>
> >>
>
> >>
>
> >>
>
> >> Many thanks, regards,
>
> >>
>
> >>
>
> >>
>
> >> Paulo
>
> >>
>
> >>
>
> >>
>
> >>
>
> >>
>
> >> Hello Leaders,
>
> >>
>
> >>
>
> >>
>
> >> I hope you are well.
>
> >>
>
> >>
>
> >>
>
> >> You better than anyone else know that OWASP as an organization has
>
> > been
>
> >> built by your continuous open contributions both by defining its
>
> >> mission, organizational structure, rules and procedures and by
>
> >> leading
>
> >
>
> >> the application security projects that are its core of activity.
>
> >>
>
> >>
>
> >>
>
> >> In my today's call for contributions, procedures regarding projects
>
> >> development's stage assessment are the main issue.
>
> >>
>
> >>
>
> >>
>
> >> As you may know, a system to evaluate OWASP projects is already in
>
> >> use
>
> >
>
> >> and actually consists in both a set of criteria
>
> >> http://www.owasp.org/index.php/Category:OWASP_Project_Assessment and
>
> >> a
>
> >
>
> >> skeleton/frame to implement it
>
> >>
>
> > http://www.owasp.org/index.php/OWASP_Live_CD_2008_Project_-_Assessment
>
> > _F
>
> > rame
>
> >> .
>
> >>
>
> >>
>
> >>
>
> >> With other few subsequent modifications, this set of criteria has
>
> > mainly
>
> >> resulted of a vigorous discussion held through this mailing list
>
> > almost
>
> >> a year ago and since then it has been used in all newly set up
>
> > projects.
>
> >>
>
> >>
>
> >> Since then this issue has been discussed consecutively in several
>
> >> different contexts. In our Summit, for example, even if we haven't
>
> >> committed a specific slot of time to deal with this matter, it has
>
> >> collaterally arisen throughout many project's presentations. In
>
> >> addition, I regularly receive from OWASP Board requests to make
>
> >> modifications, a systemic reflection is being held within the
>
> > Project's
>
> >> Committee and, as result of my daily handling of projects under
>
> > review,
>
> >> I am obtaining some feedback from project leaders and reviewers.
>
> >>
>
> >>
>
> >>
>
> >> Overall, the people with whom I've discussed this issue usually say
>
> > that
>
> >> the procedure can be improved and IMHO, even if I think*/ /*the
>
> >> Assessment Criteria is working and actually has been of great help,
>
> > they
>
> >> are right.
>
> >>
>
> >>
>
> >>
>
> >>  From these discussions, I've retained that a handful of criteria
>
> >> have
>
> >
>
> >> been proposed but haven't been implemented yet as forthcoming:
>
> >>
>
> >> -          OWASP writing style (Tool projects/Release Quality),
>
> >>
>
> >> -          Translation (Tools and Documentation/Release Quality),
>
> >>
>
> >> -          Bi-monthly periodic news (Tools and Documentation/non
>
> >> specified Quality status),
>
> >>
>
> >> -          5 slide deck for OWASP Boot Camp project (Tools and
>
> >> Documentation/Beta status),
>
> >>
>
> >> -          Attribution rules (Tools and Documentation/non specified
>
> >> Quality status),
>
> >>
>
> >> -          Compulsory Project Skeleton/Frame (Tools and
>
> >> Documentation/all Quality status),
>
> >>
>
> >> -           Reviewer role - addition and clarification,
>
> >>
>
> >> http://owaspsoc2008.wordpress.com/2008/07/15/assessment-guidance/
>
> >>
>
> >> -          Mentor role addition and definition.
>
> >>
>
> >> In addition, as far as I am concerned, a few more structural comments
>
> >> have also been made. Even without pointing out alternative technical
>
> >> solutions, at least a  couple of them have questioned the rationale
>
> >> of
>
> >
>
> >> working with tables in wiki text and others have pointed out the
>
> >> willingness of having a project's page similar to, for example, this
>
> > one
>
> >> http://www.hdiv.org/.
>
> >>
>
> >>
>
> >>
>
> >> Having said all the above with the intention of giving you a picture
>
> > of
>
> >> the current situation, I ask for your contribution so as to update
>
> >> the
>
> >
>
> >> OWASP Assessment Criteria.
>
> >>
>
> >>
>
> >>
>
> >> In operational terms, I've replicated the Assessment Criteria page
>
> >>
>
> > http://www.owasp.org/index.php/Category:OWASP_Project_Assessment_-_Upd
>
> > at
>
> > e
>
> >> and propose you introduce your changes directly on it. As soon as we
>
> >> finish the discussion phase, all the contributions will be moved to
>
> > the
>
> >> original wiki page. With the goal of enhancing the discussion, I also
>
> >> propose you use this mailing list to inform which changes are being
>
> >> proposed and the reason or goal for doing so. We are also building a
>
> >> Google questionnaire to collect your opinions and contributions and,
>
> > as
>
> >> soon as it is finished, it will be sent off.
>
> >>
>
> >>
>
> >>
>
> >> Please do have into account that you proposals can have implications
>
> > in
>
> >> the assessment frame that we are currently using and, if it happens,
>
> >> please present a compatible solution.
>
> >>
>
> >>
>
> >>
>
> >> To conclude, I would like to inform you that the Project's Committee
>
> >> propose that, as soon as we finish this discussion, we establish as a
>
> >> rule to apply to all OWASP Projects that the quality categorization
>
> > must
>
> >> respect the revised assessment criteria which eventually will mean
>
> > that
>
> >> all projects not assessed under these rules will be placed under
>
> >> Alpha
>
> >
>
> >> Quality status.
>
> >>
>
> >> */ /*
>
> >>
>
> >> I thank you all in anticipation and look forward to having your
>
> >> indispensable feedback.
>
> >>
>
> >> Regards,
>
> >>
>
> >>
>
> >>
>
> >> Paulo Coimbra,
>
> >>
>
> >> OWASP Project Manager <https://www.owasp.org/index.php/Main_Page>
>
> >>
>
> >>
>
> >>
>
> >>
>
> >>
>
> > ----------------------------------------------------------------------
>
> > --
>
> >> _______________________________________________
>
> >> Global_tools_and_project_committee mailing list
>
> >> Global_tools_and_project_committee at lists.owasp.org
>
> >>
>
> > https://lists.owasp.org/mailman/listinfo/global_tools_and_project_comm
>
> > it
>
> > tee
>
> _______________________________________________
> OWASP-Leaders mailing list
> OWASP-Leaders at lists.owasp.org
> https://lists.owasp.org/mailman/listinfo/owasp-leaders
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: https://lists.owasp.org/pipermail/owasp-leaders/attachments/20090310/3e43e74c/attachment-0001.html 


More information about the OWASP-Leaders mailing list