[Owasp-leaders] CALL FOR CONTRIBUTIONS TO UPDATE THE ASSESSMENT CRITERIA

dinis cruz dinis.cruz at owasp.org
Thu Mar 5 19:25:23 EST 2009


For the ones that don't understand Portuguese my comment below sort of
translates to "... here is the missing email :) ..." (not sure how to
translate 'Malandro' in this context, since these translations
http://translate.google.com/translate_t?hl=en#pt|en|malandro are not the
best to represent the context (which was the fact that Paulo was worried
that the email had not been sent, since a couple minutes nobody on Paulo's
immediate reach (me, Leonardo, etc..) had received it) ). Of course that
that comment should had gone only to Paulo, but hey, we all make those
mistakes :)
Now, on the topic at hand.

Have i mentioned (see my previous email to this list), about HOW IMPORTANT
IT IS THAT ALL PROJECT LEADERS READ THIS EMAIL.

>From my experience in the past (namely the Summit), I know that most of you
don't really read (properly) the emails sent (In fact, I am probably the
worse one since my email 'coverage' only lasts one day (or one page of
GMail) and I do read a lot of stuff diagonally (which btw, means that If I
haven't replied to you in a couple days, you will need to SPAM me until I do
:)  )).

This email is different! You WILL have to read it, understand it and
(ultimately) agree with the rules proposed.

In the last two years (staring at the OWASP European conference in Italy)
there as been an enormous focus, energy and investment by OWASP to raise the
quality and professionalism of our Projects (Tools & Documents).

In practical terms this was reflected in a (sort of mature)
http://www.owasp.org/index.php/Category:OWASP_Project_Assessment<http://www.owasp.org/index.php/Category:OWASP_Project_Assessment
 > criteria
which we NOW want to make official and enforce its use on ALL Owasp
projects.

The good news is that we (namely me, Paulo and the recently formed OWASP
Global Projects
Committee<https://www.owasp.org/index.php/Global_Projects_Committee> )
have a bit of experience on the effectiveness (and operational problems) of
these assessment criterias, so we should be able to provide constructive
comments on the viability and scalability of your proposals. It is important
to make the point (namely for the ones that were not involved in the last
Season of Code (where one of the requirements was that the delivered
projects had to be of BETA or QUALITY level)) that this assessment criteria
WORKS quite well. In fact the only time we had 'problems' with it, was when
the people involved (like me for example) couldn't find the time to perform
the allocated tasks (for example to perform a review). Ultimately this is
about scalability. My view is that only very defined, pragmatic and
easy-to-understand 'rules of engagement' will allow our projects to grow in
a controlled and 'quality focused' way.

Remember that this will apply to ALL projects that we currently have
https://www.owasp.org/index.php/Category:OWASP_Project (122 as per that
page) with the ultimate objective to make our projects as EASY TO USE (and
discoverable) as possible, for our wide spread community of users. For
example, once this is sorted out, the RELEASE quality projects will be given
much more visibility and exposure since they will be the easiest to use and
should be the ones with better 'user experience'.

This thread is probably one of the most important ones in the history of
OWASP, since this "project assessment criteria" is what will set the OWASP
agenda and focus for the next years (and the plan is that once it is in
place, we will only make changes to deal with specific issues).

One point that I would like to make is that we have NO intention to put any
limitations or 'bureaucratic process' (as some might view this) to the
creation and development of new/existing OWASP projects. The way this has
been designed, is to have a very relaxed ALPHA Quality level, a
semi-demanding BETA-Quality and a very 'quality focused' QUALITY level.
Think of this in terms of project maturity and usability.

We could also view this as: ALPHA is the place for innovation, BETA the
place for consolidation and RELEASE the place for professionalism /
usability.

Another key idea that we are embedding in this criteria is the role
of reviewers (and maybe even project mentors). From a project development /
improvement point of view, at OWASP, we are always trying to find ways to
increase the feedback that project leaders receive. We found that the role
of the REVIEWER has been (in most cases) spectacularly successful at that
(thanks to all SoC 08 reviewers :) ), so please take the time to understand
its role and how it fits within your project (also, if you have the cycles,
please DO put your name forward as a reviewer for OWASP projects (remember
at 1, 2 (or more) reviewers per project, at 120 projects we will need
hundreds of reviewers :)  (you can email Paulo with your availability and
areas you would like to be involved))

Ultimately we have to raise the quality, usability and practicality of our
projects so that they are as good (if not better) to what is available out
there (commercially or by other Open Source projects).

I want OWASP tools and documents that I can proudly recommend and use on
professional / commercial engagements. If we want to change the world of
security, we have to make sure that the goodness of OWASP reaches the right
audience :)


OK, so, your next steps are:

   1. the read email below from Paulo
   2. read the proposed assessment criteria:
   http://www.owasp.org/index.php/Category:OWASP_Project_Assessment_-_Update

   3. think about how it relates to your project
   4. if you didn't participate on the OWASP Summer of Code
2008<https://www.owasp.org/index.php/OWASP_Summer_of_Code_2008> (and
   even if you did) please DO make the effort to browse the following pages
   since they show a REAL-WORLD example of the assessment criteria being used:
      1. SoC 08 Approved projects, authors, status target and
reviewers<https://www.owasp.org/index.php/OWASP_Summer_of_Code_2008_Projects_Authors_Status_Target_and_Reviewers>

      2. OWASP Live CD 2008
Project<https://www.owasp.org/index.php/Category:OWASP_Live_CD_2008_Project>
,
       Project assessment
table<https://www.owasp.org/index.php/OWASP_Live_CD_2008_Project_-_Assessment_Frame>
        , Self-Evaluation
(B)<https://www.owasp.org/index.php/Project_Information:template_Live_CD_2008_Project_-_Final_Review_-_Self_Evaluation_-_B>
       , First Reviewer
(D)<https://www.owasp.org/index.php/Project_Information:template_Live_CD_2008_Project_-_Final_Review_-_First_Reviewer_-_D>
       , Second Reviewer
(F)<https://www.owasp.org/index.php/Project_Information:template_Live_CD_2008_Project_-_Final_Review_-_Second_Reviewer_-_F>
       , Board Member's Review
(G)<https://www.owasp.org/index.php/Project_Information:template_Live_CD_2008_Project_-_Final_Review_-_OWASP_Board_Member_-_G>
      3. OWASP Application Security Verification
Standard<https://www.owasp.org/index.php/Category:OWASP_Application_Security_Verification_Standard_Project>
        <https://www.owasp.org/index.php/Category:OWASP_Application_Security_Verification_Standard_Project>
      , Project assessment
table<https://www.owasp.org/index.php/Project_Information:template_Application_Security_Verification_Standard_-_Proposal_-_Assessment_Frame>
       , Final Review/SelfEvaluation
(B)<https://www.owasp.org/index.php/Project_Information:template_Application_Security_Verification_Standard_-_Final_Review_-_Self_Evaluation_-_B>
       , Final Review/1st Reviewer (D)
<https://www.owasp.org/index.php/Project_Information:template_Application_Security_Verification_Standard_-_Final_Review_-_First_Reviewer_-_D>
       , Final Review/2nd Reviewer
(F)<https://www.owasp.org/index.php/Project_Information:template_Application_Security_Verification_Standard_-_Final_Review_-_Second_Reviewer_-_F>

      4. OWASP Spanish Project<https://www.owasp.org/index.php/OWASP_Spanish>
       , See&Edit: Final Review/SelfEvaluation
(B)<https://www.owasp.org/index.php/Project_Information:template_Spanish_Project_-_Final_Review_-_Self_Evaluation_-_B>
       , See&Edit: Final Review/1st Reviewer
(D)<https://www.owasp.org/index.php/Project_Information:template_Spanish_Project_-_Final_Review_-_First_Reviewer_-_D>
       , See&Edit: Final Review/2nd Reviewer
(F)<https://www.owasp.org/index.php/Project_Information:template_Spanish_Project_-_Final_Review_-_Second_Reviewer_-_F>
      5. etc...
   5. read the criteria again
   http://www.owasp.org/index.php/Category:OWASP_Project_Assessment_-_Update

   6. identify your areas of concern and find alternative solutions for it
   7. write your comments and solutions:
      - in a post to an owasp-leaders thread (use this thread or start a new
      one if your topic / issue is quite specific)
      - in your your blog (with link send to this list)
      - in a forum (for the super-proactive ones that will take on the
      challenge in my previous email and organize a forum solution for
OWASP :)  )


Looking forward to your comments, feedback, ideas, criticisms, suggestions,
good humor and pragmatic responses :)

Dinis Cruz


2009/3/5 Dinis Cruz <dinis.cruz at owasp.org>

> Aqui ta' o malandro :)
>
> Dinis Cruz
>
> On 5 Mar 2009, at 14:54, "Paulo Coimbra" <paulo.coimbra at owasp.org> wrote:
>
>  Hello Leaders,
>
>
>
> I hope you are well.
>
>
>
> You better than anyone else know that OWASP as an organization has been
> built by your continuous open contributions both by defining its mission,
> organizational structure, rules and procedures and by leading the
> application security projects that are its core of activity.
>
>
>
> In my today’s call for contributions, procedures regarding projects
> development’s stage assessment are the main issue.
>
>
>
> As you may know, a system to evaluate OWASP projects is already in use and
> actually consists in both a set of criteria
> <http://www.owasp.org/index.php/Category:OWASP_Project_Assessment>
> http://www.owasp.org/index.php/Category:OWASP_Project_Assessment and a
> skeleton/frame to implement it
> <http://www.owasp.org/index.php/OWASP_Live_CD_2008_Project_-_Assessment_Frame>
> http://www.owasp.org/index.php/OWASP_Live_CD_2008_Project_-_Assessment_Frame.
>
>
>
> With other few subsequent modifications, this set of criteria has mainly
> resulted of a vigorous discussion held through this mailing list almost a
> year ago and since then it has been used in all newly set up projects.
>
>
>
> Since then this issue has been discussed consecutively in several different
> contexts. In our Summit, for example, even if we haven’t committed a
> specific slot of time to deal with this matter, it has collaterally arisen
> throughout many project’s presentations. In addition, I regularly receive
> from OWASP Board requests to make modifications, a systemic reflection is
> being held within the Project’s Committee and, as result of my daily
> handling of projects under review, I am obtaining some feedback from project
> leaders and reviewers.
>
>
>
> Overall, the people with whom I’ve discussed this issue usually say that
> the procedure can be improved and IMHO, even if I think* *the Assessment
> Criteria is working and actually has been of great help, they are right.
>
>
>
> From these discussions, I’ve retained that a handful of criteria have been
> proposed but haven’t been implemented yet as forthcoming:
>
> -          OWASP writing style (Tool projects/Release Quality),
>
> -          Translation (Tools and Documentation/Release Quality),
>
> -          Bi-monthly periodic news (Tools and Documentation/non specified
> Quality status),
>
> -          5 slide deck for OWASP Boot Camp project (Tools and
> Documentation/Beta status),
>
> -          Attribution rules (Tools and Documentation/non specified
> Quality status),
>
> -          Compulsory Project Skeleton/Frame (Tools and Documentation/all
> Quality status),
>
> -           Reviewer role - addition and clarification,
>
> <http://owaspsoc2008.wordpress.com/2008/07/15/assessment-guidance/>
> http://owaspsoc2008.wordpress.com/2008/07/15/assessment-guidance/
>
> -          Mentor role addition and definition.
>
> In addition, as far as I am concerned, a few more structural comments have
> also been made. Even without pointing out alternative technical solutions,
> at least a  couple of them have questioned the rationale of working with
> tables in wiki text and others have pointed out the willingness of having a
> project’s page similar to, for example, this one <http://www.hdiv.org/>
> http://www.hdiv.org/.
>
>
>
> Having said all the above with the intention of giving you a picture of the
> current situation, I ask for your contribution so as to update the OWASP
> Assessment Criteria.
>
>
>
> In operational terms, I’ve replicated the Assessment Criteria page
> <http://www.owasp.org/index.php/Category:OWASP_Project_Assessment_-_Update>
> http://www.owasp.org/index.php/Category:OWASP_Project_Assessment_-_Updateand propose you introduce your changes directly on it. As soon as we finish
> the discussion phase, all the contributions will be moved to the original
> wiki page. With the goal of enhancing the discussion, I also propose you use
> this mailing list to inform which changes are being proposed and the reason
> or goal for doing so. We are also building a Google questionnaire to collect
> your opinions and contributions and, as soon as it is finished, it will be
> sent off.
>
>
>
> Please do have into account that you proposals can have implications in the
> assessment frame that we are currently using and, if it happens, please
> present a compatible solution.
>
>
>
> To conclude, I would like to inform you that the Project’s Committee
> propose that, as soon as we finish this discussion, we establish as a rule
> to apply to all OWASP Projects that the quality categorization must respect
> the revised assessment criteria which eventually will mean that all projects
> not assessed under these rules will be placed under Alpha Quality status.
>
> * *
>
> I thank you all in anticipation and look forward to having your
> indispensable feedback.
>
> Regards,
>
>
>
> Paulo Coimbra,
>
> OWASP Project Manager <https://www.owasp.org/index.php/Main_Page>
>
>
>
> _______________________________________________
> OWASP-Leaders mailing list
> OWASP-Leaders at lists.owasp.org
> https://lists.owasp.org/mailman/listinfo/owasp-leaders
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: https://lists.owasp.org/pipermail/owasp-leaders/attachments/20090306/58c08ff1/attachment-0001.html 


More information about the OWASP-Leaders mailing list