[Owasp-leaders] [Global_tools_and_project_committee] [Owasp-board] FW: REQUESTFOR DECISION/CALL FOR CONTRIBUTIONS TO UPDATE THE ASSESSMENT CRITERIA

Paulo Coimbra paulo.coimbra at owasp.org
Mon Mar 9 13:20:50 EDT 2009


Leaders,

 

As Matt has said, the discussion of downgrading to Alpha grew out of the
committee trying to figure out how to handle non-responsive project leads
and/or dead projects which had never been assessed.  

 

In fact we’ve never thought about downgrading projects already evaluated. We
were simply thinking in proposing a division in two categories –projects
with and without assessment. 

 

For what it's worth, I agree with Stephan when he says “Before a new
assessment criteria version becomes official or released, there would be a 3
or 6 month period allowed for project owners to "certify" their projects for
the new version.”

 

In addition, I believe the suggestion made by Stephan/Mike is also worth
following – assessment references included in the OWASP Projects page should
prominently inform about the followed assessment criteria version - i.e.
“Assessment Criteria version 1.0” if the projects were assessed under the
assessment criteria currently in use or “Assessment Criteria version 2.0”
for those ones that are yet to be assessed accordingly with the, in time to
come, updated assessment criteria version. 

 

If you allow me, I take the opportunity to challenge you all again to find
the spare cycles to think about the assessment criteria itself -
<http://www.owasp.org/index.php/Category:OWASP_Project_Assessment_-_Update>
http://www.owasp.org/index.php/Category:OWASP_Project_Assessment_-_Update  -
until now just one change as been done. 

 

Thanks,

 

Paulo Coimbra,

 <https://www.owasp.org/index.php/Main_Page> OWASP Project Manager

 

From: Matt Tesauro [mailto:mtesauro at gmail.com] 
Sent: segunda-feira, 9 de Março de 2009 14:10
To: Boberski, Michael [USA]
Cc: Dave Wichers; paulo.coimbra at owasp.org;
global_tools_and_project_committee at lists.owasp.org
Subject: Re: [Global_tools_and_project_committee] [Owasp-board] FW:
REQUESTFOR DECISION/CALL FOR CONTRIBUTIONS TO UPDATE THE ASSESSMENT CRITERIA

 

 

Boberski, Michael [USA] wrote:

> One last thought re "As for downgrading projects, I think it is a 

> necessary"

> 

> Would you consider for example legal project now alpha status, since 

> the only update in years has been the one I did for ASVS?

Definitely not.  Before any downgrade occurs, I'd look at several factors
beyond just pre-dating the assessment criteria:

* non-responsive project lead

* long period of inactivity on the mail list

* reduced relevance due to changing technology

* out of date information

 

Considering the fact that I recommended the Legal Annex where I work on
Friday, I'd definitely not consider downgrading it.

 

My hope is that the vast majority of projects won't be in this situation.
All of last years SoC won't be.  From my perspective, the downgrading was
only to address the situation of older, unmaintained projects of reduced
relevance.  Hopefully these will be the exception rather then the rule - a
few statistical outliers.

 

Actually in preparing for the DHS event on Friday, I looked up the previous
OWASP Live CD (2007) again.  OWASP.org has a page for it but when you follow
the links to download it, the external linked site no longer hosts any ISOs.
This is a project that should be archived somehow but not prominently
displayed as it would appear to only have historical value.

 

http://www.owasp.org/index.php/Category:OWASP_Live_CD_2007_Project

 

> 

> The answer I would argue is no. The quality of a tool or document 

> doesn't degrade over time.

Absolutely true in this case.  If we had a hardening guide for Tomcat 3.x,
it would be less relevant now then when created.  I'd consider moving
something like that from Release to Alpha in hopes that a new project lead
would grab it and get it up to date.  That's why I listed the factors above
- downgrading is much more than assessed or not.

 

Thanks for all the great back and forth on this - the process will be much
better off for this dialog.

 

-- Matt Tesauro

OWASP Live CD Project Lead

http://www.owasp.org/index.php/Category:OWASP_Live_CD_2008_Project

http://mtesauro.com/livecd/ - Documentation Wiki

 

> 

> I've gone and made some updates over the weekend to the projects page 

> and to ESAPI and ASVS pages, let's start perhaps from what is proposed 

> there, with the criteria versioning and such.

> 

> Best,

> 

> Mike B.

>  

> 

> -----Original Message-----

> From: Matt Tesauro [mailto:mtesauro at gmail.com]

> Sent: Saturday, March 07, 2009 1:53 PM

> To: Boberski, Michael [USA]

> Cc: Dave Wichers; paulo.coimbra at owasp.org; 

> global_tools_and_project_committee at lists.owasp.org

> Subject: Re: [Global_tools_and_project_committee] [Owasp-board] FW:

> REQUESTFOR DECISION/CALL FOR CONTRIBUTIONS TO UPDATE THE 

> ASSESSMENTCRITERIA

> 

> Responses inline below:

> 

> Boberski, Michael [USA] wrote:

>> Team, OWASP is getting overly bureaucratic, it seems to me.

> Trust me I hate bureaucracy.  This is about providing a process to 

> gauge quality.  The proposal represents a large, one-time effort plus 

> small efforts going forward.  If OWASP wants quality, there is work to 

> be done and a time frame to do it in.  We're trying to get the bulk 

> done upfront and provide a means to never again accumulate a backlog of
work.

> 

>>  

>> I'd rather see people putting time/energy into tightening up their 

>> project pages, tools, and project presentations/datasheets. An 

>> example

> 

>> are PHP and .NET ESAPI, there's no published mapping of Java ESAPI to 

>> PHP/ESAPI, that also should then identify which interfaces are being 

>> targeted for which releases. I'm going to try to work with Andrew to

> fix

>> that problem for PHP since I may have a need for a PHP ESAPI for a 

>> customer engagement, but it's still a good example.

> Under the assessment criteria, there are incentives to do this work.  

> If

> 

>   a project is Beta and wants to be Release, these are the types of 

> things that need to happen.  Release project = prominence, 

> particularly on the project page.  If you want these things done, how 

> better then provide a clear incentive for the project leads?

> 

>>  

>> The more complete and professional a page/doc/tool looks, the easier

> it

>> is to identify the status and content of a doc/tool, the easier is to 

>> figure out its usefulness and to promote its adoption. That a 

>> doc/tool

> 

>> has correct content or works is taken as a given, that is completely 

>> secondary to the initial figuring out if a doc/tool is a potential 

>> solution to one's problem of the day.

> I'd suggest that the process we're proposing will get you there.  One 

> aspect of looking professional is consistency.  The criteria and 

> project

> 

> page requirements are aimed an getting that consistent representation 

> while allowing a project to further explain its relevance to its 

> target audience.  That's why we're shooting for a standard frame + 

> whatever else the project wants/needs.

> 

> As for "correct content or works is taken as a given", I'd ask for 

> what project and what categorization?  Can this be said for an Alpha
project?

> 

>   Beta or Release?  How about the many non-assessed projects?  Once we 

> have all projects assessed, statements like that can be made with 

> certainty.

> 

> I know I actually liked the fact that I could read the assessment 

> criteria and know explicitly where my project would fall based on how 

> I fulfilled the listed criteria.  I have N things to do in Y time and 

> I can plan according to make the SoC deadline with the quality rating 

> I'm shooting for.  No guessing.

> 

>>  

>> I would also caution against downgrading projects, which is what one

> of

>> the comments seems to imply could happen. If you must address some 

>> perceived contention over project assessment criteria, you should

> simply

>> put dates against ratings, and identify the criteria version that a 

>> project was assessed against, then leave that rating alone as the 

>> criteria continues to evolve over time. That is what more 

>> well-established and formal testing programs for instance like Common 

>> Criteria and FIPS 140 do. I hope I am misreading comments on this

> point

>> however.

> I really like the idea of dating the assessment ratings.  Please join 

> the discussion and add those to the page Paulo has created.  That's 

> fantastic stuff and why we've posed this to the collective 

> intelligence of OWASP.

> http://www.owasp.org/index.php/Category:OWASP_Project_Assessment_-_Upd

> at

> e

> 

> As for downgrading projects, I think it is a necessary thing to 

> determine projects that are dormant, abandoned or otherwise moribund.  

> I

> 

> personally would like to start an "archive" "deprecated" or similar 

> location for projects that aren't actively developed and/or no longer 

> timely (think PHP 3 coding guidelines) but may still have value.  

> Also, those abandoned but potentially useful projects could become SoC 

> projects so their quality will improve.

> 

> Finally, by engaging the project leads, OWASP could also find projects 

> were the lead no longer wants or can be the maintainer.  Identifying 

> those projects would allow someone else to take up that project and 

> move

> 

> it forward.

> 

> -- Matt Tesauro

> OWASP Live CD Project Lead

> http://www.owasp.org/index.php/Category:OWASP_Live_CD_2008_Project

> http://mtesauro.com/livecd/ - Documentation Wiki

> 

>>  

>> Best,

>>  

>> Mike B.

>>  

>>  

>> 

>> 

> ----------------------------------------------------------------------

> --

>> *From:* global_tools_and_project_committee-bounces at lists.owasp.org

>> [mailto:global_tools_and_project_committee-bounces at lists.owasp.org]

> *On

>> Behalf Of *Dave Wichers

>> *Sent:* Wednesday, March 04, 2009 5:34 PM

>> *To:* paulo.coimbra at owasp.org; 'OWASP Foundation Board List'; 

>> global_tools_and_project_committee at lists.owasp.org

>> *Subject:* Re: [Global_tools_and_project_committee] [Owasp-board] FW: 

>> REQUESTFOR DECISION/CALL FOR CONTRIBUTIONS TO UPDATE THE

> ASSESSMENTCRITERIA

>> I'm OK with this, although I'm not a big fan of many of the

> suggestions

>> below. But that's OK. Lets get the ideas out there and we can then

> make

>> some decisions.

>> 

>>  

>> 

>> -Dave

>> 

>>  

>> 

>> *From:* owasp-board-bounces at lists.owasp.org

>> [mailto:owasp-board-bounces at lists.owasp.org] *On Behalf Of *Paulo

> Coimbra

>> *Sent:* Wednesday, March 04, 2009 11:25 AM

>> *To:* 'OWASP Foundation Board List'; 

>> global_tools_and_project_committee at lists.owasp.org

>> *Subject:* [Owasp-board] FW: REQUEST FOR DECISION/CALL FOR

> CONTRIBUTIONS

>> TO UPDATE THE ASSESSMENT CRITERIA

>> *Importance:* High

>> 

>>  

>> 

>> Board, Project's Committee,

>> 

>>  

>> 

>> In consequence of the comments received in the last Committee 

>> meeting,

> 

>> I've introduced the changes yellow underlined.  Please let me know if 

>> this email can be sent off.*/ /*

>> 

>>  

>> 

>> Many thanks, regards,

>> 

>>  

>> 

>> Paulo

>> 

>>  

>> 

>>  

>> 

>> Hello Leaders,

>> 

>>  

>> 

>> I hope you are well.

>> 

>>  

>> 

>> You better than anyone else know that OWASP as an organization has

> been

>> built by your continuous open contributions both by defining its 

>> mission, organizational structure, rules and procedures and by 

>> leading

> 

>> the application security projects that are its core of activity.

>> 

>>  

>> 

>> In my today's call for contributions, procedures regarding projects 

>> development's stage assessment are the main issue.

>> 

>>  

>> 

>> As you may know, a system to evaluate OWASP projects is already in 

>> use

> 

>> and actually consists in both a set of criteria 

>> http://www.owasp.org/index.php/Category:OWASP_Project_Assessment and 

>> a

> 

>> skeleton/frame to implement it

>> 

> http://www.owasp.org/index.php/OWASP_Live_CD_2008_Project_-_Assessment

> _F

> rame

>> .

>> 

>>  

>> 

>> With other few subsequent modifications, this set of criteria has

> mainly

>> resulted of a vigorous discussion held through this mailing list

> almost

>> a year ago and since then it has been used in all newly set up

> projects.

>>  

>> 

>> Since then this issue has been discussed consecutively in several 

>> different contexts. In our Summit, for example, even if we haven't 

>> committed a specific slot of time to deal with this matter, it has 

>> collaterally arisen throughout many project's presentations. In 

>> addition, I regularly receive from OWASP Board requests to make 

>> modifications, a systemic reflection is being held within the

> Project's

>> Committee and, as result of my daily handling of projects under

> review,

>> I am obtaining some feedback from project leaders and reviewers.

>> 

>>  

>> 

>> Overall, the people with whom I've discussed this issue usually say

> that

>> the procedure can be improved and IMHO, even if I think*/ /*the 

>> Assessment Criteria is working and actually has been of great help,

> they

>> are right.

>> 

>>  

>> 

>>  From these discussions, I've retained that a handful of criteria 

>> have

> 

>> been proposed but haven't been implemented yet as forthcoming:

>> 

>> -          OWASP writing style (Tool projects/Release Quality),

>> 

>> -          Translation (Tools and Documentation/Release Quality),

>> 

>> -          Bi-monthly periodic news (Tools and Documentation/non 

>> specified Quality status),

>> 

>> -          5 slide deck for OWASP Boot Camp project (Tools and 

>> Documentation/Beta status),

>> 

>> -          Attribution rules (Tools and Documentation/non specified 

>> Quality status),

>> 

>> -          Compulsory Project Skeleton/Frame (Tools and 

>> Documentation/all Quality status),

>> 

>> -           Reviewer role - addition and clarification,

>> 

>> http://owaspsoc2008.wordpress.com/2008/07/15/assessment-guidance/

>> 

>> -          Mentor role addition and definition.

>> 

>> In addition, as far as I am concerned, a few more structural comments 

>> have also been made. Even without pointing out alternative technical 

>> solutions, at least a  couple of them have questioned the rationale 

>> of

> 

>> working with tables in wiki text and others have pointed out the 

>> willingness of having a project's page similar to, for example, this

> one

>> http://www.hdiv.org/.

>> 

>>  

>> 

>> Having said all the above with the intention of giving you a picture

> of

>> the current situation, I ask for your contribution so as to update 

>> the

> 

>> OWASP Assessment Criteria.

>> 

>>  

>> 

>> In operational terms, I've replicated the Assessment Criteria page

>> 

> http://www.owasp.org/index.php/Category:OWASP_Project_Assessment_-_Upd

> at

> e

>> and propose you introduce your changes directly on it. As soon as we 

>> finish the discussion phase, all the contributions will be moved to

> the

>> original wiki page. With the goal of enhancing the discussion, I also 

>> propose you use this mailing list to inform which changes are being 

>> proposed and the reason or goal for doing so. We are also building a 

>> Google questionnaire to collect your opinions and contributions and,

> as

>> soon as it is finished, it will be sent off.

>> 

>>  

>> 

>> Please do have into account that you proposals can have implications

> in

>> the assessment frame that we are currently using and, if it happens, 

>> please present a compatible solution.

>> 

>>  

>> 

>> To conclude, I would like to inform you that the Project's Committee 

>> propose that, as soon as we finish this discussion, we establish as a 

>> rule to apply to all OWASP Projects that the quality categorization

> must

>> respect the revised assessment criteria which eventually will mean

> that

>> all projects not assessed under these rules will be placed under 

>> Alpha

> 

>> Quality status.

>> 

>> */ /*

>> 

>> I thank you all in anticipation and look forward to having your 

>> indispensable feedback.

>> 

>> Regards,

>> 

>>  

>> 

>> Paulo Coimbra,

>> 

>> OWASP Project Manager <https://www.owasp.org/index.php/Main_Page>

>> 

>>  

>> 

>> 

>> 

> ----------------------------------------------------------------------

> --

>> _______________________________________________

>> Global_tools_and_project_committee mailing list 

>> Global_tools_and_project_committee at lists.owasp.org

>> 

> https://lists.owasp.org/mailman/listinfo/global_tools_and_project_comm

> it

> tee

-------------- next part --------------
An HTML attachment was scrubbed...
URL: https://lists.owasp.org/pipermail/owasp-leaders/attachments/20090309/7b3c171b/attachment-0001.html 


More information about the OWASP-Leaders mailing list