[Owasp-leaders] FW: REQUESTFOR DECISION/CALL FOR CONTRIBUTIONS TO UPDATE THE ASSESSMENTCRITERIA

Stephen Craig Evans stephencraig.evans at gmail.com
Mon Mar 9 02:30:20 EDT 2009


Hi Matt,

Using my suggestion, the main project page would link to and feature
the "current projects" which would be those that adhere to the current
assessment version. One link on that page could link to projects that
adhere to older assessment versions.

Before a new assessment criteria version becomes official or released,
there would be a 3 or 6 month period allowed for project owners to
"certify" their projects for the new version.

Doing it this way, the work is bottom up instead of top down, meaning
that the onus is on the project owner to do the work and not the
global committee. So, already, the work that you have done to-date
working on what is an Alpha project (as you stated in the paragraph in
your email) is unnecessary.

> As to your and Buanzo's projects (and likely others) not neatly fitting
> into the Tool or Document bucket, its obvious from the feedback we've
> gotten that this situation will need to be considered.

This issue of which bucket a project goes into is completely a
separate one than the assessment criteria issue.

I don't think at all that my project is diminished because it doesn't
easily fit into a bucket (and I am sure that Buanzo doesn't either). I
haven't read Pravir's idea but IMO from the viewpoint of OWASP, it
might be better to have just 2 categories and perhaps have to make
some hard choices occasionally about what bucket a project goes into
instead of expanding the number of categories, which seems like it
will create more work and maintenance than it's worth... From the
viewpoint of a project owner, whatever the outcome is (2 or more
categories) doesn't really matter that much.

Stephen

On Mon, Mar 9, 2009 at 11:55 AM, Matt Tesauro <mtesauro at gmail.com> wrote:
> You raise a situation I'd not considered - primarily because the
> situation you describe is not what the "downgrade to Alpha" idea was
> created to handle.  At least that's not my understanding.
>
> The discussion of downgrading to Alpha grew out of the committee trying
> to figure out how to handle non-responsive project leads and/or dead
> projects which had never been assessed.  In the case that:
> * a project lead does not respond to queries
> * the mail list has prolonged inactivity
> * the project pre-dates the assessment criteria
> * and a project appears to be otherwise be dead or no longer relevant
> Then moving them to Alpha seemed like an appropriate response.  Alpha is
> a very open space for projects and good for holding projects of unknown
> status in the near term.
>
> Considering the idea of appending a version to the ratings (props to
> Mike), I can't see a reason to _ever_ downgrade a project in your
> situation.  Obviously you'll respond to queries so I can't image your
> project being called dead.  And as you stated below, if a project lead
> sees value in going from beta criteria v1.0 to beta criteria 2.0, they
> are free to make that choice and do the work.
>
> To me, the idea of was to keep the project page from suffering from the
>  "sourceforge" effect where projects get created, possibly releases
> some early code and then go stagnant.  Going forward, I suspect that
> we'll have to figure out how to determine that a project should be
> archived and/or deprecated.  I'm not in favor of removing projects -
> just making the in-active, no longer relevant ones less prominent.  My
> example has been a PHP 3 secure coding guide.  Might be a decent basis
> for an update to PHP 5 but I pity the person coding in PHP 3.
>
> As to your and Buanzo's projects (and likely others) not neatly fitting
> into the Tool or Document bucket, its obvious from the feedback we've
> gotten that this situation will need to be considered.  Pravir has
> posted a sound idea for this and I suspect this will be discussed in
> tomorrow's committee meeting.
>
> Much like Buanzo's, your project may not easily be categorized but that
> doesn't diminish its value.  It simply means that we need to better
> figure out how to handle those that are not clearly tools or documents
> - which is fantastic because that is why we asked for feedback.
>
> -- Matt Tesauro
> OWASP Live CD Project Lead
> http://www.owasp.org/index.php/Category:OWASP_Live_CD_2008_Project
> http://mtesauro.com/livecd/ - Documentation Wiki
>
> Stephen Craig Evans wrote:
>> Hi,
>>
>>>From Paulo's original email:
>> "I would like to inform you that the Project’s Committee propose that,
>> as soon as we finish this discussion, we establish as a rule to apply
>> to all OWASP Projects that the quality categorization must respect the
>> revised assessment criteria which eventually will mean that all
>> projects not assessed under these rules will be placed under Alpha
>> Quality status."
>>
>> I do not agree with this and I pretty much agree with what Mike has
>> suggested with the following modification:
>>
>> The existing Assessment Criteria is version 1.0. The new one under
>> current discussion could be version 2.0.
>>
>> As Mike suggested, the main project page should prominently display
>> Quality Level with Assessment Criteria version number (and hotlinks to
>> the definitions of those 2).
>>
>> For example, my SoC 2008 project, Securing WebGoat using ModSecurity,
>> automatically gets the designation Tools Project with rating Beta
>> Quality, Assessment Criteria version 1.0.
>>
>> If it does not meet the new standards and I want to meet them, then
>> I'll do the necessary work and then it will be given a rating of Beta
>> Quality, Assessment Criteria version 2.0.
>>
>> Doing it this way:
>> - We avoid all of the hassle of going back and re-assessing projects
>> previous to version 2.0
>> - We don't lessen the value of version 1.0 projects - an insult the
>> project owneres - by degrading them to Alpha Quality
>> - We give project owners the option and opportunity to upgrade their
>> version 1.0 projects to version 2.0
>> - We have a flexible process that allows for future Assessment Criteria versions
>>
>>
>> @Buanzo:
>>
>> Hey Roomie!
>>
>> To address your and Matt's comments:
>> "
>>> With all this, I basicly want to say that mine (and other projects, too) are not necessarily 'tools'
>>> per se, and maybe there is need to assess this situation more throughly. I call for help in this
>>> matter, or for someone to hit me in the face with a 'hey, you got it wrong, Buanzo! this is the
>>> deal' explanation :P
>> This time no smacking is needed.  I would agree that your project
>> isn't really a tool nor is it a document.  Its a specification with a
>> reference implementation.  How that fits the assessment criteria, I
>> honestly don't know but it is something we will have to account for
>> going forward.
>> "
>>
>> Yes, your project is in a gray area betwen Tools and Documentation. I
>> consider my project mainly as a research project since it evolved into
>> using Lua scripts in ModSecurity, which really hasn't been done
>> before. I provide ModSecurity rulesets and complete fully functional
>> Lua scripts, plus I rebuilt the Lua standalone executable and shared
>> object libraries from source to include extra functionality. It's not
>> a Tools project but not a Documentation project, but it's fairly clear
>> that it is not a Tools project so it falls under Documentation.
>>
>> If your project is a reference implementation, then IMHO it falls more
>> under Documentation category. Yes, "Documentation", meaning non-Tools,
>> might not be the best name and that could be changed; however, I can't
>> think of a name better than "Non-Tools" off the top of my head.
>>
>> So, under my proposed modification at the beginning of this email,
>> your project could either be Beta Tools Quality, Assessment Criteria
>> version 1.0 -or- Beta Documentation Quality, Assessment Criteria
>> version 2.0.
>>
>> Your Season of Quality suggestions still hold; in my case, I could do
>> the same and offer a VMWare image with Ubuntu, WebGoat, ModSecurity,
>> and include my re-built Lua version (with MD5 and SQL database access
>> as extra features), my Lua scripts, and ModSec rulesets.
>>
>> Again, your project is in a gray area. I don't think there should be
>> an extra category besides Tools and Documentation (even though
>> "Documentation" could be renamed) - it will just introduce more room
>> for confusion. But, you can always choose what category you want for
>> your project based on the criteria.
>>
>> Cheers,
>> Stephen
>>
>>
>> On Sun, Mar 8, 2009 at 12:25 PM, Matt Tesauro <mtesauro at gmail.com> wrote:
>>> Mike Boberski wrote:
>>>> The only way to eat an elephant is one bite at a time.
>>>>
>>>> Let us start with the first bite.
>>>>
>>>> Re: "Please note that not all the projects below have been evaluated
>>>> under this criteria and might be re-classified once that process is
>>>> completed" which is now on each tab of the projects page. We need to
>>>> discuss this!
>>>>
>>>> Overturning verdicts/assessments does not provide consumers in any
>>>> context with confidence that the rating organization has its stuff
>>>> together or that its ratings mean anything. It is also completely
>>>> demoralizing to contributors.
>>>>
>>>> I wrote earlier:
>>>>
>>>>
>>>>  >     should simply put dates against ratings, and identify the criteria
>>>>  >     version that a project was assessed against, then leave that rating
>>>>  >     alone as the criteria continues to evolve over time. That is what
>>>>  >     more well-established and formal testing programs for instance like
>>>>  >     Common Criteria and FIPS 140 do.
>>>>
>>>> I propose the following actions:
>>>>
>>>> 1. Delete "Please note that not all the projects below have been
>>>> evaluated under this criteria and might be re-classified once that
>>>> process is completed" from the tabs on the projects page.
>>> I agree with this.  I'd rather have to clear up and answer specific
>>> questions then globally spread doubt.  Hell, its a wiki so why not make
>>> the change, its all versioned anyway..
>>>
>>>> 2. Append to each project's short description on the projects page
>>>> "(Assessment Criteria <version>)". Going with version only instead of
>>>> version and date will simplify matters.
>>> I also like this as it both shows per project where they are and also
>>> (by its lack) would show projects that pre-date the criteria.  This is
>>> useful information to both internal and external audiences.
>>>
>>>> 3. (Optional step) On each project's project page, wherever it currently
>>>> identifies its release status at the top of the page, append
>>>> "(Assessment Criteria <version>)"
>>> I also think this is a stellar idea.
>>>
>>>> I am assuming you're maintaining version control for project assessment
>>>> criteria. If not the time is now. Mark the current criteria v1.0, make
>>>> sure it's saved off and made accessible on the site somewhere, and
>>>> assign versions when new versions are released as time goes on.
>>> I haven't heard an explicit version number used - most frequently it is
>>> spoken about with the SoC as its context.
>>>
>>> Since the one used for 2008 SoC is the first assessment criteria, v1.0
>>> makes sense to me.
>>>
>>>> If there has been no criteria versioning to date, or if you can tell me
>>>> the current version, I will go an update all listings on the projects
>>>> page, then you guys/whomever if you want to perform optional step (3)
>>>> above, can email project leads and ask them to update their project page
>>>> to match.
>>> Again, v1.0 makes the most sense for the criteria version number.  Paulo
>>> will hopefully chime in if I'm incorrect in my thinking.
>>>
>>>> If there's agreement, let me know what criteria version to put, and I'll
>>>> take care of updating the projects page listings. Then we can move onto
>>>> the next bite.
>>> If you don't get an explicit request to NOT do this by the time you wake
>>> up on Tuesday, March 10th (whatever the timezone), then consider
>>> agreement to be reached.  I've CC'ed the Global Project Committee in
>>> case they disagree with this.
>>>
>>> -- Matt Tesauro
>>> OWASP Live CD Project Lead
>>> http://www.owasp.org/index.php/Category:OWASP_Live_CD_2008_Project
>>> http://mtesauro.com/livecd/ - Documentation Wiki
>>>
>>>> Mike
>>>>
>>>>
>>>> ------------------------------------------------------------------------
>>>>
>>>> _______________________________________________
>>>> OWASP-Leaders mailing list
>>>> OWASP-Leaders at lists.owasp.org
>>>> https://lists.owasp.org/mailman/listinfo/owasp-leaders
>>> _______________________________________________
>>> OWASP-Leaders mailing list
>>> OWASP-Leaders at lists.owasp.org
>>> https://lists.owasp.org/mailman/listinfo/owasp-leaders
>>>
>>
>>
>>
> _______________________________________________
> OWASP-Leaders mailing list
> OWASP-Leaders at lists.owasp.org
> https://lists.owasp.org/mailman/listinfo/owasp-leaders
>



-- 
http://www.linkedin.com/in/stephencraigevans


More information about the OWASP-Leaders mailing list