<html><head><meta http-equiv="content-type" content="text/html; charset=utf-8"></head><body dir="auto"><div>> <span style="font-family:arial,sans-serif">Well , how do you make sure you have an "objective" technical review?</span></div>
<div><br></div><div>By measuring objective things like time to fix bugs, number of unfixed bugs, status of documentation, time since last substantial update, number of updates, etc for starters. These are things we can review with minimal tech expertise.</div>
<div><br></div><div>Admittedly, it's takes a lot of technical expertise to evaluate other quality characteristics of OWASP projects. One if my beefs with ESAPI is that they use the same instance of SecureRandom for all random number generation, which is deterministic. This is really bad! Security libraries need good random number generation or the entire crypto API is undermined! Admittedly, only a Java and Security expert would even recognize these problems. That's not an easy problem to deal with, but maybe we can file those under "number of bugs, etc". We also have live CVE's of broken crypto in ESAPI. These are serious unfixed problems....</div>
<div><br></div><div>••• My goal here is to be honest about OWASP project quality •••</div><div><br></div><div>If we can't measure project quality fairly, I'd much rather just say that instead of making claims of flagship (and production quality). </div>
<div><br></div><div>Several large companies have done reviews of our flagships and deemed them far from production quality when we claimed they were. This ties into fundraising. Fundraising is one of my roles at board members. </div>
<div><br></div><div>So I will support any honest direction you want to go in Johanna. Does my position here make more sense?</div><div><br></div><div>With respect,</div><div><div>--</div><div>Jim Manico</div><div>@Manicode</div>
<div>(808) 652-3805</div></div><div><br>On May 4, 2014, at 4:54 PM, johanna curiel curiel <<a href="mailto:johanna.curiel@owasp.org">johanna.curiel@owasp.org</a>> wrote:<br><br></div><blockquote type="cite"><div><div dir="ltr">
<span style="font-family:arial,sans-serif;font-size:13px">I think objective technical review is much more important than a "community vote" where the system can easily be gamed.</span><br><div><span style="font-family:arial,sans-serif;font-size:13px"><br>

</span></div><div><font face="arial, sans-serif">Well , how do you make sure you have an "objective" technical review?</font></div><div><font face="arial, sans-serif"><br></font></div></div><div class="gmail_extra">

<br><br><div class="gmail_quote">On Thu, Apr 17, 2014 at 12:00 PM, Jim Manico <span dir="ltr"><<a href="mailto:jim.manico@owasp.org" target="_blank">jim.manico@owasp.org</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">


  
    
  
  <div bgcolor="#FFFFFF" text="#000000">
    I see the community vote for project review to be a "popularity
    vote" and not a "quality review". I would recommend having three
    main technical reviewers: One for assessment projects, one for code
    projects and another for documentation. These three need very
    different criteria for quality and flagship type classification. <br>
    <br>
    I think objective technical review is much more important than a
    "community vote" where the system can easily be gamed.<div class=""><br>
    <br>
    > Testing and fixing is the only way to verify how stable a
    release is.<br>
    <br></div>
    An easier way is to review google code or GIT and check out the
    history of reported and fixed bugs for tool and code projects. <br>
    <br>
    For example take ESAPI for Java: They claim it's production but its
    full of existing critical bugs:
<a href="https://code.google.com/p/owasp-esapi-java/issues/list?can=2&q=&sort=priority&colspec=ID%20Type%20Status%20Priority%20Milestone%20Component%20Owner%20Summary" target="_blank">https://code.google.com/p/owasp-esapi-java/issues/list?can=2&q=&sort=priority&colspec=ID%20Type%20Status%20Priority%20Milestone%20Component%20Owner%20Summary</a><br>


    <br>
    Take OWASP Java Encoder where historically we have 7 bugs reported..
<a href="https://code.google.com/p/owasp-java-encoder/issues/list?can=1&q=&colspec=ID+Type+Status+Priority+Milestone+Owner+Summary&cells=tiles" target="_blank">https://code.google.com/p/owasp-java-encoder/issues/list?can=1&q=&colspec=ID+Type+Status+Priority+Milestone+Owner+Summary&cells=tiles</a><br>


    <br>
    But worked hard to fix them all:
    <a href="https://code.google.com/p/owasp-java-encoder/issues/list" target="_blank">https://code.google.com/p/owasp-java-encoder/issues/list</a><br>
    <br>
    These are much more valuable data points than community vote, in my
    opinion.<br>
    <br>
    But I do think that USERS of the project voting is valuable.<br>
    <br>
    My 1 cent,<br>
    Jim<div><div class="h5"><br>
    <br>
    <div>On 4/16/14, 7:34 PM, johanna curiel
      curiel wrote:<br>
    </div>
    <blockquote type="cite">
      <div dir="ltr">Well, that's why is so important to have a review
        system and community providing input
        <div>How can we know a project release is what they claim?<br>
          <div><br>
          </div>
          <div>Just as Microsoft can claims "beta" or final release
            still,  many patches have to been release afterwards to fix
            issues.<br>
            <div><br>
            </div>
            <div>If a project claims is "beta" or final  release but is
              full with bugs it will find out by itself.</div>
          </div>
          <div>Testing and fixing is the only way to verify how stable a
            release is.</div>
          <div>
            <br>
          </div>
          <div>A new project with a first time release most of the time
            will be between alpha and beta.</div>
          <div>After a clear use of the project and feedback from the
            community, and multiple fixes, it can move to a stable
            release</div>
          <div><br>
          </div>
          <div>regards</div>
          <div><br>
          </div>
          <div>Johanna</div>
        </div>
      </div>
      <div class="gmail_extra"><br>
        <br>
        <div class="gmail_quote">On Wed, Apr 16, 2014 at 9:15 PM, Jim
          Manico <span dir="ltr"><<a href="mailto:jim.manico@owasp.org" target="_blank">jim.manico@owasp.org</a>></span>
          wrote:<br>
          <blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
            <div bgcolor="#FFFFFF" text="#000000"> What happens when
              products classify themselves as release quality when they
              are really alpha quality?<span><font color="#888888"><br>
                  - Jim</font></span>
              <div>
                <div><br>
                  <br>
                  <br>
                  <div>On 4/14/14, 6:48 PM, Samantha Groves wrote:<br>
                  </div>
                  <blockquote type="cite">
                    <div dir="ltr">Yes, please just to clarify, we
                      already have this. Project products must
                      self-classify as release, beta or alpha quality. </div>
                    <div class="gmail_extra"><br>
                      <br>
                      <div class="gmail_quote">On Mon, Apr 14, 2014 at
                        3:50 PM, Dinis Cruz <span dir="ltr"><<a href="mailto:dinis.cruz@owasp.org" target="_blank">dinis.cruz@owasp.org</a>></span>
                        wrote:<br>
                        <blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
                          <div dir="ltr">Hi Jim, when you in <a href="http://lists.owasp.org/pipermail/owasp-board/2014-April/013567.html" target="_blank">http://lists.owasp.org/pipermail/owasp-board/2014-April/013567.html</a>
                            say:
                            <div> <br>
                            </div>
                            <blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
                              <i>Something like: </i>
                              <ul>
                                <li><i>OWASP Project - Release Quality </i></li>
                              </ul>
                              <ul>
                                <li><i>OWASP Product - BETA Quality </i></li>
                              </ul>
                              <ul>
                                <li><i>OWASP Product - ALPHA Quality </i></li>
                              </ul>
                              <ul>
                                <li><i>Incubator Project</i></li>
                              </ul>
                            </blockquote>
                            <div>
                              <div><br>
                              </div>
                              <div>You mean something like: <a href="https://www.owasp.org/index.php/Assessing_Project_Releases" target="_blank">https://www.owasp.org/index.php/Assessing_Project_Releases</a></div>
                            </div>
                            <div><br>
                            </div>
                            <div>And when talking about OWASP Project
                              Health, what about: <a href="https://www.owasp.org/index.php/Assessing_Project_Health" target="_blank">https://www.owasp.org/index.php/Assessing_Project_Health</a></div>
                            <div><br>
                            </div>
                            <div>Note that those are pages created by
                              Paulo in 2009, Johanna and Samantha used
                              them has bases of their more recent
                              project assessment work</div>
                            <span><font color="#888888">
                                <div><br>
                                </div>
                                <div>Dinis</div>
                              </font></span></div>
                        </blockquote>
                      </div>
                      <br>
                      <br clear="all">
                      <div><br>
                      </div>
                      -- <br>
                      <div dir="ltr">
                        <p style="font-size:13px;font-family:arial,sans-serif;margin:0px"><font color="#ff6600"><b>Samantha Groves, MBA</b></font></p>
                        <p style="font-size:13px;font-family:arial,sans-serif;margin:0px"><i><font color="#ff6600">OWASP Projects Manager</font></i></p>
                        <p style="font-size:13px;font-family:arial,sans-serif;margin:0px"><i><font color="#ff6600"><br>
                            </font></i></p>
                        <p style="font-size:13px;font-family:arial,sans-serif;margin:0px"><font color="#666666">The OWASP Foundation</font></p>
                        <p style="font-size:13px;font-family:arial,sans-serif;margin:0px"><font color="#666666">Phoenix, USA</font></p>
                        <p style="font-size:13px;font-family:arial,sans-serif;margin:0px"><span style="color:rgb(102,102,102)">Email: <a href="mailto:samantha.groves@owasp.org" target="_blank">samantha.groves@owasp.org</a></span></p>


                        <p style="font-size:13px;font-family:arial,sans-serif;margin:0px"><font color="#666666">Skype: samanthahz </font></p>
                        <p style="font-size:13px;font-family:arial,sans-serif;margin:0px"><font color="#666666"><br>
                          </font></p>
                        <p style="font-size:13px;font-family:arial,sans-serif;margin:0px"><a href="https://www.owasp.org/index.php/Category:OWASP_Project" style="background-color:transparent" target="_blank">OWASP Global Projects</a></p>


                        <p style="font-size:13px;font-family:arial,sans-serif;margin:0px"><a href="http://goo.gl/mZXdZ" target="_blank">Book
                            a Meeting with Me</a></p>
                        <p style="font-size:13px;font-family:arial,sans-serif;margin:0px"><font color="#666666"><a href="http://owasp4.owasp.org/contactus.html" target="_blank">OWASP Contact US Form</a></font></p>
                        <p style="font-size:13px;font-family:arial,sans-serif;margin:0px"><a href="http://www.tfaforms.com/263506" target="_blank">New Project Application Form</a></p>
                        <p style="font-size:13px;font-family:arial,sans-serif;margin:0px"><br>
                        </p>
                        <p style="color:rgb(80,0,80);font-size:13px;font-family:arial,sans-serif;margin:0px"><br>
                        </p>
                      </div>
                    </div>
                  </blockquote>
                  <br>
                </div>
              </div>
            </div>
          </blockquote>
        </div>
        <br>
      </div>
    </blockquote>
    <br>
  </div></div></div>

</blockquote></div><br></div>
</div></blockquote></body></html>