[Owasp-cert] Is this an evil thought?
chw at hauser-wenz.de
Mon Jul 28 01:22:35 EDT 2008
Generally, I agree with a closed process. However I am still convinced that we need a „core group”. What we did with the other certification is that there was one person responsible for each topic area of the exam. This was both for competency reasons (of course everybody is very knowledgeable in OWASP’s main topics, but of course everybody has their strength points), and to ensure that there were enough questions per topic areas – for some topics it is very hard to generate enough questions for, I predict some will find out during writing questions J
So far, it would be sufficient if there was one person (or one group) per topic area. However I still think that everyone responsible for a topic needs to see all or most of the questions. 1) The wording of the questions and the type of questions would be more or less consistent. 2) most importantly: there are no repeated questions. Some of the topic areas do overlap (think “The OWASP Top Ten”) with others, so chances are that the same question is added twice, maybe with a different answer set.
I am very much in favor of OWASP’s open approach, but in order to have a reasonable certification process, we need to keep the questions secret. After a few months they’ll appear as braindumps anyway, so we should try to update the exam every year or so anyway. Even the LPIC exam is secret.
One thing my experiences in writing exams show is that the core group should be selected specifically for this task. There are more OWASP chapter leaders than we’d need, and one thing that happened in another quite well-known exam was that the reviewers that were on board due to their job title sometimes did not have sufficient time to review, so they just waved questions through, including those that were objectionable.
Alternatively, as suggested, we could refrain from using multiple choice. However in my opinion, this would prevent the exam from being successful, since it would be hard to actually take the exam. You need to be able to take the exam electronically (only then it scales) and get immediate results. I think the LPIC exams can be taken on paper (and you get your results one or two months later), but they are also using multiple choice. Of course multiple choice is not always sufficient or fair, but all other types of exams, including “live exams” in front of some judges, is unfair as well. Therefore MC is the best compromise, in my opinion.
Synopsis: Core group for questions (consisting of people on this list, obviously), closed process for questions, multiple choice questions. This may not be the most popular opinion, but it’s the one that’ll work :-)
In order to protect the integrity of the exam, should I be the only person that knows all of the questions/answers? Likewise, security is ensured if the actual questions/answers aren't sent to this listserv as there may be lurkers? What do you think of question creators sending directly to reviewers with only a CC to me?
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the Owasp-cert