[Owasp-cert] Deadlines for August

Matthew Chalmers matthew.chalmers at owasp.org
Wed Jul 30 14:50:04 EDT 2008


 Well put, Gary.

For the CISA, a proctored, pencil & paper exam, I had to speak with a
proctor at the end who advised me to note my exam number, the numbers of the
questions on which I had comments, and my argument. I was not allowed to
copy the questions or their choices. I was told to take this info home with
me and write to ISACA (snail mail). I did end up taking home a lot of notes
on questions I thought had issues but I never did write to ISACA; I passed
the exam anyway.

SANS has gone through 2-3 major iterations of exam styles over the years but
how Gary explained it is basically how I recall it. There are pros and cons.
Being able to submit comments electronically right after answering the
question was good--but there were issues with being able to copy exam
content (which hurts them) and not being able to go back through the exam
(which hurts us as exam-takers). I, too, got very similar feedback from my
comments. It was nice to know that someone actually considered them even if
I didn't agree with the response.

For the CEH, it was a hybrid of the two: it was a test taken on computer,
but in a room with other test-takers and a proctor. (The SANS test, back
then, was taken over the Internet from wherever I wanted to do it, whenever
I wanted to do it.) I do not, however, recall there being an explicit,
documented challenge process. And there were some questions on that exam
that I really had issues with. Of course as with the others, having passed I
promptly stopped caring.

I completely agree with point #2 below. I do hope we can devise some method
of testing candidates that doesn't translate well into an Exam-cram or
Dummies style study guide. We should make every reasonable effort to keep
exam content (whether they be questions, possible answers, scenarios, etc.)
secret until a candidate comes to write the exam--and at that time, every
reasonable effort should have been taken to make it as unlikely as possible
for someone to memorise exam content and regurgitate it later, because every
time that happens the pieces are assembled until practically the entire pool
of possible content is known and made public.

On point #3 I too am for the idea of some kind of long essay, thesis,
research/white paper, etc. at an expert level of certification which is
reviewed by a 'board'. There should be some more objective points that must
be included in the work, but the way in which it's included and the
intellectual content of the work will have to be at least somewhat
subjectively scored. If by an odd-numbered panel, the majority can rule on
pass/fail. Perhaps failing work should be published just like passing work
so that potential candidates can see what kind of work will not pass. The
one problem I saw with the SANS paper requirement was that it had/has a
multi-month timeframe--it would be difficult to prove beyond a doubt that
the exam candidate did the work alone, without others' help. Subtle
plagiarism may also be difficult to ascertain.

Matt

  On Tue, Jul 29, 2008 at 9:33 PM, Gary Palmer <owasp at getmymail.org> wrote:

>  #1) For the CISSP I had to complete a paper form that was provided as
> part of the test.  I had to complete that form within the time allotted and
> turn it into the reviewer.  It was reviewed independently and outside the
> test room and I was not informed of the results.  For the SANS online
> testing, it was structured to present one question at a time with the answer
> options.  Each question had a checkbox you could use to tag the question.
> There was also a checkbox to gat the prior question in case you clicked
> submit and then realized you wanted to challenge.  At the test completion,
> after the allotted time, you were given a chance to review the disputed
> questions and submit an argument on the problem.  I have done that and
> gotten responses that range from "yeah but... you lose" to "very good point,
> well stated and we see the confusion, you will receive credit and we will
> adjust the question database."  If we worry about someone capturing the
> questions, then it must be a proctored exam, even the most secure website
> would not prevent real time screen recording!
>
> #2) Being a student, exam taker, and teacher I know too well the "those who
> cant, teach" statement.  That is why I start classes with credentials,
> regardless of experience the class seems more comfortable because I have
> degrees and certifications.  But it really gets the points home when I give
> real world examples from experience.  I very much focus on concepts and then
> their implementation.  Understanding the principles behind XSS is very
> important because any new approaches for an exploit can be quickly seem and
> mitigated.  People who only learn the established body of knowledge can
> protect against all know, but no new, attacks.  If we publish content areas,
> someone will publish the book of knowledge (dummy book? :-) to study our
> knowledge areas.  I am not saying good or bad, but the reality I believe in
> is:
> 1- we publish content areas
> 2- people take the exam and the certification becomes desired and well
> respected
> 3- more people want to take and pass the exam
> 4- some smart writers publish books on the exam and content areas and what
> is needed to pass
> 5- the book writers make lots of money
> 6- we are left with an exam by the numbers.
>
> Asking for concepts requires more thought, back to my essay suggestion.
>
> #3) I agree with open and public processes.  SANS posts the papers for all
> to see and that has become an excellent resources for many professionals.  I
> think there are some who do not want to publish junk because everyone else
> can see it-- hit them in their pride!  And maybe that is how to distinguish
> the highest level of certification, publish a paper on the OWASP site which
> advances or simplifies some area of content.  Have strict guidelines for
> form and content and reference.  As long as "someone votes" it is
> subjective.  If there is a nay vote, then maybe a need exists to justify why
> it was rejected so others could argue or agree.
>
> I hate the secret approach, the most objective way to do that is by
> questions with explicit answers, no essays.  multiple choice, matching,
> timeline event sequencing, true/false, etc.
>
> Cheers,
> Gary
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: https://lists.owasp.org/pipermail/owasp-cert/attachments/20080730/58c26f42/attachment.html 


More information about the Owasp-cert mailing list