[Owasp-cert] Deadlines for August

Gary Palmer owasp at getmymail.org
Tue Jul 29 22:33:11 EDT 2008


#1) For the CISSP I had to complete a paper form that was provided as part
of the test.  I had to complete that form within the time allotted and turn
it into the reviewer.  It was reviewed independently and outside the test
room and I was not informed of the results.  For the SANS online testing, it
was structured to present one question at a time with the answer options.
Each question had a checkbox you could use to tag the question.  There was
also a checkbox to gat the prior question in case you clicked submit and
then realized you wanted to challenge.  At the test completion, after the
allotted time, you were given a chance to review the disputed questions and
submit an argument on the problem.  I have done that and gotten responses
that range from "yeah but... you lose" to "very good point, well stated and
we see the confusion, you will receive credit and we will adjust the
question database."  If we worry about someone capturing the questions, then
it must be a proctored exam, even the most secure website would not prevent
real time screen recording!
 
#2) Being a student, exam taker, and teacher I know too well the "those who
cant, teach" statement.  That is why I start classes with credentials,
regardless of experience the class seems more comfortable because I have
degrees and certifications.  But it really gets the points home when I give
real world examples from experience.  I very much focus on concepts and then
their implementation.  Understanding the principles behind XSS is very
important because any new approaches for an exploit can be quickly seem and
mitigated.  People who only learn the established body of knowledge can
protect against all know, but no new, attacks.  If we publish content areas,
someone will publish the book of knowledge (dummy book? :-) to study our
knowledge areas.  I am not saying good or bad, but the reality I believe in
is:
1- we publish content areas
2- people take the exam and the certification becomes desired and well
respected
3- more people want to take and pass the exam
4- some smart writers publish books on the exam and content areas and what
is needed to pass
5- the book writers make lots of money
6- we are left with an exam by the numbers.
 
Asking for concepts requires more thought, back to my essay suggestion.
 
#3) I agree with open and public processes.  SANS posts the papers for all
to see and that has become an excellent resources for many professionals.  I
think there are some who do not want to publish junk because everyone else
can see it-- hit them in their pride!  And maybe that is how to distinguish
the highest level of certification, publish a paper on the OWASP site which
advances or simplifies some area of content.  Have strict guidelines for
form and content and reference.  As long as "someone votes" it is
subjective.  If there is a nay vote, then maybe a need exists to justify why
it was rejected so others could argue or agree.
 
I hate the secret approach, the most objective way to do that is by
questions with explicit answers, no essays.  multiple choice, matching,
timeline event sequencing, true/false, etc.
 
Cheers,
Gary
 

  _____  

From: owasp-cert-bounces at lists.owasp.org
[mailto:owasp-cert-bounces at lists.owasp.org] On Behalf Of
james at architectbook.com
Sent: Sunday, July 27, 2008 5:28 AM
Cc: owasp-cert at lists.owasp.org
Subject: Re: [Owasp-cert] Deadlines for August


Gary, I have a few questions

1. If someone wanted to dispute the question, I am assuming that they did so
outside of taking the exam. Wouldn't that have required them to actually
write down the question which can potentially weaken the security of the
exam?

2. I am all for publishing the content areas to be used for the exam, but am
somewhat against having a specific curriculum aligned to it. I do believe
that students who take certification should be able to see whether the
instructor has taken and passed the exams themselves and the survey results
seem to indicate this is positive as well. The phrase: Those who can't do,
teach is one of the first things our exam will conquer.

3. I have a preference of making certification wherever possible more open
than any other process on the planet. I am revoking my initial comments
regarding having a separate review committee as this really isn't
transparent. I am now thinking that the best method would be to post essays
to the OWASP site where each and every OWASP chapter leader gets to vote on
the quality. More Yeas than Nays determines outcome. No secret societies.



-------- Original Message --------
Subject: Re: [Owasp-cert] Deadlines for August
From: "Gary Palmer" <owasp at getmymail.org>
Date: Sat, July 26, 2008 1:29 pm
To: <owasp-cert at lists.owasp.org>



First, for simplicity in email, when responding, I recommend we all just
respond to the lists email address and not a second copy to the author or
the replied to post.  My mail now takes care of than but it might help
others...
 
For the SANS stuff, I remember when I did my research paper.  There were
several (I think 3) people who reviewed it.  They voted and the overall
grade is what I received.  I remember having some conversations with a few
of them because they had questions.  Because of those conversations I felt
they were being objective and reasonable.  I also appreciated that the
online test allowed me to mark objectionable questions and after the test I
could formulate my abjections.  I always received a response and several
times they agreed and gave me credit AND improved the question.
 
The CISSP also allows you to dispute a question and between being able to
dispute and my SANS feedback experience I think we should also have a
dispute process.  I was less enamored with the CISSP process because I never
heard back from them to validate or dismiss my concern so I actually could
not learn.  SANS, on the other hand, did reply and explain their position so
I did learn.
 
Isn't the point here to measure what we know and offer opportunity to learn
and grow?  When I teach I typically tell the class "here is everything I
will test you on".  I try to give specific examples.  If it were math I
would give exact problems and on the test just change the values, not the
structure.  I want to test what they know and give a good grade if they
understand and convey the concepts.  I hate trick questions and I never
asked questions I did not warn them about.  I told them what was important
and let them focus on it.  If they got that, the other details could sort
themselves out!
 
So with all these words I am suggesting having a process to dispute
questions (which allows us to review and improve) and a feedback process to
let the subject know results (to help them learn and to validate that their
complaint was indeed heard and considered.
 
Cheers,
Gary
 

  _____  

From: owasp-cert-bounces at lists.owasp.org [
<mailto:owasp-cert-bounces at lists.owasp.org>
mailto:owasp-cert-bounces at lists.owasp.org] On Behalf Of Matthew Chalmers
Sent: Friday, July 25, 2008 8:54 PM
To: james at architectbook.com
Cc: owasp-cert at lists.owasp.org
Subject: Re: [Owasp-cert] Deadlines for August


I definitely see your point about writing the .NET version of WebGoat...of
course it's designed to be flawed, haha. ;-P I just thought of something
else too. When I got my GSNA it was back when SANS/GIAC was still requiring
a research paper rather than having it be optional for an extra gold star.
They obviously figured out some way to grade these papers. Is anyone from
SANS on this list? I wouldn't want people who failed to automatically start
ranting about a lack of objectivity...
 
Matt


On Fri, Jul 25, 2008 at 10:23 PM, < <mailto:james at architectbook.com>
james at architectbook.com> wrote:


Several thoughts:

1. For the record, I think we need to test above and beyond the OWASP
Application Security Desk Reference Project. While it is over 1K pages, I
still think it is incomplete.
2. I think it is safe to assume that most IT folks internationally or even
next door just plain suck at written communication. This is a historical
problem ever since we were called data processing.
3. In terms of essays, I think there is no need to worry. The last time I
visited HR, I asked for a count of the number of languages spoken by my IT
peers and at last count it was 43 distinct. During the day, I run around
being enterprisey and don't have any issues with voluntelling some of my
peers that they are now official OWASP reviewers. Folks from Microsoft,
Oracle, IBM probably have counts even higher. So, the action item I guess is
to noodle how to get country diversity into the mix. The problem you
describe usually doesn't occur in the America's or Europe and could be
isolated to Asian & African countries.
4. I would actually say that one form of waiver from writing comprehensive
documentation is to instead deliver working software. If someone who can't
write a lick of English but decides to mercilessly contribute to writing the
.NET version of WebGoat most certainly understands web application
security...



-------- Original Message --------
Subject: Re: [Owasp-cert] Deadlines for August

From: "Matthew Chalmers" < <mailto:matthew.chalmers at owasp.org>
matthew.chalmers at owasp.org>
Date: Fri, July 25, 2008 10:18 pm
To: "Gary Palmer" < <mailto:owasp at getmymail.org> owasp at getmymail.org>
Cc:  <mailto:owasp-cert at lists.owasp.org> owasp-cert at lists.owasp.org


I like the idea of understanding in multiple distinct areas, kind of like
domains in a 'CBOK'. I also like the idea of being able to redo pieces
(after a certain amount wait time maybe) if we do go with some formula that
gets you a higher level. For example if there are five domains...five exams
maybe...we say you have to get at least x% in each to be a 'master' but if
you do great on four of them but only average on the 5th, you're not a
master until you retake that 5th part and get the minimum qualifying score.
It's an idea anyway.
 
I'm not so sure about the essay. I like essays and fill-in-the-blank in
general, however, we're talking about a very wide audience, much wider than
that of the CBEST. People who will want (and may deserve) an OWASP cert will
not necessarily speak English, not necessarily have good written
communication skills, etc. We're not testing those things, we're just
testing...whatever we're testing relating to web application security. ;-)
Also, if we grade tests rather than a third party like VUE, we may not get
competent volunteers who speak every language in which we may offer the
test. Plus even if we had the luxury of being able to require English, even
good English, of our test-takers, that would mean we'd have to ensure our
test-graders were that much more competent since they're reading essays
rather than just checking to make sure the correct box was ticked. I wonder
if an essay just for the highest level would be do-able even.
 
Matt


On Fri, Jul 25, 2008 at 8:50 PM, Gary Palmer < <mailto:owasp at getmymail.org>
<mailto:owasp at getmymail.org> owasp at getmymail.org> wrote:


Interestingly enough, I am preparing for the CBEST which is the first step
in obtaining a teaching credential.  The CBEST is a 3 part exam, the first
deals with reading and comprehension, the second is math and the third is an
essay.  The R&C is split between knowing what you read, understanding what
you read, and then critical analysis/research analysis/authors
perspective/main point/feelings/etc.  The math seems to be fairly straight
forward.  The essay is 2 compositions, the first is analyze a given
situation and the other is to write about a personal experience.
 
The grading is scaled and the combined score must be passing.  There is a
minimum for each section to pass, but just missing in one and exceeding in
another to have a combined passing score will work too.
 
More at:  <http://www.cbest.nesinc.com/CA14_overview.asp>
<http://www.cbest.nesinc.com/CA14_overview.asp>
http://www.cbest.nesinc.com/CA14_overview.asp
 
Maybe we should consider this type of approach, the essay would certainly
challenge the person and provide a record versus a personal meeting that may
be biased by personalities.  The different sections could measure depth and
understanding of knowledge for multiple degrees of certification.  Also the
CBEST allows repeats where you can focus on one section or repeat all, in
order to improve your scores.  An interesting approach and unique when I
compare to the assorted other certification exams I have taken.
 
Cheers,
Gary Palmer
 

  _____  

From:  <mailto:owasp-cert-bounces at lists.owasp.org>
<mailto:owasp-cert-bounces at lists.owasp.org>
owasp-cert-bounces at lists.owasp.org [mailto:
<mailto:owasp-cert-bounces at lists.owasp.org>
<mailto:owasp-cert-bounces at lists.owasp.org>
owasp-cert-bounces at lists.owasp.org] On Behalf Of Matthew Chalmers
Sent: Friday, July 25, 2008 5:53 PM
To:  <mailto:dmalloc at users.sourceforge.net>
<mailto:dmalloc at users.sourceforge.net> dmalloc at users.sourceforge.net 

Cc:  <mailto:owasp-cert at lists.owasp.org>
<mailto:owasp-cert at lists.owasp.org> owasp-cert at lists.owasp.org

Subject: Re: [Owasp-cert] Deadlines for August


No offense to James but again I agree with David. I think pure
multiple-guess should be avoided--not to mention a pure multiple-guess
product that's been rushed because of concerns with time to market or
delivering 'something ok' sooner rather than 'something good' later.
 
Matt


On Fri, Jul 25, 2008 at 7:40 PM, David H. <
<mailto:dmalloc at users.sourceforge.net>
<mailto:dmalloc at users.sourceforge.net> dmalloc at users.sourceforge.net> wrote:


> If we stick to simple multiple choice for the first exam, it means that we
> have delivered something of value and can in the meantime buy us more time
> to work on more complex scenarios.
>

That is exactly what I try to dispute. I do not see any value in
Multiple Choice Questionaires not even when you are purely testing for
relearned value. That stems from a long history with Multiple Choice
Tests and the desire to create meaningful exams for a new paradigm of
learning.

--
Sent from gmail so do not trust this communication.
Do not send me sensitive information here, ask for my none-gmail accounts.

"Therefore the considerations of the intelligent always include both
benefit and harm." - Sun Tzu




_______________________________________________
Owasp-cert mailing list
 <mailto:Owasp-cert at lists.owasp.org>  <mailto:Owasp-cert at lists.owasp.org>
Owasp-cert at lists.owasp.org
 <https://lists.owasp.org/mailman/listinfo/owasp-cert>
<https://lists.owasp.org/mailman/listinfo/owasp-cert>
https://lists.owasp.org/mailman/listinfo/owasp-cert




  _____  

_______________________________________________
Owasp-cert mailing list
Owasp-cert <http://email.secureserver.net/pcompose.php#Compose>
@lists.owasp.org
 <https://lists.owasp.org/mailman/listinfo/owasp-cert>
<https://lists.owasp.org/mailman/listinfo/owasp-cert>
https://lists.owasp.org/mailman/listinfo/owasp-cert



  _____  

_______________________________________________
Owasp-cert mailing list
Owasp-cert <http://email.secureserver.net/pcompose.php#Compose>
@lists.owasp.org
 <https://lists.owasp.org/mailman/listinfo/owasp-cert>
https://lists.owasp.org/mailman/listinfo/owasp-cert


-------------- next part --------------
An HTML attachment was scrubbed...
URL: https://lists.owasp.org/pipermail/owasp-cert/attachments/20080729/aad308cf/attachment-0001.html 


More information about the Owasp-cert mailing list