[Owasp-cert] Deadlines for August

Gary Palmer owasp at getmymail.org
Sat Jul 26 13:29:20 EDT 2008


First, for simplicity in email, when responding, I recommend we all just
respond to the lists email address and not a second copy to the author or
the replied to post.  My mail now takes care of than but it might help
others...
 
For the SANS stuff, I remember when I did my research paper.  There were
several (I think 3) people who reviewed it.  They voted and the overall
grade is what I received.  I remember having some conversations with a few
of them because they had questions.  Because of those conversations I felt
they were being objective and reasonable.  I also appreciated that the
online test allowed me to mark objectionable questions and after the test I
could formulate my abjections.  I always received a response and several
times they agreed and gave me credit AND improved the question.
 
The CISSP also allows you to dispute a question and between being able to
dispute and my SANS feedback experience I think we should also have a
dispute process.  I was less enamored with the CISSP process because I never
heard back from them to validate or dismiss my concern so I actually could
not learn.  SANS, on the other hand, did reply and explain their position so
I did learn.
 
Isn't the point here to measure what we know and offer opportunity to learn
and grow?  When I teach I typically tell the class "here is everything I
will test you on".  I try to give specific examples.  If it were math I
would give exact problems and on the test just change the values, not the
structure.  I want to test what they know and give a good grade if they
understand and convey the concepts.  I hate trick questions and I never
asked questions I did not warn them about.  I told them what was important
and let them focus on it.  If they got that, the other details could sort
themselves out!
 
So with all these words I am suggesting having a process to dispute
questions (which allows us to review and improve) and a feedback process to
let the subject know results (to help them learn and to validate that their
complaint was indeed heard and considered.
 
Cheers,
Gary
 

  _____  

From: owasp-cert-bounces at lists.owasp.org
[mailto:owasp-cert-bounces at lists.owasp.org] On Behalf Of Matthew Chalmers
Sent: Friday, July 25, 2008 8:54 PM
To: james at architectbook.com
Cc: owasp-cert at lists.owasp.org
Subject: Re: [Owasp-cert] Deadlines for August


I definitely see your point about writing the .NET version of WebGoat...of
course it's designed to be flawed, haha. ;-P I just thought of something
else too. When I got my GSNA it was back when SANS/GIAC was still requiring
a research paper rather than having it be optional for an extra gold star.
They obviously figured out some way to grade these papers. Is anyone from
SANS on this list? I wouldn't want people who failed to automatically start
ranting about a lack of objectivity...
 
Matt


On Fri, Jul 25, 2008 at 10:23 PM, <james at architectbook.com> wrote:


Several thoughts:

1. For the record, I think we need to test above and beyond the OWASP
Application Security Desk Reference Project. While it is over 1K pages, I
still think it is incomplete.
2. I think it is safe to assume that most IT folks internationally or even
next door just plain suck at written communication. This is a historical
problem ever since we were called data processing.
3. In terms of essays, I think there is no need to worry. The last time I
visited HR, I asked for a count of the number of languages spoken by my IT
peers and at last count it was 43 distinct. During the day, I run around
being enterprisey and don't have any issues with voluntelling some of my
peers that they are now official OWASP reviewers. Folks from Microsoft,
Oracle, IBM probably have counts even higher. So, the action item I guess is
to noodle how to get country diversity into the mix. The problem you
describe usually doesn't occur in the America's or Europe and could be
isolated to Asian & African countries.
4. I would actually say that one form of waiver from writing comprehensive
documentation is to instead deliver working software. If someone who can't
write a lick of English but decides to mercilessly contribute to writing the
.NET version of WebGoat most certainly understands web application
security...



-------- Original Message --------
Subject: Re: [Owasp-cert] Deadlines for August

From: "Matthew Chalmers" <matthew.chalmers at owasp.org>
Date: Fri, July 25, 2008 10:18 pm
To: "Gary Palmer" <owasp at getmymail.org>
Cc: owasp-cert at lists.owasp.org


I like the idea of understanding in multiple distinct areas, kind of like
domains in a 'CBOK'. I also like the idea of being able to redo pieces
(after a certain amount wait time maybe) if we do go with some formula that
gets you a higher level. For example if there are five domains...five exams
maybe...we say you have to get at least x% in each to be a 'master' but if
you do great on four of them but only average on the 5th, you're not a
master until you retake that 5th part and get the minimum qualifying score.
It's an idea anyway.
 
I'm not so sure about the essay. I like essays and fill-in-the-blank in
general, however, we're talking about a very wide audience, much wider than
that of the CBEST. People who will want (and may deserve) an OWASP cert will
not necessarily speak English, not necessarily have good written
communication skills, etc. We're not testing those things, we're just
testing...whatever we're testing relating to web application security. ;-)
Also, if we grade tests rather than a third party like VUE, we may not get
competent volunteers who speak every language in which we may offer the
test. Plus even if we had the luxury of being able to require English, even
good English, of our test-takers, that would mean we'd have to ensure our
test-graders were that much more competent since they're reading essays
rather than just checking to make sure the correct box was ticked. I wonder
if an essay just for the highest level would be do-able even.
 
Matt


On Fri, Jul 25, 2008 at 8:50 PM, Gary Palmer < <mailto:owasp at getmymail.org>
owasp at getmymail.org> wrote:


Interestingly enough, I am preparing for the CBEST which is the first step
in obtaining a teaching credential.  The CBEST is a 3 part exam, the first
deals with reading and comprehension, the second is math and the third is an
essay.  The R&C is split between knowing what you read, understanding what
you read, and then critical analysis/research analysis/authors
perspective/main point/feelings/etc.  The math seems to be fairly straight
forward.  The essay is 2 compositions, the first is analyze a given
situation and the other is to write about a personal experience.
 
The grading is scaled and the combined score must be passing.  There is a
minimum for each section to pass, but just missing in one and exceeding in
another to have a combined passing score will work too.
 
More at:  <http://www.cbest.nesinc.com/CA14_overview.asp>
http://www.cbest.nesinc.com/CA14_overview.asp
 
Maybe we should consider this type of approach, the essay would certainly
challenge the person and provide a record versus a personal meeting that may
be biased by personalities.  The different sections could measure depth and
understanding of knowledge for multiple degrees of certification.  Also the
CBEST allows repeats where you can focus on one section or repeat all, in
order to improve your scores.  An interesting approach and unique when I
compare to the assorted other certification exams I have taken.
 
Cheers,
Gary Palmer
 

  _____  

From:  <mailto:owasp-cert-bounces at lists.owasp.org>
owasp-cert-bounces at lists.owasp.org [mailto:
<mailto:owasp-cert-bounces at lists.owasp.org>
owasp-cert-bounces at lists.owasp.org] On Behalf Of Matthew Chalmers
Sent: Friday, July 25, 2008 5:53 PM
To:  <mailto:dmalloc at users.sourceforge.net> dmalloc at users.sourceforge.net 

Cc:  <mailto:owasp-cert at lists.owasp.org> owasp-cert at lists.owasp.org

Subject: Re: [Owasp-cert] Deadlines for August


No offense to James but again I agree with David. I think pure
multiple-guess should be avoided--not to mention a pure multiple-guess
product that's been rushed because of concerns with time to market or
delivering 'something ok' sooner rather than 'something good' later.
 
Matt


On Fri, Jul 25, 2008 at 7:40 PM, David H. <
<mailto:dmalloc at users.sourceforge.net> dmalloc at users.sourceforge.net> wrote:


> If we stick to simple multiple choice for the first exam, it means that we
> have delivered something of value and can in the meantime buy us more time
> to work on more complex scenarios.
>

That is exactly what I try to dispute. I do not see any value in
Multiple Choice Questionaires not even when you are purely testing for
relearned value. That stems from a long history with Multiple Choice
Tests and the desire to create meaningful exams for a new paradigm of
learning.

--
Sent from gmail so do not trust this communication.
Do not send me sensitive information here, ask for my none-gmail accounts.

"Therefore the considerations of the intelligent always include both
benefit and harm." - Sun Tzu




_______________________________________________
Owasp-cert mailing list
 <mailto:Owasp-cert at lists.owasp.org> Owasp-cert at lists.owasp.org
 <https://lists.owasp.org/mailman/listinfo/owasp-cert>
https://lists.owasp.org/mailman/listinfo/owasp-cert




  _____  

_______________________________________________
Owasp-cert mailing list
Owasp-cert <http://email.secureserver.net/pcompose.php#Compose>
@lists.owasp.org
 <https://lists.owasp.org/mailman/listinfo/owasp-cert>
https://lists.owasp.org/mailman/listinfo/owasp-cert



-------------- next part --------------
An HTML attachment was scrubbed...
URL: https://lists.owasp.org/pipermail/owasp-cert/attachments/20080726/972b57df/attachment.html 


More information about the Owasp-cert mailing list