[Owasp-cert] ISO 17024

bill pankey bpankey at tunitas.com
Tue Aug 5 16:20:37 EDT 2008


James

For ANSI 17024 certification the overall requirement is that the
certification be "fair, valid and reliable".  To ensure this ANSI will
examine characteristics of the certifying body, its management processes as
well as the content and administration of the certification examination.

The application form for 10724 certification is found at:
http://publicaa.ansi.org/sites/apdl/Documents/Conformity%20Assessment/Personnel%20Certifier%20Accreditation/Documents%20related%20to%20accreditation%20under%20ANSI-ISO-IEC%2017024/Public%20Documents/ANSI-PCAC-FR-504.doc

A couple points stood out to me:

1) The certification process has to be 'well managed' ... "well managed' is
a term which implies formal policy and procedures with process controls,
audit and an improvement program.  Candidates who fail must be able to
appeal results / question the scoring of specifc test items.

2) Independence is a big deal ... and something that I think may be
'touchy'.  While the certifying body cannot have an economic interest in
'who get certified', it does have an economic interest in the cert itself
... too few certs and its business model is broken.

3) Periodic verification of the validity and reliability of the exams
through statistical studies.  As part of the 'well managed process'
the methodology has to be designed at the outset.

4) Certification validity concerns the degree to which the empirical
evidence and theoretical rationale supports the interpretation given to the
certificate, eg that the cert holder is "competent to ..."   For ANSI the
key issue is demonstration of job relevance

5) Exams must be *criterion* rather than *norm* referenced.  Minimum passing
scores have to be tied to competent performance of job responsibilities
rather than 'grading on the curve'.   This implies that if OWASP does a
competent job 'qualifying' certification candidates, pass rates could be
high.

6) The certifying body has to maintain an test item database with a history
of all test results.  This is needed for item analysis and statistical
studies.

7) Exam security is a big deal of course.  ANSI does not say what security
procedures must be employed but it does list topics to be addressed in a
security plan.

So ....

a) it makes sense for item writers and reviewers to document what job
competency is being addressed by the knowledge / skill associated with the
item.

b) one approach might be for the group to write a ' job description'  for
which the cert holder is, by knowledge and skill, ideally qualified.  Item
writers merely need reference the specific requirement(s).

c) controls will be a big thing, so, if ISO/ANSI certification is an
important target, work should begin to define and document the appropriate
control environment.

d) its is probably easier to obtain ISO certification for an existing cert
than it is with a new one, especially if the existing cert process was
designed with 10724 in mind.

Bill
-------------- next part --------------
An HTML attachment was scrubbed...
URL: https://lists.owasp.org/pipermail/owasp-cert/attachments/20080805/3438fda5/attachment.html 


More information about the Owasp-cert mailing list