[Owasp-testing] Observations on the current Testing Guide (Part 1)

Safuat Hamdy safuat.hamdy at secorvo.de
Fri Jan 16 10:11:32 UTC 2015

Hello all,

I've been using the testing guide a lot during which I noticed many
inconsistencies and oddities that should be fixed:

- The way headings are typeset follow no recognizable rule and is partly
- References to tools sould be consolidated in a single section to
remove a lot of redundancy; ancient and outdated tools like WebScarab
shouldn't be mentioned.

INFO-XXX for 001, 002, 004, 006, 007, 008, 009 and 010:
These items have unclear expectation of testing outcome in the sense
that no verdict can be stated. It is conceivable to give a verdict at
002, 008, 009 if software is discoverd for which vulnerabilities are
known; likewise at 002 and 003 for excessive information disclosure. If
this is intended, then say so explicitely. However, in general
information gathering and reconnaissance should be a prerequisite for
testing but not testing items themselves.

Subsections "Test Objectives" and "How to Test" are typeset differently
compared to other sections.

INFO-004, INFO-008:
"Summary" is not a summary

The Title "Map Application Architecture" is largely misleading. From the
text: "... it is necessary to map the network and application
architecture. ..." and later "... Is there a firewalling system
protecting the webserver? ..." and so on.

This section is simply not clear. No wonder - the subsection "Test
Objective" is missing and it seems that INFO-010 has indeed none.

"Summary" is not a summary
No "Test Objective" subsection
In fact, the expected outcome is not clear

No "Test Objective" subsection
I believe that testing for directory browsing belongs here, should be added.

"Summary" is not a summary
No "Test Objective" subsection

No "Test Objective" subsection

IDENT-001, IDENT-004:
"Summary" is not a summary

AUTHN-001, AUTHN-002, AUTHN-004, AUTHN-005, AUTHN-008, AUTHN-010:
"Summary" is not a summary
No "Test Objective" subsection

"Summary" is not a summary

No "Test Objective" subsection

AUTHZ-001 (appears twice), AUTHZ-002, AUTHZ-003, AUTHZ-004, SESS-001,
SESS-002 and many others (too many to mention them all):
"Summary" is not a summary
No "Test Objective" subsection

"Brief Summary" (huh?!) is not a brief summary
No "Test Objective" subsection

SESS-006, SESS-007:
The distinction between SESS-006, item "Testing for Session Timeout" and
SESS-007 is not at all convincing.  They are simply identical.

States expected results explicitely. Good idea - why not for other test

Identical to CONFIG-006 (?)

How does that possibly make sense for web app testing?! All you can do
here is fuzzing and a check for unusual behavior.

Page 211, Table:
Lists non-existent items IDENT-006 and IDENT-007

Page 212, table:
INPVAL is not correctly enumarated (INPVAL-005?)

More will follow.

S. Hamdy


There are 10 types of people - those who understand ternary,
those who confuse it with binary, and those who don't get it at all.

More information about the Owasp-testing mailing list