[Owasp-testing] Observations on the current Testing Guide (Part 1)

Safuat Hamdy safuat.hamdy at secorvo.de
Fri Jan 16 10:11:32 UTC 2015


Hello all,

I've been using the testing guide a lot during which I noticed many
inconsistencies and oddities that should be fixed:


ALL ITEMS
- The way headings are typeset follow no recognizable rule and is partly
inconsistent.
- References to tools sould be consolidated in a single section to
remove a lot of redundancy; ancient and outdated tools like WebScarab
shouldn't be mentioned.


INFO-XXX for 001, 002, 004, 006, 007, 008, 009 and 010:
These items have unclear expectation of testing outcome in the sense
that no verdict can be stated. It is conceivable to give a verdict at
002, 008, 009 if software is discoverd for which vulnerabilities are
known; likewise at 002 and 003 for excessive information disclosure. If
this is intended, then say so explicitely. However, in general
information gathering and reconnaissance should be a prerequisite for
testing but not testing items themselves.


INFO-002:
Subsections "Test Objectives" and "How to Test" are typeset differently
compared to other sections.


INFO-004, INFO-008:
"Summary" is not a summary


INFO-010:
The Title "Map Application Architecture" is largely misleading. From the
text: "... it is necessary to map the network and application
architecture. ..." and later "... Is there a firewalling system
protecting the webserver? ..." and so on.

This section is simply not clear. No wonder - the subsection "Test
Objective" is missing and it seems that INFO-010 has indeed none.


CONFIG-001:
"Summary" is not a summary
No "Test Objective" subsection
In fact, the expected outcome is not clear


CONFIG-002:
No "Test Objective" subsection
I believe that testing for directory browsing belongs here, should be added.


CONFIG-003, CONFIG-004, CONFIG-006, CONFIG-007:
"Summary" is not a summary
No "Test Objective" subsection


CONFIG-005, CONFIG-008:
No "Test Objective" subsection


IDENT-001, IDENT-004:
"Summary" is not a summary


AUTHN-001, AUTHN-002, AUTHN-004, AUTHN-005, AUTHN-008, AUTHN-010:
"Summary" is not a summary
No "Test Objective" subsection


AUTHN-003:
"Summary" is not a summary


AUTHN-006
No "Test Objective" subsection


AUTHZ-001 (appears twice), AUTHZ-002, AUTHZ-003, AUTHZ-004, SESS-001,
SESS-002 and many others (too many to mention them all):
"Summary" is not a summary
No "Test Objective" subsection


SESS-003:
"Brief Summary" (huh?!) is not a brief summary
No "Test Objective" subsection


SESS-006, SESS-007:
The distinction between SESS-006, item "Testing for Session Timeout" and
SESS-007 is not at all convincing.  They are simply identical.


SESS-006:
States expected results explicitely. Good idea - why not for other test
items?


INPVAL-003:
Identical to CONFIG-006 (?)


INPVAL-014:
How does that possibly make sense for web app testing?! All you can do
here is fuzzing and a check for unusual behavior.


Page 211, Table:
Lists non-existent items IDENT-006 and IDENT-007


Page 212, table:
INPVAL is not correctly enumarated (INPVAL-005?)



More will follow.


S. Hamdy

-- 

There are 10 types of people - those who understand ternary,
those who confuse it with binary, and those who don't get it at all.


More information about the Owasp-testing mailing list