[Owasp-testing] OWASP Testing Guide V3 - Index Brainstorming - "Spidering and Googling"

christian.heinrich at cmlh.id.au christian.heinrich at cmlh.id.au
Thu May 8 02:22:09 EDT 2008


To follow on from my recent presentation at the OWASP Conference in
Australia, can I recommend that the "Spidering and Googling" section of the
OWASP Testing Guide V2 be split into two sections i.e. "Spiders, Robots and
Crawlers" (before) and "Search Engine Discovery/Reconnaissance" (after) for
the OWASP Testing Guide V3?

The reason for this is two fold:
1. "Spiders, Robots and Crawlers" is a separate process (e.g. recursively
indexing directories outside of robots.txt) and "Googlebot" is dependant on
this technology, hence it would provide a better flow into the "Search
Engine Discovery/Reconnaissance" section, which is why I listed it as
"(before)" "Search Engine Discovery/Reconnaissance" in the above paragraph.
2. The renaming of "Googling" to "Search Engine Discovery/Reconnaissance"
would allow for the inclusion of other search engines, such as Live, Yahoo!,
etc.  Obviously there would be sub-sections under this addressing the
nuances of each search engine.

Please let me know your thoughts or suggestions?

Christian Heinrich
OWASP Individual Member
Sydney, Australia Chapter

More information about the Owasp-testing mailing list