[Owasp-o2-platform] finding robots.txt via O2

dinis cruz dinis.cruz at owasp.org
Wed Dec 8 23:16:32 EST 2010


See below Matt's script to use google to find robots.txt.

Next step on this script is to expand it to download a bund of those
robots.txt and do an analysis on them (would be interresting to come up with
some rules for expected and not expected robots.txt content)

Dinis Cruz

---------- Forwarded message ----------
From: Matt Parsons <mparsons1980 at gmail.com>
Date: 8 December 2010 23:09


Today I discuss a simple script that you can write in 02 to find
robots.txt.   As we all know sometimes these scripts have sensitive
information in them.   Feel free to comment or e-mail me.   02 is a free
platform and an OWASP project.


http://parsonsisconsulting.wordpress.com/2010/12/08/how-to-find-robots-txt-with-02/


Matt Parsons, MSM, CISSP
> 315-559-3588 Blackberry
> 817-294-3789 Home office
> "Do Good and Fear No Man"
> Fort Worth, Texas
> A.K.A The Keyboard Cowboy
> mailto:mparsons1980 at gmail.com <mparsons1980 at gmail.com>
> http://www.parsonsisconsulting.com
> http://www.parsonsisconsultingblog.com
> <http://www.o2-ounceopen.com/o2-power-users/>
> http://www.linkedin.com/in/parsonsconsulting
> http://www.vimeo.com/8939668
> http://twitter.com/parsonsmatt
>
>
>
>
>
>
>
>


-- 

Matt Parsons, MSM, CISSP
315-559-3588 Blackberry
817-294-3789 Home office
"Do Good and Fear No Man"
Fort Worth, Texas
A.K.A The Keyboard Cowboy
mailto:mparsons1980 at gmail.com <mparsons1980 at gmail.com>
http://www.parsonsisconsulting.com
http://www.parsonsisconsultingblog.com
<http://www.o2-ounceopen.com/o2-power-users/>
http://www.linkedin.com/in/parsonsconsulting
http://www.vimeo.com/8939668
http://twitter.com/parsonsmatt








_______________________________________________
Owasp-codereview mailing list
Owasp-codereview at lists.owasp.org
https://lists.owasp.org/mailman/listinfo/owasp-codereview
-------------- next part --------------
An HTML attachment was scrubbed...
URL: https://lists.owasp.org/pipermail/owasp-o2-platform/attachments/20101209/c8cb5c27/attachment-0001.html 


More information about the Owasp-o2-platform mailing list