[Owasp-appsensor-project] Detection or protection from web scraping
colin.watson at owasp.org
Fri Aug 16 14:34:01 UTC 2013
Thank you for the suggestions. I have updated UT2 on the wiki page
(and the equivalent text in the draft v2 Guide book) to mention
content scraping explicitly.
On 12 July 2013 05:16, Junior Lazuardi <junior.lazuardi at owasp.org> wrote:
> Hi Dennis and Ryan,
> thanks for the info, i see some of them are covered already, also on
> AppSensor cheat sheet
> Just curious that it's not "easy" to search on scraping explicitly, compared
> to those commercial ones.
> People might not be informed of it.
> Anyway, is scraping classified as an attack?
> and websites allowing scraping to be having vulnerabilities?
> a couple of search results at www.owasp.org
> "Unacceptable behavior could include unauthorized scraping of content,
> searching for vulnerabilities, and attempts undertake fraud."
> "But signatures have become less effective at detecting threats over time,
> and aren’t sufficient to address the sophisticated abusive behavior that
> large, publicly exposed Web applications are subject to, including page
> scraping, logic abuse, malicious automation, phishing, and malware
> maybe add "content scraping / abuse" in UT2? or unnecessary (inclusive)?
> UT2: Speed of Application Use
> The speed of requests from a user indicates that an automated tool is being
> used to access the site. The use of a tool undertaking a high number of
> requests quickly may indicate reconnaissance for an attack or attempts to
> identify vulnerabilities in the site.
> On Thu, Jul 11, 2013 at 7:26 PM, Ryan Barnett <ryan.barnett at owasp.org>
>> Web Scraping would be covered by this AppSensor Detection Point -
>> From: Junior Lazuardi <junior.lazuardi at owasp.org>
>> Date: Thursday, July 11, 2013 4:26 AM
>> To: <owasp-appsensor-project at lists.owasp.org>
>> Subject: [Owasp-appsensor-project] Detection or protection from web
>> Hello project members,
>> I'm curious about web scraping, and can't find many on OWASP website, or
>> on available projects.
>> Some characteristics of web scraping might have been covered already in
>> AppSensor Detection Points, but the word/phrase itself is so rarely
>> I found that some commercial products (WAF) sell web scraping protection
>> as a feature, and they even have specific whitepaper on it.
>> Please let me know what you think of it, and whether it fits on appsensor
>> best regards,
>> _______________________________________________ Owasp-appsensor-project
>> mailing list Owasp-appsensor-project at lists.owasp.org
> Owasp-appsensor-project mailing list
> Owasp-appsensor-project at lists.owasp.org
More information about the Owasp-appsensor-project