[Owasp-board] FYI: Tons of wiki spam just cleaned up

Jim Manico jim.manico at owasp.org
Tue Apr 19 17:44:08 UTC 2016


Matt,

Thanks for this update. Super insightful to know what you're spending
time on. Insane....

Regarding this:

"BTW, to Jim's point on the scope of the wiki clean-up effort.  While
looking for an account with no contributions, I ran a maintenance script
which lists active accounts which have never made wiki edits - 9,912
exist or ~ 10K stale wiki accounts.  Our infrastructure needs a lot of
de-crufting."

Consider just wiping those accounts. They're not being used after all...

Thanks again for this.

Aloha,
Jim


On 4/19/16 7:18 AM, Matt Tesauro wrote:
> As an update, here's a perfect example of my IT hours being consumed
> early in the week.
>
> As of Sunday, I've:
> * Setup a redirect site for https://www.appsecusa.org =>
> https://2016.appsecusa.org + purchased the SSL cert for that site
> * Renewed the SSL cert for www.owasp.org <http://www.owasp.org>
> * Renewed the SSL cert for lists.owasp.org <http://lists.owasp.org>
> (purchased and will be installed this evening)
> * Cleaned up SPAM on the wiki after a couple of bogus accounts slipped
> by our approval process.
>
> Those 4 bullets don't sound like much but the last one took 6+ hours
> of work and required me to write some code to automate cleaning up the
> latest bit of wiki spam.
>
> BTW, to Jim's point on the scope of the wiki clean-up effort.  While
> looking for an account with no contributions, I ran a maintenance
> script which lists active accounts which have never made wiki edits -
> 9,912 exist or ~ 10K stale wiki accounts.  Our infrastructure needs a
> lot of de-crufting.
>
> Back on point - 6+ hours to clean up Wiki SPAM, why?
>
> In cleaning up the SEO link bait created by these wiki spammers, I
> found out that the 'standard' wiki clean-up tools didn't work great in
> this situation.[1]
>
> The tools will either delete a page only authored by a wiki user or
> revert the last edit of that wiki user.
>
> This worked for a couple hundred of the SPAM instances but left a
> couple hundred pages which had multiple spammer edits from 2+ wiki
> spammer accounts.  To the standard clean-up tools, this looked like a
> real page with multiple authors/edits so it skipped them.
>
> After deleting a few of these manually and seeing the scope of that
> work, I wrote some code to clean up the mess on our wiki.  I've posted
> that code to GitHub so that more then me has access to use it.  I had
> to run it several times with new users as it kept leading to new
> spammer accounts which shared edits with the user being cleaned up. 
> Yes, all user accounts I found had their accounts blocked indefinitely.
>
> You can find the code at
> https://github.com/mtesauro/random-docs/tree/master/scripts/mediawiki/spam-cleanup
>
> The bulk of the work is done by clean-spam.sh:
> https://github.com/mtesauro/random-docs/blob/master/scripts/mediawiki/spam-cleanup/clean-spam.sh
>
> That code, when run by wrapper.sh will take the first contribution of
> the spammer, check if its authored only by spammers (or wiki scripts)
> and delete it if that is true.  It logs all its actions and places the
> URL of pages that cause problems in a separate file for manual review.
>
> THIS is the kind of work I'm prefer to be doing and is much more
> valuable to the Foundation than some of the general IT maintenance
> work I spoke of in the other thread.
>
> Since this happened later in the day of our previous thread on OWASP
> IT, I wanted to share this example of my 10 hours getting mostly
> consumed in the first couple days of the week.
>
> Cheers!
>
> [1] https://www.mediawiki.org/wiki/Manual:Combating_vandalism#Standard_cleanup_tools
>
> --
> -- Matt Tesauro
> OWASP AppSec Pipeline Lead
> https://www.owasp.org/index.php/OWASP_AppSec_Pipeline 
> OWASP WTE Project Lead
> _https://www.owasp.org/index.php/OWASP_Web_Testing_Environment_Project_
> http://AppSecLive.org <http://appseclive.org/> - Community and
> Download site
>
>
> _______________________________________________
> Owasp-board mailing list
> Owasp-board at lists.owasp.org
> https://lists.owasp.org/mailman/listinfo/owasp-board

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.owasp.org/pipermail/owasp-board/attachments/20160419/1a4d533a/attachment.html>


More information about the Owasp-board mailing list