[Owasp-board] FYI: Tons of wiki spam just cleaned up

Matt Tesauro matt.tesauro at owasp.org
Tue Apr 19 17:18:39 UTC 2016


As an update, here's a perfect example of my IT hours being consumed early
in the week.

As of Sunday, I've:
* Setup a redirect site for https://www.appsecusa.org =>
https://2016.appsecusa.org + purchased the SSL cert for that site
* Renewed the SSL cert for www.owasp.org
* Renewed the SSL cert for lists.owasp.org (purchased and will be installed
this evening)
* Cleaned up SPAM on the wiki after a couple of bogus accounts slipped by
our approval process.

Those 4 bullets don't sound like much but the last one took 6+ hours of
work and required me to write some code to automate cleaning up the latest
bit of wiki spam.

BTW, to Jim's point on the scope of the wiki clean-up effort.  While
looking for an account with no contributions, I ran a maintenance script
which lists active accounts which have never made wiki edits - 9,912 exist
or ~ 10K stale wiki accounts.  Our infrastructure needs a lot of
de-crufting.

Back on point - 6+ hours to clean up Wiki SPAM, why?

In cleaning up the SEO link bait created by these wiki spammers, I found
out that the 'standard' wiki clean-up tools didn't work great in this
situation.[1]

The tools will either delete a page only authored by a wiki user or revert
the last edit of that wiki user.

This worked for a couple hundred of the SPAM instances but left a couple
hundred pages which had multiple spammer edits from 2+ wiki spammer
accounts.  To the standard clean-up tools, this looked like a real page
with multiple authors/edits so it skipped them.

After deleting a few of these manually and seeing the scope of that work, I
wrote some code to clean up the mess on our wiki.  I've posted that code to
GitHub so that more then me has access to use it.  I had to run it several
times with new users as it kept leading to new spammer accounts which
shared edits with the user being cleaned up.  Yes, all user accounts I
found had their accounts blocked indefinitely.

You can find the code at
https://github.com/mtesauro/random-docs/tree/master/scripts/mediawiki/spam-cleanup

The bulk of the work is done by clean-spam.sh:
https://github.com/mtesauro/random-docs/blob/master/scripts/mediawiki/spam-cleanup/clean-spam.sh

That code, when run by wrapper.sh will take the first contribution of the
spammer, check if its authored only by spammers (or wiki scripts) and
delete it if that is true.  It logs all its actions and places the URL of
pages that cause problems in a separate file for manual review.

THIS is the kind of work I'm prefer to be doing and is much more valuable
to the Foundation than some of the general IT maintenance work I spoke of
in the other thread.

Since this happened later in the day of our previous thread on OWASP IT, I
wanted to share this example of my 10 hours getting mostly consumed in the
first couple days of the week.

Cheers!

[1]
https://www.mediawiki.org/wiki/Manual:Combating_vandalism#Standard_cleanup_tools

--
-- Matt Tesauro
OWASP AppSec Pipeline Lead
https://www.owasp.org/index.php/OWASP_AppSec_Pipeline
OWASP WTE Project Lead
*https://www.owasp.org/index.php/OWASP_Web_Testing_Environment_Project
<https://www.owasp.org/index.php/OWASP_Web_Testing_Environment_Project>*
http://AppSecLive.org <http://appseclive.org/> - Community and Download site
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.owasp.org/pipermail/owasp-board/attachments/20160419/8434bf72/attachment.html>


More information about the Owasp-board mailing list