[Owasp-board] FYI-Status Only --- Update on wiki server issues

Tom Brennan - OWASP tomb at owasp.org
Wed Mar 9 01:01:01 UTC 2016


Akamai long time supporter has bartered/donated its CDN platform to OWASP
including its KONA system that is now contributed to managed now by long
time OWASP guy Ryan Barnett of the CRS Project (who now works for them).

My question is are we now using the CDN?

Are we now using KONA too?

Are the reports available for traffic such as analytics and related items.
It would seem that items #1, #2 can be addressed.

Item #3 there are a large number of pages that are now forwarding to other
pages etc.. This data should be very userful as we get ready to award the
contract to one of the (6) bidders on the Website RFQ

Thanks Paul/Matt for the update and info in advance.

Tom Brennan
Global Board of Directors
(d) 973-506-9304

On Tue, Mar 8, 2016 at 6:08 PM, Paul Ritchie <paul.ritchie at owasp.org> wrote:

> To OWASP Board:    Here is quick update from Matt T on some research &
> work he had to do the past couple days on the WIKI do to HIGH Loads.
> FYI Only - This is the kind of work Matt "should" be doing.  We spent time
> this weekend at the Staff Summit identifying some areas that can be
> off-loaded to other IT resource so Matt can focus on these mission critical
> issues.
> Best Regards, Paul Ritchie
> OWASP Executive Director
> paul.ritchie at owasp.org
> ---------- Forwarded message ----------
> From: Matt Tesauro <matt.tesauro at owasp.org>
> Date: Tue, Mar 8, 2016 at 4:42 PM
> Subject: Update on wiki server issues
> To: Paul Ritchie <paul.ritchie at owasp.org>, Kate Hartmann <
> kate.hartmann at owasp.org>, Noreen Whysel <noreen.whysel at owasp.org>, Laura
> Grau <laura.grau at owasp.org>, Claudia Casanovas <
> claudia.aviles-casanovas at owasp.org>, Kelly Santalucia <
> kelly.santalucia at owasp.org>
> Paul and company,
> There continue to be brief periods of high CPU & load on the OWASP wiki
> server. I've been digging through the 8.4+ million lines of the March log
> files for the last 2 hours and have discovered 3 contributing factors:
> (1) Internet putzes running SSL configuration scanners against us.
>        These are hard to deal with as they don't appear in the log files
> depending on which scanner someone chooses to use.
>     *What to do*:  Nothing for the time being - this is already dying
> down (those I can see) and I expect it to continue to lesson as we get
> father from the latest SSL vulnerability we get.
> (2) Internet putzes running web crawlers on the wiki
>        Web crawlers start on the main page of a web site and follow every
> link they find.  Beyond just being annoying, these crawlers will 'view'
> every change on every page for the wiki.  Each crawler can generate several
> thousand requests to the server.  One crawler from Taiwan requested 7,000
> pages in under 10 minutes.
>     What to do:  I will either write a fail2ban rule or add an Apache
> module that will restrict or ban for ~10 minutes any person that makes too
> many requests per minute.  The threshold will be set far faster than any
> normal user would click on links.
> (3) The wiki is taking to itself - a lot
>           I also noticed a large number of requests in a short period
> where the wiki is making requests to itself.  These are requests that come
> from the wiki server to the wiki server.  Logging in MediaWiki isn't very
> good so I cannot tell exactly why these requests are being made.  It
> appears that there is 1 or more MediaWiki extension that is having problems
> and probably needs an update.
>      What to do:  Unfortunately, there's no automated process to keep
> MediaWiki extensions updated so I'll need to manually verify the version of
> each of our extensions to see which has updates.
> I've also adjusted some settings on Apache and our DB which should give us
> some breathing room while I continue to work on this issue.
> Cheers!
> --
> -- Matt Tesauro
> OWASP AppSec Pipeline Lead
> https://www.owasp.org/index.php/OWASP_AppSec_Pipeline
> OWASP WTE Project Lead
> *https://www.owasp.org/index.php/OWASP_Web_Testing_Environment_Project
> <https://www.owasp.org/index.php/OWASP_Web_Testing_Environment_Project>*
> http://AppSecLive.org <http://appseclive.org/> - Community and Download
> site
> _______________________________________________
> Owasp-board mailing list
> Owasp-board at lists.owasp.org
> https://lists.owasp.org/mailman/listinfo/owasp-board

The information contained in this message and any attachments may be 
privileged, confidential, proprietary or otherwise protected from 
disclosure. If you, the reader of this message, are not the intended 
recipient, you are hereby notified that any dissemination, distribution, 
copying or use of this message and any attachment is strictly prohibited. 
If you have received this message in error, please notify the sender 
immediately by replying to the message, permanently delete it from your 
computer and destroy any printout.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.owasp.org/pipermail/owasp-board/attachments/20160308/eaac495e/attachment.html>

More information about the Owasp-board mailing list