[Owasp-leaders] Fwd: Pingdom Alert: Incident #3 for OWASP (www.owasp.org), has been assigned to you.

Matt Tesauro matt.tesauro at owasp.org
Sat Jan 24 22:16:35 UTC 2015


No I don't.

I'm 95%+ certain that yesterday's was due to my inaccurate estimate of the
number of Apache workers needed for the radically different new wiki
setup.  Apache logged as much yesterday:
----
[Fri Jan 23 07:56:30.986686 2015] [mpm_prefork:error] [pid 10052] AH00161:
server reached MaxRequestWorkers setting, consider raising the
MaxRequestWorkers setting
----

It showed up again and I've bumped up MaxWorkers a second time.

For those with free time, the logs files for today up to a couple of
minutes ago can be had by downloading the tar.bz file at:

https://b2babeda35d8eca56de1-a20d474ee4eefa67424901ff4d5c1eb0.ssl.cf2.rackcdn.com/site-down-jan24.tar.bz

If you find something interesting and actionable, feel free to email me
directly.


--
-- Matt Tesauro
OWASP WTE Project Lead
http://www.owasp.org/index.php/Category:OWASP_Live_CD_Project
http://AppSecLive.org - Community and Download site
OWASP OpenStack Security Project Lead
https://www.owasp.org/index.php/OWASP_OpenStack_Security_Project

On Sat, Jan 24, 2015 at 11:46 AM, Jerry Hoff <jerry.hoff at whitehatsec.com>
wrote:

>  Hi all,
>
>  Matt - do we know the reason for the outages? Is this due to
> infrastructure issues or is this an DOS/DDOS type event?
>
>  Thank you,
> Jerry
>
>
> On Jan 24, 2015, at 10:43, Matt Tesauro <matt.tesauro at owasp.org> wrote:
>
>   Regrettably, this is the second time its had issues between 5:30 AM and
> 6 AM where I live so I'm sleeping.
>
>  Last weekend, the architecture for the wiki was radically changed, the
> MediaWiki source updated to the latest, and the underlying OS updated to
> the latest.
>
>  * The serving of www.owasp.org was moved from a single large VM holding
> Apache + MySQL on one host was moved into two VMs, one for the web head and
> anther running MariaDB.  (MariaDB is what Wikipedia uses for their
> install).  The VMs available during this migration were also substantially
> better - most importantly full SSD based disks.
>
>  * MediaWiki was upgraded to the latest version of the 1.23.x version
>
>  * Ubuntu Server was updated to 14.04 LTS from 12.04 LTS to get the
> latest OpenSSL options available.  The wiki now supports forward secrecy
> [1].
>
>  Yesterday's event was caused by MaxWorkers being reached by Apache, the
> settings were increased to account for the growing traffic on the wiki.
>
>  I've not had a chance to fully investigate today's event, but I did
> notice that the error log for Apache was huge (7.6 GB) due to wiki skins
> using a deprecated method existing on the server (see
> https://www.mediawiki.org/wiki/Manual:Skin_autodiscovery) which I've
> fixed.
>
>  Additionally,  there was an old Twitter feed plugin written by a member
> of the community years ago which was making deprecated PHP calls and also
> causing Apache errors - including fopen'ing non-existent files and causing
> unnecessary file reads with each request.  it has been removed.
>
>  I'm going to eat a quick breakfast and go watch my son play basketball.
> I'll look for any other contributing factors later after we finish
> celebrating my daughters 13th birthday, which is today.
>
>  [1] https://www.ssllabs.com/ssltest/analyze.html?d=owasp.org
>
>  Cheers!
>
>  --
> -- Matt Tesauro
> OWASP WTE Project Lead
> http://www.owasp.org/index.php/Category:OWASP_Live_CD_Project
> http://AppSecLive.org - Community and Download site
> OWASP OpenStack Security Project Lead
> https://www.owasp.org/index.php/OWASP_OpenStack_Security_Project
>
> On Sat, Jan 24, 2015 at 5:53 AM, psiinon <psiinon at gmail.com> wrote:
>
>>  Looks like owasp.org is down again :(
>>
>>  Simon
>>
>> ---------- Forwarded message ----------
>> From: <alert at pingdom.com>
>> Date: Sat, Jan 24, 2015 at 10:19 AM
>> Subject: Pingdom Alert: Incident #3 for OWASP (www.owasp.org), has been
>> assigned to you.
>> To: psiinon at gmail.com
>>
>>
>> Hi Simon Bennetts,
>>
>> This is a notification sent by Pingdom.
>>
>> Incident 3, OWASP (www.owasp.org),
>> has been assigned to you.
>>
>> Log in to your account at https://my.pingdom.com/ to acknowledge, see
>> further details and take
>> the necessary actions.
>>
>>
>> Best regards,
>> The Pingdom Team
>>
>>
>>
>>
>>
>> --
>> OWASP ZAP <https://www.owasp.org/index.php/ZAP> Project leader
>>
>> _______________________________________________
>> OWASP-Leaders mailing list
>> OWASP-Leaders at lists.owasp.org
>> https://lists.owasp.org/mailman/listinfo/owasp-leaders
>>
>>
>    _______________________________________________
> OWASP-Leaders mailing list
> OWASP-Leaders at lists.owasp.org
> https://lists.owasp.org/mailman/listinfo/owasp-leaders
>
>
>
>
> ______________________________________________________________________
>
> The contents of this electronic message, including any attachments, are
> intended only for the use of the individual or entity to which they are
> addressed and may contain confidential information. If you are not the
> intended recipient, you are hereby notified that any use, dissemination,
> distribution, or copying of this message or any attachment is strictly
> prohibited. If you have received this transmission in error, please send an
> e-mail to postmaster at whitehatsec.com and delete this message, along with
> any attachments, from your computer.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.owasp.org/pipermail/owasp-leaders/attachments/20150124/2d950a02/attachment.html>


More information about the OWASP-Leaders mailing list