[Esapi-user] RegEx

Jeff Williams jeff.williams at owasp.org
Wed Jan 26 17:11:53 EST 2011


I don't fix hypothetical performance problems.  You're guessing.  And even
if you did find a regex dos problem, the solution isn't to not validate.
The solution is fixing the regex.

Did you know that validating headers makes some web applications faster?
Think I/O-bound, not CPU bound.  These are the things you find out when you
actually test stuff.

I do get your point about only needing validation/encoding when the data is
untrusted.  The problem is that short of RED/BLACK architectures at NSA,
nobody keeps their untrusted data separate from the trusted stuff.  It's an
unscrambling the egg problem.

And you don't have to keep telling me about performance critical
applications. Over the last 12 years, I've worked security on many of the
most performance critical financial systems in existence.

--Jeff


-----Original Message-----
From: esapi-user-bounces at lists.owasp.org
[mailto:esapi-user-bounces at lists.owasp.org] On Behalf Of Jim Manico
Sent: Wednesday, January 26, 2011 4:56 PM
To: Jeff Williams
Cc: ESAPI-Developers; ESAPI Users List
Subject: Re: [Esapi-user] RegEx

I would like to appeal this judgment, your honor. We have additional
supporting evidence that does demonstrate that Regular Expressions can lead
to significant performance problems in some cases. We have new evidence not
brought before the court before. We would also like to challenge the courts
testing methodologies on appeal. Thank you for considering this request.

> My main point is that validation is critically important to security, 
> and people often push back because they perceive a possible 
> performance
problem.
> When you look at the time it takes to connect to a database, execute 
> queries, generate UI components, etc...

You know what Jeff, I agree with the statement above 100%.

> the idea that you can better your
> performance by skimping on validation is dangerous lunacy.

But I do not agree with your definition of lunacy. This might be true 99.99%
of the time. But please do not discount edge cases. There are times when 3-4
layers of RegEx's might be overkill. Saving a few millisecond in some cases
is a positive and important thing. Real time systems, online video games
(multi-billion $ market), etc.

Sometimes, the hammer is best. Other times, you need the scalpel and only
want to validate as minimally as is necessary. Remember, security is not
always the prime driver for an applications success. Depends on the threat
model.

So Jeff, you want to validate every header getting populate by the app
server. I want to sanitize headers only when untrusted data is placed into a
header, or I want to encode only when header data is used (IN SOME
SITUATIONS ONLY). These are both valid based on the situation.

I'm not trying to discount your overall rule, it's solid. I would rather
have good security at the expense of perfect latency. I am talking about
edge cases, not your standard every day web app.

But let's not discount RegExDOS. It's real, and we should explore your test
case more carefully.

I (do not ever) rest my case, your honor.

- Jim





> Come on Jim, this regex DOS isn't a problem with the speed of regex, 
> it's a problem of using safe regex in the first place.
> 
> Also - the example just doesn't work.  I tested the ESAPI 
> Validator.URL regex against the provided attack string a million times 
> and I get the exact same speed as before.  Takes 0. 000295 ms per 
> match operation.  Always check your evidence counselor.  I find you in 
> contempt of court.  Judgement: his honor.  Please see the clerk to pay 
> your fine and court costs. Do not pass go and do not collect $200.
> 
> My main point is that validation is critically important to security, 
> and people often push back because they perceive a possible performance
problem.
> When you look at the time it takes to connect to a database, execute 
> queries, generate UI components, etc...  the idea that you can better 
> your performance by skimping on validation is dangerous lunacy.
> 
> --Jeff
> 
> 
> -----Original Message-----
> From: esapi-user-bounces at lists.owasp.org
> [mailto:esapi-user-bounces at lists.owasp.org] On Behalf Of Jim Manico
> Sent: Wednesday, January 26, 2011 3:50 PM
> To: Jeff Williams
> Cc: ESAPI-Developers; ESAPI Users List
> Subject: [Esapi-user] RegEx
> 
> 
>> A regex doesn't take anywhere close to a tenth of a second.  
> 
> PS: This is absolutely, positively an incorrect statement. (Or worse, 
> a misleading statement).
> 
> The evidence, your honor:
> http://code.google.com/p/owasp-esapi-java/issues/detail?id=158
> 
> Maybe when you get up to "billions of request" you will start using 
> test cases of this nature. (joking)
> 
> ;)  Jim
> _______________________________________________
> Esapi-user mailing list
> Esapi-user at lists.owasp.org
> https://lists.owasp.org/mailman/listinfo/esapi-user
> 

_______________________________________________
Esapi-user mailing list
Esapi-user at lists.owasp.org
https://lists.owasp.org/mailman/listinfo/esapi-user



More information about the Esapi-user mailing list