[Esapi-user] Do we need to enable canonicalization in Java?

Jeff Williams jeff.williams at aspectsecurity.com
Mon Nov 10 14:33:23 UTC 2014


Incorrect.  Java EE does one level of URL decoding on *some* parts of the HTTP request.  It does not decode HTML entities, Javascript, Database, or any other type of encoding.

As I mentioned before, there’s no need to use Encoder.canonicalize() on its own.

If you’re using ESAPI.isValid* methods, it does canonicalization under the hood.  I recommend using getValid* so that you can use the canonical form downstream.

Maybe that’s what you’re seeing?  Good luck,

--Jeff

From: Zenz Daniel [mailto:daniel.zenz at lotterien.at]
Sent: Monday, November 10, 2014 8:52 AM
To: Jeff Williams; Eduardo Macarron; esapi-user at lists.owasp.org
Subject: AW: [Esapi-user] Do we need to enable canonicalization in Java?

Hi Jeff,

what you are saying is 100% correct. And I am fully aware of that.

I just made the experience that a Java EE server does this canonicalization automatically. We had the canonicalization from ESAPI in our code, but it just made too much and the data was corrupted - so we removed it and we made several tests with multi and different encoding. As long as it was a valid (multi-)encoding, we got the decoded values from our JEE server.

Therefor I think, in real world JEE it is not needed to do the canonicalization by hand, because it is already done by the server.

Regards, Daniel

Von: Jeff Williams [mailto:jeff.williams at aspectsecurity.com]
Gesendet: Freitag, 07. November 2014 17:09
An: Zenz Daniel; Eduardo Macarron; esapi-user at lists.owasp.org<mailto:esapi-user at lists.owasp.org>
Betreff: RE: [Esapi-user] Do we need to enable canonicalization in Java?

I think you aren’t fully understanding.   The appserver may do one level of URL decoding, depending on the type of request and part of the HTTP request involved.  But that’s not what we are talking about here.

Attackers can encode their attacks, using a variety of encoding and escaping formats.  They can use repeated encodings, or nested encodings, or multiple different encodings to hide their attacks.   I once enumerated over 80 legal ways to encode the “<” character.  That’s just the single encodings – not multiple.

The attacker’s goal is to bypass your validation by encoding the data so that it looks innocuous.  Depending on where that data goes in your application, it can get decoded.  For example, say the data gets URL decoded by the container, then it goes to a database that unescapes ‘’ and eventually it gets to the command line or RSS parser/generator that unencodes the data again.  I’ve seen three-level decoding attacks before and there are probably deeper ones.

The defense is canonicalization.

--Jeff


From: Zenz Daniel [mailto:daniel.zenz at lotterien.at]
Sent: Friday, November 07, 2014 6:32 AM
To: Jeff Williams; Eduardo Macarron; esapi-user at lists.owasp.org<mailto:esapi-user at lists.owasp.org>
Subject: AW: [Esapi-user] Do we need to enable canonicalization in Java?

Hi,

as far as I understood the question from Eduardo, he wants to know if Java (application server) provides the data from the request already canonicalized, or if it is necessary to canonicalize it “by hand”.

My experience is that, in a Java EE environment, it is not needed to do the canonicalization, because it is already done by the application server.

But this is one of the questions I ask myself very often, if it is really not needed in Java EE. So I am not 100% sure about that.

Regards, Daniel


Von: esapi-user-bounces at lists.owasp.org<mailto:esapi-user-bounces at lists.owasp.org> [mailto:esapi-user-bounces at lists.owasp.org] Im Auftrag von Jeff Williams
Gesendet: Montag, 03. November 2014 21:05
An: Eduardo Macarron; esapi-user at lists.owasp.org<mailto:esapi-user at lists.owasp.org>
Betreff: Re: [Esapi-user] Do we need to enable canonicalization in Java?

Hi Eduardo,

I’m not sure if anyone answered your question.  For some background, read this article, “The Only Two Things Every Developer Needs to Know About Injection<http://www.darkreading.com/application-security/the-only-2-things-every-developer-needs-to-know-about-injection/a/d-id/1269091>.”

I didn’t get into canonicalization in the article, but it’s critically important.  The reason is that attacks can be encoded into many forms.  To prove the point, I once encoded an attack in Morse Code<http://www.zdnet.com/blog/security/morse-code-rickroll-0-day-no-seriously-i-mean-it/1071>.

The point is that encoded data might easily pass your validation mechanism, only to be transformed later by a decoder into an attack.  The solution is to canonicalize first, then validate, and use the canonical form everywhere downstream.

If you’re using ESAPI.isValid* methods, it does canonicalization under the hood.  I recommend using getValid* so that you can use the canonical form downstream.

--Jeff


From: esapi-user-bounces at lists.owasp.org<mailto:esapi-user-bounces at lists.owasp.org> [mailto:esapi-user-bounces at lists.owasp.org] On Behalf Of Eduardo Macarron
Sent: Monday, October 27, 2014 2:17 AM
To: esapi-user at lists.owasp.org<mailto:esapi-user at lists.owasp.org>
Subject: [Esapi-user] Do we need to enable canonicalization in Java?

Hello everybody in the list.

We are adding ESAPI 2.x to a Spring MVC+Spring Security+MyBatis application.

We only want ESAPI for XSS protection (Canonicalize, Validate, Encode). Not for SQL injection, authentication or authorization.

To implement the XSS protection we are validating inputs with calls to Validator.isValid* methods.

We are not encoding output with ESAPI because input data is supposed to be trusted after validation and also because Spring does some encoding by default.

My question is about canonicalization. Sorry if this same question has been made millions of times. I have not been able to find a good reply to it yet.

Do we need canonicalization?

I can not understand how an encoded input can be a threat. Can anybody point to a sample of an attack using encoded data in Java?

thank you!!!

Österreichische Lotterien Ges.m.b.H., Rennweg 44, A-1038 Wien
FN 54472 g, Handelsgericht Wien, DVR-Nr.: 0476706, UID ATU 15666508
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.owasp.org/pipermail/esapi-user/attachments/20141110/01df1850/attachment.html>


More information about the Esapi-user mailing list