[Esapi-user] Canonicalization and ESAPI's Input Validation Layer

Wed Sep 3 22:10:44 UTC 2014

Well we have the JSON Sanitizer which could be integrated:

https://www.owasp.org/index.php/OWASP_JSON_Sanitizer

Aloha,
Jim

On 8/31/14, 9:05 PM, Fabio Cerullo wrote:
> Jeff
>
> How difficult would it be to implement a JSON validator in ESAPI? 
> Would that address the concerns raised by Jim?
>
> Thanks
> Fabio
>
> On Saturday, August 30, 2014, Jeff Williams 
> <jeff.williams at aspectsecurity.com 
> <mailto:jeff.williams at aspectsecurity.com>> wrote:
>
>     Hi Jerry,
>
>     Imagine a JSON value containing an escaped quote.  Canonicalizing
>     would unescape it and break the syntax. We don't have a json
>     codec, so this is only a problem because it is very similar to CSS
>     escaping syntax.
>
>     That's why you shouldn't attempt to canonicalize or validate
>     complex data without a specialized library like AntiSamy.  Which
>     is wrapped in the getValidSafeHTML() method in ESAPI.  We don't
>     yet have a method to get valid safe JSON.
>
>     --Jeff
>
>
>     On Aug 30, 2014, at 12:57 PM, "Jerry Hoff" <jerryhoff at gmail.com
>     <javascript:_e(%7B%7D,'cvml','jerryhoff at gmail.com');>> wrote:
>
>>     Interesting discussion gentlemen!
>>
>>     Jim - could you provide an example where canonicalization breaks
>>     html and json input?  I want to make sure I am following correctly.
>>
>>     Thank you,
>>     Jerry
>>
>>     On Aug 30, 2014, at 19:39, Jim Manico <jim.manico at owasp.org
>>     <javascript:_e(%7B%7D,'cvml','jim.manico at owasp.org');>> wrote:
>>
>>>     So what is the line drawn between an attack and acceptably
>>>     encoded data in ESAPI?
>>>
>>>     Canonicalization also "breaks" some input types (html, json,
>>>     etc) so canonicalization of .all. input is equally idiotic.
>>>     (which is why all of ESAPI's IV functions can disable
>>>     canonicalization)
>>>
>>>     --
>>>     Jim Manico
>>>     @Manicode
>>>     (808) 652-3805
>>>
>>>     On Aug 30, 2014, at 5:07 AM, Jeff Williams
>>>     <jeff.williams at aspectsecurity.com
>>>     <javascript:_e(%7B%7D,'cvml','jeff.williams at aspectsecurity.com');>>
>>>     wrote:
>>>
>>>>     There are many situations in which apps get encoded data that
>>>>     does not represent an attack. However, any kind of multiply
>>>>     encoded data is clearly an attack.  So the line we drew in
>>>>     ESAPI is, I think, the right one. We only throw an intrusion
>>>>     detection whenever there is absolutely a clear attack.
>>>>      Otherwise we clean up and try to make the data work.
>>>>
>>>>     Turning off canonicalization, for the vast majority of inputs
>>>>     would be idiotic.  You simply can't reliably validate encoded
>>>>     data. If you follow this plan, the only CVE you will need to
>>>>     worry about is "Canonicalize, Validate, Escape"
>>>>
>>>>     --Jeff
>>>>
>>>>
>>>>     On Aug 30, 2014, at 3:22 AM, "Fabio Cerullo"
>>>>     <fcerullo at gmail.com
>>>>     <javascript:_e(%7B%7D,'cvml','fcerullo at gmail.com');>> wrote:
>>>>
>>>>>     Hi
>>>>>
>>>>>     What about someone who "needs" encoded data to be accepted by
>>>>>     the webapp?
>>>>>
>>>>>     For example, a web form with a field that is used for
>>>>>     completion percentage of something. So in that field you could
>>>>>     input "80% complete".
>>>>>
>>>>>     That would be a legit case of canonicalise first, and then
>>>>>     validate because that value needs to be stored as it is.
>>>>>
>>>>>     I believe the approach taken by ESAPI is: treat everything as
>>>>>     potentially malicious, canonicalise just in case, and then
>>>>>     validate  all entries.
>>>>>
>>>>>     Otherwise, how would you differentiate between an attack and a
>>>>>     valid entry in example above?
>>>>>
>>>>>     Fabio
>>>>>
>>>>>     On Friday, August 29, 2014, Jim Manico <jim.manico at owasp.org
>>>>>     <javascript:_e(%7B%7D,'cvml','jim.manico at owasp.org');>> wrote:
>>>>>
>>>>>         Or even better, if encoded data is detected, even if
>>>>>         things like %80 get rejected, I prefer to reject right
>>>>>         then and there. Pretty sure we Cannonicalize before we
>>>>>         validate though, what code are you looking at?
>>>>>
>>>>>         Aloha,
>>>>>         --
>>>>>         Jim Manico
>>>>>         @Manicode
>>>>>         (808) 652-3805
>>>>>
>>>>>         On Aug 29, 2014, at 6:26 AM, Matt Seil <mseil at acm.org> wrote:
>>>>>
>>>>>>         In the context of your question Jim, I think it makes
>>>>>>         more sense to canonicalize BEFORE you validate--not the
>>>>>>         other way around.  I noticed too, that all the
>>>>>>         "getValidInput" methods call Encoder.canonicalize()
>>>>>>         /after/ validation and meant to ask why we would want
>>>>>>         that approach.
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>         On Wed, Aug 27, 2014 at 6:51 PM, Jim Manico
>>>>>>         <jim.manico at owasp.org> wrote:
>>>>>>
>>>>>>             ESAPI community,
>>>>>>
>>>>>>             I am very concerned about the design of the ESAPI
>>>>>>             validation layer and how it handles canonicalization
>>>>>>             for API's like:
>>>>>>
>>>>>>             |*isValidInput
>>>>>>             <https://owasp-esapi-java.googlecode.com/svn/trunk_doc/latest/org/owasp/esapi/reference/DefaultValidator.html#isValidInput%28java.lang.String,%20java.lang.String,%20java.lang.String,%20int,%20boolean%29>*(java.lang.String context,
>>>>>>             java.lang.String input, java.lang.String type,
>>>>>>             int maxLength, boolean allowNull)|
>>>>>>
>>>>>>             Right now, before validation, ESAPI will try to
>>>>>>             decode user input to it's normalized form and THEN it
>>>>>>             will try to validate the input.
>>>>>>             http://owasp-esapi-java.googlecode.com/svn/trunk/src/main/java/org/owasp/esapi/reference/validation/StringValidationRule.java
>>>>>>
>>>>>>             This seems rather dangerous in that encoded attacks
>>>>>>             will be "fixed" and will not alert of potential
>>>>>>             malicious input. If input is detected to be encoded,
>>>>>>             then I would suggest that we throw an
>>>>>>             IntrusionDetectionException and stop processing. Why
>>>>>>             do we "clean up" encoded data and then validate in
>>>>>>             these API's?
>>>>>>
>>>>>>             Aloha,
>>>>>>             Jim
>>>>>>
>>>>>>             PS: You certainly can disable canonicalization as it
>>>>>>             stands today via:
>>>>>>
>>>>>>             |*isValidInput
>>>>>>             <https://owasp-esapi-java.googlecode.com/svn/trunk_doc/latest/org/owasp/esapi/reference/DefaultValidator.html#isValidInput%28java.lang.String,%20java.lang.String,%20java.lang.String,%20int,%20boolean,%20boolean%29>*(java.lang.String context,
>>>>>>             java.lang.String input, java.lang.String type,
>>>>>>             int maxLength, boolean allowNull,
>>>>>>             _*boolean canonicalize*_)|
>>>>>>
>>>>>>
>>>>>>             _______________________________________________
>>>>>>             Esapi-dev mailing list
>>>>>>             Esapi-dev at lists.owasp.org
>>>>>>             https://lists.owasp.org/mailman/listinfo/esapi-dev
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>         -- 
>>>>>>         Matt Seil
>>>>>>         Software Engineer
>>>>>>         ACM/IEEE
>>>>>
>>>>>     _______________________________________________
>>>>>     Esapi-user mailing list
>>>>>     Esapi-user at lists.owasp.org
>>>>>     <javascript:_e(%7B%7D,'cvml','Esapi-user at lists.owasp.org');>
>>>>>     https://lists.owasp.org/mailman/listinfo/esapi-user
>>>     _______________________________________________
>>>     Esapi-dev mailing list
>>>     Esapi-dev at lists.owasp.org
>>>     <javascript:_e(%7B%7D,'cvml','Esapi-dev at lists.owasp.org');>
>>>     https://lists.owasp.org/mailman/listinfo/esapi-dev
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.owasp.org/pipermail/esapi-user/attachments/20140903/21c50251/attachment.html>