[Esapi-user] [Esapi-dev] Canonicalization and ESAPI's Input Validation Layer

Fabio Cerullo fcerullo at owasp.org
Thu Sep 4 09:49:27 UTC 2014


Great stuff... can we do it then?

I think would be a nice addition to ESAPI.

Fabio


On Thu, Sep 4, 2014 at 10:38 AM, Jim Manico <jim.manico at owasp.org> wrote:

> It's one function call.... easy to integrate.
>
>
> --
> Jim Manico
> @Manicode
> (808) 652-3805
>
> On Sep 3, 2014, at 11:37 PM, Fabio Cerullo <fcerullo at owasp.org> wrote:
>
> I meant the integration with ESAPI.. would Mike be interested to do it?
>
> Fabio
>
>
> On Thu, Sep 4, 2014 at 10:32 AM, Jim Manico <jim.manico at owasp.org> wrote:
>
>> It's usable today...
>>
>>
>> --
>> Jim Manico
>> @Manicode
>> (808) 652-3805
>>
>> On Sep 3, 2014, at 10:33 PM, Fabio Cerullo <fcerullo at owasp.org> wrote:
>>
>> Ah... cool. Would it be something that Mike Samuel (Proj leader) might be
>> interested to do?
>>
>> Thanks,
>> Fabio
>>
>>
>> On Wed, Sep 3, 2014 at 11:10 PM, Jim Manico <jim.manico at owasp.org> wrote:
>>
>>>  Well we have the JSON Sanitizer which could be integrated:
>>>
>>> https://www.owasp.org/index.php/OWASP_JSON_Sanitizer
>>>
>>> Aloha,
>>> Jim
>>>
>>>
>>>
>>> On 8/31/14, 9:05 PM, Fabio Cerullo wrote:
>>>
>>> Jeff
>>>
>>>  How difficult would it be to implement a JSON validator in ESAPI?
>>> Would that address the concerns raised by Jim?
>>>
>>>  Thanks
>>> Fabio
>>>
>>> On Saturday, August 30, 2014, Jeff Williams <
>>> jeff.williams at aspectsecurity.com> wrote:
>>>
>>>>  Hi Jerry,
>>>>
>>>>  Imagine a JSON value containing an escaped quote.  Canonicalizing
>>>> would unescape it and break the syntax. We don't have a json codec, so this
>>>> is only a problem because it is very similar to CSS escaping syntax.
>>>>
>>>>  That's why you shouldn't attempt to canonicalize or validate complex
>>>> data without a specialized library like AntiSamy.  Which is wrapped in the
>>>> getValidSafeHTML() method in ESAPI.  We don't yet have a method to get
>>>> valid safe JSON.
>>>>
>>>> --Jeff
>>>>
>>>>
>>>> On Aug 30, 2014, at 12:57 PM, "Jerry Hoff" <jerryhoff at gmail.com> wrote:
>>>>
>>>>   Interesting discussion gentlemen!
>>>>
>>>>  Jim - could you provide an example where canonicalization breaks html
>>>> and json input?  I want to make sure I am following correctly.
>>>>
>>>>  Thank you,
>>>> Jerry
>>>>
>>>> On Aug 30, 2014, at 19:39, Jim Manico <jim.manico at owasp.org> wrote:
>>>>
>>>>   So what is the line drawn between an attack and acceptably encoded
>>>> data in ESAPI?
>>>>
>>>>  Canonicalization also "breaks" some input types (html, json, etc) so
>>>> canonicalization of •all• input is equally idiotic. (which is why all of
>>>> ESAPI's IV functions can disable canonicalization)
>>>>
>>>>  --
>>>> Jim Manico
>>>> @Manicode
>>>> (808) 652-3805
>>>>
>>>> On Aug 30, 2014, at 5:07 AM, Jeff Williams <
>>>> jeff.williams at aspectsecurity.com> wrote:
>>>>
>>>>   There are many situations in which apps get encoded data that does
>>>> not represent an attack. However, any kind of multiply encoded data is
>>>> clearly an attack.  So the line we drew in ESAPI is, I think, the right
>>>> one. We only throw an intrusion detection whenever there is absolutely a
>>>> clear attack.  Otherwise we clean up and try to make the data work.
>>>>
>>>>  Turning off canonicalization, for the vast majority of inputs would
>>>> be idiotic.  You simply can't reliably validate encoded data. If you follow
>>>> this plan, the only CVE you will need to worry about is "Canonicalize,
>>>> Validate, Escape"
>>>>
>>>> --Jeff
>>>>
>>>>
>>>> On Aug 30, 2014, at 3:22 AM, "Fabio Cerullo" <fcerullo at gmail.com>
>>>> wrote:
>>>>
>>>>  Hi
>>>>
>>>>  What about someone who "needs" encoded data to be accepted by the
>>>> webapp?
>>>>
>>>>  For example, a web form with a field that is used for completion
>>>> percentage of something. So in that field you could input "80% complete".
>>>>
>>>>  That would be a legit case of canonicalise first, and then validate
>>>> because that value needs to be stored as it is.
>>>>
>>>>  I believe the approach taken by ESAPI is: treat everything as
>>>> potentially malicious, canonicalise just in case, and then validate
>>>>  all entries.
>>>>
>>>>  Otherwise, how would you differentiate between an attack and a valid
>>>> entry in example above?
>>>>
>>>>  Fabio
>>>>
>>>> On Friday, August 29, 2014, Jim Manico <jim.manico at owasp.org> wrote:
>>>>
>>>>>  Or even better, if encoded data is detected, even if things like %80
>>>>> get rejected, I prefer to reject right then and there. Pretty sure we
>>>>> Cannonicalize before we validate though, what code are you looking at?
>>>>>
>>>>>  Aloha,
>>>>> --
>>>>> Jim Manico
>>>>> @Manicode
>>>>> (808) 652-3805
>>>>>
>>>>> On Aug 29, 2014, at 6:26 AM, Matt Seil <mseil at acm.org> wrote:
>>>>>
>>>>>     In the context of your question Jim, I think it makes more sense
>>>>> to canonicalize BEFORE you validate--not the other way around.  I noticed
>>>>> too, that all the "getValidInput" methods call Encoder.canonicalize()
>>>>> *after* validation and meant to ask why we would want that approach.
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On Wed, Aug 27, 2014 at 6:51 PM, Jim Manico <jim.manico at owasp.org>
>>>>> wrote:
>>>>>
>>>>>> ESAPI community,
>>>>>>
>>>>>> I am very concerned about the design of the ESAPI validation layer
>>>>>> and how it handles canonicalization for API's like:
>>>>>>
>>>>>> *isValidInput
>>>>>> <https://owasp-esapi-java.googlecode.com/svn/trunk_doc/latest/org/owasp/esapi/reference/DefaultValidator.html#isValidInput%28java.lang.String,%20java.lang.String,%20java.lang.String,%20int,%20boolean%29>*(java.lang.String context,
>>>>>> java.lang.String input, java.lang.String type, int maxLength,
>>>>>> boolean allowNull)
>>>>>>
>>>>>> Right now, before validation, ESAPI will try to decode user input to
>>>>>> it's normalized form and THEN it will try to validate the input.
>>>>>> http://owasp-esapi-java.googlecode.com/svn/trunk/src/main/java/org/owasp/esapi/reference/validation/StringValidationRule.java
>>>>>>
>>>>>> This seems rather dangerous in that encoded attacks will be "fixed"
>>>>>> and will not alert of potential malicious input. If input is detected to be
>>>>>> encoded, then I would suggest that we throw an IntrusionDetectionException
>>>>>> and stop processing. Why do we "clean up" encoded data and then validate in
>>>>>> these API's?
>>>>>>
>>>>>> Aloha,
>>>>>> Jim
>>>>>>
>>>>>> PS: You certainly can disable canonicalization as it stands today via:
>>>>>>
>>>>>>   *isValidInput
>>>>>> <https://owasp-esapi-java.googlecode.com/svn/trunk_doc/latest/org/owasp/esapi/reference/DefaultValidator.html#isValidInput%28java.lang.String,%20java.lang.String,%20java.lang.String,%20int,%20boolean,%20boolean%29>*(java.lang.String context,
>>>>>> java.lang.String input, java.lang.String type, int maxLength,
>>>>>> boolean allowNull, *boolean canonicalize*)
>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> Esapi-dev mailing list
>>>>>> Esapi-dev at lists.owasp.org
>>>>>> https://lists.owasp.org/mailman/listinfo/esapi-dev
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>>  Matt Seil
>>>>> Software Engineer
>>>>>  ACM/IEEE
>>>>>
>>>>>    _______________________________________________
>>>> Esapi-user mailing list
>>>> Esapi-user at lists.owasp.org
>>>> https://lists.owasp.org/mailman/listinfo/esapi-user
>>>>
>>>>    _______________________________________________
>>>> Esapi-dev mailing list
>>>> Esapi-dev at lists.owasp.org
>>>> https://lists.owasp.org/mailman/listinfo/esapi-dev
>>>>
>>>>
>>>
>>> _______________________________________________
>>> Esapi-dev mailing list
>>> Esapi-dev at lists.owasp.org
>>> https://lists.owasp.org/mailman/listinfo/esapi-dev
>>>
>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.owasp.org/pipermail/esapi-user/attachments/20140904/23deddd4/attachment.html>


More information about the Esapi-user mailing list