[Esapi-dev] Question about ESAPI encoding

Wed Sep 22 21:23:45 EDT 2010

I agree, August. ++  

Bringing ESAPI to "them" is a great way to speed adoption. We offer low
level utilities that any framework should be able to utilize in some way.

- Jim

From: esapi-dev-bounces at lists.owasp.org
[mailto:esapi-dev-bounces at lists.owasp.org] On Behalf Of augustd
Sent: Tuesday, September 21, 2010 7:07 AM
To: Rohit Sethi
Cc: ESAPI-Developers
Subject: Re: [Esapi-dev] Question about ESAPI encoding

I've been thinking that ESAPI output encoding needs to be integrated
directly into the major frameworks. For example you could have Struts and
Spring taglibs with ESAPI already baked in. 

Then the developer need only call (In Struts) <html:text
value="<whatever>"/> as they normally would, and behind the scenes, the tag
automatically runs encodeForHTMLAttribute(<whatever>) when rendering the tag
body.  

Simply swapping out the original struts-taglib-x.x.x.jar for the ESAPI
enabled version would give you an instant leg up against XSS. 

-August 

On Mon, Sep 20, 2010 at 12:33 PM, Rohit Sethi <rklists at gmail.com> wrote:

Agreed Jeff. It's just so much easier to say "apply the correct
context Esapi encoding tag" rather than worrying about if the ESAPI
encoding will break the tag output.

Jim, I have to admit that not being an ESAPI developer I don't know
how much complexity this will create for you. I recognize that there
are probably edge cases I haven't yet thought of and that this will
make testing more difficult. Let me know how best I can help.

On Mon, Sep 20, 2010 at 12:11 PM, Jeff Williams
<jeff.williams at aspectsecurity.com> wrote:
> I think the problem that Rohit is focused on is that developers might try
the ESAPI encoders only to find that their output is double encoded because
their GUI framework encodes some stuff and not other stuff. Then they'll
abandon ESAPI ultimately continuing the hamster-wheel-of-XSS that we're
currently experiencing.
>
> Even if we provide double-encoding protection, it won't really solve this
problem. Developers wouldn't be able to encode first and pass into framework
second. That'll produce double-encoding.  They could modify the framework to
call this new method, and then at least it won't break stuff.
>
> [Rohit - do you agree?]
>
> Ultimately, the right solution is to make GUI frameworks that encode
right.  That's when XSS dies.
>
> --Jeff
>
>
> -----Original Message-----
> From: Jim Manico [mailto:jim.manico at owasp.org]
> Sent: Monday, September 20, 2010 2:26 PM
> To: 'Rohit Sethi'; 'Jeff Williams'; Jeff Williams; 'ESAPI-Developers'
> Subject: RE: [Esapi-dev] Question about ESAPI encoding
>
> My only concern is that this addition may make our output encoding more
> fragile. There is chance that we may add edge-case encoding errors if we
get
> overly crazy with changes in this direction.
>
> I just ask that we add even more complex unit tests to the encoder (yes
> Jeff, I said) to account for this added complexity.
>
> To be honest, I don't care if developers double-encode. That's their bad
> code. (Maybe those .NET guys double encode a lot, but I don't see it a lot
> in the Java world ;)
>
> I just care that we stop XSS. And I do not want double-escape protection
to
> sacrifice any of the existing simplicity and security power of our
Encoder.
>
> Fair?
>
>  - Jim
>
> -----Original Message-----
> From: esapi-dev-bounces at lists.owasp.org
> [mailto:esapi-dev-bounces at lists.owasp.org] On Behalf Of Rohit Sethi
> Sent: Monday, September 20, 2010 5:17 AM
> To: Jeff Williams; Jeff Williams; ESAPI-Developers
> Subject: Re: [Esapi-dev] Question about ESAPI encoding
>
> I think its an option on existing methods, and an optional attribute
> for the tag libraries. 'DoubleEscapeProtection' or something similar
> to the anti-xss library would work.
>
> In order to preserve backwards compatibility I'd set the default for
> this option to false, and advise users through the javadoc about the
> benefits of turning it to true.
>
> On 9/20/10, Jeff Williams <jeff.williams at owasp.org> wrote:
>> I totally agree that the taglib issue is a big deal.
>>
>> Your idea here is good. If we had a way to escape for a particular
context
>> without fear of double escaping, we could prevent most screwed up content
>> and stop XSS.
>>
>> I was hoping to take this one step further, and really clean up the data
>> before we send it to a browser. That's why I'm focused on canonicalize.
> I've
>> seen too many cases where a downstream process decodes and "reactivates"
> an
>> attack. But I'm willing to let this go in order to get the big benefit
> now.
>>
>> Let's get the API worked out! Do you think this is a new method, an
option
>> on the existing escaping methods, or something else?
>>
>> Thanks,
>>
>> --Jeff
>>
>>
>> -----Original Message-----
>> From: esapi-dev-bounces at lists.owasp.org
>> [mailto:esapi-dev-bounces at lists.owasp.org] On Behalf Of Rohit Sethi
>> Sent: Monday, September 20, 2010 1:33 AM
>> To: Jeff Williams
>> Cc: ESAPI-Developers
>> Subject: Re: [Esapi-dev] Question about ESAPI encoding
>>
>> I would certainly like to leverage the existing codecs.
>>
>> My objection to the canonicalize method is exactly the problem you
>> discussed earlier - suppose a developer wants to show HTML syntax and
>> outputs &amp;lt; - the canonicalized version would incorrectly remove
>> the double-encoding. The result would be &lt; which is not what the
>> developer intended. Correct me if I'm wrong here - I'm trying to
>> understand this based on the current canonicalize method in  the
>> DefaultEncoder class.
>>
>> I think that canonicalizing all output prior to encoding will probably
>> work correctly in most cases, but will definitely break in some cases
>> (such as the example you articulated). The approach I'm suggesting
>> will do exactly what the codec was intended to do (i.e. output
>> HTML/HTML Attribute/JavaScript/CSS escaped data  - without concern if
>> that data has other, mixed encodings within it) albeit with the
>> provision that it won't re-encode something that's already been
>> escaped for that particular output type (e.g. HTML codec will not
>> re-encode HTML entities).
>>
>> This would be a major boon because we could apply it seamless to most
>> tag libraries that may output untrusted data without concern about
>> double encoding and breaking the data. Without it, we have to expect
>> developers to know the default encoding behavior of every tag library
>> they use. Sure, some diligent developers will go out of their way to
>> research this but I'm sure many others will just abandon trying to
>> escape all together because they run into pesky double encoding.
>>
>> On Sun, Sep 19, 2010 at 9:56 PM, Jeff Williams
>> <jeff.williams at aspectsecurity.com> wrote:
>>> You're right that it *shouldn't* be overly complex.  It just is.  There
>>> are roughly 70 different ways to encode the < character using the escape
>>> formats you mentioned.
>>>
>>> I suggest you consider using the existing ESAPI codecs to do what you're
>>> suggesting. What is your objection to using the existing canonicalize
>>> method - which handles this already?
>>>
>>> --Jeff
>>>
>>>
>>> -----Original Message-----
>>> From: esapi-dev-bounces at lists.owasp.org
>>> [mailto:esapi-dev-bounces at lists.owasp.org] On Behalf Of Rohit Sethi
>>> Sent: Monday, September 20, 2010 12:32 AM
>>> To: Jeff Williams; ESAPI-Developers
>>> Subject: Re: [Esapi-dev] Question about ESAPI encoding
>>>
>>> (Joined the esapi-dev list on a different email account)
>>>
>>> Jeff, I'm proposing that doing this for just the html, html attribute,
>>> javascript, and css codecs *shouldnt* be overly complex. I'm also
>>> confused on how this idea would break in multiple / nested encodings
>>> anymore than the current codecs already do.
>>>
>>> Maybe the best way to do this is to test it out .. I'd like to
>>> volunteer for this but I can't realistically say when I'll get around
>>> to it. If anyone else cares to do this in the interim I'd be much
>>> obliged.
>>>
>>>
>>>
>>> On 9/19/10, Jeff Williams <jeff.williams at owasp.org> wrote:
>>>> Hi Rohit, I think we're proposing essentially the same thing, except
>>> that
>>>> your approach doesn't work for all the different escaping formats
>>> because it
>>>> would be prohibitively complex to write a parser to handle them all.
>>> The
>>>> ESAPI Codecs are intentionally very simple to hide some of this
>>> complexity.
>>>> It also won't really do what you want if there are double or nested
>>> encoding
>>>> scenarios.  You'll end up with messed up HTML.  My assertion is that
>>> 99% of
>>>> the time, you want to canonicalize fully, then escape properly.
>>>>
>>>>
>>>>
>>>> --Jeff
>>>>
>>>>
>>>>
>>>> From: Sethi, Rohit [mailto:rohit at securitycompass.com]
>>>> Sent: Sunday, September 19, 2010 3:15 PM
>>>> To: 'jeff.williams at owasp.org'; 'jim at manico.net'
>>>> Subject: Re: Question about ESAPI encoding
>>>>
>>>>
>>>>
>>>> Jeff, forgive my ignorance on the subject. I'm not sure I understand
>>> the
>>>> need to canonicalize prior to encoding. If there was a simple option
>>> to not
>>>> encode existing escaped sequences for the specific codec in question
>>> then I
>>>> believe this - without canonicalization - will still achieve the goal.
>>> For
>>>> example looking at &amp;lt; the encoder would spot "&amp;", avoid
>>> encoding
>>>> it, and the characters rendered in the browser would be "&lt;" which
>>> is the
>>>> display html syntax you discussed below.
>>>>
>>>> I think I'm missing your point about nested encoding schemes. In what
>>>> scenario would the above solution fail with nested encodings?
>>>>
>>>>
>>>>
>>>>   _____
>>>>
>>>> From: Jeff Williams <jeff.williams at owasp.org>
>>>> To: James Manico <jim at manico.net>
>>>> Cc: Sethi, Rohit
>>>> Sent: Sat Sep 18 21:37:24 2010
>>>> Subject: Re: Question about ESAPI encoding
>>>>
>>>> The right way to do this is to canonicalize before encode. The
>>> approach
>>>> suggested won't work if there are multiple or nested encoding schemes
>>> used
>>>> in the data. ESAPI used to do exactly this. But there was some
>>> pushback
>>>> because a site that wants to display html syntax (for example) won't
>>> work
>>>> anymore as I requires double escaping.  I like the idea of making an
>>> option
>>>> for this at the risk of confusing some.  Actually I like better the
>>> idea of
>>>> making this the default and providing an option to allow multiple
>>> encoding
>>>> of a single type.
>>>>
>>>> --Jeff
>>>>
>>>>
>>>>
>>>> Jeff Williams
>>>>
>>>> Aspect Security
>>>>
>>>> work: 410-707-1487
>>>>
>>>> main: 301-604-4882
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> On Sep 18, 2010, at 7:46 PM, James Manico <jim at manico.net> wrote:
>>>>
>>>> Jeff,
>>>>
>>>>
>>>>
>>>> Any thoughts here sir? Good idea from Rohit....
>>>>
>>>>
>>>>
>>>> Rohit, can I send this to the ESAPI-dev list?
>>>>
>>>>
>>>>
>>>> - Jim
>>>>
>>>>
>>>>
>>>> From: Sethi, Rohit [mailto:rohit at securitycompass.com]
>>>> Sent: Thursday, September 16, 2010 5:07 AM
>>>> To: James Manico
>>>> Subject: Question about ESAPI encoding
>>>>
>>>>
>>>>
>>>> Jim, when creating the ESAPI encoding  has anyone thought about adding
>>> an
>>>> optional configuration to avoid double encoding? I believe Microsoft's
>>>> AntiXSS library has this concept.
>>>>
>>>>
>>>> Basically, when encoding if I find the existence of a potential escape
>>>> character (e.g. &) then I traverse the next few characters to see if
>>> it
>>>> matches a white list &lt; gt; &quot; &amp; , etc. and
>>> &#<digit><digit>;  -
>>>> in those cases I avoid encoding those specific haracters. This allows
>>> us to
>>>> safely wrap any tag library that might output HTML without worrying
>>> about
>>>> double encoding.
>>>>
>>>>
>>>> You could provide equivalent functionality to other codecs such as
>>> XML,
>>>> HTMLAttributes, JavaScript, etc.
>>>>
>>>>
>>>>
>>>> Rohit Sethi
>>>>
>>>> Director, Professional Services
>>>>
>>>> Security Compass
>>>>
>>>> http://www.securitycompass.com
>>>>
>>>> Direct : 888-777-2211 ext. 102
>>>>
>>>> Mobile: 732.546.4473
>>>>
>>>> Twitter: rksethi
>>>>
>>>>
>>>>
>>>>
>>> ************************************************************************
>>> ************************************************************************
>>> **************
>>>>
>>>> The information in this email is confidential and may be legally
>>> privileged.
>>>> Access to this email by anyone other than the intended addressee is
>>>> unauthorized.  If you are not the intended recipient of this message,
>>> any
>>>> review, disclosure, copying, distribution, retention, or any action
>>> taken or
>>>> omitted to be taken in reliance on it is prohibited and may be
>>> unlawful. If
>>>> you are not the intended recipient, please reply to or forward a copy
>>> of
>>>> this message to the sender and delete the message, any attachments,
>>> and any
>>>> copies thereof from your system.
>>>>
>>>>
>>> ************************************************************************
>>> ************************************************************************
>>> **************
>>>>
>>>>
>>>>
>>>>
>>>
>>> --
>>> Sent from my mobile device
>>>
>>> Rohit Sethi
>>> Security Compass
>>> http://www.securitycompass.com
>>> twitter: rksethi
>>> _______________________________________________
>>> Esapi-dev mailing list
>>> Esapi-dev at lists.owasp.org
>>> https://lists.owasp.org/mailman/listinfo/esapi-dev
>>>
>>
>>
>>
>> --
>> Rohit Sethi
>> Security Compass
>> http://www.securitycompass.com
>> twitter: rksethi
>> _______________________________________________
>> Esapi-dev mailing list
>> Esapi-dev at lists.owasp.org
>> https://lists.owasp.org/mailman/listinfo/esapi-dev
>>
>>
>
> --
> Sent from my mobile device
>
> Rohit Sethi
> Security Compass
> http://www.securitycompass.com
> twitter: rksethi
> _______________________________________________
> Esapi-dev mailing list
> Esapi-dev at lists.owasp.org
> https://lists.owasp.org/mailman/listinfo/esapi-dev
>
>

--
Rohit Sethi
Security Compass
http://www.securitycompass.com
twitter: rksethi
_______________________________________________
Esapi-dev mailing list
Esapi-dev at lists.owasp.org
https://lists.owasp.org/mailman/listinfo/esapi-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: https://lists.owasp.org/pipermail/esapi-dev/attachments/20100922/5d69e8e6/attachment-0001.html 