[Esapi-user] [Esapi-dev] Why ESAPI crypto uses a custom serialization scheme

Chris Schmidt chrisisbeef at gmail.com
Sat May 1 12:07:14 EDT 2010

Kevin and I had several conversations about the use of crypto on the  
client side - the end game is basically that yes, as html5 continues  
to be implemented in browsers, specifically in the area of local  
database and offline webapp functionality,having the abilityto encrypt/ 
decrypt on the client side will become much mire important. One of the  
biggest design issues I came across (which coincedentally I have not  
solved yet) is what to do for managing the keys for said encryption.  
This is also something that I have brought up in emails to the ff,  
chrome, opera, safari, and ie teams via emails. The main problem here  
is that the js sandbox is generally designed to be isolated from the  
operating system layer (with good cause) Which means that precluding  
any 'native' key management solution, keys would have to be stored in  
said sandbox which is bad for many reasons.

Basically the esapi4js has plans on implementing crypto on the client  
side but there is no realistic way to do this reliably and securely at  
this point. On that note I will follow up with the afforementioned  
browser teams to see if any additional thought has been put into the  

Great email Kevin - although I remembered a great deal of the  
reasoning behind that decision it was awesome to see it laid out so  
clearly an having this kind of information documented is important in  
my mind

Sent from my iPwn

On May 1, 2010, at 12:23 AM, Jim Manico <jim.manico at owasp.org> wrote:

> Encrypting on the client is a fundamental - hard core fundamental -
> need as we approach the web 3.0 world of client side apps interacting
> directly with webservices.
> Sure, those of you still playing with HTML forms or web 2.0 AJAX might
> scoff at most client side security controls.
> But those of us who live in the world of an entire massive web app
> only downloading one small HTML page and all other communication to
> the server is via REST/SOAP services - directly from a browser -
> understand the need to do a variety of client side encryption of
> various aspects of the XML/JSON/etc request payload - in order to
> support server-side webservice best practices.
> And I would have called the info above insanity just a few months ago
> (and called out Zalewski from Google on just this topic) until I was
> faced with the same issue myself.
> PS: Kevin, I do believe you "rock the house" and do not let this
> minimize knowing that I have platonic crypto-love for you.
> Cheers,
> Jim Manico
> On Apr 30, 2010, at 7:48 PM, "Jeff Williams" <jeff.williams at aspectsecurity.com
>> wrote:
>> Kevin,
>> Great email! Writing down these design goals makes all the tradeoffs
>> involved in great security controls visible. What a great way to
>> support
>> OWASP's mission.  I'm inspired to write down all the background for
>> Validator and Encoder.  A+
>> --Jeff
>> -----Original Message-----
>> From: esapi-dev-bounces at lists.owasp.org
>> [mailto:esapi-dev-bounces at lists.owasp.org] On Behalf Of Kevin W. Wall
>> Sent: Friday, April 30, 2010 5:03 PM
>> To: ESAPI-Developers; ESAPI-Users; Mike Boberski
>> Subject: [Esapi-dev] Why ESAPI crypto uses a custom serialization
>> scheme
>> As promised... (Or should that be, "as threatened?" ;-)
>> In another thread (Call for review of crypto code), Mike Boberski
>> asked why I chose a custom serialization scheme rather than something
>> like CMS or PKCS#7.
>> Great question. (WARNING: If you fall asleep easily at boring
>> technical
>> details, you may want to grab a cup of coffee.  OTOH, if you are
>> suffering from insomnia, read on. :)
>> First, let me say, I never really seriously considered using PKCS#7.
>> CMS (Cryptographic Message Syntax, RFC 5652) is derived from the
>> latest version of PKCS#7 (v1.5), and since that time there have been
>> 3 or 4 revisions of CMS.  So for the most part PKCS#7 has been
>> superceded
>> by CMS.
>> Secondly, let me state my reason and the design goals for some
>> serialization scheme, whether that be CMS or JSON or XML Encrypt
>> or some other custom serialization scheme.
>> Reason:
>>   We needed a _portable_ way to transport ciphertext over an
>>   insecure communications channel that was independent of OS
>>   or hardware architecture but that would not only allow the
>>   recipient to decrypt it, but also allow the recipient to
>>   detect whether or not an adversary had tampered with the
>>   data stream. (Recall the assumption was that this might
>>   not be transported over a secure channel.)
>> Design Goals:
>>   1)  Portable across different hardware architectures (e.g.,
>>       big-endian vs. little-endian issues)
>>   2)  Portable across programming languages. Should be independent
>>       of size of 'int' types, whether integers are signed or
>> unsigned,
>>       etc.
>>   3)  Should have libraries available (preferably native to the
>>       programming language) or should be easy to build in all that
>>       is required to support it for all programming languages that
>>       ESAPI supports. Where such libraries already exist, they should
>>       be fairly easy to use.
>>   4)  Should be able to be "self-contained" in that it should store
>>       everything that is needed to decrypt it except for the
>> encryption
>>       key itself. This would include the cipher algorithm, the cipher
>>       mode, the padding scheme, the IV, and a MAC to ensure
>> authenticity.
>>   5)  Representation of the encrypted serialized data should be
>> compact
>>       as possible. (At Qwest, there was a *lot* of pushback from
>>       development teams when they discovered that SSN or CC#s took
>>       more space to store when encrypted than as plaintext. Most of
>> this
>>       is because of the padding scheme and IV, but that didn't
>> matter.)
>>   6)  There should be minimal additional processing overhead in
>>       interacting with this encrypted serialized data. Keep in mind
>> there
>>       are applications where they may encrypt / decrypt several
>> million
>>       data items in a tight loop during some batch processing so
>>   7)  Should be extensible and the extensibility should be able to
>> support
>>       backward compatibility with earlier versions.
>> As I started looking into seeing if I could use CMS for this I
>> realized
>> a couple of things.
>>   a) CMS / PKCS#7 are much more complicated than what we needed. It
>> does
>>      much more than to encrypt arbitrary message content.. It can
>> also
>>      be used to support digital signatures, digests, authenticate.,
>> etc.
>>      There have been 3 or 4 revisions of the RFC for this, depending
>> on how
>>      you count. That complexity makes it hard to implement correctly
>> and
>>      it would likely result in incompatibility across different
>> versions
>>      of CMS.
>>   b) I did not find it widely implemented. In Java, the SunJCE has
>> support
>>      for portions of CMS, but they do not have any explicity java.*
>> or
>>      javax.* CMS related classes. (There may be some com.sun.*
>> classes
>>      to support it, but using those directly is probably best avoided
>>      anyway.) I believe that Bouncy Castle has more generic support
>> for
>>      CMS, but we felt that we did not want to rely on any particular
>> JCE
>>      provider as most folks will just wish to stick with SunJCE.
>>   c) After thinking about all the programming languages that ESAPI is
>>      in the works to support (Java, .NET, PHP, classic ASP,
>> ColdFusion/CFML,
>>      Python, Haskell, and whatever language is used on
>> SalesForce.com;
>>      note: JavaScript not included because in general, it would be
>> absurd
>>      to encrypt on the client side in almost all cases), I realized
>> that
>>      something as complex as CMS would never be implemented on all
>> these
>>      languages. But because CMS is so complex (compared to the custom
>>      serialization scheme that I chose), it would be much harder to
>>      implement ESAPI encryption so we could build it in such a manner
>> that
>>      was interoperable across all the ESAPI versions. In fact,
>> implementing
>>      CMS in a language where you do not have an ASN.1 parser
>> available
>>      would make it very difficult to implement straight from the RFC.
>>      BER and DER encoding is used throughout CMS. There quite likely
>> is
>>      some FOSS version of CMS for Java and I think that .NET has at
>> least
>>      partial support for CMS, but implementing this for PHP, Python,
>>      Haskell, etc. would take quite a major effort.
>>   d) In addition to CMS, I also (very briefly) considered XML Encrypt
>> for
>>      serialization. It is a much simpler standard and likely has
>> broader
>>      implementation support than does CMS, but it's big problem is
>> that
>>      its resulting size is huge by comparison to other possible
>>      serialization schemes.
>> So instead, last October, I sought out advice of cryptographers on  
>> the
>> cryptography mailing list on Metzdowd.com. And one cryptographer,
>> Ian Griggs, responded by point me at some similar work that he had
>> done.
>> That became the inspiration for the current custom serialization
>> scheme
>> we are using today.  While that serialization scheme is not
>> implemented
>> on any other programming language of ESAPI today, I do believe that  
>> it
>> is simple enough to implement practically anywhere.
>> Finally, the use of this current custom serialization scheme does not
>> preclude the use of CMS or XML Encrypt or anything else.
>> If anyone has any other specific questions about this, I'll try to
>> answer them as best I can.
>> OK, time to wake up now!!!
>> -kevin
>> -- 
>> Kevin W. Wall
>> "The most likely way for the world to be destroyed, most experts
>> agree,
>> is by accident. That's where we come in; we're computer  
>> professionals.
>> We cause accidents."        -- Nathaniel Borenstein, co-creator of
>> _______________________________________________
>> Esapi-dev mailing list
>> Esapi-dev at lists.owasp.org
>> https://lists.owasp.org/mailman/listinfo/esapi-dev
>> _______________________________________________
>> Esapi-user mailing list
>> Esapi-user at lists.owasp.org
>> https://lists.owasp.org/mailman/listinfo/esapi-user
> _______________________________________________
> Esapi-dev mailing list
> Esapi-dev at lists.owasp.org
> https://lists.owasp.org/mailman/listinfo/esapi-dev

More information about the Esapi-user mailing list