[Esapi-user] [Esapi-dev] ESAPI 2.0 crypto documentation

Jim Manico jim.manico at owasp.org
Sat Apr 17 15:49:17 EDT 2010


I actually think that the ESAPI crypto storage mechanism is overly  
simplified. I will be pushing for a full key management lifecycle  
solution for ESAPI 3.0.

http://www.owasp.org/index.php/Cryptographic_Storage_Cheat_Sheet

Jim

On Apr 17, 2010, at 4:50 AM, "Kevin W. Wall" <kevin.w.wall at gmail.com>  
wrote:

> Mike Boberski wrote:
>> I've been putting some thought into ESAPI crypto lately, as I've been
>> working on the PHP port...
>>
>> I think we've been not looking at crypto in as big-picture a way as  
>> the
>> other controls have been looked at (I am certainly guilty of that,  
>> latching
>> onto e.g. FIPS 140-related implementation details), and I think the
>> below/something along those lines will further promote the use of  
>> ESAPI to
>> wrap crypto.
>>
>> In other words, while the 2.0 crypto is technically excellent by  
>> anyone's
>> measure, it is not ESAPI-ish (i.e., "make it as easy as possible")  
>> enough.
>
> Well, I agree in part with what you are saying here. _Overall_  
> (speaking
> more broadly than just the symmetric encryption methods) I would agree
> that some things are not as "easy" as the rest of ESAPI, and probably
> this is true of the functionality related to Encryptor in general.
> (Neither is it as flexible as it should be, but that's a completely
> different topic, so I will not digress here.)
>
> To a large degree, this apparent complexity in the crypto related
> functionality of ESAPI has to do with the general development  
> community
> not having a great deal of awareness about cryptographic primitives  
> and
> what they are useful for. A good example is the use of the method name
> 'hash' in Encryptor. Most developers are going to think of "hash" as
> something that happens inside a Hashtable or HashMap rather than  
> creating a
> cryptographically-strong, secure, one way message digest. And yet, if
> these same developers would read any of the cryptographic  
> literature--even
> that aimed at the generic development community such as you might
> find in _Communications of the ACM_--you will find that those authors
> use the same terminology. So with crypto, I think there's a larger  
> than
> normal knowledge gap to cross compared to the concepts in the rest of
> ESAPI.
>
>> *Discussion item #1. *I think the following function names would be  
>> more
>> ESAPI-ish:
>>
>>   - protectFromDisclosure
>>   - protectFromDisclosureAndModification
>>   - protectFromDisclosureWhileAtRest
>>   - protectFromDisclosureAndModificationWhileAtRest
>>   - protectFromDisclosureWhileInTransit
>>   - protectFromDisclosureAndModificationWhileInTransit
>>   -
>>   - unprotectFromDisclosure
>>   - unprotectFromDisclosureAndModification
>>   - unprotectFromDisclosureWhileAtRest
>>   - unprotectFromDisclosureAndModificationWhileAtRest
>>   - unprotectFromDisclosureWhileInTransit
>>   - unprotectFromDisclosureAndModificationWhileInTransit
>>   - ... maybe then additional qualifier-ish names,
>>   e.g. protectFromDisclosureInOracle (maybe output format is in  
>> some optimized
>>   format for Oracle for example), protectFromDisclosureInPDF (maybe  
>> wrap
>>   interesting things like this), etc. ...
>>   - ... maybe then additional variations,
>>   e.g. protectFromDisclosureUsingMaximumKeySize, etc. ...
>
> I am open to having a discussion on this, however, I think you need  
> to be
> a bit more specific of what methods you are referring to.
>
> Kindly expound upon the above suggested names if you would, say  
> something
> old_name --> new_name.  For example, I might, for the the purposes  
> of this
> discussion at least, presume that you are suggesting that the various
> Encyptor.encrypt() methods be renamed to  
> Encryptor.protectFromDisclosure() ???
>
> If that's the case, I would think that the resulting signature like
>    CipherText Encryptor.protectFromDisclosure(SecretKey, PlainText)
>
> also would also need changed to rename the classes CipherText,  
> SecretKey,
> and PlainText to something else as well.  And at that point, while you
> _might_ get something that hides the jargon from those with no prior
> exposure to cryptography, you also _might_ risk loosing those who have
> some reasonable fluency with the subject. (Certainly my first reaction
> if I ran across an API with names like this might be to think of the
> interfaces need to be simplified to this extent, then how do I know
> that the "protecting from disclosure" isn't simply something based
> on some home-grown obfuscation. I probably would be reluctant to use
> an API with names like this if I did not almost immediately run across
> some very clear documentation that ensured me that it was using proven
> encryption and not just obfuscation under the hood.)
>
> Back in 1999, I started developing what 2 years later became a  
> proprietary
> Java class library of something that was very much ESAPI-like.  My  
> team has
> been refining this class library and its documentation and supported  
> examples
> ever since 2001 or so. (We also support a similar .NET  
> implementation with
> corresponding class and method names.) For the first 5 years or so,  
> this
> class library was about 80% cryptographic related and 20% data  
> validation
> related. During that time, we had many discussions, not only amongst  
> our
> many team members (who all started out as developers with no prior  
> security
> or cryptography experience), but also with many dozens of developers  
> from
> the IT department at Qwest. Those discussions resulted in the  
> following
> key observations:
>    1) Most IT developers preferred that we use names that other APIs  
> such as
>       the Microsoft Cryptography APIs, Suns Java Cryptography  
> Extensions, etc.
>       used rather than coming up with something different.
>    2) Make the API defaults secure (e.g,. use AES by default), but  
> allow
>       developers to use the API with other various, perhaps less  
> secure,
>       cryptographic options (.e.g., DES) in case they need to inter- 
> operate
>       with legacy applications or third-party software.
>    3) As much as possible, remove the inherent complexities of  
> cryptography.
>       For example, don't make developers learn about cipher modes,  
> IVs,
>       padding schemes, how to validate certificates, etc., unless this
>       understanding is essential in accomplishing the desired  
> functionality
>       securely. (This is closely related to secure defaults.)
>    4) Do not assume that you know the context of why or where  
> developers are
>       using your methods. For example, don't have one method to  
> encrypt a
>       string and another method to encrypt a URL. Don't have one  
> method to
>       encrypt something to store to a file and another to encrypt  
> something
>       to store in a database. Instead, the dev teams wanted us to  
> give them
>       the necessary primitives and they would wrap our APIs with  
> appropriate
>       wrappers of their own making.
>    5) Provide dev teams with APIs that will allow them to do things  
> portably.
>       For example, give them the ability to encrypt anything on a  
> big-endian
>       SPARC V9 running Solaris and send that to a little-endian x86  
> running
>       Windows where they can decrypt it and don't make them have to  
> handle
>       differences in endian-ness, or signed vs. unsigned. etc.
>
> Of all these observations, the one that initially surprised me the  
> most was
> #1, followed by #4.
>
> I think that in some regards, that observations #1 and #4 are  
> related. First of
> all, there is this tendency by the development community to think  
> that they may
> need to provide some related functionality that your API does not  
> provide or
> that at some point in the future, your API may no longer be  
> supported. So there
> this tendency to wrap your API (#4) whether it initially makes a lot  
> of sense
> or not. This application wrapping of your APIs not only allows them to
> provide some application context (e.g., an  
> encryptAndSecurelyStoreCreditCard()
> method), but it also provides a layer of de-coupling between their  
> code and
> your API. By doing so, they can easily swap out your API for another  
> one
> at almost anytime. This only makes sense. It would not make much  
> sense for
> us to provide a bunch of "encryptAndSecurelyStoreX()" methods  
> because not
> only are there a lot of common 'X' (SSN, CC#, PII, etc.), but also  
> where/how
> these things are stored differs greatly from application to  
> application.
> Perhaps when we observe that we have a certain level of applications  
> all
> doing the exact same thing in this regard (perhaps because of some  
> specific
> compliance or regulatory issue or because of some InfoSec policy or  
> business
> requirement) we will offer to incorporate said functionality into  
> our API,
> but in general we try not to cross that line and know specifically  
> how the
> application will be using our building blocks. Usually we find it  
> best to
> leave the higher level abstractions to the application development  
> team.
>
> Secondly, I think that this at least in part explains observation  
> #1. By
> using the same terminology of other similar APIs, especially in the
> case of cryptographic terminology, it makes it much easier for them
> to resort to using another API such as MS-CAPI or Sun JCE or Peter  
> Gutmann's
> cryptlib, or openSSL, etc.
>
> Since most of these other cryptographic APIs also use the  
> terminology of
> the cryptographic community, using these terms in our API serves to  
> introduce
> developers to these concepts so then if they run across an article in
> something like _CACM_ or an IEEE CS proceedings on cryptography they  
> will
> have already been exposed to some of the fundamental terminology and  
> they
> will then find themselves on more familiar ground.  So, while using
> domain specific jargon may not seem like the right thing to do in the
> short run, it seems to generally pay off in the long haul.
>
>
> Soooo... I am not saying that we should discard what you are  
> suggesting, but
> rather that we need to proceed cautiously.
>
> Perhaps we need both "layers"...one that is more by intent, from a  
> functionality
> perspective, similar to what you suggest and the other using the more
> traditional nomenclature like presently have. That's why I asked you  
> for more
> specific 'oldName --> newName' suggestions because it would help me  
> to see that.
>
> There certainly are places in ESAPI (and not just related to crypto)  
> that
> we need to think revising names. We've already had a lot of  
> discussion on both
> of these lists about this issue. I think that we need to encourage  
> continued
> discussion in this area. However, we need to also keep in mind that  
> at some
> point we need to come to closure and solidify what is in the  
> official 2.0
> release and push out other plans to either a later point release  
> (2.1) or
> later major release (3.0). That is needed because there are many  
> companies
> who are nervous (mine being one of them) of deploying "release  
> candidate"
> FOSS to a production environment. *BUT*, once we have something in an
> official release, such as the 2.0 GA release, then we need to make a  
> commitment
> to any API changes and follow a standard deprecation policy so we  
> aren't just
> yanking the rug out from underneath our user base.  I know that you  
> (Mike)
> understand this, but am just mentioning it for those who have not  
> been following
> our discussions on deprecation policies, etc. for the past 4 to 6  
> months or so.
>
>
>> *Discussion item #2. *I think we need to do better with key  
>> management;
>> without going into gory detail or proposing something too hard to  
>> implement,
>> I think crypto keys need to be associated with User objects as
>> properties/attributes .
>>
>>
>>   - Users (likely in many cases users representing server apps)  
>> will need
>>   to be authenticated and logged in with all of the existing ESAPI  
>> plumbing
>>   (logging, intrusion, etc.) before able to access keys
>>   - This will make clear associations of keys, allow easily for  
>> multiple
>>   keys (Encryptor would just use keys of whomever is logged in),  
>> and again
>>   leverage all the existing plumbing.
>
> I'm glad that you agree with this. I think that this is a good  
> direction
> to head in. In particular, I would prefer that my PII and other  
> sensitive
> information be encrypted with a key that is different than the one  
> encrypting
> your key. I've been arguing this point at Qwest since about 2001,  
> but so far,
> it's fallen mostly on deaf ears and has to do mostly with legacy  
> code and
> legacy reports.
>
> But there will also be the need to have "per application" keys as  
> well. Both are
> needed. At my day job, we almost exclusively use the application  
> keys and very
> seldom use per user keys. But I do think that it's a good idea and  
> can take
> using cryptography to the next level.
>
> Fortunately, now that we have cryptographic primitives that can take  
> a SecretKey
> object, we have the ability to build out these others. The original  
> presumption
> back in ESAPI 1.4 and carried initially into the early 2.0 release  
> candidates
> that there was only a single master key per application and  
> everything to be
> encrypted or decrypted had use that specific key was way too  
> simplistic to
> say the least.
>
> The down side to all these additional keys is that you need a way to  
> securely
> manage them.
>
> Ideally we need to allow ESAPI to be used with some sort of secure key
> management system. I think this is doable, whether it is using  
> something
> like KeyCzar (http://www.keyczar.org/) or
> StrongKey (http://sourceforge.net/projects/strongkey/) or your  
> favorite
> proprietary key management system or something yet to be determined,  
> is,
> well something TBD. But let me assure you that I have been thinking
> about it. In fact, not only have I been thinking about it, but I  
> have already
> done something like this (and beyond) at my day job.  So it's on the  
> radar,
> but certainly will *NOT* be a part of 2.0.
>
>
>> *Discussion item #2. *I think we need to standardize on PKCS#7  
>> (ideally CMS)
>> formats
>
> Am assuming you meant to label this one 'item #3' since the previous  
> discussion
> was labeled '#2'.
>
>>
>>   - Then, include a parameter in e.g. protectFromDisclosure to return
>>   binary or base 64.
>>   - Perhaps further optimize with functions where this parameter  
>> would then
>>   be hardcoded as mentioned above for the
>>   example protectFromDisclosureInOracle
>
> Yes, I would like to start using more standards such as CMS (RFC  
> 3852) or
> others.
>
> What makes a lot of that difficult is the ambitions to support ESAPI  
> across so
> many programming languages. Many of these support C or C++  
> interfaces where
> there's lots of support for these things, but some perhaps not so  
> much.
> But if we had to implement things like CMS ourselves that can get  
> into quite
> a bit of scaffold building. And I would like to at least ensure that  
> we have
> some level of portability of supporting encryption / decryption  
> across different
> programming language versions of ESAPI. It would be presumptuous to  
> assume that
> any given IT department only uses on programming language to the  
> exclusion of
> all others.
>
> As far as support for asymmetric cryptographic primitives goes (e.g.,
> signing / validating digital signatures, public key encryption, etc.),
> I'd like to see that functionality extended to be able to use
> X.509 certificates from PKCS#12 key stores.  That is much more generic
> than what we presently have and provides a higher degree of assurance
> as well.
>
>
> Anyway, good beginnings of a discussion. Keep it going.
>
> -kevin
> -- 
> Kevin W. Wall
> "The most likely way for the world to be destroyed, most experts  
> agree,
> is by accident. That's where we come in; we're computer professionals.
> We cause accidents."        -- Nathaniel Borenstein, co-creator of  
> MIME
> _______________________________________________
> Esapi-user mailing list
> Esapi-user at lists.owasp.org
> https://lists.owasp.org/mailman/listinfo/esapi-user


More information about the Esapi-user mailing list