[Esapi-user] [Esapi-dev] ESAPI 2.0 crypto documentation
mike.boberski at gmail.com
Sat Apr 17 17:05:13 EDT 2010
> not having a great deal of awareness about
The exact same is true in terms of validating and encoding, yet all that was
wrapped up into "isValidxx" and "encodeForxx".
Essentially we need one more layer on top of abstraction, not discarding
what's been done, not at all.
That's basically all I have in response to this one... Jim's email is up
On Sat, Apr 17, 2010 at 10:50 AM, Kevin W. Wall <kevin.w.wall at gmail.com>wrote:
> Mike Boberski wrote:
> > I've been putting some thought into ESAPI crypto lately, as I've been
> > working on the PHP port...
> > I think we've been not looking at crypto in as big-picture a way as the
> > other controls have been looked at (I am certainly guilty of that,
> > onto e.g. FIPS 140-related implementation details), and I think the
> > below/something along those lines will further promote the use of ESAPI
> > wrap crypto.
> > In other words, while the 2.0 crypto is technically excellent by anyone's
> > measure, it is not ESAPI-ish (i.e., "make it as easy as possible")
> Well, I agree in part with what you are saying here. _Overall_ (speaking
> more broadly than just the symmetric encryption methods) I would agree
> that some things are not as "easy" as the rest of ESAPI, and probably
> this is true of the functionality related to Encryptor in general.
> (Neither is it as flexible as it should be, but that's a completely
> different topic, so I will not digress here.)
> To a large degree, this apparent complexity in the crypto related
> functionality of ESAPI has to do with the general development community
> not having a great deal of awareness about cryptographic primitives and
> what they are useful for. A good example is the use of the method name
> 'hash' in Encryptor. Most developers are going to think of "hash" as
> something that happens inside a Hashtable or HashMap rather than creating a
> cryptographically-strong, secure, one way message digest. And yet, if
> these same developers would read any of the cryptographic literature--even
> that aimed at the generic development community such as you might
> find in _Communications of the ACM_--you will find that those authors
> use the same terminology. So with crypto, I think there's a larger than
> normal knowledge gap to cross compared to the concepts in the rest of
> > *Discussion item #1. *I think the following function names would be more
> > ESAPI-ish:
> > - protectFromDisclosure
> > - protectFromDisclosureAndModification
> > - protectFromDisclosureWhileAtRest
> > - protectFromDisclosureAndModificationWhileAtRest
> > - protectFromDisclosureWhileInTransit
> > - protectFromDisclosureAndModificationWhileInTransit
> > -
> > - unprotectFromDisclosure
> > - unprotectFromDisclosureAndModification
> > - unprotectFromDisclosureWhileAtRest
> > - unprotectFromDisclosureAndModificationWhileAtRest
> > - unprotectFromDisclosureWhileInTransit
> > - unprotectFromDisclosureAndModificationWhileInTransit
> > - ... maybe then additional qualifier-ish names,
> > e.g. protectFromDisclosureInOracle (maybe output format is in some
> > format for Oracle for example), protectFromDisclosureInPDF (maybe wrap
> > interesting things like this), etc. ...
> > - ... maybe then additional variations,
> > e.g. protectFromDisclosureUsingMaximumKeySize, etc. ...
> I am open to having a discussion on this, however, I think you need to be
> a bit more specific of what methods you are referring to.
> Kindly expound upon the above suggested names if you would, say something
> old_name --> new_name. For example, I might, for the the purposes of this
> discussion at least, presume that you are suggesting that the various
> Encyptor.encrypt() methods be renamed to Encryptor.protectFromDisclosure()
> If that's the case, I would think that the resulting signature like
> CipherText Encryptor.protectFromDisclosure(SecretKey, PlainText)
> also would also need changed to rename the classes CipherText, SecretKey,
> and PlainText to something else as well. And at that point, while you
> _might_ get something that hides the jargon from those with no prior
> exposure to cryptography, you also _might_ risk loosing those who have
> some reasonable fluency with the subject. (Certainly my first reaction
> if I ran across an API with names like this might be to think of the
> interfaces need to be simplified to this extent, then how do I know
> that the "protecting from disclosure" isn't simply something based
> on some home-grown obfuscation. I probably would be reluctant to use
> an API with names like this if I did not almost immediately run across
> some very clear documentation that ensured me that it was using proven
> encryption and not just obfuscation under the hood.)
> Back in 1999, I started developing what 2 years later became a proprietary
> Java class library of something that was very much ESAPI-like. My team has
> been refining this class library and its documentation and supported
> ever since 2001 or so. (We also support a similar .NET implementation with
> corresponding class and method names.) For the first 5 years or so, this
> class library was about 80% cryptographic related and 20% data validation
> related. During that time, we had many discussions, not only amongst our
> many team members (who all started out as developers with no prior security
> or cryptography experience), but also with many dozens of developers from
> the IT department at Qwest. Those discussions resulted in the following
> key observations:
> 1) Most IT developers preferred that we use names that other APIs such
> the Microsoft Cryptography APIs, Suns Java Cryptography Extensions,
> used rather than coming up with something different.
> 2) Make the API defaults secure (e.g,. use AES by default), but allow
> developers to use the API with other various, perhaps less secure,
> cryptographic options (.e.g., DES) in case they need to inter-operate
> with legacy applications or third-party software.
> 3) As much as possible, remove the inherent complexities of
> For example, don't make developers learn about cipher modes, IVs,
> padding schemes, how to validate certificates, etc., unless this
> understanding is essential in accomplishing the desired functionality
> securely. (This is closely related to secure defaults.)
> 4) Do not assume that you know the context of why or where developers
> using your methods. For example, don't have one method to encrypt a
> string and another method to encrypt a URL. Don't have one method to
> encrypt something to store to a file and another to encrypt something
> to store in a database. Instead, the dev teams wanted us to give them
> the necessary primitives and they would wrap our APIs with
> wrappers of their own making.
> 5) Provide dev teams with APIs that will allow them to do things
> For example, give them the ability to encrypt anything on a
> SPARC V9 running Solaris and send that to a little-endian x86 running
> Windows where they can decrypt it and don't make them have to handle
> differences in endian-ness, or signed vs. unsigned. etc.
> Of all these observations, the one that initially surprised me the most was
> #1, followed by #4.
> I think that in some regards, that observations #1 and #4 are related.
> First of
> all, there is this tendency by the development community to think that they
> need to provide some related functionality that your API does not provide
> that at some point in the future, your API may no longer be supported. So
> this tendency to wrap your API (#4) whether it initially makes a lot of
> or not. This application wrapping of your APIs not only allows them to
> provide some application context (e.g., an
> method), but it also provides a layer of de-coupling between their code and
> your API. By doing so, they can easily swap out your API for another one
> at almost anytime. This only makes sense. It would not make much sense for
> us to provide a bunch of "encryptAndSecurelyStoreX()" methods because not
> only are there a lot of common 'X' (SSN, CC#, PII, etc.), but also
> these things are stored differs greatly from application to application.
> Perhaps when we observe that we have a certain level of applications all
> doing the exact same thing in this regard (perhaps because of some specific
> compliance or regulatory issue or because of some InfoSec policy or
> requirement) we will offer to incorporate said functionality into our API,
> but in general we try not to cross that line and know specifically how the
> application will be using our building blocks. Usually we find it best to
> leave the higher level abstractions to the application development team.
> Secondly, I think that this at least in part explains observation #1. By
> using the same terminology of other similar APIs, especially in the
> case of cryptographic terminology, it makes it much easier for them
> to resort to using another API such as MS-CAPI or Sun JCE or Peter
> cryptlib, or openSSL, etc.
> Since most of these other cryptographic APIs also use the terminology of
> the cryptographic community, using these terms in our API serves to
> developers to these concepts so then if they run across an article in
> something like _CACM_ or an IEEE CS proceedings on cryptography they will
> have already been exposed to some of the fundamental terminology and they
> will then find themselves on more familiar ground. So, while using
> domain specific jargon may not seem like the right thing to do in the
> short run, it seems to generally pay off in the long haul.
> Soooo... I am not saying that we should discard what you are suggesting,
> rather that we need to proceed cautiously.
> Perhaps we need both "layers"...one that is more by intent, from a
> perspective, similar to what you suggest and the other using the more
> traditional nomenclature like presently have. That's why I asked you for
> specific 'oldName --> newName' suggestions because it would help me to see
> There certainly are places in ESAPI (and not just related to crypto) that
> we need to think revising names. We've already had a lot of discussion on
> of these lists about this issue. I think that we need to encourage
> discussion in this area. However, we need to also keep in mind that at some
> point we need to come to closure and solidify what is in the official 2.0
> release and push out other plans to either a later point release (2.1) or
> later major release (3.0). That is needed because there are many companies
> who are nervous (mine being one of them) of deploying "release candidate"
> FOSS to a production environment. *BUT*, once we have something in an
> official release, such as the 2.0 GA release, then we need to make a
> to any API changes and follow a standard deprecation policy so we aren't
> yanking the rug out from underneath our user base. I know that you (Mike)
> understand this, but am just mentioning it for those who have not been
> our discussions on deprecation policies, etc. for the past 4 to 6 months or
> > *Discussion item #2. *I think we need to do better with key management;
> > without going into gory detail or proposing something too hard to
> > I think crypto keys need to be associated with User objects as
> > properties/attributes .
> > - Users (likely in many cases users representing server apps) will
> > to be authenticated and logged in with all of the existing ESAPI
> > (logging, intrusion, etc.) before able to access keys
> > - This will make clear associations of keys, allow easily for multiple
> > keys (Encryptor would just use keys of whomever is logged in), and
> > leverage all the existing plumbing.
> I'm glad that you agree with this. I think that this is a good direction
> to head in. In particular, I would prefer that my PII and other sensitive
> information be encrypted with a key that is different than the one
> your key. I've been arguing this point at Qwest since about 2001, but so
> it's fallen mostly on deaf ears and has to do mostly with legacy code and
> legacy reports.
> But there will also be the need to have "per application" keys as well.
> Both are
> needed. At my day job, we almost exclusively use the application keys and
> seldom use per user keys. But I do think that it's a good idea and can take
> using cryptography to the next level.
> Fortunately, now that we have cryptographic primitives that can take a
> object, we have the ability to build out these others. The original
> back in ESAPI 1.4 and carried initially into the early 2.0 release
> that there was only a single master key per application and everything to
> encrypted or decrypted had use that specific key was way too simplistic to
> say the least.
> The down side to all these additional keys is that you need a way to
> manage them.
> Ideally we need to allow ESAPI to be used with some sort of secure key
> management system. I think this is doable, whether it is using something
> like KeyCzar (http://www.keyczar.org/) or
> StrongKey (http://sourceforge.net/projects/strongkey/) or your favorite
> proprietary key management system or something yet to be determined, is,
> well something TBD. But let me assure you that I have been thinking
> about it. In fact, not only have I been thinking about it, but I have
> done something like this (and beyond) at my day job. So it's on the radar,
> but certainly will *NOT* be a part of 2.0.
> > *Discussion item #2. *I think we need to standardize on PKCS#7 (ideally
> > formats
> Am assuming you meant to label this one 'item #3' since the previous
> was labeled '#2'.
> > - Then, include a parameter in e.g. protectFromDisclosure to return
> > binary or base 64.
> > - Perhaps further optimize with functions where this parameter would
> > be hardcoded as mentioned above for the
> > example protectFromDisclosureInOracle
> Yes, I would like to start using more standards such as CMS (RFC 3852) or
> What makes a lot of that difficult is the ambitions to support ESAPI across
> many programming languages. Many of these support C or C++ interfaces where
> there's lots of support for these things, but some perhaps not so much.
> But if we had to implement things like CMS ourselves that can get into
> a bit of scaffold building. And I would like to at least ensure that we
> some level of portability of supporting encryption / decryption across
> programming language versions of ESAPI. It would be presumptuous to assume
> any given IT department only uses on programming language to the exclusion
> all others.
> As far as support for asymmetric cryptographic primitives goes (e.g.,
> signing / validating digital signatures, public key encryption, etc.),
> I'd like to see that functionality extended to be able to use
> X.509 certificates from PKCS#12 key stores. That is much more generic
> than what we presently have and provides a higher degree of assurance
> as well.
> Anyway, good beginnings of a discussion. Keep it going.
> Kevin W. Wall
> "The most likely way for the world to be destroyed, most experts agree,
> is by accident. That's where we come in; we're computer professionals.
> We cause accidents." -- Nathaniel Borenstein, co-creator of MIME
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the Esapi-user