[Esapi-user] ESAPI Technology Compatibility Kit (TCK)

Boberski, Michael [USA] boberski_michael at bah.com
Wed Jan 13 08:43:59 EST 2010

I'll throw my hat in the ring for your consideration. This would be helpful for me to have for the PHP port for example. There are places where we're needing to go off-road for language-specific reasons, in those circumstances it would be particularly helpful to make sure the security checks/effects are normalized based on a common spec.

Tangential, related to terminology/concepts throughout the email chain below, I'd like to continue to draw people's attention to this document: http://www.owasp.org/images/8/82/Esapi-design-patterns.pdf

Vote Mike.

Mike B.

From: esapi-user-bounces at lists.owasp.org [mailto:esapi-user-bounces at lists.owasp.org] On Behalf Of Jeff Williams
Sent: Wednesday, January 13, 2010 12:13 AM
To: mike.boberski at gmail.com; Dinis Cruz
Cc: rob.spremulli+esapi at gmail.com; ESAPI-Developers; esapi-user at lists.owasp.org
Subject: [Esapi-user] ESAPI Technology Compatibility Kit (TCK)

(Pulled off of SC-L)

I suggest that we should *not* call this the ESTAPI.  It's confusing and inaccurate.  As I've mentioned before, I really think the right model for this is the Java Technology Compatibility Kit (TCK).  http://jcp.org/en/resources/tdk.

I think separating the interfaces from the concrete classes within the Java implementation is a weak attempt to achieve this. The right approach is to really write a technology-independent specification and create the rest of the TCK.  This should be relatively easy since we have several different working implementations.  Incidentally, writing the spec fairly late in the process is the recipe for most successful technologies.

Is anyone interested in leading this important effort (with support from the rest of the team)?  I'm seeking someone with the right combination of technical, managerial, and writing skills to pull this off effectively.


From: esapi-user-bounces at lists.owasp.org [mailto:esapi-user-bounces at lists.owasp.org] On Behalf Of Mike Boberski
Sent: Tuesday, January 12, 2010 7:47 PM
To: Dinis Cruz
Cc: rob.spremulli+esapi at gmail.com; ESAPI-Developers; esapi-user at lists.owasp.org; SC-L at securecoding.org
Subject: Re: [Esapi-user] [Esapi-dev] Recommending ESAPI?

> we start to create standards for how Security Controls should behave [and basically the rest of the post]

I submit ASVS for your consideration. If one is further concerned about building blocks in the environment, check out Common Criteria and FIPS 140-2.


There have also been discussions about creating standardized test suites for ESAPI implementations to ensure consistent security checks/effects across ESAPI language implementations, but I don't think that's what you're getting at.

It's not hard (with respect) to differentiate interfaces from reference implementations from adapters (customized controls), please see the design patterns doc I wrote that's posted to the project page. I'm not sure I see advantages to further rearranging and splitting out the interfaces.



On Tue, Jan 12, 2010 at 7:38 PM, Dinis Cruz <dinis.cruz at googlemail.com<mailto:dinis.cruz at googlemail.com>> wrote:
My view is that the key to make this work is to create the ESTAPI, which is the Enterprise Security Testing API

This way we would have (for every language):

 *   ESAPI Interfaces - which describe the functionality that each security control should have
 *   ESTAPI - Unit Tests that check the behaviour of the security controls
 *   ESAPI Reference Implementation(s) - Which are (wherever possible) 'production ready' versions of those security controls (and  in most cases a one-to-one mapping to the ESAPI Interfaces)
 *   Framework XYZ ESAPI 'connectors' - Which wrap (or expose) the security controls defined in the ESAPI Interfaces in Framework XYZ
What I really like about this world, is that we (Application Security Consultants) we start to create standards for how Security Controls should behave. and (as important) are able to work with the Framework developers without they felling that ESAPI is a 'competitor' to they Framework. After all, the way we will really change the market is when the Frameworks used by the majority of developers adopt ESAPI (or its principles)

Of course that the Framework developers are more than welcomed to grab large parts (or even all) of the code provided by the ESAPI reference implementation(s). But the key is that they (the framework developers) must: a) take ownership of the code and b) respect the ESAPI Interfaces.

And hey, if the Framework developers decide NOT to implement a particular security control, that is fine too.


I would at least expect them to provide detailed information why they made that decision and why they chose NOT to implement or support it (which would allow us (Security community) to respectably agree or disagree with their choices (hey for some Frameworks, being insecure is a feature :) )

Finally, In addition to all the advantages that we will have when frameworks adopt these security controls, there is one that for me is probably the MOST important one: An 'ESAPI compliant app' (which btw is a term we still have to agree what exactly means), is an app that is providing explicit information about where they (the developers) think their (the app) security controls are located.

In another works, via the ESAPI Interfaces (and the ESTAPI tests) the developers are actually telling us (the security consultants):
  a) what they think their application's attack surface is and
  b) what is the security behaviour that they have already tested for

Of course that they can game the system, which is why we (Security Consultants) will still be needed (we will also need to make sure that they implemented the security controls properly). But compare that to today's (2009) world, were we are lucky to get an up-to-date application diagram and a reasonable accurate description of how the application was actually coded and behaves.

This would also (finally) give the application security tools (white, black, glass, gray, pink, blue) a fighting change to automatically, or operator-driven, understand what is going on and report back:
  - what it knows (security vulnerabilities) and (as important)
  - what it doesn't know / understand
(ok there is a lot more that these tools will provide us (for example ESTAPI tests) but that is a topic for another post)

So, for me, the key added value of the ESAPI Interfaces, is that it will provide us (Security Consultants) a way to understand how the app works (from a security point of view) and to be able to finally be able to give the clients what they want: Visibility, Assurance and the ability to make 'knowledgeable Risk-based decisions'.

Dinis Cruz

2010/1/12 Jim Manico <jim.manico at owasp.org<mailto:jim.manico at owasp.org>>
Very well said.

On this note, I think we may wish to consider formally splitting the interfaces from the reference implementation. We could then build a test framework that's tests those interfaces - so we can verify different implementations of ESAPI. Expand this out in a cross-language way, and we have some serious magic to work with.

This is Dinis' idea, but I'm starting to see the light.

- Jim

An FYI from personal experience, be extra careful with the dependencies, particularly if you develop on an appserver that optimized for debug and production.

You may need these libraries even if you are not using the area of the ESAPI RI that uses them.  The -Xverify:none JVM argument changes how the classloader pre-caches some classes, particularly Exceptions.  Despite not needing to use safe file upload capabilities, without that JVM arg is was looking for Exceptions found in the commons-uploads jar

On Tue, Jan 12, 2010 at 6:54 AM, Jim Manico <jim.manico at owasp.org<mailto:jim.manico at owasp.org>> wrote:
You know Dinis, when I first read your email I was bit offended. Same with much of John Stevens' email.

But you know? You are trying to help us. These kinds of pragmatic questions need to be answered.

So here goes.
Following the recent thread on Java 6 security and ESAPI, I just would like to ask the following clarifications:

1) For an existing web application currently using a MVC framework (like Spring or Struts) are we today (9th Jan 2009) officially recommending that this web application development team adds OWASP's ESAPI.jar to the list of 'external' APIs (i.e. libs) they use, support and maintain?

I can personally attest for ESAPI 2.0 rc4 integration into Struts 1.3.x, where I've used ESAPI for several years, from the early days. I'm not deeply familiar with Spring. I would not say this is a trivial exercise, but it certainly is possible.

2) When adopting the OWASP ESAPI's J2EE implementation, is ESAPI.jar ALL they need to add? or are there other dependencies (i.e. jars) that also need to be added, supported and maintained? (for example on the 'Dependencies' section of the ESAPI Java EE<http://www.owasp.org/index.php/Category:OWASP_Enterprise_Security_API#tab=Java_EE> page (i.e. Tab) it seems to imply that there are other *.jars needed)

ESAPI.jar has significant dependencies - something that is a problem, in general, in the Java world. I'm optimistic about the new Java 7 component framework - but that is a long way off.  In the meantime:
(from http://www.owasp.org/index.php/Category:OWASP_Enterprise_Security_API#tab=Java_EE)

There are no dependencies on the ESAPI interfaces other than standard Java EE. However, the reference implementation does have dependencies that are detailed below. The reference implementation takes advantage of a few existing libraries. This list may not be totally complete.

 *   DefaultAccessController needs...

    *   Commons-Configuration 1.5

 *   DefaultValidator needs...

    *   AntiSamy 1.2 (there may be a few transitive dependencies here)

    *   NekoHTML 0.9.5
    *   Xerces 2.9.1

 *   Log4J Logger needs...

    *   Log4j 1.2.12

 *   DefaultHTTPUtilities needs...

    *   Commons-FileUpload 1.2

 *   WAF needs

    *   XOM 1.0 (there may be a few transitive dependencies here)

    *   Commons-FileUpload 1.2

3) Where can I find detailed information about each of the 9 Security Controls that ESAPI.jar currently supports: 1) Authentication, 2) Access control, 3) Input validation, 4) Output encoding/escaping, 5) Cryptography, 6) Error handling and logging, 7) Communication security, 8) HTTP security and 9) Security configuration? (I took this list of controls from the Introduction to ESAPI pdf)<http://www.owasp.org/images/8/81/Esapi-datasheet.pdf>

Detailed from a marketing perspective? :) The best technical information is our Javadoc pages at http://owasp-esapi-java.googlecode.com/svn/trunk_doc/2.0-rc4/index.html which are not complete, but are fairly decent. We have also been very good about answering questions, fast, on esapi-users and esapi-dev. But you are right - docs are evolving, but we need more.
4) When adopting EASPI.jar, are we recommending that the developers should adopt or retrofit their existing code on the areas affected by those 9 Security Controls? (i.e. code related to: Authentication, Access control, Input validation, Output encoding/escaping, Cryptography, Error handling and logging, Communication security, HTTP security and Security configuration)
It really depends on the situation. But I get your point - I've seen the Validator, Encoder, Utils and Error Handling modules used in retrofitting situations successfully. I'm not so sure about the others.

5) Should we recommend the adoption of ALL 9 Security Controls? or are there some controls that are not ready today (9 Jan 2009) for production environments and should not be recommended? (for example is the 'Authentication' control as mature as the 'Error handling and logging' control?)
I personally grade the reference 2.0 implementation as follows:
1) Authentication   C (Needs deeper enterprise integration)
2) Access control   B- (This is just a really tough issue, and usually requires deep application-specific context. Plus we have some good ideas on the table from Beef that I'd like to consider)
3) Input validation   A- (needs better messaging and internationalization (thanks Sklarew for making us think in the right direction about this)
4) Output encoding/escaping   A (Go Jeff, my only A. :) We do need a performance tuning pass (easy) and DOM XSS encoding functions)
5) Cryptography  A- (Great work Kevin, this is a huge huge improvement from 1.4)
6) Error handling and logging  B+ (Nice work on designing this from Wichers)
7) Communication security ?
8) HTTP security B- (Great utilities! I'd like to see some of these decoupled a bit more)
9) Security configuration ?

Digging deeper....
I personally use almost all of ESAPI. I've written my own Hibernate Authentication layer - but it's very specific to my data model. It's very difficult to decouple this from my app and would be difficult to donate it to the project effectively. Same with access control. My data model is VERY complex, and donating it without SQL scripts, hibernate configuration, and a whole lot of other code - is a great challenge. (Not to mention that my employer owns the code ;) The flat-file authenticator is just a proof of concept and should never be used in a production environment of any kind, IMO. The thread-local nature of the authenticator, while I use it and love it, needs to be reconsidered since other classes, like the loggers, depend on it. Error handling is fairly solid - and is only a thin layer on top of known logging methods + security specific messaging. The encoder was handed down from Gosling himself - given to Jeff - who gifted it to us. :) I want the encoder to be a hard-coded part of ESAPI. :) The validator and encoder can be dropped into any project fairly easy. Same with much of the HTTP Utils. The Encryptor from 1.4 should be avoided, which impacts other portions of the codebase. http://owasp-esapi-java.googlecode.com/svn/trunk/documentation/esapi4java-core-2.0-readme-crypto-changes.html  . 2.0 is going to be a very big milestone; I'm pretty stoked about what I'm seeing from the team.

Most importantly, it's easy to use the ESAPI configuration layer to over-ride any of the reference implementation with your personal authenticator or access controller (so long as you implement the ESAPI interfaces), as I have for my projects.

6) Are there commercial (i.e. paid) support services available for the companies who want to add ESAPI.jar to they application?

I hesitate to mention this, and I'm not trying to pimp - but I'm respectfully answering all of your questions. Aspect offers these services. I've been working with Jeff on some of those efforts. It's working out well for Aspects clients, I'd dare say. If someone else wishes to speak up on this topic, please do. Open.

7) What is the version of ESAPI.jar that we should recommend? the version 1.4 (which looks like a stable release) or the version 2.0 rc4 (which looks like it is a Release Candidate)
ESAPI 1.4.1 is very far behind 2.0 rc4.  Java 4 is way past end of lifecycle - but it's still in very wide use, so we plan to back-port all of ESAPI 2.0 to 1.4. Or at least as much as we can. I'm making some changes this week and plan on releasing 1.4.2 this week.

8) Where can I find the documentation of where and how ESAPI should be used? More importantly, where can I find the information of how it CAN NOT or SHOULD NOT be used (i.e. the cases where even when the EASPI.jar are used, the application is still vulnerable)
Yea, Docs. We need more docs. Boberski has done incredible work in this area.

9) if there list of companies that have currently added ESAPI.jar to their applications and have deployed it? (i.e. real world usage of EASPI)
Under "users and adopters" http://www.owasp.org/index.php/Category:OWASP_Enterprise_Security_API#tab=Contributors.2FUsers

10) Has the recommended ESAPI.jar (1.4 or 2.0 rc4) been through a security review? and if so where can I read its report?
Yes and see #8 sentence 2.

11) when Jim says "... you can build a new secure app without an ESAPI. But libs like OWASP ESAPI will get you there faster and cheaper....",  do we have peer-reviewed data that suports this claim?
Nope. I'm shooting from the hip and I consider this as common sense. But I agree, we REALLY need more assurance evidence that is published on the wiki - perhaps we should run o2 against the ESAPI codebase for starters. Or maybe someone can donate code review services and publish that report on our wiki. I hear you. Assurance, published assurance, is fundamental.

12) Is there a roadmap or how-to for companies that wish to adopt ESAPI.jar on an a) new application or b) existing real-world application'?
See #8 sentence 2.

13) What about the current implementations of ESAPI for the other languages. Are we also recommending their use?
Most are beta or alpha - with sparkles of 1.0. But I'd love to hear the other language leaders chime in here. I focus on the Java version of ESAPI.

14) If a development team decides to use (for example) Spring and ESAPI together in their (new or existing) application, what are the recommended 'parts' from each of those APIs (Spring and EASPI) that the developers should be using? (for example: a) use Encoding from ESAPI, b) use Authentication from Spring, c) use Authorization from ESAPI, d) use Error Handling from Spring, e) use Logging from ESAPI, etc...)

I just don't know how to answer this question. I think for starters, the completeness of our encoder helps stop XSS cold in a way that is a bit better than the frameworks. And Jeff authorer a great cheat sheet to go alongside it. http://www.owasp.org/index.php/XSS_%28Cross_Site_Scripting%29_Prevention_Cheat_Sheet


Dinis Cruz

I'm not going to shy away from these emails any longer. Is this all you got, Dinis? John Steven? Bring it on, I'll do my best to answer as honestly as I can.

But let me tell you, Dinis. I would not consider building any Java app without ESAPI. :) (please note the "I" statement - I've been deep in the code for years, I'm not saying its easy - it requires significant investment of time to use all of ESAPI as it stands today).

Another 18 hour day - I need sleep. :)

- Jim

Esapi-dev mailing list
Esapi-dev at lists.owasp.org<mailto:Esapi-dev at lists.owasp.org>


Jim Manico

OWASP Podcast Host/Producer


Esapi-user mailing list
Esapi-user at lists.owasp.org<mailto:Esapi-user at lists.owasp.org>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: https://lists.owasp.org/pipermail/esapi-user/attachments/20100113/863a3419/attachment.html 

More information about the Esapi-user mailing list