[Esapi-user] [Fwd: Problems with Logging and a response to Encrypting the parameters of a URL]

Kevin W. Wall kevin.w.wall at gmail.com
Sun Sep 5 00:05:40 EDT 2010


Sean,

I'm CC: the ESAPI lists on this response as well as forwarding your attachment
because for context and also because I am not as handy with log4j configuration
as some of the other ESAPI developers. (I get most of it to do what I want
by trial and error when I need to get something in log4j working.)  For ESAPI,
generally, all I work with is this minimal default log4j.xml file which
*prior to* 2.0-rc7 used to be included in the ESAPI jar file:

    <?xml version="1.0" encoding="UTF-8" ?>
    <!DOCTYPE log4j:configuration SYSTEM "log4j.dtd">
    <!-- main resources -->
    <log4j:configuration xmlns:log4j="http://jakarta.apache.org/log4j/">
      <appender name="console" class="org.apache.log4j.ConsoleAppender">
        <param name="Target" value="System.out"/>
        <layout class="org.apache.log4j.PatternLayout">
          <param name="ConversionPattern" value="%-5p %m%n"/>
        </layout>
      </appender>
      <root>
        <priority value ="debug" />
        <appender-ref ref="console" />
      </root>
    </log4j:configuration>

But anyhow, I'm hoping some folks on this list can better answer
your answer question about about Log4j use with ESAPI.
> How do I customize ESAPI for logging? I am not able to customize the
> output, customize appenders, and output to a file (I only receive
> console output). The following is my configuration. [See 'Original Message'
> below -kww]
> (Does ESAPI only
> look at the ESAPI.properties file and ignore the log4j.properties file?
> Is there a precedence)

I do know that all ESAPI 1.4 & 2.0 versions earlier than 2.0-rc7 had
either a log4j.properties or log4j.xml included in the esapi jar file.
(It was removed in the latest 2.0-rc7 jar.) But that might be what is
causing your problem. If you didn't specify the  -Dlog4j.configuration=
on the java command line or as a System property, it is probably using
the one built into ESAPI rather than yours. That is one reason that
we removed it...it was just too confusing for people. If that does not
fix your problem, please repost to the ESAPI lists for assistance as
that's about the extent of my expertise.


OK, now onto your first comment

> The ESAPI API, I feel, is not flexible in terms of encrypting a String
> or numbers. Encrypting String and number objects in the Jasypt API is
> easier. (I might try URL rewriting with Struts2 for my problem but I am
> not sure.)

I have never used Jasypt. I did look at it a l-o-n-g time ago and decided
that it was no more secure than the encryption in ESAPI 1.4 because it
encrypts text using PBE (password based encryption), which by default for
the standard SunJCE at least uses ECB mode and also password-based encryption
is vulnerable to dictionary attacks. ESAPI 1.4 had this problem and I was on
a crusade (some would say) to fix that as well as to address the whole
issue of ensuring message authenticity. Personally, if you are looking for
an alternative to ESAPI's encryption, I would recommend that you use
KeyCzar instead. It at least is encryption done correctly.

Secondly, if your goal is to encrypt a number as  a number so that when you
decrypt it, you automagically *know* that it is an integer or float or long
or whatever, then I would agree. At the most basic level, encryption using block
ciphers and block cipher modes such as CBC or ECB is about encrypting bytes, so
the generally it is up to the developer to be aware that a number is being
encrypted and once it is decrypted, to convert those bytes properly back into
the number. In other words, any data serialization is the responsibility of
the developer; it is not something that ESAPI encryption handles. ESAPI only
attempts to provide simple encryption _primitives_.

That said, have you tried the hex-encoding rather than the base64-encoding
as I suggested?  So does anyone else who has the patience to read through
the this thread and knows more about Struts 2 than me (and that should include
most of you!) can offer an explanation to Simon why the same code that works
stand-alone is failing when run using Struts 2. I speculated what was happening
was that Struts 2 was somehow URL-encoding the base64-encoded '/', '+', and '='
characters and that might be messing up things, but it is just that...
speculation. Can anyone offer a better explanation?

Thanks,
-kevin

-------- Original Message --------
Subject: Problems with Logging and a response to Encrypting the parameters of a URL
Date: Sat, 04 Sep 2010 11:21:08 -0600
References: <4C5F4AF5.5010103 at greymonk.net> <4C5F58B6.7080908 at gmail.com>
<4C5F6DB9.4010405 at greymonk.net> <4C60C7D6.6000609 at gmail.com>
<4C610BC2.8050200 at greymonk.net> <4C61489B.2060605 at gmail.com>
<436998748.299233.1281535647977.JavaMail.open-xchange at oxusltgw12.schlund.de>
<4C62BA96.8060301 at gmail.com> <4C707919.40509 at greymonk.net>
<4C71F241.3080803 at gmail.com> <4C747552.2000401 at greymonk.net>
<4C7479C3.7060001 at gmail.com> <4C7B04C8.4060509 at gmail.com>

Kevin --

The ESAPI API, I feel, is not flexible in terms of encrypting a String
or numbers. Encrypting String and number objects in the Jasypt API is
easier. (I might try URL rewriting with Struts2 for my problem but I am
not sure.)

How do I customize ESAPI for logging? I am not able to customize the
output, customize appenders, and output to a file (I only receive
console output). The following is my configuration. (Does ESAPI only
look at the ESAPI.properties file and ignore the log4j.properties file?
Is there a precedence)

Environment:

Linux (Ubuntu 10.04)
Java 1.6.0_20
ESAPI 2.0r6
log4j 1.2.12

ESAPI.properties:

# Log4JFactory Requires log4j.xml or log4j.properties in classpath -
http://www.laliluna.de/log4j-tutorial.html
ESAPI.Logger=org.owasp.esapi.reference.Log4JLogFactory
# ESAPI.Logger=org.owasp.esapi.reference.JavaLogFactory

#===========================================================================

# ESAPI Logging
# Set the application name if these logs are combined with other
applications
Logger.ApplicationName=Simon
# If you use an HTML log viewer that does not properly HTML escape log
data, you can set LogEncodingRequired to true
Logger.LogEncodingRequired=false
# Determines whether ESAPI should log the application name. This might
be clutter in some single-server/single-app environments.
Logger.LogApplicationName=false
# Determines whether ESAPI should log the server IP and port. This might
be clutter in some single-server environments.
Logger.LogServerIP=true
# LogFileName, the name of the logging file. Provide a full directory
path (e.g., C:\\ESAPI\\ESAPI_logging_file) if you
# want to place it in a specific directory.
Logger.LogFileName=results.log
# MaxLogFileSize, the max size (in bytes) of a single log file before it
cuts over to a new one (default is 10,000,000)
Logger.MaxLogFileSize=10000000
# Logging level, supported values are OFF, FATAL, ERROR, WARNING (ESAPI
default), INFO, DEBUG, TRACE, ALL
Logger.LogLevel=ERROR

log4j properties file (log4j.properties - is on the classpath):

# Set root logger level to DEBUG and its only appender to A1.
log4j.rootLogger=OFF, A1

# A1 is set to be a ConsoleAppender.
log4j.appender.A1=org.apache.log4j.ConsoleAppender

# A1 uses PatternLayout.
log4j.appender.A1.layout=org.apache.log4j.PatternLayout
log4j.appender.A1.layout.ConversionPattern=%-4r [%t] %-5p %c %x - %m%n

Implementation of logger in Java:

Please see attached.

Thanks again Kevin,
Sean

On 10-08-29 07:09 PM, Kevin W. Wall wrote:
> Sean,
>
> Any of these suggestions help?
> -kevin
>
> Kevin W. Wall wrote:
>   
>> I'd recommend using hex encoding. If you use base64-encoding, which is
>> a bit more efficient, then you will also have to URL encode that as
>> '+', '/', and '=' are valid base64 chars but are special to URLs.
>>
>> There's a good example of this in the file
>> 	src/examples/java/PersistedEncryptedData.java
>> that I just committed last night. You can get it from the ESAPI SVN
>> repository using this link:
>> <http://code.google.com/p/owasp-esapi-java/source/browse/trunk/src/examples/java/PersistedEncryptedData.java>
>>
>> One thing that may be happening is that Struts 2 is tweaking the values
>> so as to try to encode it itself somehow. By you explicitly encoding it
>> using hex, hopefully it would prevent that.  You did make sure that it's
>> not being rejected by a regex in one of your Struts validators, right?
>>
>> I've never used Struts 2 other than just playing around w/ it briefly.
>> However, if you are *still* unable to get it working, if you can send me
>> a small example of some the Struts 2 source code of how you intend to use it
>> along with instructions to compile to a war file (e.g., send w/ either ant's
>> build.xml or Maven's pom.xml and tell me how to execute to build a .war file),
>> then I will gladly try to drop that war file in my local Tomcat and see if I can
>> get it working for you.
>>
>> That's about the best I can do. I do know that Strut's does some odd / unusual
>> things with encoding. There was a JavaEE servlet filter that my team wrote at
>> Qwest and it broke the filter because some encoding issues and the way it was
>> wrapping servlet request / responses.
>>
>> -kevin
>> sean wrote:
>>     
>>> Hi Kevin --
>>>
>>> Thanks for your continuing assistance on this problem. Unfortunately,
>>> the problem is not solved.
>>> I should explain exactly what I would like to accomplish and any
>>> constraints placed on the problem.
>>>
>>> I am using Struts2. I wish to encrypt parameters used in a URL. I want
>>> to decrypt those values from the URL parameters when the URL is clicked
>>> by the user. When Struts2 executes an action, it puts variables into a
>>> value stack. I need to have the encrypted value as a String for the URL.
>>> When a user clicks on the URL, I need to take the parameter values that
>>> are on the value stack and decrypt them. (I can avoid the value stack
>>> but it makes the code messy and kinda violates
>>> the benefits of using Struts2.) I can't use a byte[] array.
>>>
>>> So... is there a way to convert a byte[] array into a String variable,
>>> use the String variable, and convert the String variable back into a
>>> byte[] array and then decrypt it?
>>>
>>> eg.
>>>
>>>         PlainText plainText = new PlainText( "2" );
>>>         CipherText ct = ESAPI.encryptor().encrypt( plainText );
>>>         byte[] encryptedBytes = ct.asPortableSerializedByteArray();
>>>         String encodedAndEncrypted = new String( encryptedBytes, "UTF-8" );
>>>        
>>>         lg.info( Logger.EVENT_SUCCESS,
>>> "-------------------------------------------------" );
>>>         lg.info( Logger.EVENT_SUCCESS, encodedAndEncrypted );
>>>         lg.info( Logger.EVENT_SUCCESS,
>>> "-------------------------------------------------" );
>>>        
>>>         CipherText restoredCipherText =
>>> CipherText.fromPortableSerializedBytes( encodedAndEncrypted.getBytes() );
>>>         PlainText returnedPT = ESAPI.encryptor().decrypt(
>>> restoredCipherText );
>>>        
>>>         lg.info( Logger.EVENT_SUCCESS,
>>> "-------------------------------------------------" );
>>>         lg.info( Logger.EVENT_SUCCESS, returnedPT.toString() );
>>>         lg.info( Logger.EVENT_SUCCESS,
>>> "-------------------------------------------------" );
>>>
>>> On 10-08-22 10:00 PM, Kevin W. Wall wrote:
>>>       
>>>> Hi Sean. Sorry for the brief delay in getting back to you, but I had
>>>> a few other things that I was trying to get finished for 2.0 RC7 release.
>>>>
>>>> I did manage to figure out your problem. You had
>>>> 	byte[] encryptedBytes = ct.asPortableSerializedByteArray();
>>>> 	String ciphertext = new String( encryptedBytes, "UTF-8" );
>>>> 	byte[] byteArray = ciphetext.getBytes( "UTF-8" );
>>>> 	CipherText restoredCipherText =
>>>> 		CipherText.fromPortableSerializedBytes(byteArray);
>>>> 	PlainText returnedPT = ESAPI.encryptor().decrypt( restoredCipherText );
>>>>
>>>> and then printed returnedPT.toString() with the expectation that it should
>>>> equal "1".
>>>>
>>>> If you simply take out that encoding to/from a UTF-8 encoding string, it
>>>> will work as expected. I.e., you want to change it to this:
>>>>
>>>> 	PlainText plainText = new PlainText( "1" );
>>>> 	CipherText ct = ESAPI.encryptor().encrypt( plainText );
>>>> 	byte[] encryptedBytes = ct.asPortableSerializedByteArray();
>>>> 	CipherText restoredCipherText =
>>>> 		CipherText.fromPortableSerializedBytes( encryptedBytes);
>>>> 	PlainText returnedPT = ESAPI.encryptor().decrypt( restoredCipherText );
>>>>
>>>> and then print returnedPT.toString() for example,
>>>>
>>>>         System.out.println("Decrypted value: " + returnedPT );
>>>>
>>>> you will see that it is equal to "1" as expected.
>>>>
>>>> I've attached the modified source. I compiled it and tested your original
>>>> program as sent to me, and got the same error message that you did. Then
>>>> I modified it as above, recompiled, and reran it and it worked as expected.
>>>> (Note: I did this with the 'src/test/resources/.esapi/ESAPI.properties' both
>>>> times, but that shouldn't matter. You could do the same to test it.)
>>>>
>>>> CipherText.asPortalSerializedByteArray already return an UTF-8 encoded
>>>> byte array. One would think that changing this into a UTF-8 encoded string
>>>> and then back to a UTF-8 encoded byte array would have no affect, but it
>>>> seems to. I have not tested if/where your 'byteArray' and the 'encryptedBytes'
>>>> byte array differ, but it appears that they are.
>>>>
>>>> Anyhow, try out the modified source and see if that works for you. I am
>>>> also planning on having an example of showing how to persist an encrypted
>>>> file and read it back for 2.0 RC7, so you can look at that. It will be
>>>> under 'src/examples/java'.
>>>>
>>>> -kevin
>>>>
>>>> sean wrote:
>>>>   
>>>>         
>>>>> Hello Kevin --
>>>>>
>>>>> Unfortunately my application is not working as intended. I have used
>>>>> your suggestions as described below.
>>>>>
>>>>> Attached are two files; the application as Simon.java and a log file as
>>>>> results.log.
>>>>>
>>>>> Let me know what you think.   
>>>>>
>>>>> Thanks again,
>>>>> -- Sean
>>>>>
>>>>> On 10-08-11 08:58 AM, Kevin W. Wall wrote:
>>>>>     
>>>>>           
>>>>>> sean at greymonk.net wrote:
>>>>>>   
>>>>>>       
>>>>>>             
>>>>>>> Hi Kevin --
>>>>>>>
>>>>>>>
>>>>>>> I was able to successfully complete the following section, "Encrypting /
>>>>>>> Decrypting with the New Methods – Storing Encrypted Data" from ESAPI 2.0
>>>>>>> Symmetric Encryption User Guide. 
>>>>>>>
>>>>>>>
>>>>>>> I didn't know how to take a String object to be used as URL, created by
>>>>>>> converting a byte[] to a String object, and convert it back into a CypherText
>>>>>>> object to be used to convert back into a PlainText object. I don't know how to
>>>>>>> convert the byte[] array, from ct.asPortableSerializedByteArray(), to something
>>>>>>> that can be used as a URL.
>>>>>>>     
>>>>>>>         
>>>>>>>               
>>>>>> That part is pretty easy.
>>>>>> 	Convert byte[] to String --->
>>>>>> 		byte[] byteArray = ...;
>>>>>> 		...
>>>>>> 		String str = new String(byteArray, "UTF-8");
>>>>>>
>>>>>> 	Convert String to byte[] --->
>>>>>> 		String str = ...;
>>>>>> 		...
>>>>>> 		byte[] byteArray = str.getBytes("UTF-8");
>>>>>>
>>>>>> 	Convert CipherText to byte[] --->
>>>>>> 		CipherText ct = ...;
>>>>>> 		...
>>>>>> 		byte[] encryptedBytes = ct.asPortableSerializedByteArray();
>>>>>>
>>>>>> 	Convert byte[] to CipherText (assuming byte array created as above) --->
>>>>>> 		byte[] encryptedBytes = ...;	// as above
>>>>>> 		...
>>>>>> 		CipherText ciphertext =
>>>>>> 			CipherText.fromPortableSerializedBytes(encryptedBytes);
>>>>>>
>>>>>> I still will write up a more complete example, but right now my first priority
>>>>>> is to get this bash script done to create the distribution .zip file.
>>>>>>
>>>>>> BTW, one last thing. When doing byte[] <--> String conversion, make sure
>>>>>> that you use "UTF-8". That way, it will work across different OSes. If you
>>>>>> just use the default encoding, and then transfer something from (say) Windows
>>>>>> to Solaris, it will fail because each use a different native encoding.
>>>>>>
>>>>>> HTH,
>>>>>> -kevin
>>>>>>   
>>>>>>       
>>>>>>             
>>>>>     
>>>>>           
>>>>   
>>>>         
>>>       
>>
>>     
>
>   



-- 
Kevin W. Wall
"The most likely way for the world to be destroyed, most experts agree,
is by accident. That's where we come in; we're computer professionals.
We cause accidents."        -- Nathaniel Borenstein, co-creator of MIME
-------------- next part --------------
A non-text attachment was scrubbed...
Name: Simon.java
Type: text/x-java
Size: 1387 bytes
Desc: not available
Url : https://lists.owasp.org/pipermail/esapi-user/attachments/20100905/8bacbbdd/attachment.bin 


More information about the Esapi-user mailing list