[Esapi-php] Encoder update with tests. Base64 and Encoder::Normalize() still pending.

[Boberski, Michael [USA]]

Jah, thanks.

Based on your research, let us go with the Java 2.0 solution, and not include it. 

The general strategy in my mind is that we want to target 1.4, but selectively borrow from 2.0 when it's to our advantage.

Sound OK?

Best,
 
Mike B.

-----Original Message-----
From: esapi-php-bounces at lists.owasp.org [mailto:esapi-php-bounces at lists.owasp.org] On Behalf Of jah
Sent: Tuesday, January 26, 2010 9:46 AM
To: ESAPI for PHP development list
Subject: [Esapi-php] Encoder update with tests. Base64 and Encoder::Normalize() still pending.

Hi folks,

I've just checked-in DefaultEncoder and associated tests.  Still to do:

normalize() - see below
encodeForBase64() - awaiting codec completion
decodeFromBase64 - ditto

There are tests for each method currently available.  I haven't as yet verified that the all of test results are valid and align with the Java version; this is mainly due to a number of bugs in various codecs which make it somewhat a tiresome task.  Once the bugs are ironed-out I'll re-enable some tests which have been commented out to avoid fatal errors (or an infinite loop in the case of canonicalisation using JavaScript
Codec) and will make sure that all Encoder test results comply with Java ESAPI.

Normalize is a tricky one to implement.  Java 1.4 ESAPI uses
Normalizer.normalize() and the "Normalization Form D" to decompose characters into a base character and any combining marks.  The combining
mark(s) are then killed - leaving the base character.
e.g. an accented e becomes a plain e.

ESAPI Java 2.0 doesn't implement a normalize() function any longer (it was removed in r952 [1]).

According to the PHP manual, there is the Normalizer class [2] which is available to PHP 5 >= 5.3.0, PECL intl >= 1.0.0.
This has a normalize function that can perform Normalization Form D.

For PHP 5.2.9 there isn't an equivalent, but iconv [3] is supposedly able to perform transliteration (which is a similar transfromation, but not the same thing: "If you append the string //TRANSLIT to out_charset transliteration is activated. This means that when a character can't be represented in the target charset, it can be approximated through one or several similarly looking characters.").

I've briefly tried iconv but can't get it to convert reliably and certainly not with the same results as the Java implementation.

What shall we do about Normalize?

Regards,

jah


[1] http://code.google.com/p/owasp-esapi-java/source/detail?r=952
[2] http://www.php.net/manual/en/class.normalizer.php
[3] http://www.php.net/manual/en/function.iconv.php
_______________________________________________
Esapi-php mailing list
Esapi-php at lists.owasp.org
https://lists.owasp.org/mailman/listinfo/esapi-php