[Owasp-london] Review of Owasp-London Chapter meeting on WAF (Web Application Firewalls)
Dinis Cruz
dinis at ddplus.net
Mon May 1 05:59:58 EDT 2006
Hello
Here is my brain dump about what happened (from my point of view of
course) last Tuesday (25th April) on the Owasp London Chapter meeting
(hosted on a central London pub).
As you can read here
(http://sourceforge.net/mailarchive/forum.php?thread_id=10209024&forum_id=40284),
here
(http://sourceforge.net/mailarchive/forum.php?thread_id=10232868&forum_id=40284)
and here
(http://sourceforge.net/mailarchive/forum.php?thread_id=10239385&forum_id=40284)
the theme of the night was "Web Application Firewalls (WAF): Where do
they add value and who should be using them". After a public invite for
participation, 4 vendors committed to come in an present their WAF
appliances: F5, Imperva, Breach and Fortify (I should actually say 3 WAF
Appliance vendors since Fortify presented a similar tool with a
different approach).
My objective with this presentation was to go beyond the Marketing BS
and discuss the type of issues that these tools:
- are able to defend out of the box (i.e. with minor configuration's
by the user)
- are able to defend with some customization
- are NOT able to defend
I was also very interested in the usability of the tools and the user
experience of the WAFifyed website (i.e. what happens when a potential
attack is detected).
To level the playing field, each vendor was given a vulnerable website
that they needed to protect. This website was Foundstone's HacmeBank
v2.0 (the original plan was to use Owasp's SiteGenerator but some
vendors had some problems in getting it to work (prob with windows 2003
SP1 I think), so due to time constrains we used the new version of
HacmeBank (2.0) which is very stable and feature (i.e. vulnerability)
rich :)
To give an extra humph to the event (and to justify the tight deadlines)
I added the element "Hacme Bank is under attack, here are the exploit
vectors being used, your mission is to patch this system as soon as
possible and stop the attackers". It should be noted that the vendors
did not had a lot of time in their hands (from receiving the list of
vulnerabilities to patch to the event).
For reference, here is a PDF (created using the new Owasp PenTest
Reporter tool) describing all vulnerabilities:
"http://209.97.215.160/install/Hacme Bank Pentest Report (created by
Owasp PenTest Reporter) - ).pdf" .
Now that the scenario is set, here are my brain dump of the night:
1) Firstly I am happy that the 4 vendors made it, that we had a good
number of attendees (although not all Owasp London Chapter members :)
and that the F5 sponsorship went ok (with paid: pub dedicated to event,
plasma screen, open bar and food)
2) The audio recording also went ok, so give me a couple days and I
will upload it to a server on MP3 format (need to do some audio
normalization and minor editing)
3) On the presentations it self (in the order they were made):
a) F5
- Brought their WAF Appliance (1U Rack) and did a live
presentation
- Showed several demos on the HacmeBank vulnerabilities
(XSS, SQL Injection, Cookies)
- Discussed briefly some advanced options that their
product has: Leveraging the BigIP scripting capabilities, protecting
admin pages by forcing a certain 'navigation path'
- Maybe because this was the first presenter, but they
really run out of time (the 20m minutes allocated were religiously
enforced). I did felt that the presenter at times went over the top on
the explanations given, and spent too much time on simple stuff.
Basically I would had preferred if he gave the presentation at twice the
pace covering twice the material.
b) Imperva
- Brought their WAF Appliance (1U Rack) and did a live
presentation
- Similar to F5, showed several demos on the HacmeBank
vulnerabilities (XSS, SQL Injection, Cookies) covering roughly the same
material
- Shown very powerful functionality on their product,
namely the Forms / Cookie rules creation user interface
- Had several problems during the presentation (e.g. some
demos that didn't work)
c) Breach
- No WAF Appliance, videos shown instead
- A couple too many slides for my taste, and not enough
demos.
- That said, I liked Ofer style and he has clearly a very
good technical understanding of the WAF world (he is also the organizer
of the Owasp Chapter in Israel)
- I didn't particularly agree with the concept of "Dinis
WAF Requirements", which imply that what I was asking the WAF vendors is
very advanced, difficult and has very little customer demand.
d) Fortify
- Fortify didn't present a WAF Appliance. They took the
concept of WAF and applied it to the actual targeted application. Steps
used by their product: 1) reverse engineering the J2EE application, 2)
identify the attack surface, 3) identify insecure coding practices, 4)
apply 'security protection layers' were necessary, and 5) monitor and
report malicious activity. Due to the fact that their .Net product is
still under development, Fortify's demo was based on the Owasp Web Goat
tool.
- I was quite impressed with their solution and can see
real-live cases where it could be very effective (especially since some
of the security vulnerabilities that I want to address via a WAF
solution can only be made at the Application layer)
- The main problem with this solution is that it requires
a binary change/redeployment of the targeted application, and it still a
very young product (e.g. not used by a large number of clients to
protect mission critical web applications)
[this next set of comments apply mainly to the 3 WAF vendors]
5) Coming back to the WAF Appliance presentations, there are a couple
of comments that I have to make:
a) The presentations were all very BASIC (as in Simplistic).
Come on!!!! doing an SQL injection or cookies demo using Paros? That is
'so' last century. I was expecting at least some Unit Tests or custom
use of Web Application Security Scanners. But Paros? (or for that mater
Web Scarab, Fiddler, Firefox/IE Tamper request, @Stake Web Proxy,
etc...). My problem here is not that such tool(s) were used, but the
fact that they don't allow for the automated test of the numerous
variations that exploits like SQL Injection have. Remember that the
WAF's must be able to detect ALL (or most) types of SQL Injections (not
only the ones with a quote). Think of the problems that IDS have in
detecting RPC traffic, the WAF will have exactly the same types of issues.
b) I might be wrong on this one, but I do get the feeling that
these WAF vendors have a good understanding of 'data validation' attacks
( SQL Injection, XSS, Cookies, etc) but fail to grasp the security
implications of (for example) application logic attacks (like the one in
Hacme Bank where user A is able to access data from user B). More
worryingly, they seem happy with this situation since (from their point
of view) their clients don't need that functionality.
c) Where are the published independent security reviews of these
products? I find amazing that vendors that are selling a 'security
product', e.g. a software application (WAF) that protects other software
applications (Websites), do not understand the value of hiring
independent 3rd party security companies to perform source code security
audits to their products (note that the final results of these audits
must be published and made available to clients). As discussed during
the panel, it is probably impossible to create bug/vulnerability free
applications, but to NOT perform independent security audits to their
code is crazy. Since these vendors are still in the 'Functionality Arms
Race' phase of their products. Basically, the development teams are more
focused on features, performance and user experience than on Security
(and I don't have to tell you how 'secure' apps developed like this tend
to be :). Maybe the solution is to put a WAF protecting a WAF protecting
a WAF protecting a website :). Note to vendors: If am am wrong in this
comment, feel free to prove me wrong and publish the security audits
performed on your current product(s).
d) another area which I still think the WAF don't get it, is the
fact that no solution allows for the easy manipulation of the data being
analyzed (both at input and at output). At the moment you only have two
choices: 1) let the request go, 2) block the request and either show a
custom error page or logout the user. This is to radical. We need the
ability to change the contents of form fields (or page contents)
dynamically. This will be the only way that some vulnerabilities can be
effectively managed and mitigated (while limiting the damage cause by
false positives). Note: same WAFs have the functionality to replace
cookies with their own (WAF controlled value)
e) Why don't they use their WAFs to protect the WAF's web
interface? Clearly a great test for the usability of a WAF product is to
use it to protect the complex GUIs used on the WAF management and
monitoring. Also, when (not if) vulnerabilities are found in their
product, they could use their WAF to mitigate those security issues.
g) And what about their website? Are the vendor's websites
protected by their WAFs?
h) To the other companies/products that want to get into the WAF
market, advice to them is: "Create a WAF that allows a good PenTester to
mitigate 90%+ of the vulnerabilities that he/she discovered". Basically,
grab HacmeBank and SiteGenerator and use it to take your product were
the current WAF vendors don't seem to want to go.
i) What about the Web Application Firewall Evaluation Criteria
(http://www.webappsec.org/projects/wafec/)? How do this WAF appliances
rate to this? I might have missed it, but where is the public disclose
of this information?
6) After thinking about what happened for a couple days, and trying
to come up with an explanation for the issues raised on item 5), I think
that I came up with an answer. The problem is that the WAF technical
teams don't have Web Penetration Testing experience. They have a
reasonable level of understanding on how to attack an web application,
but they are missing the advanced material (I am also puzzled by the
lack of use of Web Application Security Scanners (WASS)). Maybe the
solution is for a WAF company and a WASS company to merge and maximize
the skills sets of both.
7) Please take my comments with a pinch of salt :) . It is to late
at night here and I am just doing a brain dump :)
8) Moving forward, here are some ideas:
a) Expand the current Hacme Bank challenge to all WAF vendors
and see who is able to mitigate what. I already asked the 4 WAF vendors
that participated on the London event to send me the list of issues
(from Hacme Bank v2.0) that they product is:
* Able to protect with default functionality
* Able to protect with Customization
* Not able to protect
b) Implement all documented HacmeBank vulnerabilities in
SiteGenerator (also add different types of navigation)
c) help to organize other Owasp chapter events around the
world on the same subject and using the same model (Owasp chapter
leaders let-me know if you are interested)
d) Implement the Foundstone Validator .Net tool on HacmeBank
to show how most of those security issues can be resolved when the WAF
has access to the application layer
e) write a white paper about WAFs focusing on where they
should be used and on where they are not effective
f) expand the analysis of the Hacme Bank vulnerabilities and
discuss how they could/should be mitigated at WAF
9) Finally, I just want to give a warning to the WAF vendors: "you
must make your appliance invisible to attackers". By invisible I mean
that it should not be possible to fingerprint your appliance remotely
from the Internet. Because if you do allow this to occur, then you
dramatically increase the risk to your clients of being exploited by a
vulnerability in your WAF (and remember the Witty worm
(http://www.newsfactor.com/story.xhtml?story_id=23559 and
http://news.zdnet.co.uk/internet/security/0,39020375,39149459,00.htm)
which was a targeted exploit on a security product).
That's it, thanks for reading, and I am looking forward to your comments.
Best regards
Dinis Cruz
Owasp .Net Project
www.owasp.net
More information about the Owasp-london
mailing list