[GPC] Project Proposal ( x 2 )

Martin Holst Swende martin.holst_swende at owasp.org
Fri Mar 11 14:12:53 EST 2011


Hi,

I have two project proposals for Owasp. I have developed something I
call the Hatkit Project, which consists of the Hatkit Proxy and the
Hatkit Datafiddler. I feel that they are both mature enough so that they
may be useful to other people, and also that it is time to try and turn
these projects into a community-development instead of a one-man sprint.
Hatkit stands for Http Analysis Toolkit, basically a toolkit for
advanced analysis of http traffic (also including layers on top of http
- basically analysis of web applications).

Project No 1:

1. Project Name:
    Hatkit Proxy
2. Project purpose/overview:
    The Hatkit Proxy is an intercepting http/tcp proxy based on the
Owasp Proxy, but with several additions. These additions are:
    - Swing-based UI
    - Interception capabilities with manual edit
    - Syntax highlightning (html/form-data/http) based on JFlex
    - Storage of http traffic into MongoDB database
    - Interception capabilities of tcp-traffic
    - Possibilities to intercept in Fully Qualified mode (like all other
http-proxies) OR Non-fully qualified mode. The latter means that
interception is performed *after* the host has been parsed, thereby
enabling the user to submit non-valid http content.

    The primary purpose of the Hatkit Proxy is to create a minimal,
lightweight proxy which stores traffic into an offline storage where
further analysis can be performed, e.g. all kinds of analysis which is
currently implemented by the proxies themselves (webscarab/burp/paros etc).
    Also, since the http traffic is stored in a MongoDB, the traffic is
stored at an object-level, retaining the structure of the parsed
traffic, which enables a user to perform advanced queries later.
    
    The proxy should also be a good choice for 'defenders' who wants to
(temporarily?) monitor traffic. The proxy itself is, as stated, very
lightweight, and the backend MongoDB storage scales very well and should
be able to handle extreme amounts of data. This would allow defenders to
perform advanced post-mortem or real-time analysis of incoming traffic.

Built in Java/Swing + MongoDB

3. Project Roadmap:
    I don't have a clear roadmap, some ideas/improvements I have are :
    * Improve TCP-interception capabilities, largely by working on the UI.
    * Improve TCP-interception by adding database storage of tcp traffic
aswell.
    * Implement support for intercepting WebSockets (although this
probably will have to be performed on the Owasp Proxy part)
    * Implement asynchronous sockets using NIO (although this probably
will have to be performed on the Owasp Proxy part)
    * Improve HTTP/HTML syntax highlightning by implementing JSON lexer

4. Project links (if any) to external sites:
    http://martin.swende.se/hg/hatkit_proxy
5. Project License:
    GPLv3
6. Project Leader name:
    Martin Holst Swende
7. Project Leader email address:
    martin.holst_swende at owasp.org
8. Project Leader wiki account - the username:
    Don't have one
9. Project Contributor(s):
    None yet
10. Project Main Links:
    http://martin.swende.se/hg/#hatkit_proxy-t1
    http://martin.swende.se/hg/hatkit_proxy

Project No 2:

1. Project Name:
    Hatkit Datafiddler
2. Project purpose/overview:
    The Datafiddler is a tool for performing advanced analysis of http
traffic. It currently consists of two main views, one table-based and
one tree-based. These views allow the user
to study different aspects of the http traffic, with very high degree of
configurability. The tool is also meant to be a framework which can
utilize existing tools to analyze traffic post mortem (or real-time).
Some documentation of the datafiddler is available at
http://www.slideshare.net/holiman/hatkit-project-datafiddler

Built in Python/Qt + MongoDB

3. Project Roadmap:
    These are some of the features I have planned:
    * Implement third-party plugin tools API
    ** Implement w3af-plugin to grep traffic (working code exists but is
not finished)
    ** Implement rat-proxy plugin to replay traffic through the passive
Rat proxy and gather results (PoC exists)
    ** Implement httprint-plugin to perform server identification (not
started)
    * Implement replayer, to replay requests to the original server
    * Implement cacher, to use the gathered data as a cache-repository
where the datafiddler acts as a forwarding proxy - but does not forward,
only fetches from db.
4. Project links (if any) to external sites:
    http://www.slideshare.net/holiman/hatkit-project-datafiddler
5. Project License:
    GPLv3
6. Project Leader name:
    Martin Holst Swende
7. Project Leader email address:
    martin.holst_swende at owasp.org
8. Project Leader wiki account - the username:
    Don't have one
9. Project Contributor(s):
    None yet
10. Project Main Links:
    http://martin.swende.se/hg/#hatkit_fiddler-t1
    http://martin.swende.se/hg/hatkit_fiddler
    http://www.slideshare.net/holiman/hatkit-project-datafiddler


Best regards,
Martin Holst Swende


More information about the Global-projects-committee mailing list