[Owasp-data-exchange-format] (lets do it!) Re: DEF Strawman characteristics

Mon Jul 25 09:48:30 EDT 2011

I really like the idea to use JSON as a way to store data,  which together
with a strong XSD/schemas should provide enough flexibility (REST interfaces
are great, but realistically they will be implemented by 'app sec
integration' tools/portals).

*In terms of trying to get more 'feedback' and create a first pass at a
'schema' I'm would advise against it.* We are now at exactly the moment
where (in my view) all past efforts to do this failed.

We don't need more ideas, planning, XSD, etc...

What we need is to start trying to make it work.

Let's take baby steps. Lets:
 - collect the data/schemas that are available TODAY
 - come up with a couple use-cases to connect them
 - implement those use cases

Just doing this ONCE would be enormously valuable.

I would say that our first step should be to collect all schemas and sample
assessment files (or whatever the tools can produce) and put them all in one
place.

This is what I tried to do with
https://www.owasp.org/index.php/Summit_2011/Open_letter_to_WebAppSec_Tool_and_Services_vendors:_Release_your_schemas_and_allow_automationand
one of its key objectives was to kick-start the conversation and point
the direction (and see any tool vendor would complain spectacularly  (which
they didn't)) Now is the perfect time to implemented it.

Take a look at what I did in O2 to add support to Fortify: Fortify « OWASP
O2 Platform Blog
<http://o2platform.wordpress.com/category/tools/fortify/> which
is what needs to happen on the SourcCode Analysis land, and if you want a
Black-box use case, what about this one: *"Here are the links to a Spring
MVC app **
http://o2platform.wordpress.com/2011/07/15/visualizing-the-links-in-jpetstore-spring-mvc/
*<http://o2platform.wordpress.com/2011/07/15/visualizing-the-links-in-jpetstore-spring-mvc/>
* how can tool xyz consume it?"*

I have been wanting to add support to the tools that are already represented
in this thread (ZAP, Netsparker), but when* I actually sit down and tried to
do it, where do it start? *
*
*
I have a bunch of emails somewhere in my inbox that have the info that I
need, but what I (and every other tool developer that wants to implement
this)  need, is a central location to get the required files.

In fact, I already have a bunch of sample files form a bunch of different
tools, so where do I upload them? (do we have a code repository for DEF?)

Dinis Cruz

On 25 July 2011 11:23, Daniel Brzozowski <daniel at brzozowski.biz> wrote:

> Just throwing some ideas in the air :)
> anyway, it wouldn't be much effort to do both - XML and JSON, the most
> important thing is content but we need to decide which way we go... So maybe
> we should create a poll to get some feedback from people?
>
> --
> Regards,
> db
>
>
> On 25 July 2011 09:51, psiinon <psiinon at gmail.com> wrote:
>
>> I specified JSON in an attempt to keep things as simple as possible, but
>> I'll admit XML came a close second.
>> And Dafydd pointed out that XML cant handle null bytes unless you do
>> something like base 64 encoding.
>> But if the majority of people think XML is more suitable then I'll be
>> happy to go with that :)
>>
>> I know quite a few products already export data in various formats.
>> It would be _really_ useful if people could email this list with examples.
>>
>> Many thanks,
>>
>> Simon
>>
>> On Sun, Jul 24, 2011 at 5:49 PM, Ferruh Mavituna <ferruh at mavituna.com>wrote:
>>
>>> I might not have the background in here, but I think REST should be
>>> completely considered as a completely separate project / extension.
>>> Secondly why not simply come up with an XSD (*which is completely
>>> designed for this purpose*) and leave it there, wouldn't that be
>>> enough?
>>>
>>> On 24 July 2011 12:05, psiinon <psiinon at gmail.com> wrote:
>>>
>>>>  Ah, yes, I should really explain the 'REST' interface a bit more:)
>>>> And, thinking about it, a few use cases wouldnt go amiss!
>>>>
>>>> My plan for the strawman was to define a hierarchical structure for
>>>> representing pentest data, covering all of the types of objects in the data
>>>> model (hosts, ports, sites (host:port), urls, issues, requests/responses).
>>>> However all of the objects will be optional, as it really depends on
>>>> whats generating it and how the data is going to be used.
>>>> So, you might want to export everything regarding a pentest session from
>>>> burp, in which case a massive file might be generated.
>>>> Or you might want to just export a single request to feed into JBroFuzz.
>>>> The file format should allow these 2 extremes and pretty much anything
>>>> inbetween.
>>>> Products could either prompt the user as to what they want to export, or
>>>> in some cases just ignore the parts that dont apply.
>>>>
>>>> However I also really like the idea of defining an API, which I know is
>>>> not really in the spirit of a data format.
>>>> Maybe this could be a separate but related doc?
>>>>
>>>> To support the REST API products would expose a base URL, eg
>>>> http://localhost:1234/myproduct/owasp-def1.0/
>>>> There would be a standard set of REST endpoints under that which would
>>>> allow other products to retrieve selected objects.
>>>> So if your do a GET on (for example)
>>>> http://localhost:1234/myproduct/owasp-def1.0/sitemap
>>>> you would get back a summary of all of the URLs the product has
>>>> discovered.
>>>> You could then do a GET on
>>>> http://localhost:1234/myproduct/owasp-def1.0/reqresp?id=1234
>>>> to get the full request/response pair for that request using the app
>>>> specific id, which would have been returned with the URL details.
>>>> All of the data returned will (or course) be in DEF format.
>>>>
>>>> Not all products would support all objects, so the top level URL would
>>>> essentially list which objects (or endpoints) are supported.
>>>>
>>>> That a bit clearer?
>>>>
>>>> And, more importantly, should it be part of the DEF, related but
>>>> separate or am I being too ambitious and therefore should it be put on hold
>>>> or scrapped?
>>>>
>>>> Re the parameters and cookies - I was thinking that they would just be
>>>> part of the request/response data.
>>>> But I can see that sometimes it might be really useful to be able to
>>>> refer to them directly.
>>>> Definitely one to think about!
>>>>
>>>> Cheers,
>>>>
>>>> Simon
>>>>
>>>>
>>>> On Sat, Jul 23, 2011 at 5:01 PM, Martin Holst Swende <
>>>> martin.holst_swende at owasp.org> wrote:
>>>>
>>>>>  ----- Ursprungsmeddelande -----
>>>>> > I've updated the Strawman tab on the project page (
>>>>> >
>>>>> https://www.owasp.org/index.php/OWASP_Data_Exchange_Format_Project#tab=Strawman)
>>>>>
>>>>> > with some proposed characteristics:
>>>>> >
>>>>> >      - The format will be JSON (to make it as simple as possible)
>>>>>
>>>>> great!
>>>>>
>>>>> >      - Products can generate and/or consume DEF
>>>>> >      - Products will be able to generate DEF via a defined REST
>>>>> interface
>>>>>
>>>>> could you please explain this a bit? Not sure I understand...
>>>>>
>>>>> >      and/or simple files - products can choose
>>>>> >      - Products which consume DEF must support both REST and file
>>>>> options
>>>>> >      - There will be minimal security (but REST based services can
>>>>> limit
>>>>> > by IP    addr)
>>>>> >      - The data model will cover: hosts, ports, sites (host:port),
>>>>> urls,
>>>>> >      issues, requests/responses
>>>>>
>>>>> How about parameters? E.g if a certain param or field is vulnerable or
>>>>> should be exported as fuzzer-target? Cookies? Perhaps also http header
>>>>> fields.
>>>>>
>>>>> >      - Products can generate a subset of DEF, the level support will
>>>>> be
>>>>> >      described in the DEF
>>>>> >
>>>>> > What do you think??
>>>>> >
>>>>> > Let me know if I've been too terse!
>>>>> >
>>>>> > Simon
>>>>>
>>>>> sounds good so far! Keep it up!
>>>>> Regards,
>>>>> Martin from n900
>>>>>
>>>>>
>>>>
>>>> _______________________________________________
>>>> Owasp-data-exchange-format mailing list
>>>> Owasp-data-exchange-format at lists.owasp.org
>>>> https://lists.owasp.org/mailman/listinfo/owasp-data-exchange-format
>>>>
>>>>
>>>
>>>
>>> --
>>> .fm
>>>
>>
>>
>> _______________________________________________
>> Owasp-data-exchange-format mailing list
>> Owasp-data-exchange-format at lists.owasp.org
>> https://lists.owasp.org/mailman/listinfo/owasp-data-exchange-format
>>
>>
>
> _______________________________________________
> Owasp-data-exchange-format mailing list
> Owasp-data-exchange-format at lists.owasp.org
> https://lists.owasp.org/mailman/listinfo/owasp-data-exchange-format
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: https://lists.owasp.org/pipermail/owasp-data-exchange-format/attachments/20110725/0240e34f/attachment.html 