[Owasp-data-exchange-format] (lets do it!) Re: DEF Strawman characteristics

Mon Jul 25 11:15:54 EDT 2011

Project set up: http://code.google.com/p/owasp-def/
Email your Google acc name to me to get added to it.

And I've created an 'examples' directory and uploaded a couple of Burp
examples Dafydd sent to me:
http://code.google.com/p/owasp-def/source/browse/trunk/examples/

I'll upload ZAP ones when I get a chance to generate them (ZAP supports both
XML and JSON;) ...

Cheers,

Simon

On Mon, Jul 25, 2011 at 2:58 PM, psiinon <psiinon at gmail.com> wrote:

> Love the idea of one central place for all of the existing data/schemas.
> I'll create a Google Code project for DEF as soon as I get a chance
> (hopefully today).
>
> All - if you havnt signed up for a Google acc please do so now and email me
> your id.
> I'll give you all write access and we can dump the examples in a suitable
> place.
>
> And I take the point about the REST interface.
> Lets follow Dini's plan below.
>
> Cheers,
>
> Simon
>
>
> On Mon, Jul 25, 2011 at 2:48 PM, dinis cruz <dinis.cruz at owasp.org> wrote:
>
>> I really like the idea to use JSON as a way to store data,  which together
>> with a strong XSD/schemas should provide enough flexibility (REST interfaces
>> are great, but realistically they will be implemented by 'app sec
>> integration' tools/portals).
>>
>> *In terms of trying to get more 'feedback' and create a first pass at a
>> 'schema' I'm would advise against it.* We are now at exactly the moment
>> where (in my view) all past efforts to do this failed.
>>
>> We don't need more ideas, planning, XSD, etc...
>>
>> What we need is to start trying to make it work.
>>
>> Let's take baby steps. Lets:
>>  - collect the data/schemas that are available TODAY
>>  - come up with a couple use-cases to connect them
>>  - implement those use cases
>>
>> Just doing this ONCE would be enormously valuable.
>>
>> I would say that our first step should be to collect all schemas and
>> sample assessment files (or whatever the tools can produce) and put them all
>> in one place.
>>
>> This is what I tried to do with
>> https://www.owasp.org/index.php/Summit_2011/Open_letter_to_WebAppSec_Tool_and_Services_vendors:_Release_your_schemas_and_allow_automationand one of its key objectives was to kick-start the conversation and point
>> the direction (and see any tool vendor would complain spectacularly  (which
>> they didn't)) Now is the perfect time to implemented it.
>>
>> Take a look at what I did in O2 to add support to Fortify: Fortify «
>> OWASP O2 Platform Blog<http://o2platform.wordpress.com/category/tools/fortify/> which
>> is what needs to happen on the SourcCode Analysis land, and if you want a
>> Black-box use case, what about this one: *"Here are the links to a Spring
>> MVC app **
>> http://o2platform.wordpress.com/2011/07/15/visualizing-the-links-in-jpetstore-spring-mvc/
>> *<http://o2platform.wordpress.com/2011/07/15/visualizing-the-links-in-jpetstore-spring-mvc/>
>> * how can tool xyz consume it?"*
>>
>>
>> I have been wanting to add support to the tools that are already
>> represented in this thread (ZAP, Netsparker), but when* I actually sit
>> down and tried to do it, where do it start? *
>> *
>> *
>> I have a bunch of emails somewhere in my inbox that have the info that I
>> need, but what I (and every other tool developer that wants to implement
>> this)  need, is a central location to get the required files.
>>
>> In fact, I already have a bunch of sample files form a bunch of different
>> tools, so where do I upload them? (do we have a code repository for DEF?)
>>
>> Dinis Cruz
>>
>> On 25 July 2011 11:23, Daniel Brzozowski <daniel at brzozowski.biz> wrote:
>>
>>> Just throwing some ideas in the air :)
>>> anyway, it wouldn't be much effort to do both - XML and JSON, the most
>>> important thing is content but we need to decide which way we go... So maybe
>>> we should create a poll to get some feedback from people?
>>>
>>> --
>>> Regards,
>>> db
>>>
>>>
>>> On 25 July 2011 09:51, psiinon <psiinon at gmail.com> wrote:
>>>
>>>> I specified JSON in an attempt to keep things as simple as possible, but
>>>> I'll admit XML came a close second.
>>>> And Dafydd pointed out that XML cant handle null bytes unless you do
>>>> something like base 64 encoding.
>>>> But if the majority of people think XML is more suitable then I'll be
>>>> happy to go with that :)
>>>>
>>>> I know quite a few products already export data in various formats.
>>>> It would be _really_ useful if people could email this list with
>>>> examples.
>>>>
>>>> Many thanks,
>>>>
>>>> Simon
>>>>
>>>> On Sun, Jul 24, 2011 at 5:49 PM, Ferruh Mavituna <ferruh at mavituna.com>wrote:
>>>>
>>>>> I might not have the background in here, but I think REST should be
>>>>> completely considered as a completely separate project / extension.
>>>>> Secondly why not simply come up with an XSD (*which is completely
>>>>> designed for this purpose*) and leave it there, wouldn't that be
>>>>> enough?
>>>>>
>>>>> On 24 July 2011 12:05, psiinon <psiinon at gmail.com> wrote:
>>>>>
>>>>>>  Ah, yes, I should really explain the 'REST' interface a bit more:)
>>>>>> And, thinking about it, a few use cases wouldnt go amiss!
>>>>>>
>>>>>> My plan for the strawman was to define a hierarchical structure for
>>>>>> representing pentest data, covering all of the types of objects in the data
>>>>>> model (hosts, ports, sites (host:port), urls, issues, requests/responses).
>>>>>> However all of the objects will be optional, as it really depends on
>>>>>> whats generating it and how the data is going to be used.
>>>>>> So, you might want to export everything regarding a pentest session
>>>>>> from burp, in which case a massive file might be generated.
>>>>>> Or you might want to just export a single request to feed into
>>>>>> JBroFuzz.
>>>>>> The file format should allow these 2 extremes and pretty much anything
>>>>>> inbetween.
>>>>>> Products could either prompt the user as to what they want to export,
>>>>>> or in some cases just ignore the parts that dont apply.
>>>>>>
>>>>>> However I also really like the idea of defining an API, which I know
>>>>>> is not really in the spirit of a data format.
>>>>>> Maybe this could be a separate but related doc?
>>>>>>
>>>>>> To support the REST API products would expose a base URL, eg
>>>>>> http://localhost:1234/myproduct/owasp-def1.0/
>>>>>> There would be a standard set of REST endpoints under that which would
>>>>>> allow other products to retrieve selected objects.
>>>>>> So if your do a GET on (for example)
>>>>>> http://localhost:1234/myproduct/owasp-def1.0/sitemap
>>>>>> you would get back a summary of all of the URLs the product has
>>>>>> discovered.
>>>>>> You could then do a GET on
>>>>>> http://localhost:1234/myproduct/owasp-def1.0/reqresp?id=1234
>>>>>> to get the full request/response pair for that request using the app
>>>>>> specific id, which would have been returned with the URL details.
>>>>>> All of the data returned will (or course) be in DEF format.
>>>>>>
>>>>>> Not all products would support all objects, so the top level URL would
>>>>>> essentially list which objects (or endpoints) are supported.
>>>>>>
>>>>>> That a bit clearer?
>>>>>>
>>>>>> And, more importantly, should it be part of the DEF, related but
>>>>>> separate or am I being too ambitious and therefore should it be put on hold
>>>>>> or scrapped?
>>>>>>
>>>>>> Re the parameters and cookies - I was thinking that they would just be
>>>>>> part of the request/response data.
>>>>>> But I can see that sometimes it might be really useful to be able to
>>>>>> refer to them directly.
>>>>>> Definitely one to think about!
>>>>>>
>>>>>> Cheers,
>>>>>>
>>>>>> Simon
>>>>>>
>>>>>>
>>>>>> On Sat, Jul 23, 2011 at 5:01 PM, Martin Holst Swende <
>>>>>> martin.holst_swende at owasp.org> wrote:
>>>>>>
>>>>>>>  ----- Ursprungsmeddelande -----
>>>>>>> > I've updated the Strawman tab on the project page (
>>>>>>> >
>>>>>>> https://www.owasp.org/index.php/OWASP_Data_Exchange_Format_Project#tab=Strawman)
>>>>>>>
>>>>>>> > with some proposed characteristics:
>>>>>>> >
>>>>>>> >      - The format will be JSON (to make it as simple as possible)
>>>>>>>
>>>>>>> great!
>>>>>>>
>>>>>>> >      - Products can generate and/or consume DEF
>>>>>>> >      - Products will be able to generate DEF via a defined REST
>>>>>>> interface
>>>>>>>
>>>>>>> could you please explain this a bit? Not sure I understand...
>>>>>>>
>>>>>>> >      and/or simple files - products can choose
>>>>>>> >      - Products which consume DEF must support both REST and file
>>>>>>> options
>>>>>>> >      - There will be minimal security (but REST based services can
>>>>>>> limit
>>>>>>> > by IP    addr)
>>>>>>> >      - The data model will cover: hosts, ports, sites (host:port),
>>>>>>> urls,
>>>>>>> >      issues, requests/responses
>>>>>>>
>>>>>>> How about parameters? E.g if a certain param or field is vulnerable
>>>>>>> or should be exported as fuzzer-target? Cookies? Perhaps also http header
>>>>>>> fields.
>>>>>>>
>>>>>>> >      - Products can generate a subset of DEF, the level support
>>>>>>> will be
>>>>>>> >      described in the DEF
>>>>>>> >
>>>>>>> > What do you think??
>>>>>>> >
>>>>>>> > Let me know if I've been too terse!
>>>>>>> >
>>>>>>> > Simon
>>>>>>>
>>>>>>> sounds good so far! Keep it up!
>>>>>>> Regards,
>>>>>>> Martin from n900
>>>>>>>
>>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> Owasp-data-exchange-format mailing list
>>>>>> Owasp-data-exchange-format at lists.owasp.org
>>>>>> https://lists.owasp.org/mailman/listinfo/owasp-data-exchange-format
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> .fm
>>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> Owasp-data-exchange-format mailing list
>>>> Owasp-data-exchange-format at lists.owasp.org
>>>> https://lists.owasp.org/mailman/listinfo/owasp-data-exchange-format
>>>>
>>>>
>>>
>>> _______________________________________________
>>> Owasp-data-exchange-format mailing list
>>> Owasp-data-exchange-format at lists.owasp.org
>>> https://lists.owasp.org/mailman/listinfo/owasp-data-exchange-format
>>>
>>>
>>
>
> _______________________________________________
> Owasp-data-exchange-format mailing list
> Owasp-data-exchange-format at lists.owasp.org
> https://lists.owasp.org/mailman/listinfo/owasp-data-exchange-format
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: https://lists.owasp.org/pipermail/owasp-data-exchange-format/attachments/20110725/155c6904/attachment.html 