[Owasp-modsecurity-core-rule-set] Woe with 920270 (Null Byte...) (was: Re: Matched rule modification)

Christian Folini christian.folini at netnea.com
Mon Jul 10 05:30:29 UTC 2017


Hey Ervin,

Like I mentioned last week, we want to come up with a solution to all
this non-ascii problems with CRS for 3.1. Chaim has explained the
problem pretty well and unfortunately, this stretches to more rules.

You manage to run curl and you have servers with non-ascii payloads.
So you will be an excellent partner when we come up with new rules
and approaches to this problem. It is likely to take a while, but 
once we have something to show, testing it on real servers
is very important.

Best,

Christian


On Fri, Jul 07, 2017 at 08:44:10PM +0200, Ervin Hegedüs wrote:
> Hi Chaim,
> 
> On Fri, Jul 07, 2017 at 10:46:22AM -0400, Chaim Sanders wrote:
> > Taking a quick look here, it appears that the problem is probably with
> > urldecodeuni but it also might not be solvable. Let me explain, If I recall
> > correctly the reason that we added request_uri and ARGS as a parameter is
> > that they require provided elements to be ASCII. The alternative is that in
> > PL1 we thought that only ASCII should be submitted as args, which seems
> > wrong.
> > However we see that there is the use of urldecodeuni, which will decode
> > even unicode chars into the base char points. Stepping aside from the other
> > Unicode problems we've been seeing as a result of some ModSec issues
> > (purportedly), the use of this transformation will defeat the purpose of
> > this rule, as it will fire whenever a unicode element has been received.
> > The real question/possible problem, is that as we learned before
> > Apache/Nginx will provide us with automatically urldecoded arguments (and
> > url's maybe). This in turn might mean that it is never possible to ensure
> > that URL's and Args were all ASCII as is required by spec.
> > Requires a little bit more testing and verification of the purpose of the
> > rule, but i'd guess this is the cause of your issue.
>  
> thanks for your detailed anwser, I think I understand most of it :).
> 
> Probably I can't do anything till you can't find the final
> solution.
> 
> What can I help you? I mean, may be I can build the old
> Modsecurity version with Apache, or Nginx, and can try with old
> CRS...
> 
> 
> Thanks,
> 
> 
> a.
> 
> > On Fri, Jul 7, 2017 at 8:24 AM, Ervin Hegedüs <airween at gmail.com> wrote:
> > 
> > > Hi Christian and Chaim,
> > > other CRS users,
> > >
> > >
> > > here is my previous e-mail with an issue (with more issues
> > > exactly, but this one whis is interesting now)
> > >
> > > On Thu, Jun 01, 2017 at 08:49:03AM +0200, Christian Folini wrote:
> > > >
> > > > On Wed, May 31, 2017 at 08:32:03AM +0200, Ervin Hegedüs wrote:
> > > > > And there is an another issue with 3.0.2 (but may be that affects
> > > > > another versions too).
> > > > >
> > > > > The request is similar that I detailed in my first post. The
> > > > > "extraParams" value (JSON field) is this:
> > > ...
> > >
> > > and you sent me your test:
> > >
> > > > I was kind of expecting problems with 920270, but probably not as
> > > > deep rooted ones as this. But first, I need to be able to reproduce
> > > > this, and I can't.
> > > >
> > > > Here is my curl call taking up your example:
> > > >
> > > > curl 'localhost/index.html' -d 'extraParams=%7B%22node%22%3A%
> > > 223%22%2C%22text%22%3A%22v%C3%A9gs%C5%91%20fejezet%22' --trace-ascii -
> > > > == Info:   Trying 127.0.0.1...
> > > > == Info: Connected to localhost (127.0.0.1) port 80 (#0)
> > > > => Send header, 153 bytes (0x99)
> > >
> > > ...
> > >
> > >
> > > now the situation is totally same like above, but the server is
> > > another, and the web application is an up-to-date RoundCube
> > > webmail.
> > >
> > > The issue is when I'ld like to compose a new mail and I'm using
> > > special hungarian characters, Modsecurity denies the request.
> > >
> > > Here is the curl test:
> > >
> > > curl "https://roundcube.mydomain.hu/" -d "_token=
> > > 0uuYa9sHayQF7AU6s5Kb4XtJeKt6PZak&_task=mail&_action=send&_
> > > id=1466771890595f68c41f875&_attachments=&_from=5&_to=airween%40gmail.com
> > > &_cc=&_bcc=&_replyto=&_followupto=&_subject=Pr%C3%B3ba+0707+1256&
> > > editorSelector=plain&_priority=0&_store_target=Sent&
> > > _draft_saveid=&_draft=&_is_html=0&_framed=1&_message=Pr%C3%B3ba+%C3%BCzenet."
> > > --trace-ascii -
> > > == Info: Hostname was NOT found in DNS cache
> > > == Info:   Trying 1.2.3.4...
> > > == Info: Connected to roundcube.mydomain.hu (1.2.3.4) port 443 (#0)
> > > == Info: successfully set certificate verify locations:
> > > == Info:   CAfile: none
> > >   CApath: /etc/ssl/certs
> > > == Info: SSLv3, TLS handshake, Client hello (1):
> > > => Send SSL data, 512 bytes (0x200)
> > > 0000: ........rJK..M`4Qg9.)..........r7.....v.0.,.(.$.........k.j.9.
> > > ...
> > > == Info:         SSL certificate verify ok.
> > > => Send header, 159 bytes (0x9f)
> > > 0000: POST / HTTP/1.1
> > > 0011: User-Agent: curl/7.38.0
> > > 002a: Host: roundcube.mydomain.hu
> > > 004a: Accept: */*
> > > 0057: Content-Length: 330
> > > 006c: Content-Type: application/x-www-form-urlencoded
> > > 009d:
> > > => Send data, 330 bytes (0x14a)
> > > 0000: _token=0uuYa9sHayQF7AU6s5Kb4XtJeKt6PZak&_task=mail&_action=send&
> > > 0040: _id=1466771890595f68c41f875&_attachments=&_from=5&_to=airween%40
> > > 0080: gmail.com&_cc=&_bcc=&_replyto=&_followupto=&_subject=Pr%C3%B3ba+
> > > 00c0: 0707+1256&editorSelector=plain&_priority=0&_store_target=Sent&_d
> > > 0100: raft_saveid=&_draft=&_is_html=0&_framed=1&_message=Pr%C3%B3ba+%C
> > > 0140: 3%BCzenet.
> > > == Info: upload completely sent off: 330 out of 330 bytes
> > > <= Recv header, 24 bytes (0x18)
> > > 0000: HTTP/1.1 403 Forbidden
> > > == Info: Server nginx/1.6.2 is not blacklisted
> > > <= Recv header, 21 bytes (0x15)
> > > 0000: Server: nginx/1.6.2
> > > <= Recv header, 37 bytes (0x25)
> > > 0000: Date: Fri, 07 Jul 2017 11:57:13 GMT
> > > <= Recv header, 25 bytes (0x19)
> > > 0000: Content-Type: text/html
> > > <= Recv header, 21 bytes (0x15)
> > > 0000: Content-Length: 168
> > > <= Recv header, 24 bytes (0x18)
> > > 0000: Connection: keep-alive
> > > <= Recv header, 2 bytes (0x2)
> > > 0000:
> > > <= Recv data, 168 bytes (0xa8)
> > > 0000: <html>
> > > 0008: <head><title>403 Forbidden</title></head>
> > > 0033: <body bgcolor="white">
> > > 004b: <center><h1>403 Forbidden</h1></center>
> > > 0074: <hr><center>nginx/1.6.2</center>
> > > 0096: </body>
> > > 009f: </html>
> > >
> > >
> > > The audit log contains these lines for that request?
> > >
> > > ModSecurity: Warning. Matched "Operator `ValidadeByteRange' with parameter
> > > `1-255' against variable `ARGS:_message' (Value: `Pr\xffffffc3\xffffffb3ba
> > > \xffffffc3\xffffffbczenet.' ) [file "/etc/nginx/owasp-modsecurity-
> > > crs/rules/REQUEST-920-PROTOCOL-ENFORCEMENT.conf"] [line "488"] [id
> > > "920270"] [rev "2"] [msg "Invalid character in request (null character)"]
> > > [data ""] [severity "2"] [ver "OWASP_CRS/3.0.0"] [maturity "9"] [accuracy
> > > "9"] [tag "application-multi"] [tag "language-multi"] [tag
> > > "platform-multi"] [tag "attack-protocol"] [tag
> > > "OWASP_CRS/PROTOCOL_VIOLATION/EVASION"] [ref "o2,1o3,1v333,16t:
> > > urlDecodeUnio2,1o3,1o7,1o8,1v459,15t:urlDecodeUni"]
> > > ModSecurity: Warning. Matched "Operator `Ge' with parameter
> > > `%{tx.inbound_anomaly_score_threshold}' against variable
> > > `TX:ANOMALY_SCORE' (Value: `8' ) [file "/etc/nginx/owasp-modsecurity-
> > > crs/rules/REQUEST-949-BLOCKING-EVALUATION.conf"] [line "36"] [id
> > > "949110"] [rev ""] [msg "Inbound Anomaly Score Exceeded (Total Score: 8)"]
> > > [data ""] [severity "2"] [ver ""] [maturity "0"] [accuracy "0"] [tag
> > > "application-multi"] [tag "language-multi"] [tag "platform-multi"] [tag
> > > "attack-generic"] [ref ""]
> > > ModSecurity: Warning. Matched "Operator `Ge' with parameter
> > > `%{tx.inbound_anomaly_score_threshold}' against variable
> > > `TX:INBOUND_ANOMALY_SCORE' (Value: `8' ) [file
> > > "/etc/nginx/owasp-modsecurity-crs/rules/RESPONSE-980-CORRELATION.conf"]
> > > [line "61"] [id "980130"] [rev ""] [msg "Inbound Anomaly Score Exceeded
> > > (Total Inbound Score: 8 - SQLI=0,XSS=0,RFI=0,LFI=0,RCE=0,PHPI=0,HTTP=0,SESS=0):
> > > Invalid character in request (null character)'"] [data ""] [severity "0"]
> > > [ver ""] [maturity "0"] [accuracy "0"] [tag "event-correlation"] [ref ""]
> > >
> > > Note, that when I put the special characters to subject of mail,
> > > and the body contains only ascii chars, the rule doesn't
> > > triggered:
> > >
> > > _token=0uuYa9sHayQF7AU6s5Kb4XtJeKt6PZak&_task=mail&_action=send&_
> > > id=1061864000595f7c3bc5c2e&_attachments =&_from=5&_to=airween%40gmail.com
> > > &_cc=&_bcc=&_replyto=&_followupto=&_subject=Pr%C3%B3ba+mail+0707+1419
> > > &editorSelector=plain&_priority=0&_store_target=Sent&
> > > _draft_saveid=&_draft=&_is_html=0&_framed=1&_message=Teszt+uzenet.
> > >
> > > but the audit log contains a line:
> > >
> > > ModSecurity: Warning. Matched "Operator `ValidadeByteRange' with parameter
> > > `1-255' against variable `ARGS:_subject' (Value: `Pr\xffffffc3\xffffffb3ba
> > > mail 0707 1419' ) [file "/etc/nginx/owasp-modsecurity-
> > > crs/rules/REQUEST-920-PROTOCOL-ENFORCEMENT.conf"] [line "488"] [id
> > > "920270"] [rev "2"] [msg "Invalid character in request (null character)"]
> > > [data ""] [severity "2"] [ver "OWASP_CRS/3.0.0"] [maturity "9"] [accuracy
> > > "9"] [tag "application-multi"] [tag "language-multi"] [tag
> > > "platform-multi"] [tag "attack-protocol"] [tag
> > > "OWASP_CRS/PROTOCOL_VIOLATION/EVASION"] [ref "o2,1o3,1v857,21t:
> > > urlDecodeUni"]
> > >
> > >
> > > What em I missing?
> > >
> > >
> > > Thanks,
> > >
> > >
> > > a.
> > >
> > >
> > >
> > 
> > 
> > -- 
> > -- 
> > Chaim Sanders
> > http://www.ChaimSanders.com

-- 
https://www.feistyduck.com/training/modsecurity-training-course
mailto:christian.folini at netnea.com
twitter: @ChrFolini


More information about the Owasp-modsecurity-core-rule-set mailing list