Cannot get REQUEST_FILENAME with %3f (encoded ?) in url path

[liudongmiao]

Describe the bug

If the url path contains %3f, cannot get real REQUEST_FILENAME.

Logs and dumps

ModSecurity: Warning. Matched "Operator `Gt' with parameter `0' against variable `REQUEST_URI_RAW' (Value: `/path1%3fpath2?query=%3f' ) [file "..."] [line "1"] [id "1"] [rev ""] [msg ""] [data ""] [severity "0"] [ver ""] [maturity "0"] [accuracy "0"] [hostname "127.0.0.1"] [uri "/path1"] [unique_id "..."] [ref "v4,24t:length"]
ModSecurity: Warning. Matched "Operator `Gt' with parameter `0' against variable `REQUEST_FILENAME' (Value: `/path1' ) [file "..."] [line "2"] [id "2"] [rev ""] [msg ""] [data ""] [severity "0"] [ver ""] [maturity "0"] [accuracy "0"] [hostname "127.0.0.1"] [uri "/path1"] [unique_id "..."] [ref "v4,14t:length"]

To Reproduce

Test for path: /path1%3fpath2?query=%3f

Expected behavior

Return urldecoded or original filename.

Server (please complete the following information):

• ModSecurity version (and connector): ModSecurity v3.0.6

Rule Set (please complete the following information):

SecRule REQUEST_URI_RAW "@gt 0" "id:1,phase:1,t:length,pass,log,auditlog"
SecRule REQUEST_FILENAME "@gt 0" "id:2,phase:1,t:length,pass,log,auditlog"

Additional context

Add any other context about the problem here.

[liudongmiao]

Current source codes.

    size_t pos_query = m_uri_decoded.find("?");

    /* ... skip multi lines ... */

    std::string path_info;
    if (pos_query == std::string::npos) {
        path_info = std::string(m_uri_decoded, 0);
    } else {
        path_info = std::string(m_uri_decoded, 0, pos_query);
    }
    if (var_size == std::string::npos) {
        var_size = uri_s.size();
    }

[martinhsv]

It has been a longstanding practice to do some automatic url decoding of REQUEST_FILENAME. One of the reasons that REQUEST_URI_RAW exists is so that an entirely undecoded string is available for those (expected to be very rare) occasions when that is needed.

Because of the availability of an alternative, the long history of decoding, as well as basic matching of web server behaviour, it probably wouldn't be a good idea for us to consider changing this functionality to never automatically perform url of REQUEST_FILENAME.

That said, the current v3 behaviour does result in two anomalies.

The first of these is what has been highlighted by the OP in this ticket: that if the path portion of the uri contains a literal '?' character (encoded in the request as %3b) then ModSecurity will truncate the REQUEST_FILENAME. This would qualify as a bug, but presumably one that would occur so rarely as to be almost never.

A second sub-issue is perhaps less of a true bug, but at least a notable difference from ModSecurity v2 behaviour. In that version, the REQUEST_FILENAME variable is indeed decoded if accessed in a phase:2 rule, but it is undecoded in a phase:1 rule.

[nitriques]

I get the same bug (I've been loosing a lot of time on that) when the query string value contains %3F. I do not even see a hit in the log, the request is just denied....

[airween]

Hi @nitriques,

I get the same bug (I've been loosing a lot of time on that) when the query string value contains %3F. I do not even see a hit in the log, the request is just denied....

Which version do you use?

It was fixed in libmodsecurity3 version 3.0.12 with PR #3048.

[nitriques]

Which version do you use?

Sorry, but I do not know how to find that. I run a whm/cpanel server.

Do you have a quick way to get it ? I can ssh/run command as root.

Thanks !!!

[nitriques]

I have this:

$ cat /etc/cpanel/ea4/modsecurity.version
2.9.7

Looks like cpanel is still distributing v2...

[nitriques]

I have a reverse-proxy in front of the Apache webserver, and I was able to replace all %3f from the url at the proxy level, which made the bug go away.

I writting this for people coming in from google like I did a couple of days ago ;)

[marcstern]

Don't you have an entry in the error log?
Very recent versions of httpd forbid rewriting URL containing %3F (unless you use a dedicated parameter UnsafeAllow3F). That may be the reason as well.

[nitriques]

@marcstern
Nothing... just as if no rule had been triggered.

unless you use a dedicated parameter UnsafeAllow3F

Wow, which I knew about that sooner. Is it available in v2 ?

[marcstern]

UnsafeAllow3F is a new parameter for RewriteRule (mod_rewrite)

[nitriques]

@marcstern Oh I see! Thanks for the info, might look into it.