Skip to content

v3/master Lua unknown error #1809

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
theMiddleBlue opened this issue Jun 20, 2018 · 13 comments
Closed

v3/master Lua unknown error #1809

theMiddleBlue opened this issue Jun 20, 2018 · 13 comments
Assignees
Labels
3.x Related to ModSecurity version 3.x RIP - libmodsecurity

Comments

@theMiddleBlue
Copy link

Hi,

I've a problem using exec: or defining a SecRuleScript with a test Lua script. Following my configurations and test:

Rule:

SecRule REQUEST_FILENAME "^/test" "phase:2,id:41,pass,exec:/opt/modseclua/test.lua

# or

SecRuleScript "/opt/modseclua/test.lua" "phase:5,pass"

My test Lua script:

function main()
	-- m.log(1, "Hello world!");
	return nil;
end

In both cases I got an empty response and an unknown error makes nginx workers to crash:

modsecurity debug log:

[4] (Rule: 41) Executing operator "Rx" with param "^/test" against REQUEST_FILENAME.
[9] Target value: "/test" (Variable: REQUEST_FILENAME)
[9] Matched vars updated.
[4] Rule returned 1.
[9] (SecDefaultAction) Running action: log
[9] Saving transaction to logs
[9] (SecDefaultAction) Running action: auditlog
[4] (SecDefaultAction) ignoring action: deny (rule does not cotains block)
[9] (SecDefaultAction) Running action: status
[4] Not running disruptive action: pass. SecRuleEngine is not On
[4] Running (non-disruptive) action: exec
[8] Running script... /opt/modseclua/test.lua

nginx error.log

2018/06/20 14:05:12 [alert] 10192#0: worker process 10276 exited on signal 11

I've try to run my script and it works:

root@mywebsite:~# cat /opt/modseclua/test.lua 
function main()
	print("test")
	-- m.log(1, "Hello world!");
	return nil
end

main()
root@mywebsite:~# 
root@mywebsite:~# 
root@mywebsite:~# lua5.2 /opt/modseclua/test.lua 
test
root@mywebsite:~# 

Any idea? Am I doing something wrong?

thanks.

@p0pr0ck5
Copy link
Contributor

@theMiddleBlue, what version of Nginx and modsecurity are you running? Can you show

@theMiddleBlue
Copy link
Author

./configure

ModSecurity - v3.0.2-67-g4e3a1f71 for Linux
 
 Mandatory dependencies
   + libInjection                                  ....v3.0.2-67-g4e3a1f71
   + SecLang tests                                 ....4e3a1f71
 
 Optional dependencies
   + GeoIP/MaxMind                                 ....found 
      * (GeoIP) v1.6.12
         -lGeoIP, -I/usr/include/
   + LibCURL                                       ....found v7.58.0 
      -lcurl,  -DWITH_CURL_SSLVERSION_TLSv1_2 -DWITH_CURL
   + YAJL                                          ....found v2.1.0
      -lyajl, -DWITH_YAJL -I/usr/include/yajl
   + LMDB                                          ....disabled
   + LibXML2                                       ....found v2.9.4
      -lxml2, -I/usr/include/libxml2 -DWITH_LIBXML2
   + SSDEEP                                        ....not found
   + LUA                                           ....found v502
      -llua5.2 -L/usr/lib/x86_64-linux-gnu/, -DWITH_LUA -DWITH_LUA_5_2 -I/usr/include/lua5.2
 
 Other Options
   + Test Utilities                                ....enabled
   + SecDebugLog                                   ....enabled
   + afl fuzzer                                    ....disabled
   + library examples                              ....enabled
   + Building parser                               ....disabled
   + Treating pm operations as critical section    ....disabled

nginx -V

nginx version: openresty/1.13.6.2
built by gcc 7.3.0 (Ubuntu 7.3.0-16ubuntu3) 
built with OpenSSL 1.1.0g  2 Nov 2017
TLS SNI support enabled
configure arguments: --prefix=/usr/local/openresty/nginx --with-cc-opt=-O2 --add-module=../ngx_devel_kit-0.3.0 --add-module=../echo-nginx-module-0.61 --add-module=../xss-nginx-module-0.06 --add-module=../ngx_coolkit-0.2rc3 --add-module=../set-misc-nginx-module-0.32 --add-module=../form-input-nginx-module-0.12 --add-module=../encrypted-session-nginx-module-0.08 --add-module=../srcache-nginx-module-0.31 --add-module=../ngx_lua-0.10.13 --add-module=../ngx_lua_upstream-0.07 --add-module=../headers-more-nginx-module-0.33 --add-module=../array-var-nginx-module-0.05 --add-module=../memc-nginx-module-0.19 --add-module=../redis2-nginx-module-0.15 --add-module=../redis-nginx-module-0.3.7 --add-module=../rds-json-nginx-module-0.15 --add-module=../rds-csv-nginx-module-0.09 --add-module=../ngx_stream_lua-0.0.5 --with-ld-opt=-Wl,-rpath,/usr/local/openresty/luajit/lib --add-module=/opt/openresty-1.13.6.2/../ModSecurity-nginx --with-stream --with-stream_ssl_module --with-http_ssl_module

Core dump:

Reading symbols from /usr/local/openresty/nginx/sbin/nginx...done.
[New LWP 13080]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Core was generated by `nginx: worker process                                                         '.
Program terminated with signal SIGSEGV, Segmentation fault.
#0  0x00007fa38ba7abf3 in ?? () from /usr/lib/x86_64-linux-gnu/liblua5.2.so.0
(gdb) backtrace full
#0  0x00007fa38ba7abf3 in ?? () from /usr/lib/x86_64-linux-gnu/liblua5.2.so.0
No symbol table info available.
#1  0x00007fa38ba6d842 in lua_setglobal () from /usr/lib/x86_64-linux-gnu/liblua5.2.so.0
No symbol table info available.
#2  0x00007fa38df9d99f in modsecurity::engine::Lua::run (this=this@entry=0x5618586627c8, t=t@entry=0x561855ba6850) at engine/lua.cc:136
        luaRet = ""
        a = 0x0
        ret = 1
        L = 0x41be7378
        rc = <optimized out>
#3  0x00007fa38df84df2 in modsecurity::actions::Exec::evaluate (this=0x561858662750, rule=<optimized out>, t=0x561855ba6850) at actions/exec.cc:55
No locals.
#4  0x00007fa38df61a9f in modsecurity::actions::Action::evaluate (ruleMessage=std::shared_ptr<modsecurity::RuleMessage> (empty) = {...}, 
    transaction=0x561855ba6850, rule=0x561858662930, this=0x561858662750) at ../headers/modsecurity/actions/action.h:67
No locals.
#5  modsecurity::Rule::executeActionsAfterFullMatch (this=this@entry=0x561858662930, trans=trans@entry=0x561855ba6850, containsBlock=false, 
    ruleMessage=std::shared_ptr<modsecurity::RuleMessage> (use count 3, weak count 0) = {...}) at rule.cc:713
        a = 0x561858662750
        __for_range = std::vector of length 2, capacity 2 = {0x561858662630, 0x561858662750}
        __for_begin = <optimized out>
        __for_end = <optimized out>
#6  0x00007fa38df69371 in modsecurity::Rule::evaluate (this=0x561858662930, trans=<optimized out>, 
    ruleMessage=std::shared_ptr<modsecurity::RuleMessage> (use count 3, weak count 0) = {...}) at rule.cc:872
        globalRet = <optimized out>
        variables = <optimized out>
        recursiveGlobalRet = <optimized out>
        containsBlock = false
        finalVars = std::vector of length 1, capacity 1 = {std::unique_ptr<modsecurity::VariableValue> = {get() = 0x56185ad6e820}}
        eparam = "\"^/test\""
#7  0x00007fa38df5ca48 in modsecurity::Rules::evaluate (this=0x56185912d830, phase=phase@entry=3, transaction=transaction@entry=0x561855ba6850)
    at rules.cc:253
        remove_rule = <optimized out>
        rule = 0x561858662930
        i = 4
        rules = Python Exception <class 'gdb.error'> value has been optimized out: 

#8  0x00007fa38df47d99 in modsecurity::Transaction::processRequestBody (this=0x561855ba6850) at transaction.cc:825
        a = std::unique_ptr<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >> = {get() = 0x0}
        fullRequest = "Host: localhost\nConnection: keep-alive\nPragma: no-cache\nCache-Control: no-cache\nUpgrade-Insecure-Requests: 1\nUser-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_5) AppleWebKit/537.36 (KHTML, like"...
        l = std::vector of length 10, capacity 16 = {0x5618559dc9a0, 0x5618559dc840, 0x5618559dc790, 0x56185ad6ec50, 0x56185ad6eb80, 0x56185ad6ea20, 
          0x56185ad6e970, 0x56185ad6e820, 0x5618581347d0, 0x56185ad6e570}
#9  0x00007fa38df48855 in modsecurity::msc_process_request_body (transaction=<optimized out>) at transaction.cc:1902
---Type <return> to continue, or q <return> to quit---
No locals.
#10 0x0000561853bca41d in ngx_http_modsecurity_pre_access_handler (r=0x561858688040)
    at /opt/openresty-1.13.6.2/../ModSecurity-nginx/src/ngx_http_modsecurity_pre_access.c:199
        ret = <optimized out>
        already_inspected = <optimized out>
        chain = <optimized out>
        ctx = 0x561858688dc0
        cf = <optimized out>
        old_pool = 0x0
#11 0x0000561853aeebef in ngx_http_core_generic_phase (r=0x561858688040, ph=0x56185ad709f8) at src/http/ngx_http_core_module.c:884
        rc = <optimized out>
#12 0x0000561853aea665 in ngx_http_core_run_phases (r=r@entry=0x561858688040) at src/http/ngx_http_core_module.c:862
        rc = <optimized out>
        ph = 0x56185ad70938
        cmcf = <optimized out>
#13 0x0000561853aea74f in ngx_http_handler (r=r@entry=0x561858688040) at src/http/ngx_http_core_module.c:845
        cmcf = <optimized out>
#14 0x0000561853af4dac in ngx_http_process_request (r=0x561858688040) at src/http/ngx_http_request.c:1950
        c = 0x5618564f8340
#15 0x0000561853af52a8 in ngx_http_process_request_headers (rev=rev@entry=0x5618565d6b30) at src/http/ngx_http_request.c:1377
        p = <optimized out>
        len = <optimized out>
        n = <optimized out>
        rc = <optimized out>
        rv = <optimized out>
        h = <optimized out>
        c = <optimized out>
        hh = <optimized out>
        r = <optimized out>
        cscf = <optimized out>
        cmcf = <optimized out>
#16 0x0000561853af5604 in ngx_http_process_request_line (rev=0x5618565d6b30) at src/http/ngx_http_request.c:1049
        n = <optimized out>
        rc = <optimized out>
        rv = <optimized out>
        host = {len = 94662556888096, 
          data = 0x561853af58d1 <ngx_http_wait_request_handler+609> "H\205\300H\211\003\017\204\264\375\377\377H\215\005\354\371\377\377H\211\357H\211E\020H\203\304\030[]A\\A]\351\326\371\377\377f\017\037D"}
        c = 0x5618564f8340
        r = 0x561858688040
#17 0x0000561853adf765 in ngx_epoll_process_events (cycle=<optimized out>, timer=<optimized out>, flags=<optimized out>)
    at src/event/modules/ngx_epoll_module.c:902
---Type <return> to continue, or q <return> to quit---
        events = 1
        revents = 1
        instance = <optimized out>
        i = 0
        level = <optimized out>
        err = 0
        rev = <optimized out>
        wev = <optimized out>
        queue = <optimized out>
        c = 0x5618564f8340
#18 0x0000561853ad6b35 in ngx_process_events_and_timers (cycle=cycle@entry=0x56185741e9f0) at src/event/ngx_event.c:252
        flags = <optimized out>
        timer = <optimized out>
        delta = 1529509585932
        q = <optimized out>
        ev = <optimized out>
#19 0x0000561853add768 in ngx_worker_process_cycle (cycle=cycle@entry=0x56185741e9f0, data=data@entry=0x0) at src/os/unix/ngx_process_cycle.c:820
        worker = 0
#20 0x0000561853adc1c6 in ngx_spawn_process (cycle=cycle@entry=0x56185741e9f0, proc=0x561853add6f0 <ngx_worker_process_cycle>, data=0x0, 
    name=0x561853bcf2d5 "worker process", respawn=respawn@entry=1) at src/os/unix/ngx_process.c:198
        on = 1
        pid = 0
        s = 1
#21 0x0000561853adea69 in ngx_reap_children (cycle=0x56185741e9f0) at src/os/unix/ngx_process_cycle.c:687
        i = 1
        live = 0
        n = <optimized out>
        ch = {command = 2, pid = 13078, slot = 1, fd = -1}
        ccf = <optimized out>
        i = <optimized out>
        n = <optimized out>
        live = <optimized out>
        ch = <optimized out>
        ccf = <optimized out>
#22 ngx_master_process_cycle (cycle=0x56185741e9f0) at src/os/unix/ngx_process_cycle.c:180
        title = <optimized out>
        p = <optimized out>
        size = <optimized out>
        i = <optimized out>
        n = <optimized out>
        sigio = 0
        set = {__val = {0 <repeats 16 times>}}
---Type <return> to continue, or q <return> to quit---
        itv = {it_interval = {tv_sec = 20, tv_usec = 41}, it_value = {tv_sec = 94662543221088, tv_usec = 0}}
        live = <optimized out>
        delay = 0
        ls = <optimized out>
        ccf = 0x56185741fc38
#23 0x0000561853ab67e5 in main (argc=<optimized out>, argv=<optimized out>) at src/core/nginx.c:384
        b = <optimized out>
        log = 0x561853e2f3a0 <ngx_log>
        i = <optimized out>
        cycle = 0x5618559e0480
        init_cycle = {conf_ctx = 0x0, pool = 0x5618559c9630, log = 0x561853e2f3a0 <ngx_log>, new_log = {log_level = 0, file = 0x0, connection = 0, 
            disk_full_time = 0, handler = 0x0, data = 0x0, writer = 0x0, wdata = 0x0, action = 0x0, next = 0x0}, log_use_stderr = 0, files = 0x0, 
          free_connections = 0x0, free_connection_n = 0, modules = 0x0, modules_n = 0, modules_used = 0, reusable_connections_queue = {prev = 0x0, 
            next = 0x0}, reusable_connections_n = 0, listening = {elts = 0x0, nelts = 0, size = 0, nalloc = 0, pool = 0x0}, paths = {elts = 0x0, nelts = 0, 
            size = 0, nalloc = 0, pool = 0x0}, config_dump = {elts = 0x0, nelts = 0, size = 0, nalloc = 0, pool = 0x0}, config_dump_rbtree = {root = 0x0, 
            sentinel = 0x0, insert = 0x0}, config_dump_sentinel = {key = 0, left = 0x0, right = 0x0, parent = 0x0, color = 0 '\000', data = 0 '\000'}, 
          open_files = {last = 0x0, part = {elts = 0x0, nelts = 0, next = 0x0}, size = 0, nalloc = 0, pool = 0x0}, shared_memory = {last = 0x0, part = {
              elts = 0x0, nelts = 0, next = 0x0}, size = 0, nalloc = 0, pool = 0x0}, connection_n = 0, files_n = 0, connections = 0x0, read_events = 0x0, 
          write_events = 0x0, old_cycle = 0x0, conf_file = {len = 42, data = 0x7ffee5b31391 ""}, conf_param = {len = 0, data = 0x0}, conf_prefix = {len = 32, 
            data = 0x7ffee5b31391 ""}, prefix = {len = 27, data = 0x561853bcb1d8 "/usr/local/openresty/nginx/"}, lock_file = {len = 0, data = 0x0}, 
          hostname = {len = 0, data = 0x0}, intercept_error_log_handler = 0x0, intercept_error_log_data = 0x0, entered_logger = 0}
        cd = <optimized out>
        ccf = <optimized out>

@p0pr0ck5
Copy link
Contributor

I suspect this is a conflict with OpenResty's integration of LuaJIT, but I'm not certain. It might be worth trying to build libmodsecurity with OpenResty's LuaJIT (yes, there's a bit of chicken-and-egg if you're using the standard OpenResty build lifecycle).

@p0pr0ck5
Copy link
Contributor

Actually, it's probably best not to try to make this work at all, given that Lua 5.1 (and thus LuaJIT) are unsupported by libmodsecurity . Whatever logic you need in Lua, probably best just to stick to OpenResty directly. Or better yet, use a WAF written specifically for OpenResty ;)

@victorhora
Copy link
Contributor

I can't remember if I've tested this before, but recent versions of LuaJIT seems compatible with Lua 5.2 according to luajit.org. You might want to try with DLUAJIT_ENABLE_LUA52COMPAT.

@victorhora victorhora added RIP - libmodsecurity 3.x Related to ModSecurity version 3.x labels Jun 20, 2018
@p0pr0ck5
Copy link
Contributor

@victorhora 5.2 compatibility is already enabled by default with all modern OpenResty releases.

@theMiddleBlue
Copy link
Author

theMiddleBlue commented Jun 20, 2018

thanks @p0pr0ck5 and @victorhora

my purpose was to generate logs in the "modsecurity auditlog format" (json) for some Lua scripts that I use with the nginx_lua_module in openresty. Maybe I need to create the same modsec auditlog JSON using nginx_lua_module :/

@victorhora
Copy link
Contributor

I might have misunderstood you (or the issue) @p0pr0ck5, but I thought that you meant that libModSecurity is not supporting LuaJIT?

I'm not affirming that it does or does not support, I was mentioning that by glancing at the current docs of LuaJIT I believe that it might work out of the box. It would be neat if you could share your thoughts here as I didn't had the chance of testing it yet. Thanks :)

@p0pr0ck5
Copy link
Contributor

I will do some testing @victorhora, but the build chain at this point offers no way for libmodsecurity to look for a custom path for Lua headers/objects, nor does it appear to support LuaJIT. So anyone wanting to leverage LuaJIT is out of luck at this point without hacking the configure script.

@victorhora victorhora self-assigned this Jun 21, 2018
@p0pr0ck5
Copy link
Contributor

Some brief updates:

I was able to reproduce the segault reported by @theMiddleBlue this morning with the following Nginx config snippet:

  modsecurity on;
        modsecurity_rules '
            SecRule REQUEST_FILENAME "^/" "phase:2,id:41,pass,exec:/home/poprocks/test.lua"
        ';
$ cat ~/test.lua 
function main()
    m.log(1, "Hello world!")
    return nil
end

And pretty easy to see why:

gdb-peda$ bt
#0  0x00007fc04cbbcc93 in ?? () from /usr/lib/x86_64-linux-gnu/liblua5.2.so.0
#1  0x00007fc04cbaf842 in lua_setglobal () from /usr/lib/x86_64-linux-gnu/liblua5.2.so.0
#2  0x00007fc04e87c2bb in modsecurity::engine::Lua::run (this=this@entry=0xa50878, t=t@entry=0xa38d10) at engine/lua.cc:136
#3  0x00007fc04e865842 in modsecurity::actions::Exec::evaluate (this=0xa50800, rule=<optimized out>, t=0xa38d10)
    at actions/exec.cc:55
#4  0x00007fc04e842fe2 in modsecurity::actions::Action::evaluate (
    ruleMessage=<error reading variable: access outside bounds of object referenced via synthetic pointer>, 
    transaction=0xa38d10, rule=0xa4f100, this=0xa50800) at ../headers/modsecurity/actions/action.h:67
#5  modsecurity::Rule::executeActionsAfterFullMatch (this=this@entry=0xa4f100, trans=trans@entry=0xa38d10, 
    containsBlock=<optimized out>, ruleMessage=std::shared_ptr (count 3, weak 0) 0xaa3260) at rule.cc:713
#6  0x00007fc04e84aeb7 in modsecurity::Rule::evaluate (this=0xa4f100, trans=0xa38d10, 
    ruleMessage=std::shared_ptr (count 3, weak 0) 0xaa3260) at rule.cc:872
#7  0x00007fc04e83e26e in modsecurity::Rules::evaluate (this=0xa4ddc0, phase=phase@entry=0x3, 
    transaction=transaction@entry=0xa38d10) at rules.cc:253
#8  0x00007fc04e827e1f in modsecurity::Transaction::processRequestBody (this=0xa38d10) at transaction.cc:792
#9  0x00007fc04e8296f5 in modsecurity::msc_process_request_body (transaction=<optimized out>) at transaction.cc:1902
#10 0x000000000054a2a3 in ngx_http_modsecurity_pre_access_handler (r=0xa4bd20)
    at /home/poprocks/src/ModSecurity-nginx/src/ngx_http_modsecurity_pre_access.c:199
#11 0x000000000045c4d3 in ngx_http_core_generic_phase (r=0xa4bd20, ph=0xa6a6b0) at src/http/ngx_http_core_module.c:884
#12 0x0000000000457e15 in ngx_http_core_run_phases (r=r@entry=0xa4bd20) at src/http/ngx_http_core_module.c:862
#13 0x0000000000457ef4 in ngx_http_handler (r=r@entry=0xa4bd20) at src/http/ngx_http_core_module.c:845
#14 0x0000000000463579 in ngx_http_process_request (r=0xa4bd20) at src/http/ngx_http_request.c:1950
#15 0x0000000000463f57 in ngx_http_process_request_line (rev=0xa6eef0) at src/http/ngx_http_request.c:1049
#16 0x000000000044bf3c in ngx_epoll_process_events (cycle=0xa33d40, timer=<optimized out>, flags=<optimized out>)
    at src/event/modules/ngx_epoll_module.c:902
#17 0x000000000044196c in ngx_process_events_and_timers (cycle=cycle@entry=0xa33d40) at src/event/ngx_event.c:252
#18 0x0000000000449895 in ngx_worker_process_cycle (cycle=0xa33d40, data=<optimized out>)
    at src/os/unix/ngx_process_cycle.c:820
#19 0x0000000000448229 in ngx_spawn_process (cycle=cycle@entry=0xa33d40, proc=proc@entry=0x449850 <ngx_worker_process_cycle>, 
    data=data@entry=0x0, name=name@entry=0x55056d "worker process", respawn=respawn@entry=0xfffffffffffffffd)
    at src/os/unix/ngx_process.c:198
#20 0x0000000000449c34 in ngx_start_worker_processes (cycle=cycle@entry=0xa33d40, n=0x1, type=type@entry=0xfffffffffffffffd)
    at src/os/unix/ngx_process_cycle.c:396
#21 0x000000000044aaa4 in ngx_master_process_cycle (cycle=cycle@entry=0xa33d40) at src/os/unix/ngx_process_cycle.c:135
#22 0x000000000041fe71 in main (argc=argc@entry=0x1, argv=argv@entry=0x7ffef4826238) at src/core/nginx.c:384
#23 0x00007fc04d842830 in __libc_start_main (main=0x41f430 <main>, argc=0x1, argv=0x7ffef4826238, init=<optimized out>, 
    fini=<optimized out>, rtld_fini=<optimized out>, stack_end=0x7ffef4826228) at ../csu/libc-start.c:291
#24 0x0000000000420049 in _start ()
gdb-peda$ x/20i $rip
=> 0x7fc04cbbcc93:	cmp    eax,DWORD PTR [rdi+0xc]
   0x7fc04cbbcc96:	jb     0x7fc04cbbcca0
   0x7fc04cbbcc98:	jmp    0x7fc04cbbc8a0
   0x7fc04cbbcc9d:	nop    DWORD PTR [rax]
   0x7fc04cbbcca0:	mov    rax,QWORD PTR [rdi+0x10]
   0x7fc04cbbcca4:	movsxd rsi,esi
   0x7fc04cbbcca7:	shl    rsi,0x4
   0x7fc04cbbccab:	lea    rax,[rax+rsi*1-0x10]
   0x7fc04cbbccb0:	ret    
   0x7fc04cbbccb1:	nop    DWORD PTR [rax+rax*1+0x0]
   0x7fc04cbbccb6:	nop    WORD PTR cs:[rax+rax*1+0x0]
   0x7fc04cbbccc0:	movzx  ecx,BYTE PTR [rdi+0xb]
   0x7fc04cbbccc4:	mov    eax,0x1
   0x7fc04cbbccc9:	shl    eax,cl
   0x7fc04cbbcccb:	sub    eax,0x1
   0x7fc04cbbccce:	and    eax,DWORD PTR [rsi+0xc]
   0x7fc04cbbccd1:	cdqe   
   0x7fc04cbbccd3:	lea    rdx,[rax+rax*4]
   0x7fc04cbbccd7:	mov    rax,QWORD PTR [rdi+0x18]
   0x7fc04cbbccdb:	lea    rax,[rax+rdx*8]
gdb-peda$ i r
rax            0x1	0x1
rbx            0x41222378	0x41222378
rcx            0x0	0x0
rdx            0x412239a0	0x412239a0
rsi            0x2	0x2
rdi            0xfffffff441225ed0	0xfffffff441225ed0
rbp            0xa50800	0xa50800
rsp            0x7ffef4823708	0x7ffef4823708
r8             0x0	0x0
r9             0xffffdfff	0xffffdfff
r10            0x7b	0x7b
r11            0x7fc04cbaf820	0x7fc04cbaf820
r12            0xa50878	0xa50878
r13            0x7fc04e8e848f	0x7fc04e8e848f
r14            0x7ffef4823980	0x7ffef4823980
r15            0xa38d10	0xa38d10
rip            0x7fc04cbbcc93	0x7fc04cbbcc93
eflags         0x10202	[ IF RF ]
cs             0x33	0x33
ss             0x2b	0x2b
ds             0x0	0x0
es             0x0	0x0
fs             0x0	0x0
gs             0x0	0x0

Meanwhile the same config with vanilla Nginx 1.15.0 works just fine, with the expected error msg:

2018/06/21 08:27:25 [info] 30065#0: *1 ModSecurity: Warning. Matched "Operator `Rx' with parameter `^/' against variable `REQUEST_FILENAME' (Value: `/' ) [file "<<reference missing or not informed>>"] [line "1"] [id "41"] [rev ""] [msg ""] [data ""] [severity "0"] [ver ""] [maturity "0"] [accuracy "0"] [hostname "127.0.0.1"] [uri "/"] [unique_id "152959484540.792096"] [ref "o0,1v4,1"], client: 127.0.0.1, server: localhost, request: "GET / HTTP/1.1", host: "localhost:8000"

Next step was to modify the autogenerated Modsecurity configure script to look for the OpenResty-included LuaJIT and attempt to build with this:

poprocks@mini-vm:~/src/ModSecurity$ grep LUA_POSSIBLE_PATHS configure | head -1
LUA_POSSIBLE_PATHS="/usr/lib /usr/local/lib /usr/local/lib64 /usr/local/lua /usr/local/liblua /usr/local /opt /usr /usr/lib64 /opt/local /home/poprocks/src/openresty-1.13.6.2/build/LuaJIT-2.1-20180420/src"

But the configuration fails:

poprocks@mini-vm:~/src/ModSecurity$ ./configure --with-lua=yes
checking for a BSD-compatible install... /usr/bin/install -c
checking whether build environment is sane... yes
checking for a thread-safe mkdir -p... /bin/mkdir -p
checking for gawk... no
checking for mawk... mawk
checking whether make sets $(MAKE)... yes
checking whether make supports nested variables... yes
checking for g++... g++
checking whether the C++ compiler works... yes
checking for C++ compiler default output file name... a.out
checking for suffix of executables... 
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C++ compiler... yes
checking whether g++ accepts -g... yes
checking for style of include used by make... GNU
checking dependency style of g++... gcc3
checking for gcc... gcc
checking whether we are using the GNU C compiler... yes
checking whether gcc accepts -g... yes
checking for gcc option to accept ISO C89... none needed
checking whether gcc understands -c and -o together... yes
checking dependency style of gcc... gcc3
checking for ar... ar
checking the archiver (ar) interface... ar
checking whether make sets $(MAKE)... (cached) yes
checking for "others/libinjection/src/libinjection_html5.c"... yes
checking for pkg-config... /usr/bin/pkg-config
checking pkg-config is at least version 0.9.0... yes
configure: Nothing about GeoIP was informed during the configure phase. Trying to detect it on the platform...
configure: using YAJL v2.1.0
configure: Nothing about GeoIP was informed during the configure phase. Trying to detect it on the platform...
configure: GeoIP library was not found
configure: Nothing about MaxMind was informed during the configure phase. Trying to detect it on the platform...
configure: MaxMind library was not found
configure: Nothing about LMDB was informed during the configure phase. Trying to detect it on the platform...
*** LOOKING AT PATH:  /usr/lib
*** LOOKING AT PATH:  /usr/local/lib
*** LOOKING AT PATH:  /usr/local/liblmdb
*** LOOKING AT PATH:  /usr/local/lmdb
*** LOOKING AT PATH:  /usr/local
*** LOOKING AT PATH:  /opt/liblmdb
*** LOOKING AT PATH:  /opt/lmdb
*** LOOKING AT PATH:  /opt
*** LOOKING AT PATH:  /usr
*** LOOKING AT PATH:  /usr/lib64
*** LOOKING AT PATH:  /opt/local
configure: LMDB library was not found
*** LOOKING AT PATH:  /usr/lib
*** LOOKING AT PATH:  /usr/local/lib
*** LOOKING AT PATH:  /usr/local/fuzzy
*** LOOKING AT PATH:  /usr/local/libfuzzy
*** LOOKING AT PATH:  /usr/local
*** LOOKING AT PATH:  /opt
*** LOOKING AT PATH:  /usr
*** LOOKING AT PATH:  /usr/lib64
*** LOOKING AT PATH:  /opt/local
configure: SSDEEP library was not found
configure: LUA support was marked as mandatory by the utilization of --with-lua=yes
*** LOOKING AT PATH:  /usr/lib
*** LOOKING AT PATH:  /usr/local/lib
*** LOOKING AT PATH:  /usr/local/lib64
*** LOOKING AT PATH:  /usr/local/lua
*** LOOKING AT PATH:  /usr/local/liblua
*** LOOKING AT PATH:  /usr/local
*** LOOKING AT PATH:  /opt
*** LOOKING AT PATH:  /usr
configure: LUA library found at: /usr/lib/x86_64-linux-gnu//liblua5.2.so.0.0.0
*** LOOKING AT PATH:  /usr/lib64
configure: LUA library found at: /usr/lib/x86_64-linux-gnu//liblua5.2.so.0.0.0
*** LOOKING AT PATH:  /opt/local
configure: LUA library found at: /usr/lib/x86_64-linux-gnu//liblua5.2.so.0.0.0
*** LOOKING AT PATH:  /home/poprocks/src/openresty-1.13.6.2/build/LuaJIT-2.1-20180420/src
configure: LUA library found at: /home/poprocks/src/openresty-1.13.6.2/build/LuaJIT-2.1-20180420/src//libluajit.so
configure: LUA headers found at: /home/poprocks/src/openresty-1.13.6.2/build/LuaJIT-2.1-20180420/src
configure: LUA_VERSION is 501 found at: /home/poprocks/src/openresty-1.13.6.2/build/LuaJIT-2.1-20180420/src
configure: error: LUA was explicitly referenced but LUA v5.1 was found and it is not currently supported on libModSecurity. LUA_VERSION: 501

Modifying the configure script to ignore 501 as an unusable version results in the following configure output:

ModSecurity - v3.0.2-64-ga3980bb for Linux
 
 Mandatory dependencies
   + libInjection                                  ....v3.0.2-64-ga3980bb
   + SecLang tests                                 ....a3980bb
 
 Optional dependencies
   + GeoIP/MaxMind                                 ....not found
   + LibCURL                                       ....found v7.47.0 
      -L/usr/lib/x86_64-linux-gnu -lcurl,  -DWITH_CURL_SSLVERSION_TLSv1_2 -DWITH_CURL
   + YAJL                                          ....found v2.1.0
      -lyajl, -DWITH_YAJL -I/usr/include/yajl
   + LMDB                                          ....not found
   + LibXML2                                       ....found v2.9.3
      -lxml2, -I/usr/include/libxml2 -DWITH_LIBXML2
   + SSDEEP                                        ....not found
   + LUA                                           ....found v501
      -lluajit -L/home/poprocks/src/openresty-1.13.6.2/build/LuaJIT-2.1-20180420/src/, -DWITH_LUA -DWITH_LUA_5_1 -I/home/poprocks/src/openresty-1.13.6.2/build/LuaJIT-2.1-20180420/src
 
 Other Options
   + Test Utilities                                ....enabled
   + SecDebugLog                                   ....enabled
   + afl fuzzer                                    ....disabled
   + library examples                              ....enabled
   + Building parser                               ....disabled
   + Treating pm operations as critical section    ....disabled

And compilation fails as follows:

libtool: compile:  g++ -DHAVE_CONFIG_H -I. -std=c++11 -I.. -g -I../others -fPIC -O3 -I../headers -DWITH_YAJL -I/usr/include/yajl -DPCRE_HAVE_JIT -DWITH_LUA -DWITH_LUA_5_1 -I/home/poprocks/src/openresty-1.13.6.2/build/LuaJIT-2.1-20180420/src -I/usr/include/libxml2 -DWITH_LIBXML2 -g -O2 -MT collection/backend/libmodsecurity_la-in_memory-per_process.lo -MD -MP -MF collection/backend/.deps/libmodsecurity_la-in_memory-per_process.Tpo -c collection/backend/in_memory-per_process.cc  -fPIC -DPIC -o collection/backend/.libs/libmodsecurity_la-in_memory-per_process.o
engine/lua.cc: In member function ‘bool modsecurity::engine::Lua::load(std::__cxx11::string, std::__cxx11::string*)’:
engine/lua.cc:88:75: error: too many arguments to function ‘int lua_dump(lua_State*, lua_Writer, void*)’
     if (lua_dump(L, Lua::blob_keeper, reinterpret_cast<void *>(&m_blob), 0)) {
                                                                           ^
In file included from /home/poprocks/src/openresty-1.13.6.2/build/LuaJIT-2.1-20180420/src/lua.hpp:4:0,
                 from ../src/engine/lua.h:17,
                 from engine/lua.cc:17:
/home/poprocks/src/openresty-1.13.6.2/build/LuaJIT-2.1-20180420/src/lua.h:208:14: note: declared here
 LUA_API int (lua_dump) (lua_State *L, lua_Writer writer, void *data);
              ^
engine/lua.cc: In member function ‘int modsecurity::engine::Lua::run(modsecurity::Transaction*)’:
engine/lua.cc:142:13: error: too many arguments to function ‘int lua_load(lua_State*, lua_Reader, void*, const char*)’
         NULL);
             ^
In file included from /home/poprocks/src/openresty-1.13.6.2/build/LuaJIT-2.1-20180420/src/lua.hpp:4:0,
                 from ../src/engine/lua.h:17,
                 from engine/lua.cc:17:
/home/poprocks/src/openresty-1.13.6.2/build/LuaJIT-2.1-20180420/src/lua.h:205:16: note: declared here
 LUA_API int   (lua_load) (lua_State *L, lua_Reader reader, void *dt,
                ^
engine/lua.cc:153:18: error: ‘LUA_ERRGCMM’ was not declared in this scope
             case LUA_ERRGCMM:
                  ^
engine/lua.cc: In static member function ‘static std::__cxx11::string modsecurity::engine::Lua::applyTransformations(lua_State*, modsecurity::Transaction*, int, std::__cxx11::string)’:
engine/lua.cc:405:37: error: ‘lua_rawlen’ was not declared in this scope
         int i, n = lua_rawlen(L, idx);
                                     ^
Makefile:2543: recipe for target 'engine/libmodsecurity_la-lua.lo' failed
make[3]: *** [engine/libmodsecurity_la-lua.lo] Error 1

I will dig into the Lua engine a bit and see what I can find.

@p0pr0ck5
Copy link
Contributor

Okay, a bit more hacking and I've got this working as PoC.

First, I edited the autogenerated configure script to replace the WITH_LUA_5_1 def with WITH_LUA_5_2, to simplify things. Then I tweaked the few places where there was a compiler failure:

$ git diff src/engine/lua.cc
diff --git a/src/engine/lua.cc b/src/engine/lua.cc
index cabdbea..0850611 100644
--- a/src/engine/lua.cc
+++ b/src/engine/lua.cc
@@ -138,8 +138,12 @@ int Lua::run(Transaction *t) {
     luaL_setfuncs(L, mscLuaLib, 0);
     lua_setglobal(L, "m");
 
+#ifdef WITH_LUA_5_2
+    int rc = lua_load(L, Lua::blob_reader, &m_blob, m_scriptName.c_str());
+#else
     int rc = lua_load(L, Lua::blob_reader, &m_blob, m_scriptName.c_str(),
         NULL);
+#endif
     if (rc != LUA_OK) {
         std::string e;
         e.assign("Failed to execute lua script: " + m_scriptName + ". ");
@@ -150,9 +154,11 @@ int Lua::run(Transaction *t) {
             case LUA_ERRMEM:
                 e.assign("Memory error. ");
                 break;
+#ifndef WITH_LUA_5_2
             case LUA_ERRGCMM:
                 e.assign("Garbage Collector error. ");
                 break;
+#endif
         }
         e.append(lua_tostring(L, -1));
 #ifndef NO_LOGS
@@ -402,7 +408,7 @@ std::string Lua::applyTransformations(lua_State *L, Transaction *t,
 
     if (lua_istable(L, idx)) {
         const char *name = NULL;
-        int i, n = lua_rawlen(L, idx);
+        int i, n = lua_objlen(L, idx);
 
         for (i = 1; i <= n; i++) {
             lua_rawgeti(L, idx, i);

This diff is absolutely not correct and needs a lot of work (particularly the #ifndef WITH_LUA_5_2), but it covers the failures. After this I was able to compile successfully. I updated the modsec rules to test transformation functions as well, since we made a small compat change in applyTransformations:

        modsecurity on;
        modsecurity_rules '
            SecDebugLog /tmp/modsec_debug.log
            SecDebugLogLevel 9
            SecRule REQUEST_FILENAME "^Qgmb" "phase:2,id:41,pass,exec:/home/poprocks/test.lua,t:sha1,t:base64encode"
        ';

And according to the debug logs we do indeed see transformations applying successfully and the Lua script executing, using the injected m global to write a log entry:

[4] Initializing transaction
[4] Transaction context created.
[4] Starting phase CONNECTION. (SecRules 0)
[9] This phase consists of 0 rule(s).
[4] Starting phase URI. (SecRules 0 + 1/2)
[4] Starting phase REQUEST_HEADERS.  (SecRules 1)
[9] This phase consists of 0 rule(s).
[4] Starting phase REQUEST_BODY. (SecRules 2)
[4] Request body processing is disabled
[9] This phase consists of 1 rule(s).
[4] (Rule: 41) Executing operator "Rx" with param "^Qgmb" against REQUEST_FILENAME.
[9]  T (0) t:sha1: "B	�J�!�?��Nl%h�����"
[9]  T (1) t:base64encode: "QgmbSvAh5T/Y/U4FbCVo18Lj/6g="
[9] Target value: "QgmbSvAh5T/Y/U4FbCVo18Lj/6g=" (Variable: REQUEST_FILENAME)
[9] Matched vars updated.
[4] Rule returned 1.
[4] Not running disruptive action: pass. SecRuleEngine is not On
[4] Running (non-disruptive) action: exec
[8] Running script... /home/poprocks/test.lua
[1] Hello world!
[9] Returning from lua script: 
[4] Starting phase RESPONSE_HEADERS. (SecRules 3)
[9] This phase consists of 0 rule(s).
[9] Appending response body: 562 bytes. Limit set to: 0.000000
[4] Starting phase RESPONSE_BODY. (SecRules 4)
[4] Response body is disabled, returning... 2
[4] Starting phase LOGGING. (SecRules 5)
[9] This phase consists of 0 rule(s).
[8] Checking if this request is suitable to be saved as an audit log.
[8] Checking if this request is relevant to be part of the audit logs.
[5] Audit log engine was not set.
[8] Request was relevant to be saved. Parts: 4430

Oh, and to be sure, we are indeed using the appropriate Lua lib:

poprocks@mini-vm:~/openresty/nginx$ ldd ./sbin/nginx | grep -i lua
	libluajit-5.1.so.2 => /home/poprocks/openresty/luajit/lib/libluajit-5.1.so.2 (0x00007f5ba93f9000)

Hopefully this is of some use folks.

@victorhora
Copy link
Contributor

Lua 5.1 is now supported. Thanks @p0pr0ck5 :)

I've just pushed PR #1854 to also support luaJIT which should hopefully solve this issue once and for all.

My tests went fine on a couple of distros, but it would be good if you folks could check on other distros to make sure we are covering the major ones.

@victorhora
Copy link
Contributor

LuaJIT should be supported as of 857bf9d

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
3.x Related to ModSecurity version 3.x RIP - libmodsecurity
Projects
None yet
Development

No branches or pull requests

4 participants