Dear Ninja team,

(edit: damn, I should have used the search function. It seems that this is a known issue:
#520
#53
So should this be fixed in Meson, then?)

while trying to port a large codebase (> 10000 files) to the Meson build system on Linux, I noticed that really long build lines can fail without providing an error message explaining the problem. This can easily be replicated by, e.g., the following (useless) Ninja build file with files main.c and test.c:

cc = gcc 

rule build
  command = $cc $in -o $out

build prog: build main.c test.c test.c test.c ... [replicate thousands of times]

$ ninja
... test.c [replicate thousands of times] ... -o prog
ninja: build stopped: subcommand failed.

More specifically, it seems to fail with line lengths longer than around 128 kiB. Having done some digging, I suspect that Ninja hits the MAX_ARG_STRLEN limit of the Linux kernel [1, 2, 3],

Number of arguments and maximum length of one argument
At least on Linux 2.6, there's also a limit on the maximum number of arguments in argv[].
On Linux 2.6.14 the function do_execve() in fs/exec.c tests if the number exceeds

PAGE_SIZE*MAX_ARG_PAGES-sizeof(void *) / sizeof(void *)

On a 32-bit Linux, this is ARGMAX/4-1 (32767). This becomes relevant if the average length of arguments is smaller than 4.

Since Linux 2.6.23, this function tests if the number exceeds MAX_ARG_STRINGS in <linux/binfmts.h> (2^32-1 = 4294967296-1).

And as additional limit since 2.6.23, one argument must not be longer than MAX_ARG_STRLEN (131072).
This might become relevant if you generate a long call like "sh -c 'automatically generated with many arguments'".
(pointed out by Xan Lopez and Ralf Wildenhues)

Without really knowing too much about the Ninja source, I would suspect the following lines to cause this,

const char* spawned_args[] = { "/bin/sh", "-c", command.c_str(), NULL };
  if (posix_spawn(&pid_, "/bin/sh", &action, &attr,
                  const_cast<char**>(spawned_args), environ) != 0)
Fatal("posix_spawn: %s", strerror(errno));

It seems that the build line is passed to /bin/sh as a single command line argument. As explained above, this might hit a limit at some point.

Is this analysis correct? If so, do I have to work around this limitation somehow or could this be changed in a future Ninja version? If I understand correctly, the argument will have to be split, or the options will have to be passed via STDIN for really long commands to work.

[1] https://www.in-ulm.de/~mascheck/various/argmax/
[2] http://lxr.free-electrons.com/source/include/uapi/linux/binfmts.h#L14
[3] short test:

$ /bin/echo "$(printf "%*s" 131071 ".")">/dev/null
$ /bin/echo "$(printf "%*s" 131072 ".")">/dev/null
-bash: /bin/echo: Argument list too long