toolchain: __used macro for GCC clashes with host (glibc) regex structure (when building for native_posix)

[wirehell]

Describe the bug

When trying to test code using regexps I discovered that when running under ztest the group matching did not work for posix target. Update: see this comment for details and reproduction.

A minimal test case is this snippet:

        int err;
	regex_t re;
	err = regcomp(&re, "He(ll)o", REG_EXTENDED);
  	printf("Err: %d\n", err);
  	printf("re_nsub: %lu\n", re.re_nsub);

My observations are:
a) If this is run as a standalone program, re.re_nsub will correctly be 1
b) If this code is run as a part of a ztest, re.re_nsub will equal 0 (any subsequent matching will fail)
c) If this code is run as a part of a test, but in an external function, it will correctly be 1
d) If this code is run as a part of a test, but from a function defined in the test file, there will be a *** stack smashing detected ***: terminated

• Target is native_posix_64 (same behavior with native_posix)
• Tried on hardware with nrf52840dk_nrf52840dk with CONFIG_PICOLIBC=y, this works as expected

To Reproduce
Steps to reproduce the behavior:

1. Clone https://github.com/wirehell/ztest-regex-issue
2. west build -p -b native_posix_64
3. ./build/zephyr/zephyr.elf
4. See output
5. (Optional) define CRASH in src/main.c and recompile/run to see the stack smashing issue.

Expected behavior
I expect regex to work correctly.

Logs and console output

*** Inconsistent regcomp ***

*** Booting Zephyr OS build zephyr-v3.3.0 ***
Running TESTSUITE parser_suite
===================================================================
START - test_regex_external_fun
Err: 0
re_nsub: 1
 PASS - test_regex_external_fun in 0.000 seconds
===================================================================
START - test_regex_inline
Err: 0
re_nsub: 0

    Assertion failed at WEST_TOPDIR/zephyr/samples/bluetooth/blep1/ztest-regex-issue/src/test.c:13: parser_suite_test_regex_inline: (re.re_nsub equal to 0)
There really is a match group
 FAIL - test_regex_inline in 0.000 seconds
===================================================================
TESTSUITE parser_suite failed.

Smash stacking issue

*** Booting Zephyr OS build zephyr-v3.3.0 ***
Running TESTSUITE parser_suite
===================================================================
START - test_regex3_fun
Err: 0
re_nsub: 0
*** stack smashing detected ***: terminated
Aborted (core dumped)

Environment (please complete the following information):

• OS: Ubuntu 22.04
• Toolchain: Zephyr SDK
• Tag v3.3.0

[wirehell]

This issue is not only for ztest, so updated name on issue. I've debugged the code and the issue seems to happen on return from regcomp, inside regcomp everything looks fine.

I found this odd thing.
In function calling regcomp:

-exec print &re
$2 = (regex_t *) 0x7ffff73fdcf0
-exec print &re->re_nsub
$3 = (size_t *) 0x7ffff73fdd18

In regcomp itself (&re is passed as *preg):

-exec print preg
$4 = (regex_t * restrict) 0x7ffff73fdcf0
-exec print &preg->re_nsub
$5 = (size_t *) 0x7ffff73fdd20

[wirehell]

While playing around it started to work when the include files was reordered.

Fails:

#include <zephyr/ztest.h>
#include <regex.h>

Works:

#include <regex.h>
#include <zephyr/ztest.h>

[wirehell]

Ok, I think I have found the actual issue now by bisecting the includes.

The error happens if this line in <zephyr/toolchain/gcc.h>
#define __used __attribute__((__used__))
is included before the definition of struct re_pattern_buffer in regex.h

To see why, let's look at the field __re_long_size_t:

  __re_long_size_t __REPB_PREFIX(used);

__REPB_PREFIX is expanded by # define __REPB_PREFIX(name) __##name to __used

The __used is expanded by the line in gcc.h to __attribute__((__used__)) which is a GCC variable attribute. This causes the field to be ignored.

This snippet illustrates the result:

#include <stdio.h>
#include <stdint.h>

struct sa {
  uint8_t __used;
  uint8_t a;
};

#include <zephyr/toolchain/gcc.h>

struct sb {
  uint8_t __used;
  uint8_t a;
};

int main() {
  printf("sa size: %d\n", sizeof(struct sa));
  printf("sb size: %d\n", sizeof(struct sb));
  return 0;
}

Output:

*** Booting Zephyr OS build zephyr-v3.3.0 ***
sa size: 2
sb size: 1

[wirehell]

Workaround for this is to always include regex.h before any zephyr headers.

[stephanosio]

Thanks for the detailed analysis in #55593 (comment).

As explained in the comment above, the issue has to do with an implementation detail of the (host) glibc regex.h -- defining a struct member named __used, which is a macro defined by the Zephyr's toolchain abstraction layer.

I am inclined to mark and close this as a known issue because:

1. This issue is only observed on the "native POSIX" targets where the host operating system uses the glibc.
2. Renaming the Zephyr toolchain abstraction layer __used because of an implementation detail on a testing platform (i.e. native_posix) is difficult to justify.
3. POSIX regex.h is not a very commonly used feature.

cc @aescolar @cfriedt @tejlmand

[github-actions]

This issue has been marked as stale because it has been open (more than) 60 days with no activity. Remove the stale label or add a comment saying that you would like to have the label removed otherwise this issue will automatically be closed in 14 days. Note, that you can always re-open a closed issue at any time.

[aescolar]

Based on the same reasoning as #55593 (comment) I would also be inclined to close this as "won't fix".
Otherwise, really this is just another side of the zephyr naming collisions issue, and I think the only way forward is to prioritize a discussion about reserved macro naming, probably together with the internal function naming ( #58007 ), and overall zephyr naming.

[github-actions]

This issue has been marked as stale because it has been open (more than) 60 days with no activity. Remove the stale label or add a comment saying that you would like to have the label removed otherwise this issue will automatically be closed in 14 days. Note, that you can always re-open a closed issue at any time.