Drop z_ prefix as identifier for internal/private functions and revert back to _

[nashif]

Introduction

Revert back to old naming conventions and use _ for private functions and drop the usage of z_ as a prefix in zephyr completely.

Problem description

The z_ as prefix for internal/private functions is misleading and giving the impression functions with z_ are public APIs.
The use of z_ was introduced to deal with some coding guideline violations. The project has since decided to not follow this rule and it is not part of the coding guidelines right now (need to check).

Proposed change

Go back to _ (underscore) as an identifier for private functions and remove all usage of z_ in zephyr.

Detailed RFC

This is going to be a treewide change that will clarify zephyr APIs and avoid any confusion about what functions should be used out of tree. This is not going to be an API change per se, it will however cause some issues with users depending on internal functions out of tree.

The scope of private functions should also be clarified as part of this RFC and if we should allow usage of internal/private calls across the tree, i.e. subsystem A calls private functions for subsystem B.

Concerns and Unresolved Questions

Major change that needs lots of preparating and discussions.

Alternatives

document clearly the z_ prefix and do not allow exporting functions with z_ in public APIs.

[carlescufi]

CC @stephanosio @aescolar @gmarull @cfriedt @andyross @galak

[cfriedt]

Just a heads up (and I know this isn't a link to the ISO C standard), but there are some related rules for prefixing symbols with an underscore. I have not looked (at least recently) to verify / locate where they are in the ISO C standard.

https://stackoverflow.com/a/39625502

• "You cannot define any identifiers in global scope whose names begin with an underscore, because these may conflict with hidden (private) library definitions"
• "You cannot define any identifiers in any scope whose names begin with two underscores, or one underscore followed by a capital letter"

It's possible that the double underscore point might have some flexibility because typically those are reserved by the "system" which would very much include Zephyr.

However, single underscore followed by a capital letter could very well result in a number of collisions (e.g. POSIX feature test macros). I'm not entirely sure if there are other standard names of the form _[A-Z][a-zA-Z0-9]* but just wanted to suggest that we do not use identifiers of that form.

[andyross]

I never liked z_ and think this would be nicer. Double or single underscore doesn't matter much to me. But I do point out that I really (really) hate the churn from treewide syntax changes like this and think we should implement some kind of blood oath to be taken by all involved to ensure this is the last time we're going to do this.

[stephanosio]

Apart from what @cfriedt already pointed out (_ prefixed identifiers being "reserved" by the ISO C standard), declaring identifiers that start with _ is a violation of the MISRA C:2012 R.21.2 "A reserved identifier or reserved macro name shall not be declared," and subsequently the Zephyr coding guidelines Rule 21.2.

[henrikbrixandersen]

Would this be a new policy going forward or do you intend to change existing code?

[gmarull]

An approach I used in other projects in the past is the one from libgit2: https://github.com/libgit2/libgit2/blob/main/docs/conventions.md

ie, public functions look like mylib_subsys_public() and internals/semi-private use mylib_subsys__internal() (double underscore). Problem is that Zephyr has never used a standard prefix like z_, so we can't have z_gpio_get() and z_gpio__something_internal()...

[aescolar]

My 2 cents:
I think this boils down to the same issue we discussed a few times:

• Any symbol/macro we define globally has a chance of colliding with another library/app symbol/macro. The only nice way to minimize this, is to namespace everything zephyr defines out of the way, as we do in many/most areas.
• namespacing symbols with just _ or __, apart from guidelines like MISRA, is also dangerous, as we are likely to hit C libraries and others who feel they own the "reserved" areas.
• z_ is indeed not evidently indicating "internal". Something like zint_ or kint_ would have been, or even _k_ or _z_
• There is the reasoning of "Zephyr is the OS and therefore it is important enough to define any reserved names it wants. And, in case of conflicts it is others who should fix their naming". but, IMHO, this is just asking for trouble.

My take would be to either not mess around with the z_ prefix or agree something that does not start with an underscore, but evidently implies it is internal and stick with it.

But I do point out that I really (really) hate the churn from treewide syntax changes like this and think we should implement some kind of blood oath to be taken by all involved to ensure this is the last time we're going to do this.

I fully agree.

An olden link:
https://github.com/zephyrproject-rtos/zephyr/wiki/Naming-Conventions

[nashif]

great input, I am well aware of all those constraints and issue with standards, but the current situation is bad enough IMO and this z_ prefix is very confusing and I have seen private functions being used as if they were public APIs. Most notable example is https://github.com/zephyrproject-rtos/zephyr/blob/main/include/zephyr/sys/fdtable.h, where some has decided to use z_ as the preferred prefix for zephyr public APIs...

[ejohnso49]

As a user, I'd say I haven't hit a ton of confusion understanding that z_ prefixed functions are private and the complications of their use. As others have mentioned, I would be worried about the churn. Regarding the prevalence of the misuse of z_ for public functions, is it feasible to change the misuse instead?

[dkalowsk]

If there is going to be a change, why make a change that is going to actively impact the efforts of multiple other individuals/teams working on enabling Zephyr for passing industry safety standards? As @andyross points out, there is likely one last chance (before end users and developers revolt) to make this type of change again. If this is going to change, picking something more clear zint_ or zpriv_ would be a better solution for all parties.

[cfriedt]

I've been doing this in a couple of places already, but for reserved, internal Zephyr API, I believe the __z_ prefix should be OK in that it does not violate any ISO C rules.

[ceolin]

Have we decided which direction we are going ?

@aescolar made good points. We have being fixing some symbols collision in an ad-hoc way, here is one more #78906 needed.

I honestly don't think we can take the approach that we are the OS and others should adapt to that. It seems to me that we need a proper namespace for pretty much everything.

[cfriedt]

I've been doing this in a couple of places already, but for reserved, internal Zephyr API, I believe the __z_ prefix should be OK in that it does not violate any ISO C rules.

Just wanted to follow-up - my __z_ statement above seems to be wrong:

in C++, identifiers with a double underscore anywhere are reserved everywhere; in C, only the ones that begin with a double underscore are reserved.

The use of two underscores (`__') in identifiers is reserved for the compiler's internal use according to the ANSI-C standard

I think _z_ may also be ok, since it's reserved, but as an OS, Zephyr should be able to use part of the reserved namespace. As Anas pointed out, ThreadX uses _tx_ (and _nx_ for NetX, IIRC).

[carlescufi]

Architecture WG:

Options:

• _z_ and __z_ are problematic because they are reserved, but we could consider an exception for those
• zpriv_ and zint_ are alternatives that can be considered
• We need to consider whether using an exception is the right path forward
• Other RTOS (e.g. ThreadX) use _

[aescolar]

I'd just forget about the _* ones to close the discussion about reserved namespace, as that discussion would keep happening every X months.
I'd recommend to just list a few options for the private one and do a vote.

[carlescufi]

Architecture WG:

• The following links are analyzed to understand whether _ as a prefix (with lower case after) could be compatible with the standards:

  • https://www.dii.uchile.cl/~daespino/files/Iso_C_1999_definition.pdf (7.1.3)
  • https://www.gnu.org/software/libc/manual/html_node/Reserved-Names.html
  • https://devblogs.microsoft.com/oldnewthing/20230109-00/?p=107685

• We delay the decision until next week to leave some time for additional investigation

[carlescufi]

Architecture WG:

• When we transitioned from underscore (_) to z_, many functions got renamed that perhaps should not have
• Vote on: Should we move away from z_ as a prefix for internal/private in not auto-generated code?
  • Motion: @carlescufi , second: @erwango
  • @henrikbrixandersen Absent
  • @pdgendt Abstain
  • @stephanosio Absent
  • @fabiobaltieri Abstain
  • @kartben Abstain
  • @nashif Yes
  • @carlescufi Yes
  • @jhedberg Yes
  • @erwango Yes
  • @MaureenHelm Yes
  • @cfriedt Abstain
  • @decsny Abstain
• Vote passes, next step is what we replace it with.

[jhedberg]

next step is what we replace it with

I think I mentioned this in some past meeting, and someone had a counterargument, which I forgot, so I'll repeat it: what we essentially have is the desire to represent two independent attributes of an API through some naming convention:

• public/private
• which subsystem the API belongs to

There seems to be a consensus that the the naming convention should be based on prefixes, however I don't see it as self-evident that the order of these prefixes should be "public/private first, subsystem second". E.g. I could perfectly well imagine using bt_priv_* for private Bluetooth APIs. Anyway, if someone wants to repeat the counterargument to this here, feel free, so we have it documented and can move on, or do people think that this alternative is worth considering as well?