[RFC] Missing parts of libc required for CivetWeb

[PiotrZierhoffer]

Introduction

This issue is created as prompted by the discussion on the Networking Forum.

We are working on implementing support for CivetWeb HTTP library (https://github.com/civetweb/civetweb), but it requires improvements in the libc layer. It's a good example of an external project using sockets APIs that should be relatively easy to integrate with Zephyr, as a proof of concept.

Problem description

CivetWeb uses different POSIX API calls, as any application or library runnable on Linux.

Using Newlib is not an option due to heavy conflicts with the POSIX layer (see #13787).

Improving minimal-libc requires some consideration to maintain compatibility with other layers.

Here we list features missing from minimal-libc that would be required for CivetWeb to work.

Detailed RFC

Here is the list of functions that are used by CivetWeb, with pull requests were applicable.
For some we will prepare stub implementations that will be enough to get the library to work.

• strcspn (Add strcspn and iscntrl to minimal libc #16451)
• iscntrl (Add strcspn and iscntrl to minimal libc #16451)
• strerror (libc: add strerror() to minimal libc #16527)
• strtoull, strtoll (libc: add strtoll and strtoull functions #16511)
• atof (we have the implementation ready, will publish it with [DNM] for reference)
• sscanf
• localtime
• gmtime
• time
• difftime

Dependencies

This is relevant to the POSIX and libc layers. Also time Infrastructure is affected.

CC: @pfalcon @nashif @rakons @andrewboie @jukkar @laperie @tgorochowik

Concerns and Unresolved Questions

The main purpose of minimal-libc is to be, well, minimal. Further work should preserve this approach.

Time related functions rely on a concept of current time. Right now there is no such thing in Zephyr.

There is a proposal by @pfalcon we were about to follow, using the work he submitted via #15602

Current status

As suggested by @nashif (see #16451), we will now proceed with including relevant functions in the sample itself.

[andrewboie]

Using Newlib is not an option due to heavy conflicts with the POSIX layer (see #13787).

To me this seems like what we should be spending effort on fixing.

[pfalcon]

Using Newlib is not an option due to heavy conflicts with the POSIX layer (see #13787).

To me this seems like what we should be spending effort on fixing.

Not only that, but the quoted issue, #13787, was made to draw attention to the problem, and gather interested parties to brainstorm and solve the issue. Not to provide interested parties with excuses for not doing that ;-).

[PiotrZierhoffer]

No excuses here, we'd love to see newlib and posix support to be top notch :)

And I think we discussed that already on the networking forum and there is an agreement in that matter. It doesn't mean though that this is the first step we should do, or can do, to keep the topic active.

@laperie asked me to create this issue to keep track of things. I added you to CC because we discussed these topics earlier.

[pfalcon]

create this issue to keep track of things. I added you to CC because we discussed these topics earlier.

This is absolutely great that you posted this ticket and also RFC on the mainling list! The same should have been be done regarding CivetWeb (for as long as that effort is supposed to influence Zephyr back, and not just independent porting effort).

However, please kindly accept the possibility to receive criticism as a result of posting this RFC - that's done both for common good, and to potentially save your own efforts from going in the wrong direction.

[pfalcon]

And I think we discussed that already on the networking forum

I find that discussion such insightful that I decided to put off other tasks assigned to me and cleaned up and flushed a number of patches I had in my queue:

• posix: Add headers related to BSD Sockets API #16621
• posix: Clean up various headers #16626
• posix: unistd.h: Add gethostname() #16009
• lib: posix: Switch to use zephyr_interface_library_named cmake directive #15937

I didn't lie however that POSIX/Newlib issues isn't going to be fixed, unless people will contribute to that fixing. It can be as small as testing and voting up, but it has to be there. For example, my patches above turn into the usual bikeshed playground (and I see the same happening with @tgorochowik's and @PiotrZierhoffer's patches). Again, there should be active interest to get that into working state, and common approach and understanding, or it will be drown in Zephyr's ever-growing patch well.

[PiotrZierhoffer]

@pfalcon, by all means, your insight and comments are always very welcome!

[pfalcon]

This is absolutely great that you posted this ticket and also RFC on the mainling list!

Oops, multitasking too much. I saw a github notification in email, and my mind tricked me into seeing the desirable - that it was also posted on the Zephyr "devel" mailing list. Well, I'd suggest to do that anyway, even if a quick link to this ticket. Doing so helps the much wider audience to stay in loop on Zephyr's developments and progress. And as pointed several times already, gathering critical mass is a muss for such projects. Thanks in advance.

[PiotrZierhoffer]

@pfalcon , yeah, about multitasking - I missed your message, sorry for that. I will post to devel at once.

[pfalcon]

@PiotrZierhoffer:

@pfalcon , yeah, about multitasking - I missed your message, sorry for that. I will post to devel at once.

Many thanks for posting, and sorry for lack of reply from my side - company sprint, followed by vacation. That's definitely helpful, as we clearly have different parties interested in this matter, while not aware of the current state of affairs, so any publicity efforts like that are required, even if seemingly don't lead to immediate responses.

I, from my side, have been working on "POSIX subsys maintenance approach RFC" which I promised, and I hope to post it shortly (definitely before going for another vacation streak by the end of next week).

[gmarull]

civetweb is no longer a module, closing