Skip to content

Zephyr SDK ships with very outdated newlib for Xtensa #64

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
pfalcon opened this issue Apr 24, 2019 · 18 comments
Closed

Zephyr SDK ships with very outdated newlib for Xtensa #64

pfalcon opened this issue Apr 24, 2019 · 18 comments

Comments

@pfalcon
Copy link
Contributor

pfalcon commented Apr 24, 2019

I'm working on elaborating POSIX subsystem in Zephyr and unifying its support across different config options (different libc's, different archs, etc.) One particular idea is to make sure that minlibs contained the same internal (implementation) headers as newlib, so that higher-level, public POSIX headers can work interchangeably with either libc. One example of such a header is sys/_timeval.h, which defines struct timeval in newlib. After introducing and switching to such header for minlibc and switching to it in (initial version of) zephyrproject-rtos/zephyr#15628 , I got a failure with qemu_xtensa and newlib: https://app.shippable.com/github/zephyrproject-rtos/zephyr/runs/40119/5/tests .

Turns out, newlib for xtensa in SDK 0.10.0 doens't have that header. But it was introduced in 2015: https://sourceware.org/git/gitweb.cgi?p=newlib-cygwin.git;a=commit;h=bb0159489785d577ad0b8061a1ba7956ee0f89d0 . That means that newlib as used by xtensa is very, very old.

Discrepancies like this make it very hard to go forward with POSIX subsystem development, as almost any change leads to failures from undermaintained architectures like xtensa.

@pfalcon pfalcon changed the title Zephyr SDK 0.10.0 ships with very outdated newlib Zephyr SDK 0.10.0 ships with very outdated newlib for Xtensa Apr 24, 2019
@pfalcon
Copy link
Contributor Author

pfalcon commented Apr 24, 2019

@galak, I believe I heard about issue this issue with different newlib version, and here it hits again.

@pabigot
Copy link
Contributor

pabigot commented Jun 27, 2019

Recording for posterity: Some research indicates that xtensa uses a custom newlib that was forked from newlib 2.0.0 around 2014-10-26 and has been unchanged since 2015-04-08. Updating to newlib 3.1.0 as used in the other targets seems unlikely.

I agree this is annoying as it requires special case code in Zephyr just to accommodate this platform, and has caused strife in reviews of zephyrproject-rtos/zephyr#16626. I suspect it will stay this way as long as xtensa is a supported platform.

@pfalcon
Copy link
Contributor Author

pfalcon commented Jun 27, 2019

@pabigot, as you did the research, perhaps you could elaborate on some points:

xtensa uses a custom newlib that was forked

Going from passive voice to active, who forked it? And where exactly? And what kind of "xtensa" is that which uses a custom newlib? (E.g., I hack xtensa in my own free time, and don't use any outdated forked newlib's.)

Updating to newlib 3.1.0 as used in the other targets seems unlikely.

For comparison, we have "minlibc" kind of libc in Zephyr, which doesn't have any custom xtensa version. Of course, that's a bit contrived example, as minlibc doesn't do anything which would require platform dependence. But some guys like @nashif, in addition (or "in replacement" is what they have in mind?) to 2 libc's we already have, want to bring 3rd: zephyrproject-rtos/zephyr#16451 (comment) . That also would not have any purposely outdated xtensa variant, or mixing up this 3rd one into the picture would be totally wrong IMHO.

seems unlikely.

Bottom line, that's fine. We just need to accept the fact Zephyr has very wide scope, and different subsystems are supported to a different extent for different targets. Here, the talk is that Xtensa target in its current shape is not ready for POSIX subsystem support. If we can't fix it now, we can remove it from CI for that subsystem. Again, "now". This can, and hopefully will, change later. But currently, need to workaround targets like Xtensa is detrimental to the further development of POSIX subsys (which needs a lot of work itself).

@pabigot
Copy link
Contributor

pabigot commented Jun 27, 2019

Zephyr is using https://github.com/jcmvbkbc/newlib-xtensa.

I can't comment on alternative approaches or the potential of a standard newlib on xtensa.

@pfalcon
Copy link
Contributor Author

pfalcon commented Jun 27, 2019

Zephyr is using https://github.com/jcmvbkbc/newlib-xtensa.

Oh, then it should be easy to ask first-hand. Hey @jcmvbkbc, do you still work on Xtensa? Any comments on the concerns expressed above? (That your fork of newlib is rather outdated; is it really the case no/not enough things were submitted upstream?) Thanks.

@nashif
Copy link
Member

nashif commented Jun 27, 2019

see jcmvbkbc/newlib-xtensa#5

@pfalcon
Copy link
Contributor Author

pfalcon commented Jun 27, 2019

see jcmvbkbc/newlib-xtensa#5

So, the picture is now clear: Xtensa couldn't be supported in Zephyr beyond minlibc. However, an old fork was found, which kinda allowed to support newlibc somewhat. Well, more thorough use leads to issues. So, looking towards proposing to TSC to do the needful on a next case Xtensa poses an issue to POSIX subsys maintenance. (So far, we have 2 workarounds, which wasn't that bad, but one is still not merged, even though 2nd is exactly the same as 1st.)

@pabigot
Copy link
Contributor

pabigot commented Jun 27, 2019

newlib 3.1.0 has some xtensa stuff, but the sdk build breaks because it's not a supported CPU.

@jcmvbkbc
Copy link

Any comments on the concerns expressed above? (That your fork of newlib is rather outdated; is it really the case no/not enough things were submitted upstream?)

That's true. I haven't touched it in a while. I guess I can update it to the most recent newlib and attempt to submit it upstream.

@nashif
Copy link
Member

nashif commented Jun 27, 2019

I guess I can update it to the most recent newlib and attempt to submit it upstream.

@jcmvbkbc that would be great, thanks. We can help with the testing.

pfalcon added a commit to pfalcon/zephyr that referenced this issue Jul 25, 2019
Unfortunately, Zephyr SDK 0.10.0 ships with outdated Newlib 2.0.0
(from 2015 or earlier) which lacks sys/_timespec.h header, requiring
ugly workaround of defining struct timespec inline (the whole idea
was to standardize on sys/_timespec.h header for different libc's).

This is similar to earlier workaround for struct timeval definition
introduced in a6aee9b. Zephyr SDK ticket for this issue
is zephyrproject-rtos/sdk-ng#64, and it
will ve possible to remove both workarounds when Xtensa toolchain
will be upgraded to newlib version consistent with other
architectures.

Signed-off-by: Paul Sokolovsky <[email protected]>
galak pushed a commit to zephyrproject-rtos/zephyr that referenced this issue Jul 25, 2019
Unfortunately, Zephyr SDK 0.10.0 ships with outdated Newlib 2.0.0
(from 2015 or earlier) which lacks sys/_timespec.h header, requiring
ugly workaround of defining struct timespec inline (the whole idea
was to standardize on sys/_timespec.h header for different libc's).

This is similar to earlier workaround for struct timeval definition
introduced in a6aee9b. Zephyr SDK ticket for this issue
is zephyrproject-rtos/sdk-ng#64, and it
will ve possible to remove both workarounds when Xtensa toolchain
will be upgraded to newlib version consistent with other
architectures.

Signed-off-by: Paul Sokolovsky <[email protected]>
LeiW000 pushed a commit to LeiW000/zephyr that referenced this issue Sep 2, 2019
Unfortunately, Zephyr SDK 0.10.0 ships with outdated Newlib 2.0.0
(from 2015 or earlier) which lacks sys/_timespec.h header, requiring
ugly workaround of defining struct timespec inline (the whole idea
was to standardize on sys/_timespec.h header for different libc's).

This is similar to earlier workaround for struct timeval definition
introduced in a6aee9b. Zephyr SDK ticket for this issue
is zephyrproject-rtos/sdk-ng#64, and it
will ve possible to remove both workarounds when Xtensa toolchain
will be upgraded to newlib version consistent with other
architectures.

Signed-off-by: Paul Sokolovsky <[email protected]>
@galak
Copy link
Contributor

galak commented Oct 4, 2019

@jcmvbkbc that would be great, thanks. We can help with the testing.

@jcmvbkbc any updates on updating newlib for xtensa?

@galak
Copy link
Contributor

galak commented Feb 2, 2020

@jcmvbkbc any updates on this?

@jcmvbkbc
Copy link

jcmvbkbc commented Feb 4, 2020

@galak it's in progress. I expect to publish updated version/start upstreaming it later this month.

@galak
Copy link
Contributor

galak commented Feb 4, 2020

@galak it's in progress. I expect to publish updated version/start upstreaming it later this month.

@jcmvbkbc Great to hear, thanks for the update!

@galak
Copy link
Contributor

galak commented Mar 11, 2020

@jcmvbkbc just touching base to see if any progress on the newlib update. thanks.

@galak
Copy link
Contributor

galak commented May 12, 2020

@jcmvbkbc any updates on newer newlib?

@stephanosio stephanosio changed the title Zephyr SDK 0.10.0 ships with very outdated newlib for Xtensa Zephyr SDK ships with very outdated newlib for Xtensa Jun 5, 2020
@galak
Copy link
Contributor

galak commented Apr 15, 2021

Closing is newlib is now in sync for Zephyr SDK across all SDK variants including xtensa.

@galak galak closed this as completed Apr 15, 2021
@pfalcon
Copy link
Contributor Author

pfalcon commented Apr 15, 2021

Great news!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

5 participants