RISC-V binary contains multiply and divide instructions despite march=rv32i only

[m42uko]

Hi,

I am on the latest release (v3.5.0) version of Zephyr with a not officially supported RV32I CPU (https://github.com/olofk/serv), built using the latest SDK (v0.16.4). When I build any example for this CPU, the emitted code contains MUL/DIV instructions and therefore fails to run on my rv32i (no m) CPU. The DTS does, however, clearly specify the RV32I instruction set: https://github.com/olofk/serv/blob/main/zephyr/dts/riscv/serv.dtsi#L18

The SERV repository is set up to be used as a west manifest, so you should be able to quickly reproduce this problem by building for the "service" target and looking at the disassembly.

From my troubleshooting so far, it looks as if the "offending" MULs/DIVs come from picolibc code (that afaik is delivered precompiled with the SDK). The -march to the rest of the code is fine, and it does not contain any MULs/DIVs.

I opened this as a discussion instead of an issue as suggested in the template to make sure there is no user error here. But we can migrate if this should turn out to be an issue in Zephyr / the related toolchain.

Thanks!
Markus

CC: @olofk

[nordicjm]

@keith-packard

[keith-packard]

Hrm. Maybe it's picking the wrong library because Zephyr doesn't know about this target? I just checked the C library using this command, which uses objdump on the entire library and looks for any instance of mul or div. There are lots of symbols with those letters, but no instructions.

$ objdump -D /opt/zephyr-sdk-0.16.4/riscv64-zephyr-elf/picolibc/riscv64-zephyr-elf/lib/rv32i_zicsr_zifencei/ilp32/libc.a | grep '\(mul\)\|\(div\)'

[m42uko]

TL;DR: Copying ~/zephyr-sdk-0.16.4/riscv64-zephyr-elf/picolibc/riscv64-zephyr-elf/lib/rv32i_zicsr_zifencei/ilp32/ directly to the lib directory "fixes" the issue. (There are multiple "lib" directories, do it in all.)

I did some more digging just now, and it seems to indeed be some problem with the toolchain / how it is used.

Here's the output from the default build when grepping for mul (just to show the issue is present). I marked the problematic ones with **. The rest is part of the debug_info section.

[markus@571aaf79e9b4 hello_world]$ riscv64-zephyr-elf-objdump -D build/zephyr/zephyr.elf |grep mul
0000022c <__mulsi3>:
     238:       00068463                beqz    a3,240 <__mulsi3+0x14>
     248:       fe0596e3                bnez    a1,234 <__mulsi3+0x8>
     68c:       ba1ff0ef                jal     ra,22c <__mulsi3>
     70c:       b21ff0ef                jal     ra,22c <__mulsi3>
     744:       ae9ff0ef                jal     ra,22c <__mulsi3>
    11a0:       88cff0ef                jal     ra,22c <__mulsi3>
  **2210:       03d78833                mul     a6,a5,t4
  **2400:       027d03b3                mul     t2,s10,t2
  **2410:       027983b3                mul     t2,s3,t2
    1cae:       23000003                lb      zero,560(zero) # 230 <__mulsi3+0x4>
    1ce2:       24000003                lb      zero,576(zero) # 240 <__mulsi3+0x14>
    1f20:       24000003                lb      zero,576(zero) # 240 <__mulsi3+0x14>
    1f56:       24000003                lb      zero,576(zero) # 240 <__mulsi3+0x14>
    3844:       02000033                mul     zero,zero,zero
    ee14:       02043eb3                mulhu   t4,s0,zero
    103e:       03002e33                mulhsu  t3,zero,a6
    26b8:       03012e33                mulhsu  t3,sp,a6
     d70:       23220083                lb      ra,562(tp) # 232 <__mulsi3+0x6>
    250c:       24320423                sb      gp,584(tp) # 248 <__mulsi3+0x1c>

I don't have a good understanding of GCC internals and its libc handling, but I played a bit around with the toolchain and managed to get rid of the MULs. It's not a fix, but it makes the problem go away, and I hope someone with a better understanding of GCC can help me come up with a proper fix. (I also played with -rpath, but couldn't get it to work that way.)

So what I did was open the toolchain lib directory and replace the default set of files with the files from the rv32i directory:

cd ~/zephyr-sdk-0.16.4/riscv64-zephyr-elf/picolibc/riscv64-zephyr-elf/lib
cp rv32i_zicsr_zifencei/ilp32/* ./

(There are multiple "lib" directories, do it in all.)

After that, building the hello_world sample again yields:

[markus@571aaf79e9b4 hello_world]$ riscv64-zephyr-elf-objdump -D build/zephyr/zephyr.elf |grep mul
0000022c <__mulsi3>:
     238:       00068463                beqz    a3,240 <__mulsi3+0x14>
     248:       fe0596e3                bnez    a1,234 <__mulsi3+0x8>
     68c:       ba1ff0ef                jal     ra,22c <__mulsi3>
     70c:       b21ff0ef                jal     ra,22c <__mulsi3>
     744:       ae9ff0ef                jal     ra,22c <__mulsi3>
    11a0:       88cff0ef                jal     ra,22c <__mulsi3>
    1cae:       23000003                lb      zero,560(zero) # 230 <__mulsi3+0x4>
    1ce2:       24000003                lb      zero,576(zero) # 240 <__mulsi3+0x14>
    1f20:       24000003                lb      zero,576(zero) # 240 <__mulsi3+0x14>
    1f56:       24000003                lb      zero,576(zero) # 240 <__mulsi3+0x14>
    3844:       02000033                mul     zero,zero,zero
    ee14:       02043eb3                mulhu   t4,s0,zero
    103e:       03002e33                mulhsu  t3,zero,a6
    26b8:       03012e33                mulhsu  t3,sp,a6
     d70:       23220083                lb      ra,562(tp) # 232 <__mulsi3+0x6>
    250c:       24320423                sb      gp,584(tp) # 248 <__mulsi3+0x1c>

The offending MULs are gone. The remaining mul/mulhu/mulhsu are only part of the debug_info, so not actually part of the binary.

Can anyone point me in the right direction on how to fix this properly? I might have some more time to dig into this in the upcoming days, but some pointers would centainly help.

Markus