Cortex-R Improvement

[stephanosio]

This is an epic for ARM Cortex-R port improvement, in preparation for the TI Hercules SoC support in #19644.

• Extend qemu_cortex_r5 test coverage #20217 Extend qemu_cortex_r5 test coverage (make CI test Cortex-R port)
  • Refactor and fix xilinx_zynqmp SoC definition #20267 Refactor and fix xilinx_zynqmp SoC definition (RPU and APU should be separate)
  • boards: arm: qemu_cortex_r5: Use arm-generic-fdt machine type. #20298 Use arm-generic-fdt machine type (support RPU PL390 GIC emulation)
  • interrupt_controller: gic: Support PPIs #19860 Support PPI for GIC
  • interrupt_controller: gic: Support multiple GIC versions #21508 Add PL390 GICv1 support
  • Implement tickless capability for xlnx_psttc_timer #19869 Implement tickless capability for xlnx_psttc_timer (for xilinx_zynqmp)
• Fix critical bugs and add missing essential arch features
  • arch: arm: Add Cortex-R floating point configuration support. #20469 Add Cortex-R floating point configuration support
  • arch: arm: Rewrite Cortex-R reset vector function. #20473 Rewrite Cortex-R reset vector function
  • arch: arm: Fix incorrect Cortex-R interrupt state control logic. #20474 Fix incorrect Cortex-R interrupt state control logic
  • arch: arm: Fix Cortex-R power management interrupt handling #21758 Fix Cortex-R power management interrupt handling
  • soc: arm: xilinx_zynqmp: Relocate platform-specific initialisation. #20478 Relocate platform-specific initialisation for xilinx_zynqmp (in arch code)
• Essential enhancements
  • arch: arm: cortex_r: Add CMSIS support #22740 Use CMSIS-Core(R) for Cortex-R
  • Refactor ARM interrupt system (Cortex-A & Cortex-R) #22718 Refactor Cortex-R interrupt system (get rid of the fake multi-level interrupt scheme)
  • #TBD Support Cortex-R vectored interrupts (b062044 4581910)
  • #TBD Support Cortex-R direct interrupt service routine (d0b4fe6)
  • arch: arm: aarch32: Allow selecting compiler instruction set #22741 Allow selecting instruction set for C code compilation
  • arch: arm: Optimise Cortex-R exception return function. #20477 Optimise Cortex-R exception return function
• Support interrupt nesting
  • Support nested interrupts on AArch32 (non-Cortex-M) #23636 Rewrite Cortex-R context preservation code to support interrupt nesting
  • Implement GIC-based ARM interrupt tests #22670 Implement Cortex-R interrupt tests (including SGI and nesting support)
• Implement Cortex-R floating-point support #19979 Implement Cortex-R floating-point support
• Cortex-R MPU support #28231 Implement Cortex-R memory protection
• Cortex-R MPU support #28231 Implement Cortex-R user mode execution

[stephanosio]

cc @bbolen @ioannisg @carlocaione

@bbolen If, by any chance, you are working on any of the above already, please let me know.

[bwasim]

@stephanosio , I'm planning to start work on Armv7-R MPU and hope to push it into the mainline in the next couple of weeks. Let me know if someone is already working on this, so I don't end up duplicating the effort..

[stephanosio]

@stephanosio , I'm planning to start work on Armv7-R MPU and hope to push it into the mainline in the next couple of weeks. Let me know if someone is already working on this, so I don't end up duplicating the effort..

@bwasim Nobody is working on/planning to upstream ARMv7-R MPU support at the moment as far as I know, though it is possible that Lexmark guys already have it (see https://lists.zephyrproject.org/g/devel/message/6695).

p.s. By the way, ARMv7-R MPU and ARMv7-M MPU are very similar (well, they are both PMSAv7 implementations), so the focus should be on trying to refactor/reuse the Cortex-M MPU code. We should not have two separate implementations of the same thing ...

[bbolen]

@stephanosio , I'm planning to start work on Armv7-R MPU and hope to push it into the mainline in the next couple of weeks. Let me know if someone is already working on this, so I don't end up duplicating the effort..

@bwasim Nobody is working on/planning to upstream ARMv7-R MPU support at the moment as far as I know, though it is possible that Lexmark guys already have it (see https://lists.zephyrproject.org/g/devel/message/6695).

Yes, we do have something working but it is not in merge worthy shape right now. If you can wait until Monday, I can send you a patch of what we have in order to take a look.

[bwasim]

@stephanosio , I'm planning to start work on Armv7-R MPU and hope to push it into the mainline in the next couple of weeks. Let me know if someone is already working on this, so I don't end up duplicating the effort..

@bwasim Nobody is working on/planning to upstream ARMv7-R MPU support at the moment as far as I know, though it is possible that Lexmark guys already have it (see https://lists.zephyrproject.org/g/devel/message/6695).

Yes, we do have something working but it is not in merge worthy shape right now. If you can wait until Monday, I can send you a patch of what we have in order to take a look.

That would be great, Thanks..

[bbolen]

@stephanosio , I'm planning to start work on Armv7-R MPU and hope to push it into the mainline in the next couple of weeks. Let me know if someone is already working on this, so I don't end up duplicating the effort..

@bwasim Nobody is working on/planning to upstream ARMv7-R MPU support at the moment as far as I know, though it is possible that Lexmark guys already have it (see https://lists.zephyrproject.org/g/devel/message/6695).

Yes, we do have something working but it is not in merge worthy shape right now. If you can wait until Monday, I can send you a patch of what we have in order to take a look.

That would be great, Thanks..

The top three commits on https://github.com/bbolen/zephyr/commits/cortex_r_mpu are what we have. This is based on v2.2.0. We started working on this on 1.14 and merging has made just grabbing patches difficult so I had to just start clean on top of v2.2. I think I got everything, but if you have problems, let me know.

[wali-ku]

@stephanosio , I'm planning to start work on Armv7-R MPU and hope to push it into the mainline in the next couple of weeks. Let me know if someone is already working on this, so I don't end up duplicating the effort..

@bwasim Nobody is working on/planning to upstream ARMv7-R MPU support at the moment as far as I know, though it is possible that Lexmark guys already have it (see https://lists.zephyrproject.org/g/devel/message/6695).

Yes, we do have something working but it is not in merge worthy shape right now. If you can wait until Monday, I can send you a patch of what we have in order to take a look.

That would be great, Thanks..

The top three commits on https://github.com/bbolen/zephyr/commits/cortex_r_mpu are what we have. This is based on v2.2.0. We started working on this on 1.14 and merging has made just grabbing patches difficult so I had to just start clean on top of v2.2. I think I got everything, but if you have problems, let me know.

@bbolen I used your commits to enable MPU in a Cortex-R5F based board. The MPU requires power-of-two alignment (Reference: ARM Cortex-R5 TRM r1p1 Table 4.34: Region Size) but when I enable CONFIG_MPU_REQUIRES_POWER_OF_TWO_ALIGNMENT in Zephyr, I notice that all the non-static (rw) global data (i.e., the one in the .datas section in include/arch/arm/aarch32/cortex_a_r/scripts/linker.ld) gets initialized to 0xFFFF_FFFF instead of its correct initial value which results in data-abort exception (with alignment fault @ 0xffffffff) during kernel boot.

Have you seen anything similar or can you please give a hint as to where should I look to debug this problem?

If I make the global data read-only (by adding static const before it), then it gets initialized to correct value. So the problem is with the non-static data initialization alone. Also, it is worth noting that the problem occurs without MPU enabled as well i.e., if I just modify the linker file to enforce power-of-two alignment without enabling MPU, the problem still persists.

Thanks in advance!

[bbolen]

Sorry I haven't responded until now. We are working to get our MPU code rebased on v2.3 and get it cleaned up for upstreaming consideration. With respect to the problem you mention, I have seen something similar. I got around the problem by using "DISCARD" on the *.got section in the cortex R linker script. The problem is that the VMA and LMA drift apart, but I don't know what the correct thing to do is.

[wali-ku]

I tried to figure out what the problem was and found the following:

In the case of power-of-two alignment, there is extra padding added before the "datas" section due to which the addresses of the initialized global (rw) data change but the code still references the old address; without the padding offset.

In this image, the date = 26 is stored in a (rw) global variable; which should go to the "datas" section. With and wo ^2 alignment, the code references the variable at the same address but in case of ^2 alignment, the address contains the padded bytes and the correct value of the variable is present at a different location (highlighted in small red rectangle in RHS image).

I was able to work-around the issue using the following patch:

diff --git a/include/arch/arm/aarch32/cortex_a_r/scripts/linker.ld b/include/arch/arm/aarch32/cortex_a_r/scripts/linker.ld
index 57c73e680a..aec33d0ecc 100644
--- a/include/arch/arm/aarch32/cortex_a_r/scripts/linker.ld
+++ b/include/arch/arm/aarch32/cortex_a_r/scripts/linker.ld
@@ -206,13 +206,12 @@ SECTIONS
          * generally is not an issue, since modern ROM and FLASH memory is
          * usually 4k aligned.
          */
-        . = ALIGN(4);
+	_image_rodata_end = .;
+	MPU_ALIGN(_image_rodata_end -_image_rom_start);
     } GROUP_LINK_IN(ROMABLE_REGION)
 
 #include <linker/cplusplus-rom.ld>
 
-    _image_rodata_end = .;
-    MPU_ALIGN(_image_rodata_end -_image_rom_start);
     _image_rom_end = .;
 
     GROUP_END(ROMABLE_REGION)

I am not exactly sure how this patch resolves the problem; I was experimenting with the linker file to see if I could get the code to generate correct addresses of global vars with ^2 alignment and making this change gave correct results.

I also had to make changes to the syscall trampoline code in the commits you referenced; to make it comply with the Zephyr v2.3 z_arm_svc exception handling code. But I was finally able to create Usermode threads for Cortex-R5 on Zephyr with MPU enabled.

Thank you for posting these patches. It would have taken me much longer to get this done on my own.

[stephanosio]

Closing since this is more or less complete. Any other remaining issues, if any, should be tracked in separate issues.