ARM Cortex A Architecture support - ARMv8-A

[timork]

ARM Cortex A Architecture support is needed for new generation of automotive devices with strong security, reliability, real-time, connectivity (automotive ethernet) requirements.
New Cortex A class devices devices are targeted for these applications (like Cortex A55).
Integration with ARM Trusted Firmware is needed for secure boot, basic system initialization and power management.

[raveslave]

+1

[timork]

@nashif Any plans for the item in V2.x?

[carlescufi]

@timork as much as we'd like to support Armv7-A and Armv8-A (i.e. the non-M architectures), we would need individuals or companies with strong interest into having Zephyr ported to those in order for the port to happen, since this being an open source project features are typically added by those that require them. If you know of such company or individual please let us know.

[raveslave]

If we could find the right person/company, we would be willing to sponsor such effort.

not knowing the exact effort, how much comes with cmsis, what needs to be done on top.. I've only seen some commercial rtos'es supporting armv8-a, but I guess there should be interest from people using hypervisors like jailhouse, acrn or similar.

[timork]

@carlescufi We working on Armv8-A port and can consider to mainline it (Marvell).
We have some doubts due to previous experience with Cortex R port that still not mainlined from Aug 18 (Lexmark and Marvell effort)...

[galak]

@carlescufi We working on Armv8-A port and can consider to mainline it (Marvell).
We have some doubts due to previous experience with Cortex R port that still not mainlined from Aug 18 (Lexmark and Marvell effort)...

The Cortex-R port got held up for 2 reasons, the 1.14/LTS effort and lack of a SoC/board port. The SoC/board port issue should be resolved by work Linaro has done to support Xilinx Ultra96 board.

[galak]

If we could find the right person/company, we would be willing to sponsor such effort.

not knowing the exact effort, how much comes with cmsis, what needs to be done on top.. I've only seen some commercial rtos'es supporting armv8-a, but I guess there should be interest from people using hypervisors like jailhouse, acrn or similar.

@raveslave can you connecting me on this.

[nashif]

@timork

Do you have everything covered in your work? i.e.

• architecture
• Soc
• Boards
• Does it run in Qemu?

Those are needed to validate the port and make sure it is maintained and kept in sync with the rest of Zephyr.

[galak]

@carlescufi We working on Armv8-A port and can consider to mainline it (Marvell).
We have some doubts due to previous experience with Cortex R port that still not mainlined from Aug 18 (Lexmark and Marvell effort)...

@timork are you willing to share the work you guys are doing?

[timork]

@nashif

@timork

Do you have everything covered in your work? i.e.

• architecture
• Soc
• Boards
• Does it run in Qemu?

Those are needed to validate the port and make sure it is maintained and kept in sync with the rest of Zephyr.

Yes we plan for:
We cover arch/plat code
SoC - Marvell Armada 3700
Board: Espresso Bin
Qemu - TBD

[timork]

@carlescufi We working on Armv8-A port and can consider to mainline it (Marvell).
We have some doubts due to previous experience with Cortex R port that still not mainlined from Aug 18 (Lexmark and Marvell effort)...

@timork are you willing to share the work you guys are doing?

Yes,
When the code will be more mature we will post it as PR.

[raveslave]

just keeping this one warm. anyone (preferably in sweden) that would be interested working on this?

[andrewboie]

When the code will be more mature we will post it as PR.

Hi @timork was just wondering if you had anything to share.

[galak]

Hi @timork was just wondering if you had anything to share.

@timork any updates on sharing anything you've got?

[timork]

Hi @timork was just wondering if you had anything to share.

@timork any updates on sharing anything you've got?
Still on-progress...
We plan make PR on December time-frame.

[stephanosio]

@timork Hi, I'm working on various improvements to the Cortex-R port at this time and wanted to know how you are handling the following issues on the Cortex-A port (since Cortex-R is very similar to Cortex-A, I would like to make sure that the "improvements" that I am about to make aren't already implemented in the Cortex-A port):

1. Context switching with Shared FP registers mode: is it implemented?

2. Nested interrupts: is it implemented? (just in terms of kernel port, disregarding interrupt controller support; i.e. is the IRQ vector function fully re-entrant?)

3. SMP (or AMP) support: is it implemented? (i.e. are z_arch_switch and other kernel port provisions for multi-processor support implemented?)

4. (assuming AArch32) User mode: is user mode support implemented? if so, are syscalls functional as well?

It would be very helpful if you could provide a brief answer to the questions above.

[vladimir-podbrezsky]

@timork Hi, I'm working on various improvements to the Cortex-R port at this time and wanted to know how you are handling the following issues on the Cortex-A port (since Cortex-R is very similar to Cortex-A, I would like to make sure that the "improvements" that I am about to make aren't already implemented in the Cortex-A port):

1. Context switching with Shared FP registers mode: is it implemented?
2. Nested interrupts: is it implemented? (just in terms of kernel port, disregarding interrupt controller support; i.e. is the IRQ vector function fully re-entrant?)
3. SMP (or AMP) support: is it implemented? (i.e. are z_arch_switch and other kernel port provisions for multi-processor support implemented?)
4. (assuming AArch32) User mode: is user mode support implemented? if so, are syscalls functional as well?

It would be very helpful if you could provide a brief answer to the questions above.

Hello,
My name is Vladimir and am working currently on support of Cortex-A55 for Zephyr RTOS.

1. Currently, we do not save/restore FP registers while context switch. If we start to use FP operations I’ll add this support
   2 .Nesting should be supported in our code, but I did not verify it. We decided not to support interrupt nesting in the project. I can check it later.
2. We have plan to support SMP. But currently we run application tasks on single CPU only.
3. We have plan to implement user mode also, but in Aarch64 mode only.

[stephanosio]

@vladimir-podbrezsky @timork Thanks for the prompt response.

1. Currently, we do not save/restore FP registers while context switch. If we start to use FP operations I’ll add this support

Okay. In that case, I will proceed with the task #19979 "Implement Cortex-R floating-point support". This should support Cortex-A too.

2. Nesting should be supported in our code, but I did not verify it. We decided not to support interrupt nesting in the project. I can check it later.

Would you mind sharing isr_wrapper.S (or something equivalent in your code) code so I can check if it can support interrupt nesting? If it is implemented like the current Cortex-R port (which cannot support interrupt nesting at all), I will proceed with the interrupt nesting support implementation so it can be used on both Cortex-R and Cortex-A.

[sandeepbrcm]

@carlocaione we did not see your PR before and have got a minimal aarch64 port functional on Broadcom SoC with Cortex-A72 with cooperative scheduling.
https://github.com/Broadcom/zephyr/tree/arm64_support

We want to discuss few topics on the arch port.

1. Context switching
   Why use SVC for context switching in Cortex-A. And PendSV is also confusing to use
   in cortex A and R while it has advantage to simplify deferring context switching in
   Cortex-M.

2. Cortex-A class processors do not support vectored interrupt. Hence the _sw_isr_table
   may not be best suited especially where the interrupt controller has high range for
   interrupt id numbers. eg: GIC SPI.

3. Zephyr may be restricted to run only at EL1NS Or EL1S to avoid fragmentation.

[stephanosio]

1. Context switching
Why use SVC for context switching in Cortex-A. And PendSV is also confusing to use
in cortex A and R while it has advantage to simplify deferring context switching in
Cortex-M.

"PendSV" nomenclature does need to be changed- this can be misleading as PendSV is a Cortex-M-specific feature and does not exist in Cortex-A (the only reason Cortex-R kept "pendsv" is that it is trying to share code with Cortex-M).

As for using SVC in terms of context switching, it should only be invoked when arch_swap is called from "user mode"; otherwise, context switching routine is executed as part of normal interrupt exit.

2. Cortex-A class processors do not support vectored interrupt. Hence the _sw_isr_table
may not be best suited especially where the interrupt controller has high range for
interrupt id numbers. eg: GIC SPI.

While Cortex-A does not support hardware interrupt vectoring, GIC does support indexed interrupts and I think using _sw_isr_table for that is fine. SPI starts only at index 32 and, given that AArch64 is not a memory-constrained environment, it should not be that much of a problem.

As for LPI starting at 8192, we should consider implementing segmented sw_isr_table (e.g. first segment covering 0-1023, second segment covering 8192-). But then, 8 * 8192 is only 64KiB and whether that matters in AArch64 is seriously debatable (e.g. page tables would eat up a lot more than that).

Also note that, practically speaking, GIC is a mandatory feature in AArch64 (given how tightly integrated GIC is to the ARMv8-A arch, there is no good reason for any SoC vendor to use a different interrupt controller), so the "fake multi-level interrupt controller" or "forward arch interrupt control functions to the SoC layer" scheme could potentially be removed. The only reason Cortex-R port currently uses this scheme is that the ARMv7-R SoC can be/are actually implemented with many different types of interrupt controllers connected to the core VIC port.

[carlocaione]

@carlocaione we did not see your PR before and have got a minimal aarch64 port functional on Broadcom SoC with Cortex-A72 with cooperative scheduling.
https://github.com/Broadcom/zephyr/tree/arm64_support

This is really cool, looking forward to your review and comments on my PR then.

1. Context switching
   Why use SVC for context switching in Cortex-A. And PendSV is also confusing to use
   in cortex A and R while it has advantage to simplify deferring context switching in
   Cortex-M.

While agree that the naming could be misleading using SVC for the context switch also simplifies a bit the code when introducing the preemptive context switching. In latter case the task swapping is performed in the ISR so you can partially reuse the entry/exit code (eret & co) for the ISR and the SVC call.

2. Cortex-A class processors do not support vectored interrupt. Hence the _sw_isr_table
   may not be best suited especially where the interrupt controller has high range for
   interrupt id numbers. eg: GIC SPI.

Expanding on what @stephanosio said (thank you) one more reason I used _sw_isr_table is to be more aligned with what was done for Cortex-R so hopefully we can share some common code in the future. I'm not a fan of the multi-level interrupt controller story either but we can change that later.

3. Zephyr may be restricted to run only at EL1NS Or EL1S to avoid fragmentation.

I'm not sure I fully understood this point but I tried to be as agnostic as possible regarding the EL, I'm currently using the same code running in EL3 or EL1 using two different QEMUs. In 9124118 I introduced an option to drop from EL3 to EL1 if needed.

[sandeepbrcm]

1. Context switching
   Why use SVC for context switching in Cortex-A. And PendSV is also confusing to use
   in cortex A and R while it has advantage to simplify deferring context switching in
   Cortex-M.

While agree that the naming could be misleading using SVC for the context switch also simplifies a bit the code when introducing the preemptive context switching. In latter case the task swapping is performed in the ISR so you can partially reuse the entry/exit code (eret & co) for the ISR and the SVC call.

I need to think on it and get back on trade-offs. May be not an item for the first PR.

[sandeepbrcm]

.

1. Cortex-A class processors do not support vectored interrupt. Hence the _sw_isr_table
   may not be best suited especially where the interrupt controller has high range for
   interrupt id numbers. eg: GIC SPI.

Expanding on what @stephanosio said (thank you) one more reason I used _sw_isr_table is to be more aligned with what was done for Cortex-R so hopefully we can share some common code in the future. I'm not a fan of the multi-level interrupt controller story either but we can change that later.

Agree. This can be changed later.
@stephanosio reserving memory space for 100s or 1000s of interrupts out of which 90% or more will never be used doesn't sound good. And segmented sw_isr_table will loose the benefit of direct indexed accessing. We can discuss this on a RFC.

[sandeepbrcm]

1. Zphyr may be restricted to run only at EL1NS Or EL1S to avoid fragmentation.

I'm not sure I fully understood this point but I tried to be as agnostic as possible regarding the EL, I'm currently using the same code running in EL3 or EL1 using two different QEMUs. In 9124118 I introduced an option to drop from EL3 to EL1 if needed.

The choice will influence rest of the software stack, boot flow and configurations.
1-EL1 non-secure Aarch64 is the best suitable if TruztZone to be used.
2-EL1 secure Aarch64 is the good enough if no TruztZone partitioning required.
I guess EL2 and EL3 will not be required in Zephyr.

may be we should have
EL1S/EL1NS right. If booted in EL3, just drop to EL1S.

[carlocaione]

may be we should have
EL1S/EL1NS right. If booted in EL3, just drop to EL1S.

Now SWITCH_TO_EL1 is enabled by default so if booted in EL3 we just drop in EL1.

[stephanosio]

Closing this issue as the basic ARMv8-A architecture support has been merged (#20263).

Further development is tracked at #22411.