Question about interrupts

[binaryfields]

I may be jumping the gun a little bit with this question....

Is there any special initialization code (raspi3_boot/etc) needed to enable interrupt handling outside of setting a particular IRQ enable bit in one of the 3 interrupt enable registers ( 0x7E00_B210, 214, 218)?

I'm trying to setup a DMA transfer that generates an IRQ when it finishes (by providing DMA Channel 0 with a control block that has INTEN bit set and enabling Irq 16 in IRQ Enable_1 register) but the handler I provide in the current_elx_irq vector never gets called.

I seem to be out of ideas. Any thoughts?

Thanks in advance.

[binaryfields]

Ok. Sometimes writing down the issue will help lead you to a solution.... Sigh

The missing ingredient in my code was the following instruction:

unsafe { asm!("msr DAIFClr, #2" :::: "volatile") }

With this call, my interrupt handler is finally happy. ;)

[binaryfields]

Having cleared the interrupt handler calling issue, another question I wanted to ask is about sharing mutable state between main context and irq context.

I have a use case where both the main context/thread and the irq handler may need to update a shared object. If the main context has a lock on the shared object, the irq handler won't be able to perform any of the house keeping that it needs to. Even worse, since they are both running on the same core, this would potentially result in a deadlock since the irq handler won't be able to aquire a lock while the main context is suspended.

I know you have an upcoming tutorial on this subject but I was hoping maybe you could shed some light on ways this could be handled?

Thanks!

[andre-richter]

The classic solution is to disable interrupts before locking when in non-IRQ context. You often see it as something by the names of spinlock_irqsave or similar.
This issue will indeed be part of the upcoming tutorials :)

[andre-richter]

If your shared object is not complex, say a counter or otherwise single integer, you can get away without any locking by using atomic operations.

[binaryfields]

Thanks for the pointer. I have yet to hand roll synchronization primitives so I'm definitely looking forward to your next tutorial. In the meantime, I will take a peak at spinlock_irqsave and see if I can adopt something like it in my solution.

[andre-richter]

Dont wait for me though, currently prioritizing on getting my backlog of undone readmes to zero. And free time is sparse...

[binaryfields]

So I spend some time trying to model a device that uses interrupt handling to perform some of its work. I think that main and irq context synchronization now makes sense to me based on your feedback.

However, I'm stuck now on how to provide an interrupt registry that I can use with current_elx_irq.

The way I try to model it is to have a global interrupt control manager that will be invoked from current_elx_irq to dispatch an interrupt to one of the registered handlers.

However, the code within my device that tries to register the interrupt handler runs into some intricate lifetime issues that I cannot seem to get around.

Specifically, the code in Snd::start() tries to create an interrupt handler that has a reference to internal device object and register it with the interrupt control object. That seems to cause problems with lifetimes not being correct.

Any thoughts on how to model this part?

    pub fn start(&self, irq_ctl: &NullLock<IrqCtl>) {
        let irq_handler = Box::new(
            SndIrqHandler {
                dev: self.dev.clone()
            }
        );
        irq_ctl.lock(|ctl| {
            ctl.register(irq_handler);
        });
    }

Here is a full sample code that shows interrupt registration:

use alloc::prelude::*;
use alloc::rc::Rc;
use core::cell::UnsafeCell;

pub fn main() -> Result<(), &'static str> {
    let irq_ctl: NullLock<IrqCtl> = NullLock::new(
        IrqCtl::new()
    );
    let buffer = [0u32; 32];
    let snd = Snd::open(&buffer)?;
    snd.start(&irq_ctl);
    snd.close(&irq_ctl);
    Ok(())
}

pub trait IrqHandler {
    fn handle_interrupt(&mut self);
}

pub struct IrqCtl {
    handler: Option<Box<dyn IrqHandler>>,
}

impl IrqCtl {
    pub const fn new() -> Self {
        IrqCtl {
            handler: None,
        }
    }

    pub fn register(&mut self, handler: Box<dyn IrqHandler>) {
        self.handler = Some(handler);
    }

    pub fn unregister(&mut self) {
        self.handler = None;
    }
}

pub struct Snd<'a> {
    pub dev: Rc<NullLock<SndDev<'a>>>,
}

impl<'a> Snd<'a> {
    pub fn open(buffer: &'a [u32]) -> Result<Snd<'a>, &'static str> {
        let dev = Rc::new(
            NullLock::new(
                SndDev::open(buffer)?
            )
        );
        Ok(Snd {
            dev,
        })
    }

    pub fn start(&self, irq_ctl: &NullLock<IrqCtl>) {
        let irq_handler = Box::new(
            SndIrqHandler {
                dev: self.dev.clone()
            }
        );
        irq_ctl.lock(|ctl| {
            ctl.register(irq_handler);
        });
    }

    pub fn close(&self, irq_ctl: &NullLock<IrqCtl>) {
        irq_ctl.lock(|ctl| {
            ctl.unregister();
        });
        self.dev.lock(|dev| {
            dev.close();
        });
    }
}

pub struct SndIrqHandler<'a> {
    dev: Rc<NullLock<SndDev<'a>>>,
}

impl<'a> IrqHandler for SndIrqHandler<'a> {
    fn handle_interrupt(&mut self) {
        self.dev.lock(|dev| {
            dev.handle_interrupt();
        });
    }
}

pub struct SndDev<'a> {
    buffer: &'a [u32],
}

impl<'a> SndDev<'a> {
    pub fn open(buffer: &'a [u32]) -> Result<SndDev<'a>, &'static str> {
        Ok(SndDev {
            buffer
        })
    }

    pub fn close(&mut self) {}

    pub fn handle_interrupt(&mut self) {}
}

pub struct NullLock<T> {
    data: UnsafeCell<T>,
}

unsafe impl<T> Sync for NullLock<T> {}

impl<T> NullLock<T> {
    pub const fn new(data: T) -> NullLock<T> {
        NullLock {
            data: UnsafeCell::new(data),
        }
    }
}

impl<T> NullLock<T> {
    pub fn lock<F, R>(&self, f: F) -> R
        where
            F: FnOnce(&mut T) -> R,
    {
        // In a real lock, there would be code around this line that ensures
        // that this mutable reference will ever only be given out to one thread
        // at a time.
        let result = f(unsafe { &mut *self.data.get() });
        result
    }
}

[binaryfields]

The problem I encountered may in itself belong more in the Rust language chapter on lifetimes, but I was curious to get your thoughts on the approach I took.

1. Using global registry (IrqCtl) to dispatch interrupts from current_elx_irq.

2. Splitting device into outer/inner object (Snd/SndDev). Rust doesn't like self referential data structures so if the device tries to register its own interrupt handler, the interrupt handler cannot refer back to the device itself. Solved by the inner object.

3. Device (outer Snd) and Interrupt Handler (SndIrqHandler) both access the inner device through SpinLock (right now NullLock). This solves the synchronization issue. Ideally, the non-irq context would disable/enable interrupt handling as part of acquiring a lock but that it still in the works.

4. My device requires some DMA memory, for the purpose of demonstration I pass a referece to a buffer in the constructor but in real code that would come from DMA_ALLOCATOR. Structures with references require lifetimes that for whatever reason seems to throw off the compiler when I try to register instance of IrqHandler with the IrqCtl. Sigh....

Anyways, if you get a chance to take a look at these points and share your thoughts, I would appreciate your help. Thanks!

[andre-richter]

https://github.com/rust-embedded/not-yet-awesome-embedded-rust#sharing-data-with-interrupts

[artit-io]

I have a working implementation that uses miniUART interrupts for raspi3:

https://github.com/artit91/rustberry_pi

First of all, I'm a JavaScript developer. Don't hate me.

I have started learning Rust, arm64 and raspberry Pi 4 weeks ago.

@andre-richter

Could you review my code, please?

[andre-richter]

Why should I hate you? :)

When I finally get to write the Interrupt chapter, it is likely that I try to show an implementation of UART IRQs as a part of it.
Is there any particular question you have about your implementation? I guess as long as it works, its was a nice exercise?

[artit-io]

It was much harder than I thought but it totally worth the time I invested in it. I can use this knowledge in many areas of computing. Thank you for your tutorials.

[andre-richter]

Hi @digitalstreamio,

a bit more than a year later, and two refactorings in between, I finally arrived at IRQ handling: https://github.com/rust-embedded/rust-raspberrypi-OS-tutorials/tree/master/14_exceptions_part2_peripheral_IRQs

Lifetime issues are less prevalent on my approach because the drivers are static.

[andre-richter]

I will close this issue for now. We can continue discussion here, or make a new issue if that's needed.

Thanks!

[rahealy]

@digitalstreamio I suggest experimenting with dispatching IRQ's in the kernel's main loop rather than the IRQ itself. Use an atomic static mut global in the IRQ module that gets set by the IRQ handler code and Read-Only accessor functions that safely return a copy of the IRQ state to the kernel code.

That way you can put all your handlers in the kernel's main loop's scope & lifetime.

[Edit: Which is what it looks like is happening in @andre-richter 's updated "14_exceptions_part2_peripheral_IRQs" code :) ]