RightPanel keeps scaring me with unexpected MemberInfos

[ara4n]

Steps to reproduce

So while the RightPanel's behaviour has improved a lot since #19660 was closed, I think we may have oversteered a bit in terms of tracking per-room rightpanel state at too fine a granularity.

Specifically, it's really tripping me up that rooms now remember the last MemberInfo you looked at. Repro steps:

1. View a room
2. Look at some random user (typically for moderation purposes)
3. Do whatever needs to be done
4. Look at other rooms
5. Several hours/days elapse
6. You happen to look at the original room again
7. You have a massive MemberInfo staring back at you of this random user, who you no longer care about at all - causing massive cognitive dissonance. "Why is @randomidiot:matrix.org the posterchild of Matrix HQ? I don't even remember viewing their member info..."

Outcome

What did you expect?

Rooms shouldn't freak you out by showing you specific old MemberInfos when you open them; only that you were doing something in the MemberList.

What happened instead?

I keep getting freaked out by randoms jumping out at me in the RightPanel whenever I view a room.

Operating system

macOS 12.3

Application version

Nightly

How did you install the app?

Nightly

Homeserver

matrix.org

Will you send logs?

No