[vm/ffi] Migrate off .addressOf struct by value tests

Instead, keep the pointers around in various places.

Issue: #40667

Change-Id: I0d661dc74fa4d9745baa03da059f2233eb0d08d0
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/181382
Reviewed-by: Aske Simon Christensen <[email protected]>

Author: dcharkes