[tmpnet] Refactor runtime configuration in preparation for kube #3867

maru-ava · 2025-04-08T08:39:35Z

PR Chain: tmpnet+kube

This PR chain enables tmpnet to deploy temporary networks to Kubernetes. Early PRs refactor tmpnet to support the addition in #3615 of a new tmpnet node runtime for kube.

Why this should be merged

Adding the kube runtime for tmpnet requires being able to configure it via cli args. This PR refactors the existing runtime configuration to better support the introduction of the kube runtime configuration.

How this works

How this was tested

CI, manual use of tmpnetctl

Need to be documented in RELEASES.md?

N/A

TODO

Merge [tmpnet] Ensure Node has a reference to Network #3870

maru-ava · 2025-04-09T06:39:07Z

tests/fixture/tmpnet/network.go

 	if err := n.EnsureNodeConfig(node); err != nil {
 		return err
 	}
 	if err := node.Write(); err != nil {
 		return err
 	}

-	// Check the VM binaries after EnsureNodeConfig to ensure node.RuntimeConfig is non-nil
-	if err := checkVMBinaries(log, n.Subnets, node.RuntimeConfig.AvalancheGoPath, pluginDir); err != nil {


Moved this check to the node

maru-ava · 2025-04-09T06:40:07Z

tests/fixture/tmpnet/network.go

@@ -298,17 +284,6 @@ func (n *Network) Create(rootDir string) error {
 	}
 	n.Dir = canonicalDir

-	// Ensure the existence of the plugin directory or nodes won't be able to start.
-	pluginDir, err := n.GetPluginDir()


Moved this to node start

maru-ava · 2025-04-09T06:45:46Z

tests/fixture/tmpnet/network.go

@@ -220,15 +218,6 @@ func (n *Network) EnsureDefaultConfig(log logging.Logger, avalancheGoPath string
 		n.DefaultFlags = FlagsMap{}
 	}

-	// Only configure the plugin dir with a non-empty value to ensure


Plugin dir is now only ever set in node flags rather than as a network default flag

Copilot

Copilot reviewed 16 out of 16 changed files in this pull request and generated no comments.

Comments suppressed due to low confidence (2)

tests/fixture/tmpnet/network.go:1039

The PluginDir field is used directly in forming the VM binary path. Consider adding a validation to ensure PluginDir is not empty (or providing a default) to prevent constructing an invalid path.

vmPath := filepath.Join(config.PluginDir, chain.VMID.String())

tests/antithesis/compose.go:76

The removal of avalancheGoPath and pluginDir parameters in initBootstrapDB and the subsequent call to BootstrapNewNetwork could lead to misconfiguration in environments that expect these values. Consider ensuring that default values or validations are in place to maintain correct behavior.

if err := initBootstrapDB(network, bootstrapVolumePath); err != nil {

tests/fixture/tmpnet/network.go

tests/fixture/tmpnet/README.md

tests/fixture/tmpnet/flags/common.go

tests/fixture/tmpnet/flags/process_runtime.go

tests/fixture/tmpnet/flags/runtime.go

qdm12 · 2025-04-12T16:15:35Z

tests/fixture/tmpnet/flags/runtime.go

+	case processRuntime:
+		processRuntimeConfig, err := v.processRuntimeVars.getProcessRuntimeConfig()
+		if err != nil {
+			return nil, err


nit it might be useful when supporting more runtimes (kube) to mention what runtime we tried to get, in case it's misconfigured:

Suggested change

return nil, err

return nil, fmt.Errorf("for runtime %s: %w", err)

qdm12 · 2025-04-12T16:22:27Z

tests/fixture/tmpnet/node_config.go

+	IsEphemeral   bool               `json:",omitempty"`
+	Flags         FlagsMap           `json:",omitempty"`
+	RuntimeConfig *NodeRuntimeConfig `json:",omitempty"`


are these omitempty tags necessary or an extra addition to the PR?

It's secondary cleanup related to how runtime configuration is serialized.

tests/fixture/tmpnet/node_process.go

qdm12 · 2025-04-12T16:34:42Z

tests/fixture/tmpnet/node.go

 type NodeRuntimeConfig struct {
+	Process *ProcessRuntimeConfig
+}


So the aim is to then have a Process and Kube field right?
Could/Should we instead have an interface here, with the process implementation and a future kube implementation? 🤔
Maybe I'm missing the point completely though, in which case, my apologies 😉

It's a struct, it has fields, and there are no fields in common between process and kube to suggest an interface that would be common between them. There is an an interface for the runtime for which there are methods common to both types.

felipemadero

LGTM. Thanks for this. Preaproving with just a couple nits/questions

felipemadero · 2025-04-14T13:53:45Z

tests/fixture/tmpnet/network.go

@@ -518,7 +479,8 @@ func (n *Network) StartNode(ctx context.Context, log logging.Logger, node *Node)

 // Restart a single node.
 func (n *Network) RestartNode(ctx context.Context, log logging.Logger, node *Node) error {
-	if node.RuntimeConfig.ReuseDynamicPorts {
+	runtimeConfig := node.getRuntimeConfig()
+	if runtimeConfig.Process != nil && runtimeConfig.Process.ReuseDynamicPorts {


Q/ couldn't ReuseDynamicPorts be attempted on kube? I wonder if this setting can be separated from the kind of runtime.

'Reuse' implies the use of dynamic ports in the first place, but dynamic ports are not compatible with kube due to the kubelet needing a fixed port to perform readiness checks against. Also not needed because a kube pod gets its own IP so there is no potential for clashing with other nodes deployed as pods.

felipemadero · 2025-04-14T14:03:21Z

tests/fixture/tmpnet/node_process.go

@@ -37,8 +37,9 @@ const (
 var (
 	AvalancheGoPluginDirEnvName = config.EnvVarName(config.EnvPrefix, config.PluginDirKey)

-	errNodeAlreadyRunning = errors.New("failed to start node: node is already running")
-	errNotRunning         = errors.New("node is not running")
+	errNodeAlreadyRunning   = errors.New("failed to start node: node is already running")


may it be desirable for tmpnet as a library to export errors?

I tend to prefer YAGI, and I'm not getting any requests to export these errors. The driver for exporting is more likely to be implementing e2e testing of tmpnet itself, which is a near-term goal to ensure behavior that isn't implicitly tested as part of avalanchego's e2e suite.

maru-ava added the testing This primarily focuses on testing label Apr 8, 2025

maru-ava self-assigned this Apr 8, 2025

maru-ava added this to avalanchego Apr 8, 2025

github-project-automation bot moved this to Backlog 🗄️ in avalanchego Apr 8, 2025

maru-ava moved this from Backlog 🗄️ to In Progress 🏗 in avalanchego Apr 8, 2025

maru-ava changed the title ~~[tmpnet] Refactor runtime handling to enable kube extension~~ [tmpnet] Refactor runtime configuration in preparation for kube Apr 8, 2025

Base automatically changed from tmpnet-use-content-keys to master April 9, 2025 04:09

maru-ava force-pushed the tmpnet-refactor-runtime-config branch from 6a78622 to ca512a2 Compare April 9, 2025 06:24

maru-ava changed the base branch from master to tmpnet-add-network-reference-to-node April 9, 2025 06:24

maru-ava force-pushed the tmpnet-refactor-runtime-config branch 6 times, most recently from 4983079 to b824686 Compare April 9, 2025 06:35

maru-ava commented Apr 9, 2025

View reviewed changes

maru-ava force-pushed the tmpnet-refactor-runtime-config branch from b824686 to ff2f225 Compare April 9, 2025 06:44

maru-ava commented Apr 9, 2025

View reviewed changes

maru-ava force-pushed the tmpnet-refactor-runtime-config branch 2 times, most recently from 77b1a04 to 7db5b52 Compare April 9, 2025 06:58

maru-ava moved this from In Progress 🏗 to In Review 👀 in avalanchego Apr 9, 2025

maru-ava requested review from qdm12, felipemadero and Copilot April 9, 2025 07:00

maru-ava marked this pull request as ready for review April 9, 2025 07:00

Copilot AI reviewed Apr 9, 2025

View reviewed changes

maru-ava mentioned this pull request Apr 9, 2025

[tmpnet] Unify start network flag usage between e2e and tmpnetctl #3871

Merged

1 task

Base automatically changed from tmpnet-add-network-reference-to-node to master April 10, 2025 22:07

[tmpnet] Refactor runtime configuration in preparation for kube

d2c2a0e

michaelkaplan13 reviewed Apr 11, 2025

View reviewed changes

tests/fixture/tmpnet/network.go Show resolved Hide resolved

fixup: Respond to review feedback

da0e7bf

michaelkaplan13 approved these changes Apr 11, 2025

View reviewed changes

qdm12 reviewed Apr 12, 2025

View reviewed changes

maru-ava added 2 commits April 12, 2025 13:14

fixup: Add context to process runtime config validation error

815402a

fixup: Removing redundant funcion Network.GetPluginDir

df11b91

felipemadero approved these changes Apr 14, 2025

View reviewed changes

maru-ava added this pull request to the merge queue Apr 14, 2025

Merged via the queue into master with commit d6ce7ec Apr 14, 2025
24 checks passed

maru-ava deleted the tmpnet-refactor-runtime-config branch April 14, 2025 16:55

github-project-automation bot moved this from In Review 👀 to Done ✅ in avalanchego Apr 14, 2025

This was referenced May 7, 2025

[tmpnet] Define reusable flags for configuring kubernetes client access #3945

Merged

[tmpnet] Separate start of prometheus and promtail collectors #3947

Merged

maru-ava mentioned this pull request May 21, 2025

[tmpnet] Ensure GetNodeURIs returns locally-accessible URIs to ensure kube compatibility #3973

Merged

	return nil, err
	return nil, fmt.Errorf("for runtime %s: %w", err)

[tmpnet] Refactor runtime configuration in preparation for kube #3867

[tmpnet] Refactor runtime configuration in preparation for kube #3867

Uh oh!

Conversation

maru-ava commented Apr 8, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

PR Chain: tmpnet+kube

Why this should be merged

How this works

How this was tested

Need to be documented in RELEASES.md?

TODO

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

felipemadero left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

maru-ava commented Apr 8, 2025 •

edited

Loading