Provide a testing framework

[jvalkeal]

Testing a shell applications has traditionally been a difficult task to do if something has to happen beyond plain unit test of a method targets. This issue is about tracking this work.

[jvalkeal]

There are two sides to testing cli apps, one being within jvm with boot style tests, second being black box testing with generated artefact.

As most of you have noticed that if taking a simple project from start.spring.io and then running build it hangs up as on default without arguments interactive shell kicks in and spring context bootstrap never completes(as it's waiting user input). While you can add args to @SpringBootTest so that non-interactive runner kicks in, this is not what we should require.

From existing @SpringBootTest and @DataJpaTest we should look what @ShellTest would look like. To be able to interact with a shell would be beneficial in terms that test would never hung up. There are challenges though what comes for interactive vs. non-interactive testing.

Secondly it is totally different testing server app vs. cli app as there is always interaction from an user one way or the other. It gets even more problematic when cli app is interactive itself by expecting some user interaction, thus getting back to interactive runner concepts. For this it would be beneficial to have black boxing test framework which doesn't operate on a jvm.

For black box tests there is existing e2e framework in spring-shell which is based on node and uses node-pty and xterm-headless modules to run either graal based native app or jvm based fatjar app with provided commands. It even support entering interactive shell where you can simulate what user would do in an interactive session.

While java devs don't like to see any JS code, it has its own benefits as it's more close to OS and node is very good on that. Terminals on a different OS's behave differently and you need tty environment from a user. These 2 node modeles just mentioned are the ones what you see these days with "online" shell's providing features running "stuff" on a shell.

[jvalkeal]

Just some updates to this issue as I've been playing with various ideas in my branches, named https://github.com/jvalkeal/spring-shell/blob/gh489-work* for those wanting to look at it.

Essentially following boot model of adding dedicated slice test support for a particular feature I have something working as:

@ShellTest
@Import(ResolvedCommands.ResolvedCommandsConfiguration.class)
public class SpringShellSampleTests {

	@Autowired
	ShellClient.Builder builder;

	@Test
	void componentSingle() {
		ShellClient client = builder.build();
		client.shell();

		client.write(client.writeSequence().command("component single").build());
		await().atMost(2, TimeUnit.SECONDS).untilAsserted(() -> {
			assertThat(client.screen()).contains("❯ key1");
		});

		client.write(client.writeSequence().keyDown().build());
		await().atMost(2, TimeUnit.SECONDS).untilAsserted(() -> {
			assertThat(client.screen()).contains("❯ key2");
		});

		client.write(client.writeSequence().cr().build());
		await().atMost(2, TimeUnit.SECONDS).untilAsserted(() -> {
			assertThat(client.screen()).contains("Got value value2");
		});
	}
}

This experiment is based of using a subset of jediterm implementation(which provided working terminal emulation). Idea is that you'd have a test client which can be used to interact with a spring-shell app. Essentially what happens with this type of integration testing is that you send stuff into a shell and stuff is read from a shell and translated via emulation so that user is expected to see what is shown in a terminal(via emulation).

Focus of this work is to provide ways to simulate what user enters and then assert what is visible on a "screen".