Simplifying cleanup after code examples

[lcawl]

Currently the examples in the Elasticsearch documentation are tested using the functionality described here: https://github.com/elastic/elasticsearch/tree/master/docs#snippet-testing

When the X-Pack code was merged into the OSS code, it seems that cleanup for the documentation tests was not carried over from the x-pack/docs folder to the docs folder. There seem to be structures created during testing (system indices, etc) that sometimes don’t get cleaned up and then cause problems with subsequent tests. As a result, cleanup has been added on an as-needed basis. For example:

https://github.com/elastic/x-pack-elasticsearch/issues/2818
#31450
#43271
#44123

Ideally the steps for cleaning up after documentation tests can be clarified or simplified so that it's easier to accomplish for new features and/or existing features (security, watcher) that are still lingering in the x-pack/docs due to testing issues.

[elasticmachine]

Pinging @elastic/es-docs (:Docs)

[elasticmachine]

Pinging @elastic/es-core-infra (:Core/Infra/Build)

[mark-vieira]

@lcawl this is labeled with :Core/Infra/Build but it's not clear to me if any build infrastructure changes really need to be made here. Rather this just describes some improvement to the docs tests themselves. I ask because this label is generally meant to indicate some issue or effort that would fall under the scope of responsibility of the build engineering team (i.e. me 😉).

Does this issue need intervention from our end, or is this just some backlog cleanup work that will primarily be handled by the docs team?

[lcawl]

@mark-vieira I think @nik9000 @polyfractal or @davidkyle are better able to answer that question than I am, since they were involved in migrating the cleanup for rollup and ML features. Anything we can do to make it simpler or faster for other features to get the cleanup in place would be great in my opinion. I have no idea what cleanup steps would be required for the security and watcher features, for example, if they finally moved from the x-pack/docs to the docs folders.

[nik9000]

fall under the scope of responsibility of the build engineering team

This is part of the test framework and we decided a while back to use core/infra/build for that. If that isn't a thing we want to do any more maybe we should add another label?

[mark-vieira]

@nik9000 we can keep the label, just wanted to know if I need to keep this on my radar or not.

[davidkyle]

It was @colings86 who added the :core/infra/build label after this issue had already been tagged with :Docs. @colings86 is this something you've automated and regarding #51576 (comment) is it necessary

[colings86]

Fixing this will require test infrastructure that actually cleans up the cluster including the x-pack features after each test. This is going to need to be a combination of Gradle and Java infrastructure so it comes under the definition of the :Core/Infra/Build label

[elasticsearchmachine]

Pinging @elastic/es-delivery (Team:Delivery)