[Transform] transform can not be disabled on a subset of master-e/data nodes

[hendrikmuhs]

Affected versions:7.2 ->

Problem: Transform uses the persistent task framework and places its task on a node out of all data nodes. The placement is based on load and can not be controlled.

Mitigation: Do not disable transform on a subset of nodes. The limitation has been document in #52194

Important remark: The transform task uses very few resources, it only coordinates the data flow between search and index operations. The resource heavy tasks, be it search/data aggregations and indexing run on the data node that holds the corresponding shards.

Solution

Transform should allow to control on which nodes it runs. This can be done similar to ML which uses a node.ml setting. Transform should add a node.transform setting to explicitly disable transform on a certain node. In addition to that xpack.transform.enabled: false should implicitly disable transform assignment on that node (although we think disabling the plugin is not the right way to do it).

Related issue: #50033

[elasticmachine]

Pinging @elastic/ml-core (:ml/Transform)