[7.x][ML] No error when datafeed stops during updating to started (#46495) (#46542)

Investigating the test failure reported in #45518 it appears that
the datafeed task was not found during a tast state update. There
are only two places where such an update is performed: when we set
the state to `started` and when we set it to `stopping`. We handle
`ResourceNotFoundException` in the latter but not in the former.

Thus the test reveals a rare race condition where the datafeed gets
requested to stop before we managed to update its state to `started`.
I could not reproduce this scenario but it would be my best guess.

This commit catches `ResourceNotFoundException` while updating the
state to `started` and lets the task terminate smoothly.

Closes #45518

Backport of #46495

Author: dimitris-athanasiou

x-pack/plugin/ml/src/main/java/org/elasticsearch/xpack/ml/datafeed/DatafeedManager.java

@@ -8,6 +8,7 @@
 import org.apache.logging.log4j.LogManager;
 import org.apache.logging.log4j.Logger;
 import org.elasticsearch.ElasticsearchStatusException;
+import org.elasticsearch.ResourceNotFoundException;
 import org.elasticsearch.action.ActionListener;
 import org.elasticsearch.client.Client;
 import org.elasticsearch.cluster.ClusterChangedEvent;
@@ -95,7 +96,12 @@ public void onResponse(PersistentTask<?> persistentTask) {
 
                         @Override
                         public void onFailure(Exception e) {
-                            finishHandler.accept(e);
+                            if (e instanceof ResourceNotFoundException) {
+                                // The task was stopped in the meantime, no need to do anything
+                                logger.info("[{}] Aborting as datafeed has been stopped", datafeedId);
+                            } else {
+                                finishHandler.accept(e);
+                            }
                         }
                     });
                 }, finishHandler::accept