Avoid forking in AbstractSearchAsyncAction

[dnhatn]

We don't need to fork when handling unassigned shard failures in AbstractSearchAsyncAction as we never call it recursively.

Relates #70792

[elasticmachine]

Pinging @elastic/es-search (Team:Search)

[jimczi]

LGTM, should also remove the fork in

elasticsearch/server/src/main/java/org/elasticsearch/action/search/AbstractSearchAsyncAction.java

Line 301 in 5baabff

fork(() -> onShardFailure(shardIndex, shard, shardIt, e));

since executeNext will fork if needed ?

[dnhatn]

@jimczi I think it's not entirely safe to remove fork here. If we have a lot of shards and we fail to get connections for many of them, then we recursively execute [performPhaseOnShard -> onShardFailure -> performPhaseOnShard -> onShardFailure] many times. This can lead to StackOverflow.

I think we should fork only after several executions instead of a single execution (the current behavior). I will look into it in a follow-up.

[jimczi]

Right, I read too quickly and thought that executeNext would handle the retry on another replica. Let's leave it for a follow up. Thanks

[dnhatn]

Thanks Jim!