[CI] Yaml tests failing due to "resource_already_exists_exception"

[mark-vieira]

We have a number of test failures that involve YAML tests failing due to attempting to create an already existing index. It seems possible that there's a fundamental issue with the YAML test infrastructure that allows this scenario to periodically occur.

https://build-stats.elastic.co/app/kibana#/discover?_g=(refreshInterval:(pause:!t,value:0),time:(from:now%2Fw,mode:quick,to:now%2Fw))&_a=(columns:!(branch,class,test,build_url,build-id),filters:!(('$state':(store:appState),meta:(alias:!n,disabled:!f,index:e58bf320-7efd-11e8-bf69-63c8ef516157,key:build-id,negate:!t,params:(query:'20190815014739-48ABA9DF',type:phrase),type:phrase,value:'20190815014739-48ABA9DF'),query:(match:(build-id:(query:'20190815014739-48ABA9DF',type:phrase))))),index:e58bf320-7efd-11e8-bf69-63c8ef516157,interval:auto,query:(language:kuery,query:"resource_already_exists_exception"),sort:!(time,desc))

[elasticmachine]

Pinging @elastic/es-core-infra

[droberts195]

#45605 is a more detailed triage of the latest instance of this. (I opened that before I saw this one.)

I looked at one more build that's picked up by the Kibana search in the description of this issue.

In https://elasticsearch-ci.elastic.co/job/elastic+elasticsearch+master+multijob+fast+part1/759/console the server-side log from the failure that occurs in SmokeTestMultiNodeClientYamlTestSuiteIT.test {yaml=search/160_exists_query/Test exists query on boolean field in empty index} is:

[2019-08-14T18:07:42,945][WARN ][o.e.i.IndicesService     ] [integTest-0] [test/ee-7zmiUQGayvOq6LgEShg] failed to delete index
org.elasticsearch.env.ShardLockObtainFailedException: [test][0]: obtaining shard lock timed out after 0ms, previous lock details: [shard creation] trying to lock for [deleting index directory]

This is the exact same thing that happens in the case I looked at in #45605. And then the server-side of the docs test again shows nothing except a big gap with no logging:

[2019-08-14T18:08:45,832][INFO ][o.e.c.m.MetaDataCreateIndexService] [node-0] [sales] creating index, cause [api], templates [], shards [1]/[1], mappings [_doc]
[2019-08-14T18:09:23,309][INFO ][o.e.c.m.MetaDataDeleteIndexService] [node-0] [sales/tZO5adjTTXq8kLb1Of1UaQ] deleting index

[alpar-t]

Let's close this as a duplicate.