Search::ElasticSearch 8 breaks connect to Elastic 5.6

[mariopaumann]

After Upgrading Search::ElasticSearch to latest release 8 we're getting

13:30:02.912 [(eval 1048):17] (2452946) INFO: Current cxns: ["http://localhost:9278"]
13:30:02.912 [(eval 1047):17] (2452946) INFO: Forcing ping before next use on all live cxns
13:30:02.912 [(eval 1048):17] (2452946) INFO: Ping [http://localhost:9278] before next request
13:30:02.929 [(eval 1048):17] (2452946) INFO: Pinging [http://localhost:9278]
13:30:02.951 [(eval 1048):17] (2452946) INFO: Marking [http://localhost:9278] as dead. Next ping at: Mon Jan  2 13:31:02 2023
13:30:02.951 [(eval 1056):17] (2452946) CRITICAL:
13:30:02.951 [Bread/Board/LifeCycle/Singleton.pm:45] (2452946) ERROR: SERR> [NoNodes] ** No nodes are available: [http://localhost:9278], called
from sub Search::Elasticsearch::Role::Client::Direct::__ANON__ at /appl/crse1/perllib/CRS/Forkable.pm line 33.

we're still using elastic 5.6 but with 5_0::Direct. After downgrading to 7.717 it works.

[Grinnz]

Same issue attempting to connect to Elasticsearch 6.8.23 using 6_0::Direct.

[ezimuel]

@mariopaumann if you want to connect to Elasticsearch 5.6 you can use elasticsearch-perl 6.81, you don't need to use 8.0. I'm surprised that you can connect using 5_0::Direct with 7.717 🤔

[ezimuel]

@Grinnz same for you, if you want to connect to Elasticsearch 6.8 you can use elasticsearch-perl 6.81 or 7.717 using 6_0::Direct.

[ezimuel]

@Grinnz , @mariopaumann the versions of elasticsearch-perl are aligned with the Elasticsearch server versions. If you want to connect to Elasticsearch x.y.z you need to use elasticsearch-perl version x.*.

[Grinnz]

That's not really a solution. CPAN modules are expected to be maintained at the latest version. That's why 5_0::Direct and 6_0::Direct were split into separate distributions to begin with.

[Grinnz]

@Grinnz , @mariopaumann the versions of elasticsearch-perl are aligned with the Elasticsearch server versions. If you want to connect to Elasticsearch x.y.z you need to use elasticsearch-perl version x.*.

If you want it to function this way, you will have to release Search::Elasticsearch 5.x and 6.x etc as separate modules with different names, and force users to change the module they are using.

[ezimuel]

@Grinnz I'm not sure I'm following, we are mantaining the latest version in CPAN as here. For historical reason, elasticsearch-php has been released each time with multiple versions included (e.g. 7_0 and 8_0 in 8.0.0). We did this for helping people in the migration of the code. Imagine that you want to switch from version 7 to 8. You can upgrade elasticsearch-perl client to 8 and use 7_0::Direct in the configuration for having the previous behaviour.

$e = Search::Elasticsearch->new(
    client => '7_0::Direct'
);

In the meantime, you can also build another client object with 8_0::Direct and use the new features. This was the reason why we are supporting the latest major version and the previous one in the same module.

[Grinnz]

I mean that CPAN modules are meant to be maintained at their latest available version by end users. So if someone is to install https://metacpan.org/pod/Search::Elasticsearch::Client::6_0::Direct it will install the latest version of Search::Elasticsearch which is unable to connect to Elasticsearch 6.x. And when the user upgrades their CPAN modules, perhaps even to connect to Elasticsearch 8.x with another project. they become unable to connect to Elasticsearch 6.x.

[berick]

Confirming I can no longer connect to ES version 6 using latest Client::6_0.

[srchulo]

I'm seeing the same issue even when I install older versions where I can't connect to Elasticsearch 6.x

[srchulo]

With 7,717 I still see the No Nodes available error using 6_0::Direct:

my $es = Search::Elasticsearch->new(
		client => '6_0::Direct',
		nodes => 'https://my-elastic-search.com',
	);

say $es->info;

When I run curl against my cluster, it outputs fine.

[srchulo]

Notes for anyone seeing my issue: I was testing on a clean machine, and it looks like LWP::Protocol::https is not declared as a dependency. Installing that fixed my issue.

[ezimuel]

@Grinnz we are supporting the last major version (8) and the previous one (7). If you are working with 6 you need to use the previous version (7). If you update all the CPAN modules you can apply version constrain in cpanfile like as follows:

requires 'Search::Elasticsearch', '>=6.0, <8.0';

Moreover, using cpanm you can specify the version that you want to install, as follows:

cpanm EZIMUEL/Search-Elasticsearch-6.81.tar.gz

If I get your point, you are suggesting that we need to include in elasticsearch-perl 8 also 6_0::Direct, 5_0::Direct and so on?

[ezimuel]

@srchulo do you want to send a PR for including LWP::Protocol::https in 7.x branch? Thanks!

[Grinnz]

@Grinnz we are supporting the last major version (8) and the previous one (7). If you are working with 6 you need to use the previous version (7). If you update all the CPAN modules you can apply version constrain in cpanfile like as follows:

As mentioned, this is unfortunately not sufficient to prevent all issues, as CPAN is not a package manager and cannot prevent Search::Elasticsearch from being upgraded to 8.0 by another process. Your suggestion only works in a constrained Carton/Carmel environment since those are able to pin versions for a particular project.

If I get your point, you are suggesting that we need to include in elasticsearch-perl 8 also 6_0::Direct, 5_0::Direct and so on?

No, the current separation of distributions works fine, the user simply declares the Direct module they need as a dependency. The problem is that it doesn't work with the latest version of Search::Elasticsearch.

[ezimuel]

No, the current separation of distributions works fine, the user simply declares the Direct module they need as a dependency. The problem is that it doesn't work with the latest version of Search::Elasticsearch.

@Grinnz ok, this means there's something wrong in the process. Can you provide me the steps to reproduce the issue starting from a clean installation? Thanks.

[Grinnz]

The only step necessary is:

cpanm Search::Elasticsearch::Client::6_0

This will install Search-Elasticsearch-Client-6_0 7.717 and Search-Elasticsearch 8.00, which is unable to connect to an Elasticsearch 6.x node until the user downgrades it.

Even if constrained by a cpanfile in the initial install, another process may upgrade Search-Elasticsearch, and Search::Elasticsearch::Client::6_0 can't be installed at the same time as Search::Elasticsearch::Client::8_0 without causing this issue. The only end user solution to reliably keep older versions of modules is dependency tree pinning such as provided by Carton/Carmel, which is of course a good solution but not necessarily a widespread one.

[trentfisher]

I am having a similar problem to others here. I have an ES cluster running 5.x, and I installed the latest Search::Elasticsearch from cpan. I have tried with and without "client => '5_0::Direct'" and either way I get the same vague NoNodes error when I try to connect (my code looks very much like srchulo's code above).

I traced it down in the code to Search::Elasticsearch::Role::Cxn::process_response() line 366, and the fact that there is no "x-elastic-product" header returned. If I comment out that section, my queries work correctly (regardless of the "client" setting mentioned above).

Also, the error message in the throw() on the next line gets lost, and only the vague "NoNodes" error is left.

So what is the right way to make this work?

[ezimuel]

@trentfisher, @Grinnz, @srchulo, @berick, @mariopaumann I'm working on a fix for this, the solution seems to be to remove the product check with x-elastic-product header for versions < 8.0. This means with 7_0::Direct or less the product check will not be executed. Sounds good?

[ezimuel]

I just sent PR #231 for fixing the product check issue. This should solve this issue, @trentfisher , @Grinnz , @srchulo , @berick , @mariopaumann can you confirm? Thanks!

[stuartskelton]

Another bit of info:

A check for under 8x wouldn't work.

Any ES under 7.14.0 would be broken as this header was introduced in that version https://www.elastic.co/guide/en/elasticsearch/reference/7.17/release-notes-7.14.0.html.

So I suspect this may be breaking quite a few people.

As an interim measure, can we just add a ENV variable to disable the check? With the usual HERE BE DRAGONS and You take your life (and by extension production) in your own hand warnings?

I am working with my managed ES provider to see if they can add it as I am on 7.10.0, but a nicer solution would be best.

Regards

Stuart

[ezimuel]

Fixed in version_8.12