make identificaiton of closed indices more clear #25253

dzmitry-lahoda · 2017-06-15T13:16:26Z

No description provided.

elasticmachine · 2017-06-15T13:16:28Z

Since this is a community submitted pull request, a Jenkins build has not been kicked off automatically. Can an Elastic organization member please verify the contents of this patch and then kick off a build manually?

dzmitry-lahoda · 2017-06-15T13:17:22Z

https://elasticsearch.na1.echosign.com/public/viewAgreement?tsid=CBFCIBAA3AAABLblqZhAw9JIc_nx4zzBm6p5I0Eb3OvgGcMdYvUBhpd4WEZqdu06WW-a3-ZojdMqUmyKveZZprmKUtIftRIY6ZHbirm_a&

javanna

hi @asd-and-Rizzo thanks for your PR I requested a change.

javanna · 2017-06-16T15:18:48Z

docs/reference/indices/open-close.asciidoc

@@ -32,3 +32,5 @@ This setting can also be changed via the cluster update settings api.

 Closed indices consume a significant amount of disk-space which can cause problems in managed environments. Closing indices can be disabled via the cluster settings
 API by setting `cluster.indices.close.enable` to `false`. The default is `true`.
+
+Closed index responses with `index_closed_exception` and `403` status on call to `_stats` API. Closed indices are not returned in cross index `_aliases` or `_stats` API responses.


there are many more APIs that return IndexClosedException and 403 status code when executed against closed indices. For instance _search. But through the ignore_unavailable parameter, you can decide whether to ignore those closed indices or get an exception back instead. I would rephrase your new paragraph without mentioning any specific APIs, rather say "most APIs don't support execution against closed indices, hence `ignore_unavailable allows to control whether closed indices are skipped or cause an IndexClosedException to be returned". What do you think?

dzmitry-lahoda · 2017-06-19T12:32:38Z

I will check ignore_unavailable behavior and update as recommend within next 2 weeks.

javanna · 2017-06-19T12:44:48Z

thanks @asd-and-Rizzo please ping me once you've done so

I have document behavior of closed index on single index call and multiple index call. I see problem here, at least for my product. How do we know that index is closed or there are closed indices via API because these are not visible from cross index wildcard calls? Should we store index list externally to ES?

dzmitry-lahoda · 2017-07-03T11:57:03Z

@javanna, Please check my comment in patch. I see your thinking of closed behavior could be different from what I see. What does ES version your knowledge steam from?

javanna

hi @asd-and-Rizzo I left a couple more comments. My feedback doesn't stem from any specific Elasticsearch version, ignore_unavailable is supported for a very long time now. Let me know if you have any further questions.

javanna · 2017-07-12T14:53:26Z

docs/reference/indices/open-close.asciidoc

@@ -32,3 +32,7 @@ This setting can also be changed via the cluster update settings api.

 Closed indices consume a significant amount of disk-space which can cause problems in managed environments. Closing indices can be disabled via the cluster settings
 API by setting `cluster.indices.close.enable` to `false`. The default is `true`.
+
+Closed index responses with `index_closed_exception` and `403` for most APIs, including `_stats` and `_search`. `ignore_unavailable` allows to control whether closed index do not cause `index_closed_exception` to be returned.  Index returns value on call to `_aliases`. 


I would slightly rephrase this sentence: "Most APIs, including _stats and _search, return index_closed_exception and 403 status code when accessing closed indices. The `ignore_unavailable querystring parameter allows to control whether closed indices cause an error to be returned or not.".

What do you mean with "Index returns value on call to _aliases" ?

javanna · 2017-07-12T14:55:12Z

docs/reference/indices/open-close.asciidoc

+
+Closed index responses with `index_closed_exception` and `403` for most APIs, including `_stats` and `_search`. `ignore_unavailable` allows to control whether closed index do not cause `index_closed_exception` to be returned.  Index returns value on call to `_aliases`. 
+
+Closed indices are not returned in most wildcard index API calls, like `_aliases`, `_stats` or `_search`, nor throw any exceptions and behave like non-existent.


I am not sure about this sentence. You control through ignore_unavailable whether closed indices throw error or not. They are unavailable indices for most of the APIs, hence either they are ignored or they cause an error (e.g. it is not technically possible to search against a closed index, so the question is just what to do instead: error or skip them?)

dzmitry-lahoda · 2017-07-22T20:02:29Z

Here is case:
/my_index
/my_closed_index

Request:
/_aliases
Response:
/my_index
Request:
/my_closed_index/_aliases
Response:
/my_closed_index
Request:
/_search
Response:
Some values from my_index
No error from my_closed_index.
Request:
/my_closed_index/_search?ignore_unavailable
Response:
Zero hits.
Request:
/my_closed_index/_search
Response:
Error.

From these I conclude next:

I should keep external list of indices to be able to delete them later (e.g. because of my retention policy we do close and then delete) or access ES cluster directly, not via API, to find the list.
ignore_unavailable does not work on multi indeces requests. So it makes ignore_unavailable useless in some situations.

So the problem is what is correct and useful specification of closed indices calls and usage of ignore_unavailable?

I will rephrase the sentence, but current behavior of closed indexes does not feel consistent for me.

javanna · 2017-07-24T09:39:21Z

hi @asd-and-Rizzo we'd like users to ask questions in our forums, as we reserve github issues for verified bug reports and feature requests.

From your example I see that ignore_unavailable does work with e.g. the search API. But I am not clear on the following statement:

ignore_unavailable does not work on multi indeces requests. So it makes ignore_unavailable useless in some situations.

Please do open another issue if you think that the above is a bug rather than commenting here.

This other issue may be interesting too: #9438

dzmitry-lahoda · 2017-08-07T10:42:29Z

@javanna, thanks for reference to forum and other issues. My pull request was only about to add documentation about current behavior, no expectancy to change this behavior. Just helping our support team and hoping that current behavior inconsistencies(as I see these) will be visible to other people.

javanna

sorry @asd-and-Rizzo it took me a while to have another look, I left some more comments. Could you address those so I can get your PR in?

javanna · 2017-08-24T17:34:52Z

docs/reference/indices/open-close.asciidoc

@@ -32,3 +32,10 @@ This setting can also be changed via the cluster update settings api.

 Closed indices consume a significant amount of disk-space which can cause problems in managed environments. Closing indices can be disabled via the cluster settings
 API by setting `cluster.indices.close.enable` to `false`. The default is `true`.
+
+Most APIs, including _stats and _search, return `index_closed_exception` and 403 status code when accessing closed index.  The `ignore_unavailable` querystring parameter allows to control whether closed index cause an error to be returned or not.  Call to index `_aliases` returns correct value without error. 


403 recently became 400 . Can you update that?

when accessing closed indices

whether closed indices cause an error to be returned or are skipped.

I find this sentence confusing: "Call to index _aliases returns correct value without error. " I would remove it.

javanna · 2017-08-24T17:37:04Z

docs/reference/indices/open-close.asciidoc

+
+Most APIs, including _stats and _search, return `index_closed_exception` and 403 status code when accessing closed index.  The `ignore_unavailable` querystring parameter allows to control whether closed index cause an error to be returned or not.  Call to index `_aliases` returns correct value without error. 
+
+Closed indices are not returned in most wildcard API calls, like `_aliases`, `_stats` or `_search`, nor throw any exceptions. Closed indices behave like non-existent, closed indices will not be returned during call to `_aliases`. Passing `ignore_unavailable` is irrelevant to wildcard indices calls.


expand_wildcards allows to control whether to expand wildcards against open, closed or both. can you update this sentence please?

javanna · 2017-11-06T09:22:03Z

There has been no activity on this for two months, I am closing. Feel free to reopen once the comments are addressed.

make identificaiton of closed indices more clear

0e626e6

javanna requested changes Jun 16, 2017

View reviewed changes

javanna added >docs General docs changes review v6.0.0 v5.6.0 v5.5.0 v5.5.1 and removed v5.5.0 labels Jun 16, 2017

javanna self-assigned this Jun 16, 2017

javanna requested changes Jul 12, 2017

View reviewed changes

clintongormley added v5.5.2 and removed v5.5.1 labels Jul 21, 2017

clintongormley added v6.1.0 and removed v5.6.0 labels Jul 25, 2017

Update open-close.asciidoc

0018532

colings86 added v5.5.3 and removed v5.5.2 labels Aug 8, 2017

javanna requested changes Aug 24, 2017

View reviewed changes

colings86 added v5.5.4 and removed v5.5.3 labels Aug 25, 2017

javanna closed this Nov 6, 2017

lcawl removed the v6.1.0 label Dec 12, 2017


		Closed index responses with `index_closed_exception` and `403` for most APIs, including `_stats` and `_search`. `ignore_unavailable` allows to control whether closed index do not cause `index_closed_exception` to be returned. Index returns value on call to `_aliases`.

		Closed indices are not returned in most wildcard index API calls, like `_aliases`, `_stats` or `_search`, nor throw any exceptions and behave like non-existent.


		Most APIs, including _stats and _search, return `index_closed_exception` and 403 status code when accessing closed index. The `ignore_unavailable` querystring parameter allows to control whether closed index cause an error to be returned or not. Call to index `_aliases` returns correct value without error.

		Closed indices are not returned in most wildcard API calls, like `_aliases`, `_stats` or `_search`, nor throw any exceptions. Closed indices behave like non-existent, closed indices will not be returned during call to `_aliases`. Passing `ignore_unavailable` is irrelevant to wildcard indices calls.

make identificaiton of closed indices more clear #25253

make identificaiton of closed indices more clear #25253

Uh oh!

Conversation

dzmitry-lahoda commented Jun 15, 2017

Uh oh!

elasticmachine commented Jun 15, 2017

Uh oh!

dzmitry-lahoda commented Jun 15, 2017

Uh oh!

javanna left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dzmitry-lahoda commented Jun 19, 2017

Uh oh!

javanna commented Jun 19, 2017

Uh oh!

dzmitry-lahoda commented Jul 3, 2017

Uh oh!

javanna left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dzmitry-lahoda commented Jul 22, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

javanna commented Jul 24, 2017

Uh oh!

dzmitry-lahoda commented Aug 7, 2017

Uh oh!

javanna left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

javanna commented Nov 6, 2017

Uh oh!

Uh oh!

dzmitry-lahoda commented Jul 22, 2017 •

edited

Loading