-
Notifications
You must be signed in to change notification settings - Fork 25.2k
[DOCS] Document hidden data streams and naming restrictions #65862
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[DOCS] Document hidden data streams and naming restrictions #65862
Conversation
Pinging @elastic/es-core-features (Team:Core/Features) |
Pinging @elastic/es-docs (Team:Docs) |
Sorry for the false start. This is ready for review at your convenience @probakowski. |
@elasticmacine update branch |
Hi @probakowski! Can you let me know what you think of these changes? I'd love to get these docs in before the 7.11 release. Thanks! |
|
||
`open`:: | ||
Expand only to open indices. | ||
Match non-hidden data streams and open, non-hidden indices. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In my testing, open
and closed
match hidden indices and data streams though I'm not sure why.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks @danhermann. That sounds like a big regression, but I wasn't able to reproduce. Here's how I tested when running the latest master branch locally using ./gradlew run -Dtests.es.xpack.security.enabled=false
:
### 1. Create a non-hidden data stream.
PUT _index_template/visible-data-stream-template
{
"index_patterns": [ "visible-data-stream" ],
"data_stream": { },
"priority": 200
}
PUT _data_stream/visible-data-stream
### 2. Create hidden data stream.
PUT _index_template/hidden-data-stream-template
{
"index_patterns": [ "hidden-data-stream" ],
"data_stream": {
"hidden": true
},
"priority": 200
}
PUT _data_stream/hidden-data-stream
### 3. Get request with `expand_wildcards=open`.
GET *-data-stream?expand_wildcards=open
# Returns info for `visible-data-stream` only.
### 4. Get data streams with `expand_wildcards=open,hidden`.
GET *-data-stream?expand_wildcards=open,hidden
# Returns info for `visible-data-stream` and `hidden-data-stream`.
### 5. Create an open, hidden index.
PUT hidden-index
{
"settings": {
"index.hidden": true
}
}
### 6. Create and close an index.
PUT closed-index
POST closed-index/_close
### 7. Get request with `expand_wildcards=closed`.
GET *-data-stream,*-index?expand_wildcards=closed
# Returns info for `closed-index` only.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hmm, interesting. I was testing primarily with *
as the wildcard. With a more restrictive wildcard, it does appear to work as you've described.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks @jrodewig and sorry for the very long delay here. Overall changes look great, left couple comments about open
/closed
for data streams
|
||
`closed`:: | ||
Expand only to closed indices. | ||
Match closed indices. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
For data streams API (get, delete, stats etc) both open
and closed
will return all data streams (excluding hidden with hidden
is not specified) - we don't have a concept of closed and open data streams. I'm not really sure how to point that out in the documentation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good point. I was initially side-stepping this, but it may be better to be explicit.
With 04cd440, we now note that data streams cannot be closed. I don't think we need that for open
, but let me know.
Expansion of wildcards will include hidden indices. | ||
Must be combined with `open`, `closed`, or both. | ||
Match hidden data streams and hidden indices. Must be combined with `open`, | ||
`closed`, or both. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
For data streams API (get, delete, stats etc) there's a shortcut - if you provide only hidden
it will work as open,hidden
- we don't have a concept of closed and open data streams so forcing user to add it seems unnecessary. Of course it's not the case for search APIs where open
and closed
apply to the backing indices of data stream.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
To address this in 04cd440, I added a new definition specifically for the data stream APIs.
Thanks for the review @probakowski. I made some changes to address your comments. Please let me know if you have any other feedback. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM, thanks @jrodewig !
Documents #63987 and #65795
Previews