Skip to content

Add new is_write_index flag to Alias MetaData #30195

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed

Conversation

talevy
Copy link
Contributor

@talevy talevy commented Apr 26, 2018

in response to #30061.

This PR rebuilds aliasAndIndexLookup every time it wants to use it for alias validation. Depends on outcome of #29575 for using a pre-built data-structure.

This PR introduces a concept in aliases called is_write_index. When true on a specific
alias definition associated with an index, index requests using the associated alias name will
resolve to this index for indexing.

Write now, if one attempts to index a document using an alias that points to multiple indices, there is no way to resolve which one, so an exception is thrown. With this flag, you can have an alias that points to a specific index in a set of indices as the one to write to.

It is possible to set an index as an alias' write-index during index creation:

PUT foo1
{
  "aliases": {
    "bar": { "is_write_index": true }
  }
}

Now, index requests like PUT bar/_doc/_id will route to index foo1. Another index, foo2, is
free to also have an alias named "bar" pointing to it. This alias needs to be configured with is_write_index set to false. is_write_index is defaulted to false if not specified, or it can be explicitly set to false.

PUT foo2
{
  "aliases": {
    "bar": {}
  }
}

Now, searches against bar will point to both indices, but index requests will be routed to foo1.

These settings can be updated in the _aliases API like so:

POST _aliases
{
  "actions": [
    {
      "add": {
        "index": "foo1",
        "alias": "bar",
        "is_write_index": false
      }
    }
  ]
}

How does this relate to Rollover? Rollover depended on rollover-aliases that only point to
one index since multiple would break indexing. Now that the same alias can point to multiple indices, the only requirement we have is that there is only one write index.

Here is an example scenario that didn't work before, but works with these changes:

PUT _template/my_template
{
  "index_patterns": ["foo-*"],
  "settings": {
    "number_of_shards": 1,
    "number_of_replicas": 1
  },
  "aliases": {
    "logs": {}
  },
  "mappings": {
    "_doc": {
      "_source": {
        "enabled": false
      }
    }
  }
}

PUT foo-000001
{
  "aliases": {
    "logs": { "is_write_index": true }
  }
}

PUT logs/_doc/_id
{
  "hello": "world"
}

POST /logs/_rollover/
{
  "conditions": {
    "max_docs":  1
  }
}

GET foo-000001 # now this is `is_write_index: false`
GET foo-000002 # now this is `is_write_index: true`

@talevy talevy added the WIP label Apr 26, 2018
@talevy talevy force-pushed the alias-write-only-index branch from ed7f92c to ece9f50 Compare April 26, 2018 23:46
@colings86 colings86 added the :Distributed Indexing/CRUD A catch all label for issues around indexing, updating and getting a doc by id. Not search. label Apr 27, 2018
@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-distributed

@talevy talevy added the :Data Management/Indices APIs APIs to create and manage indices and templates label Apr 27, 2018
@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-core-infra

@talevy talevy removed the :Distributed Indexing/CRUD A catch all label for issues around indexing, updating and getting a doc by id. Not search. label Apr 27, 2018
@talevy
Copy link
Contributor Author

talevy commented May 17, 2018

Closing in favor of new PR

@talevy talevy closed this May 17, 2018
@talevy
Copy link
Contributor Author

talevy commented May 17, 2018

I've reopened a more complete version of this feature here: #30703

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
:Data Management/Indices APIs APIs to create and manage indices and templates >feature WIP
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants