Make indices APIs consistent

[clintongormley]

An index accepts settings, mappings, warmers and aliases. However, some use singular and some use plural, eg:

Mappings:

PUT /index/type/_mapping
{ type: { .... }}

GET /index/{type|*|blank}/_mapping

Settings:

PUT /index/_settings
GET /index/_settings

Warmers:

PUT /index/_warmer/name
{ ... }
GET /index/_warmer/{name|*|blank}

Aliases:

POST /_aliases
{ actions: [....] }

 PUT /index/_alias/alias
 { ... }

 GET /{index|*|_all}/_alias/{alias|*}

There are a couple of possibilities:

Distinguish between singular and plural when PUT/POSTing

Create/update a mapping, warmer or alias (not settings) with:

PUT /index/_mapping|_alias|_warmer/{name}
{ ... }

This breaks bwc for mapping (/index/type/_mapping vs /index/_mapping/type) - possibly support both variants for all actions?

Create/update mappings, warmers, aliases, settings with:

POST /index/_mappings|_settings|_warmers|_aliases
{ name: {....}, name2: {....} ... }

Retrieve via singular or plural:

GET /index/_mapping|_alias|_warmer|_setting/{name|*|_all|blank}
GET /index/_mappings|_aliases|_warmers|_settings/{name|*|_all|blank}

Alternatively, make singular/plural synonyms, so either can be used in any of the above examples

One final thing, all indices can be specified as _all or * (depending on the API), but eg warmers and aliases only understand wildcards. I think _all should be an acceptable alias for *for all of them

[brwe]

[...]

_I think all should be an acceptable alias for * for all of them

I agree.

Just for completeness: we also currently have the DELETE for alias

  DELETE /{index}/_alias/name

and also for warmers:

 DELETE index/_warmer/name

 DELETE index/_warmer/prefix*

 DELETE index/_warmer/

For mappings, delete is:

DELETE index/type

For settings there is none and would probably not make sense to have that.

[brwe]

So we have PUT/POST, GET and DELETE for all three we have several options. I list some here again. Please feel free to add your own option and vote.

I vote for

PUT/POST: Option 1

GET: Option 2

DELETE: Option 1

PUT/POST:

Option 1:

Remove all singular PUTs, do not distinguish between PUT and POST and only allow putting multiple items for all 4 APIs like this:

   POST (or PUT) /index/_mappings|_settings|_warmers|_aliases
   { name: {....}, name2: {....} ... }

The PUT like

 PUT /index/_alias|_warmer/{name}

would be removed completely. This might cause some overhead when typing requests but at least you can remember the syntax even when you are braindead.

Option 2:

Make the distinction singular/plural (PUT/POST) like @clintongormley wrote:

 PUT /index/_mapping|_alias|_warmer/{name}
 { ... }

 POST /index/_mappings|_settings|_warmers|_aliases
 { name: {....}, name2: {....} ... }

This would cause the syntax to be slightly more complex but is still sort of intuitive.

Option 3:

Like option 2 but allow singlar and plural for PUT and POST. Would not break bwc.

GET

Option 1

allow plural and singular but only GET /index/_api/{name|*|_all|blank}

 GET /index/_mapping|_alias|_warmer|_setting/{name|*|_all|blank}
 GET /index/_mappings|_aliases|_warmers|_settings/{name|*|_all|blank}

Option 2

allow only plural

GET /index/_mappings|_aliases|_warmers|_settings/{name|*|_all|blank}

Option 3

allow plural and singular with GET /index/_api/{name|*|_all|blank} and GET /index/{name|*|_all|blank}/_api

 GET /index/_mapping|_alias|_warmer|_setting/{name|*|_all|blank}
 GET /index/_mappings|_aliases|_warmers|_settings/{name|*|_all|blank}

 GET /index/{name|*|_all|blank}/_mapping|_alias|_warmer|_setting
 GET /index/{name|*|_all|blank}/_mappings|_aliases|_warmers|_settings

Would not break bwc.

Option 4

allow only plural

GET /index/_mappings|_aliases|_warmers|_settings/{name|*|_all|blank}
GET /index/{name|*|_all|blank}/_mappings|_aliases|_warmers|_settings

DELETE

Option 1

Make the DELETE available for 3 APIs like in _warmer:

 DELETE index/_mapping|_alias|_warmer/name

 DELETE index/_mapping|_alias|_warmer/prefix*

 DELETE index/_mapping|_alias|_warmer/

Option 2:

Leave everything as is.

[brwe]

After discussion here is what I will implement:

PUT/POST: Option 2. both plural and singular version should be supported for PUT and POST but only singular for PUT and plural for POST documented.

For PUT _mapping remove "type" from body, because it has to be defined in url anyway and is therefore redundant.

GET: Option 1 but also support two different orderings for _mapping in order to not break bwc. Do not document this. Make sure that for index * and _all is supported. Make sure that for type *, _all and blank is supported.
Return data structure should be the same for getting a single item or multiple ones. For example _mapping: Getting a single type and multiple types should be

{
     "index": {
         "type" : {
                 .....
        }
    }
}

DELETE: Should be like GET but do not support blank as item name in order to remove all, only _all or * allowed.

[electrical]

Just like to give my £0.02

few years ago i was designing REST API's and was following some webinars by a company who design api's ( http://www.youtube.com/apigee ) and following in a few nice discussions on the google groups ( https://groups.google.com/forum/#!forum/api-craft )

One of those items was the singular/plural discussion and at the end of it the 'pragmatic' approach would be to use only plural names.
Having singular and plural endpoints can become very confusing especially if its for same sort of endpoint ( GET /index/_mapping & POST /index/_mappings for example )

An other thing is the ordering, by allowing different methods it can become confusing again.
Personally i like the GET /index/_mappings/name because it makes it very clear what you expect.. ( GET me mapping 'name' from 'index' )

Cheers.

[electrical]

Forgot to address the single or multiple items in an endpoint call.
Personally i wouldn't differentiate between a single or multiple items in a call but always name them like its done with multiple items right now.

POST /index/_mappings|_settings|_warmers|_aliases
{ name: {....}, name2: {....} ... }

this makes things consistent across the board and avoid having different endpoints for pretty much the same thing.

[brwe]

In principle I agree to the plural only. However, since some APIs are already singular, it might be good have plural and singular as synonyms?

There is several issues we would like to fix:

1. plural and singular should be unified.
2. index and item name definition should be treated the same way. That is, *, _all and blank should be synonyms, except for DELETE where we should not allow blank
3. Order of index/_api/item_name should be consistent.
4. PUT and POST should be equally supported by all four APIs.

Assuming that we agree on making plural and singular synonyms, this would require the following list of changes:

_mapping

PUT

PUT {index|_all|*|prefix*}/type/_mapping
{
    "type" : {
          "properties": {...}
    }
}

would become

PUT {index|_all|*|prefix*|blank}/_mapping/type
{
     "properties": {...}
}

and _mappings as synonym.

We would keep the old behavior but remove the documentation.

Note that currently there is also the option to write

PUT {index|_all|*|prefix*}/_mapping?type=typename

although this is not documented.

POST

Currently exactly the same as PUT.

We would add

POST {index|_all|*|prefix*|blank}/_mappings
{
    "type1" :
         "properties": {...}
    },
    "type2" :
         "properties": {...}
    },
    ...
}

and _mapping as synonym.

GET

GET {index|_all|*|prefix*|blank}/{type|*|blank}/_mapping

would become

GET {index|_all|*|prefix*|blank}/_mappings/{type|_all|*|prefix*|blank}

and _mapping as synonym.

We would keep the old behavior but remove the documentation.

DELETE

DELETE index/type

would become:

DELETE {index|_all|*|prefix*|blank}/_mapping/{type|_all|*|prefix*}

We would add _mappings as synonym.

We would keep the old behavior but remove the documentation.

_settings

PUT

PUT {index|_all|*|prefix*|blank}/_settings {
    ...
}

would remain.

We would add _setting as synonym

POST

Does not exist yet.

We would add

POST {index|_all|*|prefix*|blank}/_settings {
    ...
}

We would add _setting as synonym.

GET

GET {index|*|_all|pref*|blank}/_settings

would remain as is.

We would add _setting as synonym.

DELETE

Does not exist and we will not add it since it is unclear what it should do.

_warmer

PUT

PUT {index|_all|*|prefix*}/{type|_all|*|prefix*|blank}/_warmer/warmer_name {
    ...
}

would become:

PUT {index|_all|*|prefix*|blank}/{type|_all|*|prefix*|blank}/_warmer/warmer_name {
    ...
}

We would add _warmers as synonym.

POST

does not exist yet. We would add

POST {index|_all|*|prefix*|blank}/{type|_all|*|prefix*|blank}/_warmers {
    "warmer1":{...},
    "warmer2":{…},
    ...
}

We would add _warmer as synonym.

GET

GET {index|*|_all|pref*}/_warmer/{warmer_name|*|pref*}

would become

GET {index|*|_all|pref*|blank}/{type|_all|*|prefix*|blank}/_warmers/{warmer_name|*|_all|pref*|blank}

We would add _warmer as synonym.

DELETE

DELETE {index|_all|*|prefix*}/_warmer/{warmer|*|prefix*}

would become

DELETE {index|_all|*|prefix*}/{type|_all|*|prefix*}/_warmers/{warmer|*|_all|prefix*}

We would add _warmer as synonym.

_alias

PUT

PUT index/_alias/alias

would become

PUT {index|_all|*|prefix*|blank}/_alias/alias

We would add _aliases as synonym.

POST

Remains as is. The alias API is a special case because post triggers different actions.

GET

GET {index|_all|*|prefix*|blank}/_alias/{alias|*|prefix*}

and

GET {index|_all|*|prefix*|blank}/_aliases

would become

GET {index|_all|*|prefix*|blank}/_aliases/{alias|*|prefix*|_all|blank}

We would add _alias as synonym.

DELETE

DELETE index/_alias/alias

would become

DELETE {index|_all|*|prefix*|blank}/_alias/{alias|*|prefix*|_all}

We would add _aliases as synonym.

Note that DELETE index/_alias/* currently returns ok but does not actually remove the aliases.

Things to keep in mind

Keep an eye on #4549

[clintongormley]

Here's a proposed roadmap of what should be done for RC1, and what can follow later:

PUT (singular)

Request                                 Status
-----------------------------------------------------------------------------------
PUT /{index}/_mapping/{type}            Add URL
{ single defn }

PUT /{index}/_warmer/{name}             Already supported
{ single defn }

PUT /{index}/_alias/{name}              Already supported
{ single defn }

PUT /{index}/_setting/{name}            Will not support
{ single defn }

Also support:

 PUT /{index}/{type}/_mapping           Already supported
 PUT & POST                             Add URLs

Where:

• {index} = blank | * | _all | prefix* | name,name
• {name} = name

POST (plural)

Request                                 Status
-----------------------------------------------------------------------------------
POST /{index}/_mappings                 Post 1.0
{ multi defn }

POST /{index}/_warmers                  Post 1.0
{ multi defn }

POST /_aliases                          Already supported
{ actions }

POST /{index}/_settings                 Already supported
{ multi defn }

Also support:

 PUT & POST                             Add URLs

Where:

• {index} = blank | * | _all | prefix* | name,name (Or should we just allow name,name? prefix*?)

Should we also support this?

POST /{index}/_aliases
{ alias: { defn},... }

Probably not as we can't delete aliases at the same time.

GET (singular or plural)

Request                                 Status
-----------------------------------------------------------------------------------
GET /{index}/_mappings/{type}           Add URL - make response consistent
GET /{index}/_mapping/{type}            Add URL - make response consistent

GET /{index}/_warmers/{name}            Add URL
GET /{index}/_warmer/{name}             Already supported

GET /{index}/_aliases/{name}            Already supported
GET /{index}/_alias/{name}              Already supported

GET /{index}/_settings/{name}           Already supported where {name}=blank
                                        Add {name} support post 1?
GET /{index}/_setting/{name}            Add URL? Or should not support?

Also support:

GET /{index}/{type}/_mapping            Already supported - make response consistent

Where:

• {index} = blank | * | _all | prefix* | name,name
• {name} = blank | * | _all | prefix* | name,name

Missing types/warmers/aliases/settings should return empty objects, not throw 404s

DELETE (singular or plural)

Request                                 Status
-----------------------------------------------------------------------------------
DELETE /{index}/_mappings/{type}        Add URL
DELETE /{index}/_mapping/{type}         Add URL

DELETE /{index}/_warmers/{name}         Add URL
DELETE /{index}/_warmer/{name}          Already supported

DELETE /{index}/_aliases/{name}         Already supported
DELETE /{index}/_alias/{name}           Already supported

DELETE /{index}/_settings/{name}        Will not be supported
DELETE /{index}/_setting/{name}         Will not be supported

Also support:

DELETE /{index}/{type}/_mapping         Already supported

Where:

• {index} = * | _all | prefix* | name,name BLANK NOT ALLOWED
• {name} = * | _all | prefix* | name,name BLANK NOT ALLOWED

If no matching types/warmers/aliases/settings found, throw 404.

[clintongormley]

The output from GET requests should have the following format:

{
    "index_name": {
        "element_name":  ...element value ...,
        ...
    },
    ....
}

In other words:

• the first level should always be the index name or names
• the second level is the name of each element, eg the name of the type for _mappings, the name of the warmer for _warmersetc
• if no matching element is found, then we return an empty object { }, we don't throw a 404
• the response object is only populated with the elements we find.

For example:

GET /index_1,index_2,alias_3/_mappings/foo

If index_1 and alias_3 (which points to index_3) both contain type foo, but index_2 doesn't, then the response would be:

{
    "index_1": {
        "foo": { .... }
    },
    "index_3": {
        "foo": { .... }
    }
}

[clintongormley]

RC1 work completed. Remaining work moved to #4743

[lfrancke]

Two questions about this.

The first is about a comment @brwe left:

For PUT _mapping remove "type" from body, because it has to be defined in url anyway and is therefore redundant.

Was this implemented in the end? When I'm posting a mapping containing a type to a URL with a non-matching I see a message acknowledging it but I don't think anything actually happened.

Second question is about deprecation. The PUT Mapping API doc mentions this:

Instead of _mapping you can also use the plural _mappings. The uri PUT /{index}/{type}/_mapping is still supported for backwards compatibility.

Are the singular versions supposed to be removed at some point?