Improve REST error handling when endpoint does not support HTTP verb, add OPTIONS support #24437

dakrone · 2017-05-02T18:21:06Z

This is a follow-up of #17916 from @jbertouch's work brought up to date with the latest master, here's a subset of original description:

First pass at closing #15335. The new behavior for the RestController#executeHandler method is as follows:

For a request to a valid endpoint with an unsupported HTTP method -> Return a 405 HTTP error including allowed methods for the endpoint in the Allow HTTP header. Refer to HTTP/1.1 - 10.4.6 - 405 Method Not Allowed.
For an OPTIONS HTTP method request to a valid endpoint -> Return a 200 HTTP response including allowed methods for the endpoint in the Allow HTTP header. Refer to HTTP/1.1 - 9.2 - Options.

It allows things like the following:

~ λ curl -v -XPOST 'localhost:9200/my_index/_settings'
*   Trying 127.0.0.1...
* TCP_NODELAY set
* Connected to localhost (127.0.0.1) port 9200 (#0)
> POST /my_index/_settings HTTP/1.1
> Host: localhost:9200
> User-Agent: curl/7.51.0
> Accept: */*
> 
< HTTP/1.1 405 Method Not Allowed
< Allow: PUT,GET
< content-type: application/json; charset=UTF-8
< content-length: 134
< 
{
  "error" : "Incorrect HTTP method for uri [/my_index/_settings] and method [POST], allowed: [PUT, GET]",
  "status" : 405
}
* Curl_http_done: called premature == 0
* Connection #0 to host localhost left intact

Where you can see the HTTP/1.1 405 Method Not Allowed response and Allow: GET header.

It also adds support for OPTIONS:

~ λ curl -v -XOPTIONS x:9200/_cluster/settings
*   Trying 127.0.0.1...
* TCP_NODELAY set
* Connected to x (127.0.0.1) port 9200 (#0)
> OPTIONS /_cluster/settings HTTP/1.1
> Host: x:9200
> User-Agent: curl/7.51.0
> Accept: */*
> 
< HTTP/1.1 200 OK
< Allow: GET,PUT
< content-type: text/plain; charset=UTF-8
< content-length: 0
< 
* Curl_http_done: called premature == 0
* Connection #0 to host x left intact

Also improved OPTIONS http method handling to better conform with the http spec.

See elastic#15335

Simple addition to surface the RestResponse object so we can run tests against it (see issue elastic#15335).

See elastic#15335.

Randomizing unit test for enhanced REST response error handling. See issue elastic#15335 for more details.

Constructor added to support RestController test cases.

# Conflicts: # core/src/main/java/org/elasticsearch/rest/RestController.java # test/framework/src/main/java/org/elasticsearch/test/rest/FakeRestRequest .java

See elastic#15335

Refactored executeHandler to prioritize explicit path matches. See elastic#15335 for more information.

In this test, an OPTIONS method request is valid, and should not return a 405 error.

Then TrieMatchingMode can be package private and not leak into RestController

dakrone · 2017-06-22T21:49:06Z

@s1monw okay! I've pushed two commits that make some drastic changes to this PR:

The first is a change from PathTrie<RestHandler> to PathTrie<MethodHandlers> so that we can have a single PathTrie for all paths, instead of a separate one for each method. It allows us to retrieve the supported methods for a path in a much easier manner (this was from your suggestion).

The second is a change in PathTrie (and of course RestController) to make it retrieve all of the RestHandlers (now MethodHandlers actually) at once, looping through the candidates and trying each to see if it is valid. This is also from your suggestion. As a byproduct, no outside classes have to know anything about TrieMatchingMode (again, your suggestion) and the class has been made package-private.

…hods

I accidentally removed this :X

s1monw

I took another look and I like this. Yet, I think we need to make the change that you did here first in a separate PR and then do the actual change based on that. Is this possible? I'd love to make is smaller so reviews are more effective.

s1monw · 2017-06-28T10:11:09Z

core/src/main/java/org/elasticsearch/common/path/PathTrie.java

+            this.path = path;
+            this.trie = trie;
+            this.paramSupplier = paramSupplier;
+            modes.add(TrieMatchingMode.EXPLICIT_NODES_ONLY);


just use Arrays.asList when this is initializaed?

The modes need to go in a specific order, this ensures that even if someone changes the order in the ENUM definition the matching will still work correctly

Oh, but I see what you're saying, you're just saying for contructing it, sure I'll change that

s1monw · 2017-06-28T10:11:33Z

core/src/main/java/org/elasticsearch/common/path/PathTrie.java

+
+        @Override
+        public boolean hasNext() {
+            if (modes.isEmpty() == false) {


return modex.isEmpty() == false?

s1monw · 2017-06-28T10:13:21Z

core/src/main/java/org/elasticsearch/rest/RestController.java

+     * @param path Path to handle (e.g., "/{index}/{type}/_bulk")
+     * @param handler The handler to actually execute
+     */
+    public void registerHandler(RestRequest.Method singleMethod, String path, RestHandler handler) {


why did you add this method and why don't you use the registerHandler method above instead?

This is a compatibility layer for all the Rest* actions, if I used only the one above I would have to change every action's registerHandler method call (which I might do, but in a subsequent PR)

s1monw · 2017-06-28T10:15:20Z

core/src/main/java/org/elasticsearch/rest/RestController.java

+        }
+
+        // Loop through all possible handlers, attempting to dispatch the request
+        Iterator<MethodHandlers> allHandlers = getAllHandlers(request);


hmm can't we do for (MethodHandlers handler : ((Iterable<MethodHandlers>) this::getAllHandlers))?

I can't cast the Iterator to Iterable (I tried it and got class cast exceptions), is there a different way to get an Iterable out of an Iterator?

…hods This changes `PathTrie` and `RestController` to use a single `PathTrie` for all endpoints, it also allows retrieving the endpoints' supported HTTP methods more easily. This is a spin-off and prerequisite of elastic#24437

dakrone · 2017-06-28T23:10:10Z

Yet, I think we need to make the change that you did here first in a separate PR and then do the actual change based on that.

I opened #25459 as a spin-off encapsulating the MethodHandlers and PathTrie changes

…hods (#25459) * Refactor PathTrie and RestController to use a single trie for all methods This changes `PathTrie` and `RestController` to use a single `PathTrie` for all endpoints, it also allows retrieving the endpoints' supported HTTP methods more easily. This is a spin-off and prerequisite of #24437 * Use EnumSet instead of multiple if conditions * Make MethodHandlers package-private and final * Remove duplicate registerHandler method * Remove public modifier

…handling

dakrone · 2017-07-06T00:00:04Z

@s1monw #25459 was merged and I've merged master into this branch, bringing it from +915 lines to +369 (much smaller!), please take another look!

s1monw

this looks good to me. Yet, I think we should have a real REST test for this somehow? WDYT @dakrone we can add a test REST handler in a smoketest qa env where we can add a rest spec that we don't implement, something along those lines? This can be a folllowup IMO

s1monw · 2017-07-07T07:44:51Z

core/src/main/java/org/elasticsearch/rest/RestController.java

+            // Get the map of matching handlers for a request, for the full set of HTTP methods.
+            final Set<RestRequest.Method> validMethodSet = getValidHandlerMethodSet(request);
+            if (validMethodSet.size() > 0
+                && !validMethodSet.contains(request.method())


I know its controversial but can we use == false ?

s1monw · 2017-07-07T07:44:58Z

core/src/main/java/org/elasticsearch/rest/RestController.java

+                // Not Allowed error.
+                handleUnsupportedHttpMethod(request, channel, validMethodSet);
+                requestHandled = true;
+            } else if (!validMethodSet.contains(request.method())


same here ^^

dakrone · 2017-07-07T15:01:14Z

Yet, I think we should have a real REST test for this somehow? WDYT @dakrone we can add a test REST handler in a smoketest qa env where we can add a rest spec that we don't implement, something along those lines?

Sure I will open an issue to add this as a follow-up

Thanks for the reviews @s1monw!

* master: Remove deprecated created and found from index, delete and bulk (elastic#25516) fix testEnsureVersionCompatibility for 5.5.0 release fix Version.v6_0_0 min compatibility version to 5.5.0 Add bwc indices for 5.5.0 Add v5_5_1 constant [DOCS] revise high level client Search Scroll API docs (elastic#25599) Improve REST error handling when endpoint does not support HTTP verb, add OPTIONS support (elastic#24437) Avoid SecurityException in repository-S3 on DefaultS3OutputStream.flush() (elastic#25254) [Tests] Add tests for CompletionSuggestionBuilder#build() (elastic#25575)

* master: (42 commits) Harden global checkpoint tracker Remove deprecated created and found from index, delete and bulk (elastic#25516) fix testEnsureVersionCompatibility for 5.5.0 release fix Version.v6_0_0 min compatibility version to 5.5.0 Add bwc indices for 5.5.0 Add v5_5_1 constant [DOCS] revise high level client Search Scroll API docs (elastic#25599) Improve REST error handling when endpoint does not support HTTP verb, add OPTIONS support (elastic#24437) Avoid SecurityException in repository-S3 on DefaultS3OutputStream.flush() (elastic#25254) [Tests] Add tests for CompletionSuggestionBuilder#build() (elastic#25575) Enable cross-setting validation [Docs] Fix typo in bootstrap-checks.asciidoc (elastic#25597) Index ids in binary form. (elastic#25352) bwc checkout should fetch from all remotes IndexingIT should check for global checkpoints regardless of master version [Tests] Add tests for PhraseSuggestionBuilder#build() (elastic#25571) Remove unused class MinimalMap (elastic#25590) [Docs] Document Scroll API for Java High Level REST Client (elastic#25554) Disable date field mapping changing (elastic#25285) Allow BWC Testing against a specific branch (elastic#25510) ...

dakrone · 2017-07-07T22:18:59Z

Yet, I think we should have a real REST test for this somehow? WDYT @dakrone we can add a test REST handler in a smoketest qa env where we can add a rest spec that we don't implement, something along those lines?

Definitely, I opened #25611 for this.

Thanks for the reviews @s1monw!

jbertouch and others added 30 commits April 5, 2016 00:29

Improved REST endpoint exception handling, see elastic#15335

540a299

Also improved OPTIONS http method handling to better conform with the http spec.

Tidied up formatting and comments

0906aa1

See elastic#15335

Tests for elastic#15335

1d3dcdb

Cleaned up comments, added section number

550cc94

Swapped out tab indents for space indents

321af02

Merge remote-tracking branch 'elastic/master' into issue_15335

f2af227

Test class now extends ESSingleNodeTestCase

20a2273

Capture RestResponse so it can be examined in test cases

447a182

Simple addition to surface the RestResponse object so we can run tests against it (see issue elastic#15335).

Refactored class name, included feedback

7d1cc14

See elastic#15335.

Unit test for REST error handling enhancements

f661a2f

Randomizing unit test for enhanced REST response error handling. See issue elastic#15335 for more details.

Cleaned up formatting

fc5bbeb

New constructor to set HTTP method

b38aed8

Constructor added to support RestController test cases.

Refactored FakeRestRequest, streamlined test case.

b08ee7f

Merge remote-tracking branch 'elastic/master' into issue_15335

e85f537

# Conflicts: # core/src/main/java/org/elasticsearch/rest/RestController.java # test/framework/src/main/java/org/elasticsearch/test/rest/FakeRestRequest .java

Cleaned up conflicts

208ba18

Tests for elastic#15335

a753560

Added functionality to ignore or include path wildcards

b0cc920

See elastic#15335

Further enhancements to request handling

cc17f3a

Refactored executeHandler to prioritize explicit path matches. See elastic#15335 for more information.

Cosmetic fixes

622093c

Refactored method handlers

daec2c4

Merge remote-tracking branch 'elastic/master' into issue_15335

6901745

Merge remote-tracking branch 'elastic/master' into issue_15335

8252921

Removed redundant import

e6d2777

Updated integration tests

4d67a91

Refactoring to address issue elastic#17853

474d001

Cleaned up test assertions

55680ee

Fixed edge case if OPTIONS method randomly selected as invalid method

c10967d

In this test, an OPTIONS method request is valid, and should not return a 405 error.

Merge remote-tracking branch 'origin/master' into pr/17916

99a80a9

Remove redundant static modifier

d56b72b

Hook the multiple PathTrie attempts into RestHandler.dispatchRequest

42aeeb3

Add PathTrie.retrieveAll where all matching modes can be retrieved

24d818a

Then TrieMatchingMode can be package private and not leak into RestController

dakrone requested a review from s1monw June 22, 2017 21:49

dakrone added 2 commits June 22, 2017 16:22

Include body of error with 405 responses to give hint about valid met…

c168066

…hods

Fix missing usageService handler addition

222bdde

I accidentally removed this :X

s1monw suggested changes Jun 28, 2017

View reviewed changes

Initialize PathTrieIterator modes with Arrays.asList

6cc244e

dakrone mentioned this pull request Jun 28, 2017

Refactor PathTrie and RestController to use a single trie for all methods #25459

Merged

Merge remote-tracking branch 'origin/master' into improve-rest-error-…

da41feb

…handling

dakrone requested a review from s1monw July 6, 2017 00:00

s1monw approved these changes Jul 7, 2017

View reviewed changes

dakrone added 2 commits July 7, 2017 08:05

Use "== false" instead of !

9524f14

Missing paren :-/

e87321f

dakrone merged commit 8aa0a5c into elastic:master Jul 7, 2017

dakrone mentioned this pull request Jul 7, 2017

Migrate RestHttpResponseHeadersIT from HttpSmokeTestCase to ESRestTestCase #25611

Closed

colings86 added v6.0.0-beta1 and removed v6.0.0 labels Jul 31, 2017

dakrone deleted the improve-rest-error-handling branch December 13, 2017 20:36

javanna mentioned this pull request Mar 2, 2018

POST index/_settings with body produces wrong exception #17853

Closed

olcbean mentioned this pull request Mar 4, 2018

Unsupported methods on REST endpoints should respond with status code 405 #15335

Closed

olcbean mentioned this pull request Mar 12, 2018

RestUpgradeAction : remove explicit http method check #28992

Closed

henningandersen mentioned this pull request Sep 9, 2019

BUG: use a single trie for all methods #46482

Closed

Improve REST error handling when endpoint does not support HTTP verb, add OPTIONS support #24437

Improve REST error handling when endpoint does not support HTTP verb, add OPTIONS support #24437

Uh oh!

Conversation

dakrone commented May 2, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

dakrone commented Jun 22, 2017

Uh oh!

s1monw left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dakrone commented Jun 28, 2017

Uh oh!

dakrone commented Jul 6, 2017

Uh oh!

s1monw left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dakrone commented Jul 7, 2017

Uh oh!

dakrone commented Jul 7, 2017

Uh oh!

Uh oh!

dakrone commented May 2, 2017 •

edited

Loading