Refactor PathTrie and RestController to use a single trie for all methods #25459

dakrone · 2017-06-28T23:09:03Z

This changes PathTrie and RestController to use a single PathTrie for all
endpoints, it also allows retrieving the endpoints' supported HTTP methods more
easily.

This is a spin-off and prerequisite of #24437

…hods This changes `PathTrie` and `RestController` to use a single `PathTrie` for all endpoints, it also allows retrieving the endpoints' supported HTTP methods more easily. This is a spin-off and prerequisite of elastic#24437

rjernst · 2017-06-29T00:28:50Z

core/src/main/java/org/elasticsearch/rest/MethodHandlers.java

+/**
+ * Encapsulate multiple handlers for the same path, allowing different handlers for different HTTP verbs.
+ */
+public class MethodHandlers {


Could we instead have RestHandler have optional overridable methods for each verb? That would seem less error prone (and greatly reduce the number of classes we define from what we currently have).

I think, @dakrone correct me if I am wrong, that in #24437 we need to know if a rest handler can handle a certain verb to ensure we send the right error back. That might be the reason why we need to be explicit about it. But maybe there is a way to do this?

I think, @dakrone correct me if I am wrong, that in #24437 we need to know if a rest handler can handle a certain verb to ensure we send the right error back. That might be the reason why we need to be explicit about it. But maybe there is a way to do this?

That's correct, that's the reason for this, the only way I can think of to do it on the handler itself is collapsing every handler for the same rest endpoint into the same class, and then duplicating methods to determine which verbs are supported. I don't think it would be cleaner at all

s1monw

left some suggestions and comments

s1monw · 2017-06-29T07:42:40Z

core/src/main/java/org/elasticsearch/common/path/PathTrie.java

            if (node == null) {
-                node = children.get(wildcard);
-                if (node == null) {
+                if (trieMatchingMode == TrieMatchingMode.WILDCARD_NODES_ALLOWED) {


maybe you wanna use a switch here?

I tried to change this to a switch, but it's not strictly a switch due to the additional if conditions in the branches below and the else condition, I think it's much cleaner as an if tree

s1monw · 2017-06-29T07:43:10Z

core/src/main/java/org/elasticsearch/common/path/PathTrie.java

-                // is a child wildcard node, use the child wildcard node
-                if (index + 1 == path.length && node.value == null && children.get(wildcard) != null) {
+                if (index + 1 == path.length && node.value == null && children.get(wildcard) != null
+                        && trieMatchingMode != TrieMatchingMode.WILDCARD_ROOT_NODES_ALLOWED


I like it when we have an explicitly named EnumSet for conditions like this. It's easier to read and understand the code?

s1monw · 2017-06-29T07:47:55Z

core/src/main/java/org/elasticsearch/common/path/PathTrie.java


 import static java.util.Collections.emptyMap;
 import static java.util.Collections.unmodifiableMap;

 public class PathTrie<T> {

+    enum TrieMatchingMode {


I wonder, if we would always retrieve all matching one (recall optimized) and then sort once we have them by the conditions we have here would that be simpler code wise? Also down the road, not in this PR should we use an FST <String, int> that we could score by some predifined value to get the sorted order by default and safe a boat load of objects? the output of the fst would then just point into an array to get the handler...

I wonder, if we would always retrieve all matching one (recall optimized) and
then sort once we have them by the conditions we have here would that be
simpler code wise?

We currently can't easily do this due to not knowing which handler matched for
the params. For example, let's say the second TrieMatchingMode matched, but
from the perspective of the caller (RestController), it needs both the
MethodHandlers object and the parameters that were parsed out of the URI for
the RestRequest. It can't use the parameters from the first handler because
they might be a different map that was parsed out.

This is why I used an Iterator with a param supplier, so that they could be
"reset" between invocations, we essentially try to match the request to a
handler multiple times, mutating the RestRequest each time and resetting it if
we didn't find an appropriate handler.

I do think it would be worth investigating this, however, I think it should be
combined with making PathTrie use a builder pattern and FST since otherwise
we'll change this whole class around only to have to basically rewrite it for
using an FST.

If it's okay with you, I'd like to tackle this in subsequent work that is more
focused, rather than blocking the improve REST PR for internal cleanup. Is that
alright?

sure thing, I didn't imply to change this now

s1monw · 2017-06-29T07:49:39Z

core/src/main/java/org/elasticsearch/common/path/PathTrie.java

+     * </pre>
+     * allowing the value to be updated if desired.
+     */
+    public void insertOrUpdate(String path, T value, BiFunction<T, T, T> updater) {


would this be a perfect example to use a bulider pattern? I mean that would make it immutable which is what we want at some point? we can keep the nodes mutability for simplicity? WDYT?

I think that would be worth trying, but it would be a large undertaking. Since this is blocking a breaking change for 6.0, would you be okay with me looking at that in a follow-up?

dakrone · 2017-07-05T16:53:05Z

@s1monw I pushed a commit addressing one of your comments and left responses for the others, thanks for taking a look!

s1monw

I like it! I left some minors

s1monw · 2017-07-05T18:53:28Z

core/src/main/java/org/elasticsearch/rest/RestController.java

+     * @param handler The handler to actually execute
+     */
+    public void registerHandler(RestRequest.Method singleMethod, String path, RestHandler handler) {
+        RestRequest.Method[] methods = new RestRequest.Method[1];


why do you construct this array rather thank just passing it on to the var args? and why do we have this extra method wouldn't it be ok to just have the one with the var args?

Whoops yeah I'll remove the extra one

s1monw · 2017-07-05T18:55:09Z

core/src/main/java/org/elasticsearch/rest/RestController.java

+        // Between retrieving the correct path, we need to reset the parameters,
+        // otherwise parameters are parsed out of the URI that aren't actually handled.
+        final Map<String, String> originalParams = new HashMap<>(request.params());
+        return handlers.retrieveAll(getPath(request), () -> {


s1monw · 2017-07-05T18:56:36Z

core/src/main/java/org/elasticsearch/rest/MethodHandlers.java

+/**
+ * Encapsulate multiple handlers for the same path, allowing different handlers for different HTTP verbs.
+ */
+public class MethodHandlers {


I havent' looked closely but can this class be package private and final?

Definitely can!

dakrone · 2017-07-05T23:33:08Z

Thanks for the review @s1monw! I opened #25563 as a followup reminder with the further PathTrie changes

* master: Refactor PathTrie and RestController to use a single trie for all methods (elastic#25459) Switch indices read-only if a node runs out of disk space (elastic#25541)

dakrone added :Core/Infra/REST API REST infrastructure and utilities v6.0.0 labels Jun 28, 2017

dakrone requested a review from s1monw June 28, 2017 23:09

dakrone mentioned this pull request Jun 28, 2017

Improve REST error handling when endpoint does not support HTTP verb, add OPTIONS support #24437

Merged

rjernst reviewed Jun 29, 2017

View reviewed changes

s1monw suggested changes Jun 29, 2017

View reviewed changes

Use EnumSet instead of multiple if conditions

8837b87

dakrone requested a review from s1monw July 5, 2017 16:53

s1monw approved these changes Jul 5, 2017

View reviewed changes

dakrone added 3 commits July 5, 2017 14:51

Make MethodHandlers package-private and final

0fc148e

Remove duplicate registerHandler method

9b58c4f

Remove public modifier

3b08571

dakrone merged commit 30b5ca7 into elastic:master Jul 5, 2017

dakrone mentioned this pull request Jul 5, 2017

Cleanup PathTrie #25563

Closed

clintongormley added the >enhancement label Jul 10, 2017

colings86 added v6.0.0-beta1 and removed v6.0.0 labels Jul 31, 2017

henningandersen mentioned this pull request Sep 9, 2019

BUG: use a single trie for all methods #46482

Closed

Refactor PathTrie and RestController to use a single trie for all methods #25459

Refactor PathTrie and RestController to use a single trie for all methods #25459

Uh oh!

Conversation

dakrone commented Jun 28, 2017

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

s1monw left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dakrone commented Jul 5, 2017

Uh oh!

s1monw left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dakrone commented Jul 5, 2017

Uh oh!

Uh oh!