Skip to content

Add enrich policy PUT API #41383

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 12 commits into from
Apr 25, 2019
Merged
Show file tree
Hide file tree
Changes from 11 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,11 @@ private XPackSettings() {
}


/**
* Setting for controlling whether or not enrich is enabled.
*/
public static final Setting<Boolean> ENRICH_ENABLED_SETTING = Setting.boolSetting("xpack.enrich.enabled", true, Property.NodeScope);

/**
* Setting for controlling whether or not CCR is enabled.
*/
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,88 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License;
* you may not use this file except in compliance with the Elastic License.
*/
package org.elasticsearch.xpack.core.enrich.action;

import org.elasticsearch.action.Action;
import org.elasticsearch.action.ActionRequestValidationException;
import org.elasticsearch.action.support.master.AcknowledgedResponse;
import org.elasticsearch.action.support.master.MasterNodeRequest;
import org.elasticsearch.common.io.stream.StreamInput;
import org.elasticsearch.common.io.stream.StreamOutput;
import org.elasticsearch.common.xcontent.XContentParser;
import org.elasticsearch.xpack.core.enrich.EnrichPolicy;

import java.io.IOException;
import java.util.Objects;

public class PutEnrichPolicyAction extends Action<AcknowledgedResponse> {

public static final PutEnrichPolicyAction INSTANCE = new PutEnrichPolicyAction();
public static final String NAME = "cluster:admin/xpack/enrich/put";

protected PutEnrichPolicyAction() {
super(NAME);
}

public static Request fromXContent(XContentParser parser, String name) throws IOException {
return new Request(name, EnrichPolicy.fromXContent(parser));
}

@Override
public AcknowledgedResponse newResponse() {
return new AcknowledgedResponse();
}

public static class Request extends MasterNodeRequest<PutEnrichPolicyAction.Request> {

private final EnrichPolicy policy;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add newline before the first field

private final String name;

public Request(String name, EnrichPolicy policy) {
this.name = Objects.requireNonNull(name, "name cannot be null");
this.policy = policy;
}

public Request(StreamInput in) throws IOException {
super(in);
name = in.readString();
policy = new EnrichPolicy(in);
}

@Override
public void writeTo(StreamOutput out) throws IOException {
super.writeTo(out);
out.writeString(name);
policy.writeTo(out);
}

public String getName() {
return name;
}

public EnrichPolicy getPolicy() {
return policy;
}

@Override
public ActionRequestValidationException validate() {
return null;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add validation here that the name is required?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

im not sure we need this since the constructor for the request has an Objects.requireNotNull. I could add something to the streaminput constructor as well, but that does not look like its typically done, as we should be able to trust that more than a "user input" constructor, so to speak.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point. Maybe instead of using Objects.requireNotNull, use an if statement that checks now null and if so then throw a IllegalArgumentException? Objects.requireNotNull throws a NPE and that will result in a 500 response code, whilst a IllegalArgumentException translates to a 400 response code and that is a better response code in this case.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Never mind. This good as it is now.

In production, the constructor can't be invoked with a null name, because if the rest handler is invoked then the name had to provided.

}

@Override
public boolean equals(Object o) {
if (this == o) return true;
if (o == null || getClass() != o.getClass()) return false;
Request request = (Request) o;
return policy.equals(request.policy) &&
name.equals(request.name);
}

@Override
public int hashCode() {
return Objects.hash(policy, name);
}
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -5,25 +5,75 @@
*/
package org.elasticsearch.xpack.enrich;

import org.elasticsearch.action.ActionRequest;
import org.elasticsearch.action.ActionResponse;
import org.elasticsearch.cluster.metadata.IndexNameExpressionResolver;
import org.elasticsearch.cluster.metadata.MetaData;
import org.elasticsearch.cluster.node.DiscoveryNodes;
import org.elasticsearch.common.ParseField;
import org.elasticsearch.common.io.stream.NamedWriteableRegistry;
import org.elasticsearch.common.settings.ClusterSettings;
import org.elasticsearch.common.settings.IndexScopedSettings;
import org.elasticsearch.common.settings.Settings;
import org.elasticsearch.common.settings.SettingsFilter;
import org.elasticsearch.common.xcontent.NamedXContentRegistry;
import org.elasticsearch.ingest.Processor;
import org.elasticsearch.plugins.ActionPlugin;
import org.elasticsearch.plugins.IngestPlugin;
import org.elasticsearch.plugins.Plugin;
import org.elasticsearch.rest.RestController;
import org.elasticsearch.rest.RestHandler;
import org.elasticsearch.xpack.core.enrich.action.PutEnrichPolicyAction;
import org.elasticsearch.xpack.enrich.action.TransportPutEnrichPolicyAction;
import org.elasticsearch.xpack.enrich.rest.RestPutEnrichPolicyAction;

import java.util.Arrays;
import java.util.Collections;
import java.util.List;
import java.util.Map;
import java.util.function.Supplier;

public class EnrichPlugin extends Plugin implements IngestPlugin {
import static java.util.Collections.emptyList;
import static org.elasticsearch.xpack.core.XPackSettings.ENRICH_ENABLED_SETTING;

public class EnrichPlugin extends Plugin implements ActionPlugin, IngestPlugin {

private final Settings settings;
private final Boolean enabled;

public EnrichPlugin(final Settings settings) {
this.settings = settings;
this.enabled = ENRICH_ENABLED_SETTING.get(settings);
}

@Override
public Map<String, Processor.Factory> getProcessors(Processor.Parameters parameters) {
return Collections.emptyMap();
}

public List<ActionHandler<? extends ActionRequest, ? extends ActionResponse>> getActions() {
if (enabled == false) {
return emptyList();
}

return Arrays.asList(
new ActionHandler<>(PutEnrichPolicyAction.INSTANCE, TransportPutEnrichPolicyAction.class)
);
}

public List<RestHandler> getRestHandlers(Settings settings, RestController restController, ClusterSettings clusterSettings,
IndexScopedSettings indexScopedSettings, SettingsFilter settingsFilter,
IndexNameExpressionResolver indexNameExpressionResolver,
Supplier<DiscoveryNodes> nodesInCluster) {
if (enabled == false) {
return emptyList();
}

return Arrays.asList(
new RestPutEnrichPolicyAction(settings, restController)
);
}

@Override
public List<NamedWriteableRegistry.Entry> getNamedWriteables() {
return Collections.singletonList(new NamedWriteableRegistry.Entry(MetaData.Custom.class, EnrichMetadata.TYPE,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@
/**
* Helper methods for access and storage of an enrich policy.
*/
public final class EnrichStore {
public class EnrichStore {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we can keep this class final now?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oh, yes, now that it holds no state, good catch!


private EnrichStore() {}

Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License;
* you may not use this file except in compliance with the Elastic License.
*/
package org.elasticsearch.xpack.enrich.action;

import org.elasticsearch.action.ActionListener;
import org.elasticsearch.action.support.ActionFilters;
import org.elasticsearch.action.support.master.AcknowledgedResponse;
import org.elasticsearch.action.support.master.TransportMasterNodeAction;
import org.elasticsearch.cluster.ClusterState;
import org.elasticsearch.cluster.block.ClusterBlockException;
import org.elasticsearch.cluster.block.ClusterBlockLevel;
import org.elasticsearch.cluster.metadata.IndexNameExpressionResolver;
import org.elasticsearch.cluster.service.ClusterService;
import org.elasticsearch.common.inject.Inject;
import org.elasticsearch.threadpool.ThreadPool;
import org.elasticsearch.transport.TransportService;
import org.elasticsearch.xpack.core.enrich.action.PutEnrichPolicyAction;
import org.elasticsearch.xpack.enrich.EnrichStore;

public class TransportPutEnrichPolicyAction extends TransportMasterNodeAction<PutEnrichPolicyAction.Request, AcknowledgedResponse> {

@Inject
public TransportPutEnrichPolicyAction(TransportService transportService,
ClusterService clusterService,
ThreadPool threadPool,
ActionFilters actionFilters,
IndexNameExpressionResolver indexNameExpressionResolver) {
super(PutEnrichPolicyAction.NAME, transportService, clusterService, threadPool, actionFilters,
PutEnrichPolicyAction.Request::new, indexNameExpressionResolver);
}

@Override
protected String executor() {
return ThreadPool.Names.SAME;
}

@Override
protected AcknowledgedResponse newResponse() {
throw new UnsupportedOperationException("usage of Streamable is to be replaced by Writeable");
}

@Override
protected void masterOperation(PutEnrichPolicyAction.Request request, ClusterState state,
ActionListener<AcknowledgedResponse> listener) {
EnrichStore.putPolicy(request.getName(), request.getPolicy(), clusterService, e -> {
if (e == null) {
listener.onResponse(new AcknowledgedResponse(true));
} else {
listener.onFailure(e);
}
});
}

@Override
protected ClusterBlockException checkBlock(PutEnrichPolicyAction.Request request, ClusterState state) {
return state.blocks().globalBlockedException(ClusterBlockLevel.METADATA_WRITE);
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License;
* you may not use this file except in compliance with the Elastic License.
*/
package org.elasticsearch.xpack.enrich.rest;

import org.elasticsearch.client.node.NodeClient;
import org.elasticsearch.common.settings.Settings;
import org.elasticsearch.common.xcontent.XContentParser;
import org.elasticsearch.rest.BaseRestHandler;
import org.elasticsearch.rest.RestController;
import org.elasticsearch.rest.RestRequest;
import org.elasticsearch.rest.action.RestToXContentListener;
import org.elasticsearch.xpack.core.enrich.action.PutEnrichPolicyAction;

import java.io.IOException;

public class RestPutEnrichPolicyAction extends BaseRestHandler {

public RestPutEnrichPolicyAction(final Settings settings, final RestController controller) {
super(settings);
controller.registerHandler(RestRequest.Method.PUT, "/_enrich/policy/{name}", this);
}

@Override
public String getName() {
return "put_enrich_policy";
}

@Override
protected RestChannelConsumer prepareRequest(final RestRequest restRequest, final NodeClient client) throws IOException {
final PutEnrichPolicyAction.Request request = createRequest(restRequest);
return channel -> client.execute(PutEnrichPolicyAction.INSTANCE, request, new RestToXContentListener<>(channel));
}

static PutEnrichPolicyAction.Request createRequest(RestRequest restRequest) throws IOException {
try (XContentParser parser = restRequest.contentOrSourceParamParser()) {
return PutEnrichPolicyAction.fromXContent(parser, restRequest.param("name"));
}
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,7 @@ protected EnrichPolicy createXContextTestInstance(XContentType xContentType) {
return randomEnrichPolicy(xContentType);
}

static EnrichPolicy randomEnrichPolicy(XContentType xContentType) {
public static EnrichPolicy randomEnrichPolicy(XContentType xContentType) {
final QueryBuilder queryBuilder;
if (randomBoolean()) {
queryBuilder = new MatchAllQueryBuilder();
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License;
* you may not use this file except in compliance with the Elastic License.
*/
package org.elasticsearch.xpack.enrich.action;

import org.elasticsearch.common.io.stream.Writeable;
import org.elasticsearch.common.xcontent.XContentType;
import org.elasticsearch.test.AbstractWireSerializingTestCase;
import org.elasticsearch.xpack.core.enrich.EnrichPolicy;
import org.elasticsearch.xpack.core.enrich.action.PutEnrichPolicyAction;

import static org.elasticsearch.xpack.enrich.EnrichPolicyTests.randomEnrichPolicy;

public class PutEnrichPolicyActionRequestTests extends AbstractWireSerializingTestCase<PutEnrichPolicyAction.Request> {

@Override
protected PutEnrichPolicyAction.Request createTestInstance() {
final EnrichPolicy policy = randomEnrichPolicy(XContentType.JSON);
return new PutEnrichPolicyAction.Request(randomAlphaOfLength(3), policy);
}

@Override
protected Writeable.Reader<PutEnrichPolicyAction.Request> instanceReader() {
return PutEnrichPolicyAction.Request::new;
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
{
"enrich.put_policy": {
"documentation": "https://www.elastic.co/guide/en/elasticsearch/reference/current/enrich-put-policy.html",
"methods": [ "PUT" ],
"url": {
"path": "/_enrich/policy/{name}",
"paths": ["/_enrich/policy/{name}"],
"parts": {
"name": {
"type" : "string",
"description" : "The name of the enrich policy"
}
},
"params": {
}
},
"body": {
"description": "The enrich policy to register"
}
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
---
"Test enrich crud apis":

- do:
enrich.put_policy:
name: policy-crud
body:
type: exact_match
index_pattern: "bar*"
enrich_key: baz
enrich_values: ["a", "b"]
schedule: "*/120"

- is_true: acknowledged
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

iirc all yaml test files are required to end with a newline?