Skip to content

Add enrich policy list API #41553

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 6 commits into from
May 1, 2019
Merged

Conversation

hub-cap
Copy link
Contributor

@hub-cap hub-cap commented Apr 25, 2019

This commit wires up the Rest calls and Transport calls for listing all
enrich policies, as well as tests and rest spec additions.

This commit wires up the Rest calls and Transport calls for listing all
enrich policies, as well  as tests and rest spec additions.
@hub-cap hub-cap added >enhancement :Data Management/Ingest Node Execution or management of Ingest Pipelines including GeoIP labels Apr 25, 2019
@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-core-features

@hub-cap
Copy link
Contributor Author

hub-cap commented Apr 25, 2019

@elasticmachine run elasticsearch-ci/1

@hub-cap hub-cap requested a review from jakelandis April 25, 2019 22:34
@hub-cap
Copy link
Contributor Author

hub-cap commented Apr 25, 2019

@elasticmachine run elasticsearch-ci/bwc

Copy link
Member

@martijnvg martijnvg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I left a few comments.

public static final ListEnrichPolicyAction INSTANCE = new ListEnrichPolicyAction();
public static final String NAME = "cluster:admin/xpack/enrich/list";

protected ListEnrichPolicyAction() {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this constructor can be private?

@Override
public void writeTo(StreamOutput out) throws IOException {
out.writeMap(policyMap, StreamOutput::writeString, (o, value) -> value.writeTo(out));

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

remove new line?

* @param state the cluster state
* @return a Map of <code>policyName, EnrichPolicy</code> of the policies
*/
public static Map<String, EnrichPolicy> getPolicies(ClusterState state) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe also add a unit test for this? (now that it is public)

{
builder.startArray("policies");
{
for (Map.Entry<String, EnrichPolicy> entry : policyMap.entrySet()) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We iterate here directly from a HashMap. The ordering of the entry set is not deterministic, so different api invocations can result in policies being returned in a different order.

In order to fix that we should make a copy here using a TreeMap:

TreeMap<String, EnrichPolicy> policyMap = new TreeMap<>(this.policyMap);
for (Map.Entry<String, EnrichPolicy> entry : policyMap.entrySet()) {

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I moved this into the constructor since the input is a map and the internal state of the object is a list.

);
}
);

static {
PARSER.declareString(ConstructingObjectParser.constructorArg(), NAME);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this should be an optional parameter. The put transport action should set the name as it can be derived from the put policy request class. We only want to make setting the name in the url path required.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed, now in NamedPolicy

@jakelandis
Copy link
Contributor

LGTM (pending Martijn's comments)

@hub-cap
Copy link
Contributor Author

hub-cap commented Apr 29, 2019

ok, readdressed all comments and worked out the EnrichPolicy.NamedPolicy, which is a representation used currently for the list operations. This allows us to inject an additional field, name, into the enrich policy's output and validate it properly in tests. This might be used in other places that require a name and a policy, such as PUT, and GET.

Copy link
Member

@martijnvg martijnvg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I left one comment about the url path, LGTM otherwise.


public RestListEnrichPolicyAction(final Settings settings, final RestController controller) {
super(settings);
controller.registerHandler(RestRequest.Method.GET, "/_enrich/policy", this);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

plural? /_enrich/policies?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if w go with this then i suggest we change all of the URIs, which i like, but I assumed we did not want to go against for now. This is how I view it. You are enacting on a single or multiple policies at any time. So, /policies gives you the list, and /policies/foo gives your single foo policy back, but the resource is the same, you are just saying "give me a singlular one of these policies". Otherwise I think we should keep it the same as policy across the board. I prefer the former, but its possible we dont want to go that route until more actual API and versioning conversations happen.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok :) Lets leave this for now and decide whether we want this in a later stage of development.

Btw I just check get mappings api (returns a hash and not a list) and there both singular and plural is supported. But it looks like most apis don't mix plural with singular nouns when it comes returning a single item or multiple (get pipeline, get repository, get tasks etc). Anyhow, language wise (i'm not a native speaker) plural made sense to me.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMHO plural makes sesne for all cases. It reads easier to me, a list of policies GET /_enrich/policies, a single policy from the list of policies GET /_enrich/policies/foo

@hub-cap hub-cap merged commit c999c09 into elastic:enrich May 1, 2019
hub-cap added a commit that referenced this pull request May 2, 2019
This commit wires up the Rest calls and Transport calls for listing all
enrich policies, as well  as tests and rest spec additions.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
:Data Management/Ingest Node Execution or management of Ingest Pipelines including GeoIP >enhancement
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants