Skip to content

Enable engine factory to be pluggable #26827

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 7 commits into from
Oct 12, 2017

Conversation

jasontedor
Copy link
Member

This commit enables the engine factory to be pluggable based on index settings used when creating the index service for an index.

This commit enables the engine factory to be pluggable based on index
settings used when creating the index service for an index.
@jasontedor jasontedor force-pushed the engine-factory-provider branch from 497a8ad to ce7e52e Compare September 29, 2017 04:32
Copy link
Contributor

@s1monw s1monw left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

left a question / suggestion

*
* @return the engine factory provider
*/
EngineFactoryProvider getEngineFactoryProvider();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do we really need the indirection here and can't we simply have a single factory method:

public EngineFactory getEngineFactory(IndexSettings settings);

I don't understand why we have to have another indirection?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@s1monw I pushed 6e2f3ec.

This commit removes a level of indirection by removing the engine
factory provider abstraction.
* xdcr:
  Maybe die before trying to log cause
  Log cause when a write and flush fails
  Die if write listener fails due to fatal error
  RecoveryIT.testHistoryUUIDIsGenerated should reduce unassigned shards delay instead of ensure green.
  Replace group map settings with affix setting (elastic#26819)
  Fix references to vm.max_map_count in Docker docs
  Add more instructions about jar hell (elastic#26837)
  Forbid negative values for index.unassigned.node_left.delayed_timeout (elastic#26828)
  Nitpicking typos in comments (elastic#26831)
  MetaData Builder doesn't properly prevent an alias with the same name as an index (elastic#26804)
@jasontedor jasontedor force-pushed the engine-factory-provider branch from 0d2bbf9 to cdf80a8 Compare October 1, 2017 14:17
Since EnginePlugin is an interface, a final modifier on a parameter in a
method declared there is redundant.
*
* @return an optional engine factory
*/
Optional<EngineFactory> getMaybeEngineFactory(IndexSettings indexSettings);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why the use of Optional? In other places we have just used null to indicate there is no engine implementation being added. Also, can I don't think we should use the Maybe style of naming. Again, in the pull based plugin APIs we have not done this anywhere else, and I don't think it really adds anything.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree on renaming getMaybeEngineFactory() to getEngineFactory() the fact that it returns an optional indicates that nothing maybe returned. Personally I'm both fine with returning an Optional or just null.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1 to null and rename to getEngineFactory

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I prefer the use of optional, but I'm fine with renaming.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should come to an agreement on whether we should use Optional for these, and be consistent. But arbitrarily adding inconsistencies into the API will just create confusion.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree on the consistency argument. We discussed Optional versus null in a separate channel that we will merge this as-is for now (with Optional), consider changing the handful of other places in the plugin API where we allow null to be returned but will have a broader discussion about this first. We can return to this change if needed.

return "[" + t.v1().getClass().getName() + "/" + t.v2().get().getClass().getName() + "]";
})
.collect(Collectors.joining(",")));
throw new IllegalStateException(message);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just an idea and not sure how feasible it is, but besides failing here, it would also be great if we can fail more directly in the create index api? (for example in MetaDataCreateIndexService#validateIndexSettings(...) method). This way the error is directly visible and an index with illegal settings will not be created.

*
* @return an optional engine factory
*/
Optional<EngineFactory> getMaybeEngineFactory(IndexSettings indexSettings);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree on renaming getMaybeEngineFactory() to getEngineFactory() the fact that it returns an optional indicates that nothing maybe returned. Personally I'm both fine with returning an Optional or just null.

@jasontedor jasontedor changed the base branch from xdcr to ccr October 6, 2017 17:40
* ccr: (42 commits)
  [DOCS] Added info about snapshotting your data before an upgrade.
  Add documentation about disabling `_field_names`. (elastic#26813)
  Remove UnsortedNumericDoubleValues (elastic#26817)
  Fix IndexOutOfBoundsException in histograms for NaN doubles (elastic#26787) (elastic#26856)
  [TEST] Added skipping the `headers` feature to the Bulk REST YAML test
  Update type-field.asciidoc
  Fix search_after with geo distance sorting (elastic#26891)
  Use proper logging placeholder for Netty logging
  Add Netty channel information on write and flush failure
  Remove deploying in JBoss documentation
  Document JVM option MaxFDLimit for macOS ()
  Add additional low-level logging handler ()
  Unwrap causes when maybe dying
  Change log level on write and flush failure to warn
  [TEST] add test to ensure legacy list syntax in yml works fine
  Bump BWC version for settings serialization to 6.1.0
  Removed void token filter entries and added two tests
  Added Bengali Analyzer to Elasticsearch with respect to the lucene update(PR#238)
  Fix toString() in SnapshotStatus (elastic#26852)
  elastic#26870 change bwc version for fuzzy_transpositions to 6.1 after backport
  ...
* ccr:
  Use LF line endings in Painless generated files (elastic#26822)
Copy link
Contributor

@s1monw s1monw left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@rjernst
Copy link
Member

rjernst commented Oct 10, 2017

LGTM too. We can discuss Optional vs null next week.

@jasontedor jasontedor merged commit 6e403b0 into elastic:ccr Oct 12, 2017
jasontedor added a commit that referenced this pull request Oct 12, 2017
This commit enables the engine factory to be pluggable based on index
settings used when creating the index service for an index.

Relates #26827
@jasontedor jasontedor deleted the engine-factory-provider branch October 12, 2017 14:52
@clintongormley clintongormley added :Distributed Indexing/Distributed A catch all label for anything in the Distributed Indexing Area. Please avoid if you can. :Distributed Indexing/Engine Anything around managing Lucene and the Translog in an open shard. and removed :Engine labels Feb 13, 2018
@clintongormley clintongormley removed the :Distributed Indexing/Distributed A catch all label for anything in the Distributed Indexing Area. Please avoid if you can. label Feb 13, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
:Distributed Indexing/Engine Anything around managing Lucene and the Translog in an open shard. >non-issue v7.0.0-beta1
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants