-
Notifications
You must be signed in to change notification settings - Fork 25.2k
Refactor SearchRequest to be parsed on the coordinating node #13859
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from 93 commits
Commits
Show all changes
99 commits
Select commit
Hold shift + click to select a range
a96f14e
new SearchSourceBuilder to use as the search request
colings86 5623dd2
implemented hashCode, equals and fromXContent in NewSearchSourceBuidler
colings86 e73052b
cleanup of commented out code
colings86 eed6ab7
various fixes after a review
colings86 3c8c6c0
Completed serialisation to/from son and transport protocol and added …
colings86 9116c1c
Renamed SearchSourceBuilder to OldSearchSourceBuilder and NewSearchSo…
colings86 e1759b1
cleanup codebase to work with SearchSourceBuilder
colings86 3cb3514
clean up for methods in SearchRequestBuilder so it works with SearchS…
colings86 b4dd6a5
fixes to tests
colings86 aa3d29d
small parsing fix for fields
colings86 41e6350
muted failing test
colings86 ab01ec5
SearchRequest.source() now returns a SearchSourceBuilder.
colings86 b98f7cf
updates to fix some of the NOCOMMITs
colings86 ac8c172
fix compile errors
colings86 6c335f4
Parse source, extra-source and template-source on the coordinating node
s1monw 083f28a
remove index from QueryParseContext Constructor
colings86 a53cf89
Parsing of rescuer and sort ByteReferences.
colings86 d7429a6
Search service can now fully populate SearchContext from SearchSource…
colings86 2e3cc50
removed sysout
colings86 585ed07
unit test fixes
colings86 52d9664
fix to sort parsing in SearchService
colings86 394e551
removed sysouts
colings86 08ae68c
Merge branch 'feature/query-refactoring' into feature/search-request-…
colings86 63da68e
test fixes
colings86 85f4c39
more test fixes
colings86 2de450c
fixes and debugging
colings86 95a7b6a
Add a late-parsing SearchSoruce wrapper for index warmers
s1monw 04864dc
Impl SearchSource#hashCode|equals|toString
s1monw 7a209cc
more test fixes
colings86 4996b07
update rest tests to expect default boost to be returned in warmer re…
colings86 c93eeb8
fix inner_hits parsing in SearchService
s1monw 2dce527
rest test fixes (hopefully)
colings86 6d307a3
delete by query rest test fix (maybe)
colings86 3c9e69e
Parse query from deleteByQuery request body
colings86 a163a0f
fixed script language plugins rest tests to use correct script syntax
colings86 ce7dde0
fix SearchSourceBuilder.fromXContent to allow sort to be an object
colings86 f541551
fix lang js rest test (hopefully)
colings86 cc9519e
correct IndexQueryParserService.parse(QueryBuilder<?>) so that it doe…
colings86 510d189
Revert "correct IndexQueryParserService.parse(QueryBuilder<?>) so tha…
colings86 851af9e
fixing up NOCOMMITs and making the IndicesQueryParserService not seri…
colings86 a9c6e4c
Merge branch 'feature/query-refactoring' into feature/search-request-…
colings86 67237ba
test fixes
colings86 bd42d26
text fix
colings86 35fc73f
fix to make parseSource in SearchService work with source overriding …
colings86 78d6d29
fix groovy tests - we don't pass json anymore so no need to double es…
s1monw f3b581a
Added ext section to search request for plugins that extend the fetch…
colings86 2b94126
removed obselete NOCOMMIT
colings86 9f08d48
Merge branch 'feature/query-refactoring' into feature/search-request-…
colings86 f1cf592
Test Fixes
colings86 8d488c5
Removed delete tests which accidentally got re-added on merge
colings86 f1da2bb
test fixes
colings86 4472270
Merge branch 'feature/query-refactoring' into feature/search-request-…
colings86 05c0031
Merge branch 'master' into feature/search-request-refactoring
s1monw a21238b
Merge branch 'master' into feature/search-request-refactoring
colings86 8426c14
fixed compile errors
colings86 e15736a
fixes for some NORELEASE comments
colings86 458f87c
removed OldSearchSourceBuilder
colings86 117d8d2
Merge branch 'master' into feature/search-request-refactoring
colings86 7fbd565
review comment fixes
colings86 ec51e8e
Merge branch 'master' into feature/search-request-refactoring
colings86 e8ef63c
compile fix
colings86 8149184
test fixes
colings86 d59e959
rename RestActions.parseQuerySource() to RestActions.urlParamsToQuery…
colings86 7055a05
test fixes
colings86 ec93531
fixes
colings86 9b1d1b2
fixes
colings86 a3a2432
Merge branch 'master' into feature/search-request-refactoring
colings86 f89de33
Merge branch 'master' into feature/search-request-refactoring
javanna f39e23b
pass in QuerySourceBuilder directly to ValidateQueryRequest
javanna ef286ce
Count api: simplify default handling and toString representation
javanna 80d236e
Revert "Count api: simplify default handling and toString representat…
javanna 669a589
fixed some NORELEASE comments in tests
colings86 86be9db
Fix some RequestBuilder#toString that produced broken json
javanna 10a5593
more fixes for tests
colings86 fcc6060
Count api: simplify default handling
javanna c2fc653
[TEST] move variable assignment one line above
javanna 108f5c5
Count Request: minScore can be null, its getter should be Float rathe…
javanna 1915c74
Merge branch 'master' into feature/search-request-refactoring
javanna cbcc23b
[TEST] Simplify SearchSourceBuilderTests setup
javanna 9d1c45f
[TEST] Minor SearchSourceBuilderTests infra changes
javanna 3313742
removed defaultRescoreWindow
colings86 5b23d9a
more test clean ups
colings86 ee5d284
more clean up
colings86 a16b025
re-enables InnerHits Tests
colings86 c69cde4
review fixes
colings86 42d7bfd
test: add explicit generic type to get around a strange issue where t…
martijnvg 7aaba20
more fixes from review
colings86 4557d1b
Merge branch 'master' into feature/search-request-refactoring
colings86 78921e0
test: removed the usage of String.valueOf(...), java 8 type inference…
martijnvg 83771d3
more test fixes
colings86 e0a78fd
Merge branch 'master' into feature/search-request-refactoring
colings86 6819224
Merge branch 'master' into feature/search-request-refactoring
colings86 ca76712
Merge branch 'master' into feature/search-request-refactoring
colings86 c618f75
Merge branch 'master' into feature/search-request-refactoring
colings86 63c51b7
review comment fixes
colings86 8a87448
Revert changes made to IdsQueryBuilder, fixed upstream meanwhile
javanna 0cab4a2
Revert changes made to IndexQueryParserService, fixed upstream meanwhile
javanna ef5fc72
reverted non-null searchSourceBuilder in searchRequest for now
colings86 cf3e92a
Merge branch 'master' into feature/search-request-refactoring
colings86 File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -19,140 +19,69 @@ | |
|
||
package org.elasticsearch.action.count; | ||
|
||
import org.elasticsearch.ElasticsearchGenerationException; | ||
import org.elasticsearch.action.search.SearchRequest; | ||
import org.elasticsearch.action.support.QuerySourceBuilder; | ||
import org.elasticsearch.action.support.broadcast.BroadcastRequest; | ||
import org.elasticsearch.client.Requests; | ||
import org.elasticsearch.common.Nullable; | ||
import org.elasticsearch.common.Strings; | ||
import org.elasticsearch.common.bytes.BytesArray; | ||
import org.elasticsearch.common.bytes.BytesReference; | ||
import org.elasticsearch.common.io.stream.StreamInput; | ||
import org.elasticsearch.common.io.stream.StreamOutput; | ||
import org.elasticsearch.common.xcontent.XContentBuilder; | ||
import org.elasticsearch.common.xcontent.XContentFactory; | ||
import org.elasticsearch.common.xcontent.XContentHelper; | ||
import org.elasticsearch.index.query.QueryBuilder; | ||
import org.elasticsearch.search.builder.SearchSourceBuilder; | ||
|
||
import java.io.IOException; | ||
import java.util.Arrays; | ||
import java.util.Map; | ||
|
||
import static org.elasticsearch.search.internal.SearchContext.DEFAULT_TERMINATE_AFTER; | ||
|
||
/** | ||
* A request to count the number of documents matching a specific query. Best created with | ||
* {@link org.elasticsearch.client.Requests#countRequest(String...)}. | ||
* <p> | ||
* The request requires the query source to be set either using {@link #source(QuerySourceBuilder)}, | ||
* or {@link #source(byte[])}. | ||
* | ||
* @see CountResponse | ||
* @see org.elasticsearch.client.Client#count(CountRequest) | ||
* @see org.elasticsearch.client.Requests#countRequest(String...) | ||
*/ | ||
public class CountRequest extends BroadcastRequest<CountRequest> { | ||
|
||
public static final float DEFAULT_MIN_SCORE = -1f; | ||
|
||
private float minScore = DEFAULT_MIN_SCORE; | ||
|
||
@Nullable | ||
protected String routing; | ||
|
||
@Nullable | ||
private String preference; | ||
|
||
private BytesReference source; | ||
|
||
private String[] types = Strings.EMPTY_ARRAY; | ||
|
||
private int terminateAfter = DEFAULT_TERMINATE_AFTER; | ||
private SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder(); | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. any chance this can be final? |
||
|
||
/** | ||
* Constructs a new count request against the provided indices. No indices provided means it will | ||
* run against all indices. | ||
*/ | ||
public CountRequest(String... indices) { | ||
super(indices); | ||
searchSourceBuilder.size(0); | ||
} | ||
|
||
/** | ||
* The minimum score of the documents to include in the count. | ||
*/ | ||
public float minScore() { | ||
return minScore; | ||
public Float minScore() { | ||
return searchSourceBuilder.minScore(); | ||
} | ||
|
||
/** | ||
* The minimum score of the documents to include in the count. Defaults to <tt>-1</tt> which means all | ||
* documents will be included in the count. | ||
*/ | ||
public CountRequest minScore(float minScore) { | ||
this.minScore = minScore; | ||
return this; | ||
} | ||
|
||
/** | ||
* The source to execute. | ||
*/ | ||
public BytesReference source() { | ||
return source; | ||
} | ||
|
||
/** | ||
* The source to execute. | ||
*/ | ||
public CountRequest source(QuerySourceBuilder sourceBuilder) { | ||
this.source = sourceBuilder.buildAsBytes(Requests.CONTENT_TYPE); | ||
return this; | ||
} | ||
|
||
/** | ||
* The source to execute in the form of a map. | ||
*/ | ||
@SuppressWarnings("unchecked") | ||
public CountRequest source(Map querySource) { | ||
try { | ||
XContentBuilder builder = XContentFactory.contentBuilder(Requests.CONTENT_TYPE); | ||
builder.map(querySource); | ||
return source(builder); | ||
} catch (IOException e) { | ||
throw new ElasticsearchGenerationException("Failed to generate [" + querySource + "]", e); | ||
} | ||
} | ||
|
||
public CountRequest source(XContentBuilder builder) { | ||
this.source = builder.bytes(); | ||
return this; | ||
} | ||
|
||
/** | ||
* The source to execute. It is preferable to use either {@link #source(byte[])} | ||
* or {@link #source(QuerySourceBuilder)}. | ||
*/ | ||
public CountRequest source(String querySource) { | ||
this.source = new BytesArray(querySource); | ||
this.searchSourceBuilder.minScore(minScore); | ||
return this; | ||
} | ||
|
||
/** | ||
* The source to execute. | ||
*/ | ||
public CountRequest source(byte[] querySource) { | ||
return source(querySource, 0, querySource.length); | ||
} | ||
|
||
/** | ||
* The source to execute. | ||
* The query to execute | ||
*/ | ||
public CountRequest source(byte[] querySource, int offset, int length) { | ||
return source(new BytesArray(querySource, offset, length)); | ||
} | ||
|
||
public CountRequest source(BytesReference querySource) { | ||
this.source = querySource; | ||
public CountRequest query(QueryBuilder<?> queryBuilder) { | ||
this.searchSourceBuilder.query(queryBuilder); | ||
return this; | ||
} | ||
|
||
|
@@ -207,15 +136,12 @@ public String preference() { | |
* Upon reaching <code>terminateAfter</code> counts, the count request will early terminate | ||
*/ | ||
public CountRequest terminateAfter(int terminateAfterCount) { | ||
if (terminateAfterCount <= 0) { | ||
throw new IllegalArgumentException("terminateAfter must be > 0"); | ||
} | ||
this.terminateAfter = terminateAfterCount; | ||
this.searchSourceBuilder.terminateAfter(terminateAfterCount); | ||
return this; | ||
} | ||
|
||
public int terminateAfter() { | ||
return this.terminateAfter; | ||
return this.searchSourceBuilder.terminateAfter(); | ||
} | ||
|
||
@Override | ||
|
@@ -230,31 +156,20 @@ public void writeTo(StreamOutput out) throws IOException { | |
|
||
@Override | ||
public String toString() { | ||
String sSource = "_na_"; | ||
try { | ||
sSource = XContentHelper.convertToJson(source, false); | ||
} catch (Exception e) { | ||
// ignore | ||
} | ||
return "[" + Arrays.toString(indices) + "]" + Arrays.toString(types) + ", source[" + sSource + "]"; | ||
return "count request indices:" + Arrays.toString(indices) + | ||
", types:" + Arrays.toString(types) + | ||
", routing: " + routing + | ||
", preference: " + preference + | ||
", source:" + searchSourceBuilder.toString(); | ||
} | ||
|
||
public SearchRequest toSearchRequest() { | ||
SearchRequest searchRequest = new SearchRequest(indices()); | ||
searchRequest.source(searchSourceBuilder); | ||
searchRequest.indicesOptions(indicesOptions()); | ||
searchRequest.types(types()); | ||
searchRequest.routing(routing()); | ||
searchRequest.preference(preference()); | ||
searchRequest.source(source()); | ||
SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder(); | ||
searchSourceBuilder.size(0); | ||
if (minScore() != DEFAULT_MIN_SCORE) { | ||
searchSourceBuilder.minScore(minScore()); | ||
} | ||
if (terminateAfter() != DEFAULT_TERMINATE_AFTER) { | ||
searchSourceBuilder.terminateAfter(terminateAfter()); | ||
} | ||
searchRequest.extraSource(searchSourceBuilder); | ||
return searchRequest; | ||
} | ||
} |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
one other note: using SearchSourceBuilder internally means that all the options that the search api supports are exposed to the count api too, via java api. But the count api only supports a small subset of those features. Are we sure we don't want to keep on using the QuerySourceBuilder?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Going to still use SearchSourceBuilder internally for simplicity but will remove the setter and getter for SearchSourceBuilder so it isn't exposed to the API