Skip to content

Commit dfeeee6

Browse files
brwedadoonet
authored andcommitted
Multi-line or multi-statement Python scripts raise NullPointerException
Calling a multi-statement score using curl e.g. `a=22; _score*a` is not possible with the current lang-python, nor is it possible to write a script using semicolons or multiple lines. I can only get compound statements (e.g. list comprehensions with embedded if statements) to work, so I'm limited in the complexity of my scoring process. I'm using ES 1.3.2 and latest lang-python: ``` $ ls -la /usr/share/elasticsearch/plugins/lang-python/ -rw-r--r-- 1 root root 10482 Aug 27 17:20 elasticsearch-lang-python-2.3.0.jar -rw-r--r-- 1 root root 14340135 Aug 27 17:20 jython-standalone-2.5.3.jar ``` Here's a worked example: ``` # Delete existing data, add 2 simple records, fetch both results to stdout curl -XDELETE "http://localhost:9200/test" curl -XPUT "http://localhost:9200/test/doc/1" -d '{ "num": 1.0 }' curl -XPUT "http://localhost:9200/test/doc/2?refresh" -d '{ "num": 2.0 }' # show our records curl -XGET 'http://localhost:9200/test/_search' -d '{ "query" : { "match_all": {}} }' ``` We'll run a simple query that uses `num` as a score modifier: ```doc["num"].value * _score``` If I create `/etc/elasticsearch/scripts/py1.py`: ``` doc["num"].value * _score ``` and wait for the script to reload (by monitoring the logs), I can call: ``` $ curl -XGET "http://localhost:9200/test/_search?pretty" -d' { "query": { "function_score": { "script_score": { "script": "py1", "lang": "python" } } } }' ``` and this will calculate the results. The same can be achieved using an in-line call (ignoring `py1.py`): ``` curl -XGET "http://localhost:9200/test/_search?pretty" -d' { "query": { "function_score": { "script_score": { "script": "doc[\"num\"].value * _score", "lang": "python" } } } }' ``` However using more than 1 statement will fail. This example uses `;` to split 2 statements (this is legal in jython 2.5): ``` curl -XGET "http://localhost:9200/test/_search?pretty" -d' { "query": { "function_score": { "script_score": { "script": "a=42; doc[\"num\"].value * _score", "lang": "python" } } } }' "reason" : "QueryPhaseExecutionException[[test][3]: query[function score (ConstantScore(*:*),function=script[a=42; doc[\"num\"].value * _score], params [null])],from[0],size[10]: Query Failed [Failed to execute main query]]; nested: NullPointerException; " and the log message: org.elasticsearch.search.query.QueryPhaseExecutionException: [test][3]: query[function score (ConstantScore(*:*),function=script[a=42; doc["num"].value * _score], params [null])],from[0],size[10]: Query Failed [Failed to execute main query] at org.elasticsearch.search.query.QueryPhase.execute(QueryPhase.java:162) at org.elasticsearch.search.SearchService.executeQueryPhase(SearchService.java:261) at org.elasticsearch.search.action.SearchServiceTransportAction$5.call(SearchServiceTransportAction.java:206) at org.elasticsearch.search.action.SearchServiceTransportAction$5.call(SearchServiceTransportAction.java:203) at org.elasticsearch.search.action.SearchServiceTransportAction$23.run(SearchServiceTransportAction.java:517) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) Caused by: java.lang.NullPointerException ``` Creating a `py2.py` in the scripts directory containing: ``` a=42; doc["num"].value * _score ``` and calling ``` $ curl -XGET "http://localhost:9200/test/_search?pretty" -d' { "query": { "function_score": { "script_score": { "script": "py2", "lang": "python" } } } }' has the same error: "reason" : "QueryPhaseExecutionException[[test][3]: query[function score (ConstantScore(*:*),function=script[py2], params [null])],from[0],size[10]: Query Failed [Failed to execute main query]]; nested: PyException; " ``` If a `py3.py` script is made with the same two statements split over two lines: ``` a=42 doc["num"].value * _score ``` then the same errors are thrown. I'll note that if I experiment with equivalent groovy scripts then both the semicolon is allowed and multi-line scripts (in /scripts) are allowed. Closes #19. (cherry picked from commit 9fca562)
1 parent bc8c065 commit dfeeee6

File tree

1 file changed

+22
-0
lines changed

1 file changed

+22
-0
lines changed

src/test/java/org/elasticsearch/script/python/PythonScriptSearchTests.java

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -288,4 +288,26 @@ public void testScriptScoresWithAgg() throws IOException {
288288
assertThat(((Terms) response.getAggregations().asMap().get("score_agg")).getBuckets().get(0).getKeyAsNumber().floatValue(), Matchers.is(1f));
289289
assertThat(((Terms) response.getAggregations().asMap().get("score_agg")).getBuckets().get(0).getDocCount(), Matchers.is(1l));
290290
}
291+
292+
/**
293+
* Test case for #19: https://github.com/elasticsearch/elasticsearch-lang-python/issues/19
294+
* Multi-line or multi-statement Python scripts raise NullPointerException
295+
*/
296+
@Test
297+
public void testPythonMultiLines() throws Exception {
298+
createIndex("test");
299+
index("test", "type1", "1", jsonBuilder().startObject().field("myfield", "foo").endObject());
300+
refresh();
301+
302+
client().prepareUpdate("test", "type1", "1").setScriptLang("python")
303+
.setScript("a=42; ctx[\"_source\"][\"myfield\"]=\"bar\"", ScriptService.ScriptType.INLINE)
304+
.execute().actionGet();
305+
refresh();
306+
307+
Object value = get("test", "type1", "1").getSourceAsMap().get("myfield");
308+
assertThat(value instanceof String, is(true));
309+
310+
assertThat((String) value, CoreMatchers.equalTo("bar"));
311+
}
312+
291313
}

0 commit comments

Comments
 (0)