Skip to content

Build: Enable testing without magic comments #46180

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Sep 4, 2019
Merged

Build: Enable testing without magic comments #46180

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
62adc68
Build: Enable testing without magic comments
nik9000 Aug 30, 2019
be3b909
Merge branch 'master' into simpler_console
nik9000 Aug 30, 2019
4001033
Merge branch 'master' into simpler_console
nik9000 Sep 4, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 3 additions & 2 deletions buildSrc/src/main/groovy/org/elasticsearch/gradle/doc/RestTestsFromSnippetsTask.groovy
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -202,11 +202,12 @@ public class RestTestsFromSnippetsTask extends SnippetsTask {
previousTest = snippet
return
}
if (snippet.testResponse) {
if (snippet.testResponse || snippet.language == 'console-result') {
response(snippet)
return
}
if (snippet.test || snippet.console) {
if (snippet.test || snippet.console ||
snippet.language == 'console') {
test(snippet)
previousTest = snippet
return
Expand Down
35 changes: 24 additions & 11 deletions docs/README.asciidoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,10 +3,14 @@ Elasticsearch documentation build process.

See: https://github.com/elastic/docs

Snippets marked with `// CONSOLE` are automatically annotated with "VIEW IN
CONSOLE" and "COPY AS CURL" in the documentation and are automatically tested
by the command `gradle :docs:check`. To test just the docs from a single page,
use e.g. `./gradlew :docs:integTestRunner --tests "*rollover*"`.
Snippets marked with `[source,console]` are automatically annotated with
"VIEW IN CONSOLE" and "COPY AS CURL" in the documentation and are automatically
tested by the command `./gradlew -pdocs check`. To test just the docs from a
single page, use e.g. `./gradlew -ddocs integTestRunner --tests "*rollover*"`.

NOTE: Previously we use `// CONSOLE` instead of `[source,console]`. This worked
well for a long time so you'll see it all over early branches but we're phasing
it out because it requires some unpleasant hackery on the docs build side.

NOTE: If you have an elasticsearch-extra folder alongside your elasticsearch
folder, you must temporarily rename it when you are testing 6.3 or later branches.
Expand Down Expand Up @@ -45,10 +49,21 @@ for its modifiers:
header. If the response doesn't include a `Warning` header with the exact
text then the test fails. If the response includes `Warning` headers that
aren't expected then the test fails.
* `// TESTRESPONSE`: Matches this snippet against the body of the response of
the last test. If the response is JSON then order is ignored. If you add
`// TEST[continued]` to the snippet after `// TESTRESPONSE` it will continue
in the same test, allowing you to interleave requests with responses to check.
* `[source,console-result]`: Matches this snippet against the body of the
response of the last test. If the response is JSON then order is ignored. If
you add `// TEST[continued]` to the snippet after `[source,console-result]`
it will continue in the same test, allowing you to interleave requests with
responses to check.
* `// TESTRESPONSE`: Explicitly marks a snippet as a test response even without
`[source,console-result]`. Similarly to `// TEST` this is mostly used for
its modifiers.
* You can't use `[source,console-result]` immediately after `// TESTSETUP`.
Instead, consider using `// TEST[continued]` or rearrange your snippets.

NOTE: Previously we only used `// TESTRESPONSE` instead of
`[source,console-result]` so you'll see that a lot in older branches but we
prefer `[source,console-result]` now.

* `// TESTRESPONSE[s/foo/bar/]`: Substitutions. See `// TEST[s/foo/bar]` for
how it works. These are much more common than `// TEST[s/foo/bar]` because
they are useful for eliding portions of the response that are not pertinent
Expand All @@ -62,8 +77,6 @@ for its modifiers:
"figures out" the path. This is especially useful for making sweeping
assertions like "I made up all the numbers in this example, don't compare
them" which looks like `// TESTRESPONSE[s/\d+/$body.$_path/]`.
* You can't use `// TESTRESPONSE` immediately after `// TESTSETUP`. Instead,
consider using `// TEST[continued]` or rearrange your snippets.
* `// TESTRESPONSE[non_json]`: Add substitutions for testing responses in a
format other than JSON. Use this after all other substitutions so it doesn't
make other substitutions difficult.
Expand Down Expand Up @@ -98,7 +111,7 @@ endyaml
```

This allows slightly more expressive testing of the snippets. Since that syntax
is not supported by CONSOLE the usual way to incorporate it is with a
is not supported by `[source,console]` the usual way to incorporate it is with a
`// TEST[s//]` marker like this:

```
Expand Down
18 changes: 6 additions & 12 deletions docs/painless/painless-guide/painless-execute-script.asciidoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ If no context is specified then this context is used by default.

Request:

[source,js]
[source,console]
----------------------------------------------------------------
POST /_scripts/painless/_execute
{
Expand All @@ -43,17 +43,15 @@ POST /_scripts/painless/_execute
}
}
----------------------------------------------------------------
// CONSOLE

Response:

[source,js]
[source,console-result]
--------------------------------------------------
{
"result": "0.1"
}
--------------------------------------------------
// TESTRESPONSE

===== Filter context

Expand All @@ -69,7 +67,7 @@ index:: The name of an index containing a mapping that is compatible with the do

*Example*

[source,js]
[source,console]
----------------------------------------------------------------
PUT /my-index
{
Expand Down Expand Up @@ -99,17 +97,15 @@ POST /_scripts/painless/_execute
}
}
----------------------------------------------------------------
// CONSOLE

Response:

[source,js]
[source,console-result]
--------------------------------------------------
{
"result": true
}
--------------------------------------------------
// TESTRESPONSE


===== Score context
Expand All @@ -125,7 +121,7 @@ query:: If `_score` is used in the script then a query can specified that will b

*Example*

[source,js]
[source,console]
----------------------------------------------------------------
PUT /my-index
{
Expand Down Expand Up @@ -159,14 +155,12 @@ POST /_scripts/painless/_execute
}
}
----------------------------------------------------------------
// CONSOLE

Response:

[source,js]
[source,console-result]
--------------------------------------------------
{
"result": 0.8
}
--------------------------------------------------
// TESTRESPONSE
3 changes: 1 addition & 2 deletions docs/reference/docs/index_.asciidoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -141,7 +141,7 @@ comma-separated list of patterns you want to allow, or prefix each pattern with
`+` or `-` to indicate whether it should be allowed or blocked. When a list is
specified, the default behaviour is to disallow.

[source,js]
[source,console]
--------------------------------------------------
PUT _cluster/settings
{
Expand All @@ -164,7 +164,6 @@ PUT _cluster/settings
}
}
--------------------------------------------------
// CONSOLE

<1> Allow auto-creation of indices called `twitter` or `index10`, block the
creation of indices that match the pattern `index1*`, and allow creation of
Expand Down