Skip to content

Commit 9bb9f63

Browse files
committed
[DOCS] Note that trim filter doesn't change offsets (#53220)
The [word delimiter graph token filter docs][0] note that the `trim` filter changes the length of tokens without changing their offsets. This explicitly mentions that in the `trim` filter docs. [0]: https://www.elastic.co/guide/en/elasticsearch/reference/master/analysis-word-delimiter-graph-tokenfilter.html
1 parent 9abf537 commit 9bb9f63

File tree

1 file changed

+6
-3
lines changed

1 file changed

+6
-3
lines changed

docs/reference/analysis/tokenfilters/trim-tokenfilter.asciidoc

Lines changed: 6 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,9 @@
44
<titleabbrev>Trim</titleabbrev>
55
++++
66

7-
Removes leading and trailing whitespace from each token in a stream.
7+
Removes leading and trailing whitespace from each token in a stream. While this
8+
can change the length of a token, the `trim` filter does _not_ change a token's
9+
offsets.
810

911
The `trim` filter uses Lucene's
1012
https://lucene.apache.org/core/{lucene_version_path}/analyzers-common/org/apache/lucene/analysis/miscellaneous/TrimFilter.html[TrimFilter].
@@ -37,8 +39,9 @@ GET _analyze
3739
}
3840
----
3941

40-
The API returns the following response. Note the `" fox "` token contains
41-
the original text's whitespace.
42+
The API returns the following response. Note the `" fox "` token contains the
43+
original text's whitespace. Note that despite changing the token's length, the
44+
`start_offset` and `end_offset` remain the same.
4245

4346
[source,console-result]
4447
----

0 commit comments

Comments
 (0)