Skip to content

Commit 9c75f14

Browse files
committed
[DOCS] Reformat classic token filter docs (#48314)
1 parent e1dd0e7 commit 9c75f14

File tree

1 file changed

+143
-5
lines changed

1 file changed

+143
-5
lines changed
Lines changed: 143 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,9 +1,147 @@
11
[[analysis-classic-tokenfilter]]
2-
=== Classic Token Filter
2+
=== Classic token filter
3+
++++
4+
<titleabbrev>Classic</titleabbrev>
5+
++++
36

4-
The `classic` token filter does optional post-processing of
5-
terms that are generated by the <<analysis-classic-tokenizer,`classic` tokenizer>>.
7+
Performs optional post-processing of terms generated by the
8+
<<analysis-classic-tokenizer,`classic` tokenizer>>.
69

7-
This filter removes the english possessive from the end of words, and
8-
it removes dots from acronyms.
10+
This filter removes the english possessive (`'s`) from the end of words and
11+
removes dots from acronyms. It uses Lucene's
12+
https://lucene.apache.org/core/{lucene_version_path}/analyzers-common/org/apache/lucene/analysis/standard/ClassicFilter.html[ClassicFilter].
13+
14+
[[analysis-classic-tokenfilter-analyze-ex]]
15+
==== Example
16+
17+
The following <<indices-analyze,analyze API>> request demonstrates how the
18+
classic token filter works.
19+
20+
[source,console]
21+
--------------------------------------------------
22+
GET /_analyze
23+
{
24+
"tokenizer" : "classic",
25+
"filter" : ["classic"],
26+
"text" : "The 2 Q.U.I.C.K. Brown-Foxes jumped over the lazy dog's bone."
27+
}
28+
--------------------------------------------------
29+
30+
The filter produces the following tokens:
31+
32+
[source,text]
33+
--------------------------------------------------
34+
[ The, 2, QUICK, Brown, Foxes, jumped, over, the, lazy, dog, bone ]
35+
--------------------------------------------------
36+
37+
/////////////////////
38+
[source,console-result]
39+
--------------------------------------------------
40+
{
41+
"tokens" : [
42+
{
43+
"token" : "The",
44+
"start_offset" : 0,
45+
"end_offset" : 3,
46+
"type" : "<ALPHANUM>",
47+
"position" : 0
48+
},
49+
{
50+
"token" : "2",
51+
"start_offset" : 4,
52+
"end_offset" : 5,
53+
"type" : "<ALPHANUM>",
54+
"position" : 1
55+
},
56+
{
57+
"token" : "QUICK",
58+
"start_offset" : 6,
59+
"end_offset" : 16,
60+
"type" : "<ACRONYM>",
61+
"position" : 2
62+
},
63+
{
64+
"token" : "Brown",
65+
"start_offset" : 17,
66+
"end_offset" : 22,
67+
"type" : "<ALPHANUM>",
68+
"position" : 3
69+
},
70+
{
71+
"token" : "Foxes",
72+
"start_offset" : 23,
73+
"end_offset" : 28,
74+
"type" : "<ALPHANUM>",
75+
"position" : 4
76+
},
77+
{
78+
"token" : "jumped",
79+
"start_offset" : 29,
80+
"end_offset" : 35,
81+
"type" : "<ALPHANUM>",
82+
"position" : 5
83+
},
84+
{
85+
"token" : "over",
86+
"start_offset" : 36,
87+
"end_offset" : 40,
88+
"type" : "<ALPHANUM>",
89+
"position" : 6
90+
},
91+
{
92+
"token" : "the",
93+
"start_offset" : 41,
94+
"end_offset" : 44,
95+
"type" : "<ALPHANUM>",
96+
"position" : 7
97+
},
98+
{
99+
"token" : "lazy",
100+
"start_offset" : 45,
101+
"end_offset" : 49,
102+
"type" : "<ALPHANUM>",
103+
"position" : 8
104+
},
105+
{
106+
"token" : "dog",
107+
"start_offset" : 50,
108+
"end_offset" : 55,
109+
"type" : "<APOSTROPHE>",
110+
"position" : 9
111+
},
112+
{
113+
"token" : "bone",
114+
"start_offset" : 56,
115+
"end_offset" : 60,
116+
"type" : "<ALPHANUM>",
117+
"position" : 10
118+
}
119+
]
120+
}
121+
--------------------------------------------------
122+
/////////////////////
123+
124+
[[analysis-classic-tokenfilter-analyzer-ex]]
125+
==== Add to an analyzer
126+
127+
The following <<indices-create-index,create index API>> request uses the
128+
classic token filter to configure a new
129+
<<analysis-custom-analyzer,custom analyzer>>.
130+
131+
[source,console]
132+
--------------------------------------------------
133+
PUT /classic_example
134+
{
135+
"settings" : {
136+
"analysis" : {
137+
"analyzer" : {
138+
"classic_analyzer" : {
139+
"tokenizer" : "classic",
140+
"filter" : ["classic"]
141+
}
142+
}
143+
}
144+
}
145+
}
146+
--------------------------------------------------
9147

0 commit comments

Comments
 (0)