Skip to content

Commit c1c48bd

Browse files
MpdreamzMpdreamz
committed
doc gen changes
1 parent 6bf8305 commit c1c48bd

File tree

56 files changed

+4665
-23
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

56 files changed

+4665
-23
lines changed

Diff for: ‎.travis.yml

+1-1
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,3 @@
11
language: csharp
22
solution: src/Elasticsearch.sln
3-
script: ./build.sh
3+
script: ./build.sh

Diff for: ‎docs/asciidoc/Aggregations/Bucket/Children/ChildrenAggregationMapping.doc.asciidoc

+16
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,16 @@
1+
To use the child aggregation you have to make sure
2+
a `_parent` mapping is in place, here we create the project
3+
index with two mapped types, `project` and `commitactivity` and
4+
we add a `_parent` mapping from `commitactivity` to `parent`
5+
6+
[source, csharp]
7+
----
8+
var createProjectIndex = TestClient.GetClient().CreateIndex(typeof(Project), c => c
9+
.Mappings(map=>map
10+
.Map<Project>(m=>m.AutoMap())
11+
.Map<CommitActivity>(m=>m
12+
.Parent<Project>()
13+
)
14+
)
15+
);
16+
----

Diff for: ‎docs/asciidoc/Aggregations/WritingAggregations.doc.asciidoc

+58
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,58 @@
1+
Aggregations are arguably one of the most powerful features of Elasticsearch.
2+
NEST allows you to write your aggregations using a strict fluent dsl, a verbatim object initializer
3+
syntax that maps verbatim to the elasticsearch API
4+
a more terse object initializer aggregation DSL.
5+
6+
Three different ways, yikes thats a lot to take in! Lets go over them one by one and explain when you might
7+
want to use which one.
8+
9+
The fluent lambda syntax is the most terse way to write arggragations.
10+
It benefits from types that are carried over to sub aggregations
11+
12+
[source, csharp]
13+
----
14+
s => s
15+
.Aggregations(aggs=>aggs
16+
.Children<CommitActivity>("name_of_child_agg", child => child
17+
.Aggregations(childAggs=>childAggs
18+
.Average("average_per_child", avg=>avg.Field(p=>p.ConfidenceFactor))
19+
.Max("max_per_child", avg=>avg.Field(p=>p.ConfidenceFactor))
20+
)
21+
)
22+
)
23+
----
24+
The object initializer syntax (OIS) is a one-to-one mapping with how aggregations
25+
have to be represented in the Elasticsearch API. While it has the benefit of being a one-to-one
26+
mapping, it being dictionary based in C# it can grow exponentially in complexity rather fast.
27+
28+
[source, csharp]
29+
----
30+
new SearchRequest<Project>
31+
{
32+
Aggregations = new ChildrenAggregation("name_of_child_agg", typeof(CommitActivity))
33+
{
34+
Aggregations =
35+
new AverageAggregation("average_per_child", "confidenceFactor") &&
36+
new MaxAggregation("max_per_child", "confidenceFactor")
37+
}
38+
}
39+
----
40+
For this reason the OIS syntax can be shortened dramatically by using `*Agg` related family,
41+
These allow you to forego introducing intermediary Dictionaries to represent the aggregation DSL.
42+
It also allows you to combine multiple aggregations using bitwise AND (`
43+
`) operator.
44+
45+
Compare the following example with the previous vanilla OIS syntax
46+
47+
[source, csharp]
48+
----
49+
new SearchRequest<Project>
50+
{
51+
Aggregations = new ChildrenAggregation("name_of_child_agg", typeof(CommitActivity))
52+
{
53+
Aggregations =
54+
new AverageAggregation("average_per_child", Field<CommitActivity>(p=>p.ConfidenceFactor))
55+
&& new MaxAggregation("max_per_child", Field<CommitActivity>(p=>p.ConfidenceFactor))
56+
}
57+
}
58+
----

Diff for: ‎docs/asciidoc/ClientConcepts/ConnectionPooling/BuildingBlocks/ConnectionPooling.Doc.asciidoc

+157
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,157 @@
1+
= Connection Pooling
2+
Connection pooling is the internal mechanism that takes care of registering what nodes there are in the cluster and which
3+
we can use to issue client calls on.
4+
5+
== SingleNodeConnectionPool
6+
The simplest of all connection pools, this takes a single `Uri` and uses that to connect to elasticsearch for all the calls
7+
It doesn't opt in to sniffing and pinging behavior, and will never mark nodes dead or alive. The one `Uri` it holds is always
8+
ready to go.
9+
10+
[source, csharp]
11+
----
12+
var uri = new Uri("http://localhost:9201");
13+
var pool = new SingleNodeConnectionPool(uri);
14+
pool.Nodes.Should().HaveCount(1);
15+
var node = pool.Nodes.First();
16+
node.Uri.Port.Should().Be(9201);
17+
----
18+
This type of pool is hardwired to optout out sniffing
19+
20+
[source, csharp]
21+
----
22+
pool.SupportsReseeding.Should().BeFalse();
23+
----
24+
and pinging
25+
26+
[source, csharp]
27+
----
28+
pool.SupportsPinging.Should().BeFalse();
29+
----
30+
When you use the low ceremony ElasticClient constructor that takes a single Uri
31+
We default to this SingleNodeConnectionPool
32+
33+
[source, csharp]
34+
----
35+
var client = new ElasticClient(uri);
36+
----
37+
[source, csharp]
38+
----
39+
client.ConnectionSettings.ConnectionPool.Should().BeOfType<SingleNodeConnectionPool>();
40+
----
41+
However we urge that you always pass your connection settings explicitly
42+
43+
[source, csharp]
44+
----
45+
client = new ElasticClient(new ConnectionSettings(uri));
46+
----
47+
[source, csharp]
48+
----
49+
client.ConnectionSettings.ConnectionPool.Should().BeOfType<SingleNodeConnectionPool>();
50+
----
51+
or even better pass the connection pool explicitly
52+
53+
[source, csharp]
54+
----
55+
client = new ElasticClient(new ConnectionSettings(pool));
56+
----
57+
[source, csharp]
58+
----
59+
client.ConnectionSettings.ConnectionPool.Should().BeOfType<SingleNodeConnectionPool>();
60+
----
61+
== StaticConnectionPool
62+
The static connection pool is great if you have a known small sized cluster and do no want to enable
63+
sniffing to find out the cluster topology.
64+
65+
[source, csharp]
66+
----
67+
var uris = Enumerable.Range(9200, 5).Select(p => new Uri("http://localhost:" + p));
68+
----
69+
a connection pool can be seeded using an enumerable of `Uri`
70+
71+
[source, csharp]
72+
----
73+
var pool = new StaticConnectionPool(uris);
74+
----
75+
Or using an enumerable of `Node`
76+
77+
[source, csharp]
78+
----
79+
var nodes = uris.Select(u=>new Node(u));
80+
----
81+
[source, csharp]
82+
----
83+
pool = new StaticConnectionPool(nodes);
84+
----
85+
This type of pool is hardwirted to optout out sniffing
86+
87+
[source, csharp]
88+
----
89+
pool.SupportsReseeding.Should().BeFalse();
90+
----
91+
but supports pinging when enabled
92+
93+
[source, csharp]
94+
----
95+
pool.SupportsPinging.Should().BeTrue();
96+
----
97+
To create a client using this static connection pool pass
98+
the connection pool to the connectionsettings you pass to ElasticClient
99+
100+
[source, csharp]
101+
----
102+
var client = new ElasticClient(new ConnectionSettings(pool));
103+
----
104+
[source, csharp]
105+
----
106+
client.ConnectionSettings.ConnectionPool.Should().BeOfType<StaticConnectionPool>();
107+
----
108+
== SniffingConnectionPool
109+
A subclass of StaticConnectionPool that allows itself to be reseeded at run time.
110+
It comes with a very minor overhead of a ReaderWriterLock to ensure thread safety.
111+
112+
[source, csharp]
113+
----
114+
var uris = Enumerable.Range(9200, 5).Select(p => new Uri("http://localhost:" + p));
115+
----
116+
a connection pool can be seeded using an enumerable of `Uri`
117+
118+
[source, csharp]
119+
----
120+
var pool = new SniffingConnectionPool(uris);
121+
----
122+
Or using an enumerable of `Node`
123+
A major benefit here is you can include known node roles when seeding
124+
NEST can use this information to favour sniffing on master eligable nodes first
125+
and take master only nodes out of rotation for issueing client calls on.
126+
127+
[source, csharp]
128+
----
129+
var nodes = uris.Select(u=>new Node(u));
130+
----
131+
[source, csharp]
132+
----
133+
pool = new SniffingConnectionPool(nodes);
134+
----
135+
This type of pool is hardwirted to optout out sniffing
136+
137+
[source, csharp]
138+
----
139+
pool.SupportsReseeding.Should().BeTrue();
140+
----
141+
but supports pinging when enabled
142+
143+
[source, csharp]
144+
----
145+
pool.SupportsPinging.Should().BeTrue();
146+
----
147+
To create a client using this siffing connection pool pass
148+
the connection pool to the connectionsettings you pass to ElasticClient
149+
150+
[source, csharp]
151+
----
152+
var client = new ElasticClient(new ConnectionSettings(pool));
153+
----
154+
[source, csharp]
155+
----
156+
client.ConnectionSettings.ConnectionPool.Should().BeOfType<SniffingConnectionPool>();
157+
----

Diff for: ‎docs/asciidoc/ClientConcepts/ConnectionPooling/BuildingBlocks/DateTimeProviders.Doc.asciidoc

+53
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,53 @@
1+
= Date time providers
2+
3+
Not typically something you'll have to pass to the client but all calls `to DateTime.Now`
4+
in the client have been abstracted by IDateTimeProvider. This allows us to unit test timeouts and clusterfailover
5+
in run time not being bound to wall clock time.
6+
7+
[source, csharp]
8+
----
9+
var dateTimeProvider = DateTimeProvider.Default;
10+
----
11+
dates are always returned in UTC
12+
13+
[source, csharp]
14+
----
15+
dateTimeProvider.Now().Should().BeCloseTo(DateTime.UtcNow);
16+
----
17+
Another responsibility of this interface is to calculate the time a node has to taken out of rotation
18+
based on the number of attempts to revive it. For very advance usecases this might be something of interest
19+
to provide a custom implementation for.
20+
21+
[source, csharp]
22+
----
23+
var dateTimeProvider = DateTimeProvider.Default;
24+
----
25+
26+
The default timeout calculation is: `min(timeout * 2 ^ (attempts * 0.5 -1), maxTimeout)`
27+
The default values for `timeout` and `maxTimeout` are
28+
29+
[source, csharp]
30+
----
31+
var timeout = TimeSpan.FromMinutes(1);
32+
----
33+
[source, csharp]
34+
----
35+
var maxTimeout = TimeSpan.FromMinutes(30);
36+
----
37+
Plotting these defaults looks as followed:
38+
[[timeout]]
39+
.Default formula, x-axis time in minutes, y-axis number of attempts to revive
40+
image::timeoutplot.png[dead timeout]
41+
The goal here is that whenever a node is resurected and is found to still be offline we send it
42+
back to the doghouse for an ever increasingly long period untill we hit a bounded maximum.
43+
44+
[source, csharp]
45+
----
46+
var timeouts = Enumerable.Range(0, 30)
47+
.Select(attempt => dateTimeProvider.DeadTime(attempt, timeout, maxTimeout))
48+
.ToList();
49+
----
50+
[source, csharp]
51+
----
52+
increasedTimeout.Should().BeWithin(maxTimeout);
53+
----

Diff for: ‎docs/asciidoc/ClientConcepts/ConnectionPooling/BuildingBlocks/KeepingTrackOfNodes.Doc.asciidoc

+103
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,103 @@
1+
=
2+
3+
4+
[source, csharp]
5+
----
6+
var node = new Node(new Uri("http://localhost:9200"));
7+
node.Uri.Should().NotBeNull();
8+
node.Uri.Port.Should().Be(9200);
9+
----
10+
By default master eligable and holds data is presumed to be true *
11+
12+
[source, csharp]
13+
----
14+
node.MasterEligable.Should().BeTrue();
15+
----
16+
[source, csharp]
17+
----
18+
node.HoldsData.Should().BeTrue();
19+
----
20+
Is resurrected is true on first usage, hints to the transport that a ping might be useful
21+
22+
[source, csharp]
23+
----
24+
node.IsResurrected.Should().BeTrue();
25+
----
26+
When instantiating your connection pool you could switch these to false to initialize the client to
27+
a known cluster topology.
28+
29+
passing a node with a path should be preserved. Sometimes an elasticsearch node lives behind a proxy
30+
31+
[source, csharp]
32+
----
33+
var node = new Node(new Uri("http://test.example/elasticsearch"));
34+
----
35+
[source, csharp]
36+
----
37+
node.Uri.Port.Should().Be(80);
38+
node.Uri.AbsolutePath.Should().Be("/elasticsearch/");
39+
----
40+
We force paths to end with a forward slash so that they can later be safely combined
41+
42+
[source, csharp]
43+
----
44+
var combinedPath = new Uri(node.Uri, "index/type/_search");
45+
----
46+
[source, csharp]
47+
----
48+
combinedPath.AbsolutePath.Should().Be("/elasticsearch/index/type/_search");
49+
----
50+
which is exactly what the `CreatePath` method does on `Node`
51+
52+
[source, csharp]
53+
----
54+
combinedPath = node.CreatePath("index/type/_search");
55+
----
56+
[source, csharp]
57+
----
58+
combinedPath.AbsolutePath.Should().Be("/elasticsearch/index/type/_search");
59+
var node = new Node(new Uri("http://localhost:9200"));
60+
node.FailedAttempts.Should().Be(0);
61+
node.IsAlive.Should().BeTrue();
62+
----
63+
64+
every time a node is marked dead the number of attempts should increase
65+
and the passed datetime should be exposed.
66+
67+
[source, csharp]
68+
----
69+
var deadUntil = DateTime.Now.AddMinutes(1);
70+
node.MarkDead(deadUntil);
71+
node.FailedAttempts.Should().Be(i + 1);
72+
node.IsAlive.Should().BeFalse();
73+
node.DeadUntil.Should().Be(deadUntil);
74+
----
75+
however when marking a node alive deaduntil should be reset and attempts reset to 0
76+
77+
[source, csharp]
78+
----
79+
node.MarkAlive();
80+
----
81+
[source, csharp]
82+
----
83+
node.FailedAttempts.Should().Be(0);
84+
node.DeadUntil.Should().Be(default(DateTime));
85+
node.IsAlive.Should().BeTrue();
86+
----
87+
Nodes are considered equal if they have the same endpoint no matter what other metadata is associated
88+
89+
[source, csharp]
90+
----
91+
var node = new Node(new Uri("http://localhost:9200")) { MasterEligable = false };
92+
----
93+
[source, csharp]
94+
----
95+
var nodeAsMaster = new Node(new Uri("http://localhost:9200")) { MasterEligable = true };
96+
(node == nodeAsMaster).Should().BeTrue();
97+
(node != nodeAsMaster).Should().BeFalse();
98+
var uri = new Uri("http://localhost:9200");
99+
(node == uri).Should().BeTrue();
100+
var differentUri = new Uri("http://localhost:9201");
101+
(node != differentUri).Should().BeTrue();
102+
node.Should().Be(nodeAsMaster);
103+
----

0 commit comments

Comments
 (0)