Router - document published metrics #5699

pecameron · 2017-10-13T13:01:32Z

OCP 3.6

Documents metrics that are published by the router
Document installation and configuration
Document related environment variables

Trello:
https://trello.com/c/EHq8dqNj/529-document-the-metrics-the-router-exposes

pecameron · 2017-10-13T13:03:21Z

@knobunc @mburke5678 @bfallonf PTAL

pecameron · 2017-10-13T13:27:08Z

Just squashed the early commit.

knobunc · 2017-10-17T16:05:41Z

architecture/topics/haproxy_template_router.adoc

+provide no router metrics at all.
+
+Prometheus format data is the default data format for metrics in {product-title}.


I'm not sure we need to say this.

Or can we integrate it with the previous paragraph.

Dropped line

knobunc · 2017-10-17T16:06:10Z

architecture/topics/haproxy_template_router.adoc

+=== HAProxy Template Router Metrics
+
+The HAProxy router, by default, exposes or publishes metrics in


"by default"

Not sure if we want to say that... why not drop it.

knobunc · 2017-10-17T16:06:54Z

architecture/topics/haproxy_template_router.adoc

+Prometheus format data is the default data format for metrics in {product-title}.
+
+The metrics are scraped from the the router controller and from HAProxy.


The metrics are from both the router controller, and contain the statistics retrieved from the HAProxy that the router controls.

This whole paragraph feels like it should be combined with the one that is two below.

knobunc · 2017-10-17T16:07:09Z

architecture/topics/haproxy_template_router.adoc

+The metrics are scraped from the the router controller and from HAProxy.
+
+The router scrapes the following HAProxy metrics every 5 seconds. To reduce resource usage when there are


s/scrapes/gathers/

This whole paragraph feels like it should be combined with the one that is two below.

reorganized discussion

knobunc · 2017-10-17T16:08:19Z

architecture/topics/haproxy_template_router.adoc

+more than 500 servers, the backends are reported instead of the servers.
+
+The router metrics start at zero when the router is deployed. The HAProxy metrics are reset to


s/metrics/counters/?

knobunc · 2017-10-17T16:10:33Z

architecture/topics/haproxy_template_router.adoc

+The following HAProxy metrics are scraped on a preiodic basis and converted to Prometheus format.
+For every frontend the "F" counters are scraped. When the total number of servers is less than
+the Server Threshold, the "b" counters are scraped for each backend and the "S" server counters


Server Threshold (default 500)

knobunc · 2017-10-17T16:11:11Z

architecture/topics/haproxy_template_router.adoc

+|===
+
+Column 3 - the counter


Inconsistent capitalization across these descriptions

mburke5678 · 2017-10-17T19:38:41Z

@pecameron Overall, this looks good. A nice addition.

We have some information on HAProxy router and template routers that appears to have been added in 3.6. We could probably combine with your doc.
https://docs.openshift.com/container-platform/3.6/architecture/networking/routes.html#routes-template-routers

I came across this blog from @cpitman about creating a GUI for the HAProxy router metrics (which I haven't gotten to work yet; but I am sure that it is just me). Is this applicable for 3.6 and something we want to advertise in the official docs?
https://docs.openshift.com/container-platform/3.6/architecture/networking/routes.html#routes-template-routers

pecameron · 2017-10-18T18:30:02Z

@gaurav-nelson @mburke5678 @knobunc PTAL

mburke5678 · 2017-10-19T14:41:23Z

install_config/router/default_haproxy_router.adoc

-Or, optionally, use the `--metrics-image` option to override the HAProxy
-defaults:
-+
+metrics are automatically available in prometheus format on the stats-port, default 1936. To suppress metrics collection,


s/prometheus/Prometheus

mburke5678 · 2017-10-19T14:46:36Z

install_config/router/default_haproxy_router.adoc

----
+The `--expose-metrics` and `--metrics-image` options are deprecated. The haproxy-exporter
+side car is now integrated into the router controller so you can delete the sidecar from existing
+router DCs. You can continue to use the side car in existing routers. New routers use the integrated metrics.


Elsewhere we refer to sidecar container and not just sidecar. Is this the same concept? Maybe use sidecar container here?
https://docs.openshift.org/latest/install_config/router/default_haproxy_router.html#exposing-the-router-metrics

mburke5678 · 2017-10-19T14:49:13Z

install_config/router/default_haproxy_router.adoc

-+
+
+
+You can extract the raw statistics in prometheus format by using the following.


s/prometheis/Prometheus

mburke5678 · 2017-10-19T14:53:16Z

architecture/topics/router_environment_variables.adoc

 |`ROUTER_LOG_LEVEL` | warning | The log level to send to the syslog server.
 |`ROUTER_MAX_CONNECTIONS`| 20000 | Maximum number of concurrent connections.
+|`ROUTER_METRICS_TYPE`| haproxy | Generate metrics for xref:../../install_config/router/default_haproxy_router.adoc#exposing-the-router-metrics[the haproxy router]. (haproxy is the only supported value)
 |`ROUTER_OVERRIDE_HOSTNAME`|  | If set `true`, override the spec.host value for a route with the template in `ROUTER_SUBDOMAIN`.


s/haproxy/haproxy

Not sure what you want to change here...

@pecameron Sorry. Maybe put the ` characters around haproxy.

mburke5678 · 2017-10-19T14:54:09Z

@pecameron A couple of comments; I can do a follow-up PR for the edits, if you prefer.

pecameron · 2017-10-19T15:00:48Z

@mburke5678 PTAL

mburke5678 · 2017-11-17T16:24:20Z

architecture/topics/haproxy_template_router.adoc

+|http_request_size_bytes|The HTTP request sizes in bytes.
+|http_response_size_bytes|The HTTP response sizes in bytes.
+|openshift_build_info|A metric with a constant '1' value labeled by major, minor, git commit & git version from which OpenShift was built.


Not sure what "from which OpenShift was built." Which cluster??
If using OpenShift, please use {product-title}

The git source control software has a version too. This just shows it.

mburke5678 · 2017-11-17T16:25:51Z

architecture/topics/haproxy_template_router.adoc

+|openshift_build_info|A metric with a constant '1' value labeled by major, minor, git commit & git version from which OpenShift was built.
+|ssh_tunnel_open_count|Counter of ssh tunnel total open attempts
+|ssh_tunnel_open_fail_count|Counter of ssh tunnel failed open attempts


s/ssh tunnel/SSH tunnel

mburke5678 · 2017-11-17T16:26:43Z

install_config/router/default_haproxy_router.adoc

+are, by default, exposed or published in
+link:https://prometheus.io/docs/concepts/data_model/[Prometheus format]
+for consumption by external metrics collection and aggregation systems (e.g. Prometheus, statsd)..
+Metrics are also available dirctly from the


(e.g. Prometheus, statsd)..
Two periods

mburke5678 · 2017-11-17T16:28:05Z

install_config/router/default_haproxy_router.adoc

----
+The `--expose-metrics` and `--metrics-image` options are deprecated. The haproxy-exporter
+side car is now integrated into the router controller so you can delete the sidecar container from existing
+router DCs. You can continue to use the side car in existing routers. New routers use the integrated metrics.


s/router DCs/router deployment configs

mburke5678 · 2017-11-17T16:39:44Z

@pecameron Text looks good. Did you work with @bfallonf to make sure you edited the right topics files? I am not familiar with how they are mapped into other topics. Or, if they still are.

Documents metrics that are published by the router Document installation and configuration Document related environment variables Trello: https://trello.com/c/EHq8dqNj/529-document-the-metrics-the-router-exposes

pecameron · 2017-12-01T21:03:54Z

@mburke5678 I made the suggested changes. PTAL

pecameron · 2018-01-30T16:57:47Z

@mburke5678 has this information been merged in a different PR?

pecameron · 2018-01-30T16:58:54Z

@mburke5678, @mrobson has been looking for the information in our docs.

mburke5678 · 2018-01-31T01:19:34Z

@pecameron I am not sure if you made the changes in the right .adoc files.

@bfallonf Can you look at what Phil did? I think the files he used are yours??

mburke5678 · 2018-01-31T13:45:29Z

@pecameron
@bfallonf says we are good to merge this. What version(s) of OpenShift does this apply to?

pecameron · 2018-01-31T14:37:34Z

@mburke5678 @bfallonf - 3.6 and newer

…bjects.adoc (cherry picked from commit e8f0ef6) xref:openshift#7009 xref:openshift#5699

Documents metrics that are published by the router Document installation and configuration Document related environment variables Trello: https://trello.com/c/EHq8dqNj/529-document-the-metrics-the-router-exposes (cherry picked from commit 2c23c51) xref:openshift#5699

mburke5678 · 2018-01-31T20:31:11Z

[rev_history]
|xref:../architecture/networking/haproxy-router.adoc#architecture-core-concepts-haproxy-router[Architecture -> HAProxy Router Plug-in]
|New section describing the xref:../architecture/networking/haproxy-router.adoc#haproxy-template-router-metrics[HAProxy Template Router Metrics].
%
|xref:../architecture/networking/routes.adoc#architecture-core-concepts-routes[Architecture -> Routes]
|Added information on the ROUTER_LISTEN_ADDR and ROUTER_METRICS_TYPE variables to the xref:../architecture/networking/routes.adoc#router-environment-variables[Router Environment Variables] section.
%
|xref:../install_config/router/default_haproxy_router.adoc#install-config-router-default-haproxy[Using the Default HAProxy Router]
|Modified some information in the xref:../install_config/router/default_haproxy_router.adoc#exposing-the-router-metrics[Router Environment Variables] section.
%

pecameron force-pushed the metrics branch from a3d353b to 0821b6d Compare October 13, 2017 13:25

knobunc reviewed Oct 17, 2017

View reviewed changes

pecameron force-pushed the metrics branch from 0821b6d to 01b5f53 Compare October 18, 2017 18:06

mburke5678 reviewed Oct 19, 2017

View reviewed changes

pecameron force-pushed the metrics branch 2 times, most recently from 03647bd to 4436175 Compare October 19, 2017 15:00

pecameron force-pushed the metrics branch from 4436175 to a7cd3cf Compare October 19, 2017 15:02

ncbaratta added the peer-review-needed Signifies that the peer review team needs to review this PR label Nov 9, 2017

mburke5678 reviewed Nov 17, 2017

View reviewed changes

Router - document published metrics

2c23c51

Documents metrics that are published by the router Document installation and configuration Document related environment variables Trello: https://trello.com/c/EHq8dqNj/529-document-the-metrics-the-router-exposes

pecameron force-pushed the metrics branch from a7cd3cf to 2c23c51 Compare December 1, 2017 21:03

openshift-ci-robot added the size/L Denotes a PR that changes 100-499 lines, ignoring generated files. label Dec 1, 2017

mburke5678 merged commit dbcbab3 into openshift:master Jan 31, 2018

mburke5678 added branch/enterprise-3.6 branch/enterprise-3.7 branch/enterprise-3.9 branch/online branch/dedicated labels Jan 31, 2018

mburke5678 pushed a commit to mburke5678/openshift-docs that referenced this pull request Jan 31, 2018

[upstream/dedicated-3.7] [dedicated-3.7] Fix tiny typo in other_api_o…

0b97d75

…bjects.adoc (cherry picked from commit e8f0ef6) xref:openshift#7009 xref:openshift#5699

mburke5678 pushed a commit to mburke5678/openshift-docs that referenced this pull request Jan 31, 2018

[dedicated-3.6] fix merge error xref:openshift#5699

ddeb1c5

mburke5678 pushed a commit to mburke5678/openshift-docs that referenced this pull request Jan 31, 2018

[dedicated-3.7] fix merge error xref:openshift#5699

60dacc9

mburke5678 pushed a commit to mburke5678/openshift-docs that referenced this pull request Jan 31, 2018

[dedicated-3.9] fix merge error xref:openshift#5699

ebc25f6

mburke5678 pushed a commit to mburke5678/openshift-docs that referenced this pull request Jan 31, 2018

[online-3.7] fix merge error xref:openshift#5699

d0e1ef8

mburke5678 pushed a commit to mburke5678/openshift-docs that referenced this pull request Jan 31, 2018

[online-3.9] fix merge error xref:openshift#5699

c87172d

		provide no router metrics at all.

		Prometheus format data is the default data format for metrics in {product-title}.

		=== HAProxy Template Router Metrics

		The HAProxy router, by default, exposes or publishes metrics in

		Prometheus format data is the default data format for metrics in {product-title}.

		The metrics are scraped from the the router controller and from HAProxy.

		The metrics are scraped from the the router controller and from HAProxy.

		The router scrapes the following HAProxy metrics every 5 seconds. To reduce resource usage when there are

		more than 500 servers, the backends are reported instead of the servers.

		The router metrics start at zero when the router is deployed. The HAProxy metrics are reset to

		+


		You can extract the raw statistics in prometheus format by using the following.

		\|===

		Column 3 - the counter

Router - document published metrics #5699

Router - document published metrics #5699

Uh oh!

Conversation

pecameron commented Oct 13, 2017

Uh oh!

pecameron commented Oct 13, 2017

Uh oh!

pecameron commented Oct 13, 2017

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mburke5678 commented Oct 17, 2017

Uh oh!

pecameron commented Oct 18, 2017

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mburke5678 commented Oct 19, 2017

Uh oh!

pecameron commented Oct 19, 2017

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!