Skip to content

Commit a2da26d

Browse files
committed
data collection article rename
1 parent 8ad3e3d commit a2da26d

33 files changed

+143
-137
lines changed

.openpublishing.redirection.azure-monitor.json

+5
Original file line numberDiff line numberDiff line change
@@ -7740,6 +7740,11 @@
77407740
"redirect_url": "/azure/azure-monitor/reference",
77417741
"redirect_document_id": false
77427742
},
7743+
{
7744+
"source_path": "articles/azure-monitor/vm/data-collection.md",
7745+
"redirect_url": "/azure/azure-monitor/vm/data-collection",
7746+
"redirect_document_id": false
7747+
},
77437748
{
77447749
"source_path": "articles/azure-monitor/agents/data-collection-windows-events.md",
77457750
"redirect_url": "/azure/azure-monitor/vm/data-collection-windows-events",

articles/azure-monitor/agents/azure-monitor-agent-extension-versions.md

+1-1
Original file line numberDiff line numberDiff line change
@@ -75,4 +75,4 @@ We strongly recommend that you always update to the latest version, or opt in to
7575
## Next steps
7676

7777
- [Install and manage the extension](./azure-monitor-agent-manage.md).
78-
- [Create a data collection rule](./azure-monitor-agent-data-collection.md) to collect data from the agent and send it to Azure Monitor.
78+
- [Create a data collection rule](../vm/data-collection.md) to collect data from the agent and send it to Azure Monitor.

articles/azure-monitor/agents/azure-monitor-agent-manage.md

+3-3
Original file line numberDiff line numberDiff line change
@@ -32,7 +32,7 @@ For any machine that isn't in Azure, the [Azure Arc agent](/azure/azure-arc/serv
3232
| Installation method | Description |
3333
|:---|:---|
3434
| Virtual machine (VM) extension | Use any of the methods described in this article to install the agent via the Azure extension framework. This method doesn't create a DCR, so you must create at least one DCR and associate it with the agent before data collection begins. |
35-
| [Create a DCR](./azure-monitor-agent-data-collection.md) | When you create a DCR in the Azure portal, the Azure Monitor Agent is installed on any machine that's added as a resource for the DCR. The agent immediately begins to collect data as defined in the DCR.
35+
| [Create a DCR](../vm/data-collection.md) | When you create a DCR in the Azure portal, the Azure Monitor Agent is installed on any machine that's added as a resource for the DCR. The agent immediately begins to collect data as defined in the DCR.
3636
| [VM insights](../vm/vminsights-enable-overview.md) | When you enable VM insights on a machine, the Azure Monitor Agent is installed and a DCR is created to collect a predefined set of data. You shouldn't modify this DCR, but you can create more DCRs to collect more data. |
3737
| [Container insights](../containers/kubernetes-monitoring-enable.md#container-insights) | When you enable Container insights on a Kubernetes cluster, a containerized version of the Azure Monitor Agent is installed in the cluster and a DCR is created to immediately begin collecting data. You can modify the DCR by using the guidance in [Configure data collection and cost optimization in Container insights by using DCRs](../containers/container-insights-data-collection-dcr.md).
3838
| [Client installer](./azure-monitor-agent-windows-client.md) | Install the agent by using a Windows MSI installer for Windows 11 and Windows 10 clients. |
@@ -48,7 +48,7 @@ For any machine that isn't in Azure, the [Azure Arc agent](/azure/azure-arc/serv
4848

4949
### [Azure portal](#tab/azure-portal)
5050

51-
To install the agent and create a DCR to collect data by using the Azure portal, use the guidance in [Collect data by using the Azure Monitor Agent](./azure-monitor-agent-data-collection.md).
51+
To install the agent and create a DCR to collect data by using the Azure portal, use the guidance in [Collect data by using the Azure Monitor Agent](../vm/data-collection.md).
5252

5353
### [Azure PowerShell](#tab/azure-powershell)
5454

@@ -530,4 +530,4 @@ Currently not supported.
530530
531531
## Related content
532532
533-
[Create a data collection rule](./azure-monitor-agent-data-collection.md) to collect data from the agent and send it to Azure Monitor.
533+
[Create a data collection rule](../vm/data-collection.md) to collect data from the agent and send it to Azure Monitor.

articles/azure-monitor/agents/azure-monitor-agent-overview.md

+2-2
Original file line numberDiff line numberDiff line change
@@ -103,9 +103,9 @@ The following tables identify the different environments and features that are c
103103

104104
## Supported data sources
105105

106-
For a list of data sources the Azure Monitor Agent can collect and to learn how to configure them, see [Collect data with the Azure Monitor Agent](./azure-monitor-agent-data-collection.md).
106+
For a list of data sources the Azure Monitor Agent can collect and to learn how to configure them, see [Collect data with the Azure Monitor Agent](../vm/data-collection.md).
107107

108108
## Related content
109109

110110
- [Install the Azure Monitor Agent](azure-monitor-agent-manage.md) on Windows and Linux virtual machines.
111-
- [Create a data collection rule](./azure-monitor-agent-data-collection.md) to collect data from the agent and send it to Azure Monitor.
111+
- [Create a data collection rule](../vm/data-collection.md) to collect data from the agent and send it to Azure Monitor.
Original file line numberDiff line numberDiff line change
@@ -1,83 +1,84 @@
1-
---
2-
title: Performance in Forwarding to Azure Monitor Agent
3-
description: Learn about performance benchmark data for the Azure Monitor Agent running in a gateway event forwarding scenario.
4-
ms.topic: conceptual
5-
author: guywi-ms
6-
ms.author: guywild
7-
ms.date: 11/14/2024
8-
ms.reviewer: jeffwo
9-
10-
# Customer intent: As a deployment engineer, I can scope the resources required to scale my gateway data colletors the use the Azure Monitor Agent.
11-
12-
---
13-
# Azure Monitor Agent performance benchmark
14-
15-
The Azure Monitor Agent can handle many thousands of events per second (EPS) in the gateway event forwarding scenario. The exact throughput rate depends on various factors like the size of each event, the specific data type, and physical hardware resources.
16-
17-
This article describes the Microsoft internal benchmark that's used for testing the agent throughput of 10,000 (10K) syslog events in the gateway forwarder scenario. The benchmark results should provide a guide to size the resources that you need in your environment.
18-
19-
> [!NOTE]
20-
> The results in this article are only informational related to the performance of Azure Monitor Agent in a gateway forwarding scenario. The results and the information in the article don't constitute any service agreement on the part of Microsoft.
21-
22-
## Best practices for agent as a forwarder
23-
24-
- The Linux Azure Monitor Agent should target 10K EPS. A 20K EPS warning might occur, but it doesn't mean that data is lost. The Azure Monitor Agent doesn't guarantee a lossless connection. Loss is more likely when EPS is over 10K.
25-
- The forwarder should be on a dedicated system to eliminate potential interference from other workloads.
26-
- The forwarder system should be monitored for CPU, memory, and disk utilization to prevent overloads from causing data loss.
27-
- The load balancer and redundant forwarder systems should be used to improve reliability and scalability. For other considerations for forwarders, see the Log Analytics gateway documentation.
28-
29-
## Agent performance
30-
31-
The benchmark is run in a controlled environment to get repeatable, accurate, and statistically significant results. The resources consumed by the agent are measured under a load of 10,000 simulated syslog events per second. The simulated load is run on the same physical hardware that the agent being tested is on. Test trials run for seven days. For each trial, performance metrics are sampled every second to collect CPU, memory, and network maximum and average usage. This approach provides the right information to help you estimate the resources needed for your environment.
32-
33-
> [!NOTE]
34-
> Performance testing results don't measure the end-to-end throughput ingested by a Log Analytics workspace (or other telemetry sinks). End-to-end variability might occur due to network and back-end pipeline performance.
35-
36-
The benchmarks are run on an Azure virtual machine Standard_F8s_v2 system using Azure Monitor Agent Linux version 1.25.2 and 10 GB of disk space for the event cache.
37-
38-
- vCPUs: Eight with Hyper-Threading (800% CPU is possible)
39-
- Memory: 16 GiB
40-
- Temp storage: 64 GiB
41-
- Max disk IOPS: 6,400
42-
- Network: 12,500 Mbps max on all four physical NICs
43-
44-
## Results
45-
46-
| Perf Metric | Ave (Max) Med |
47-
|:---|:---:|
48-
| **CPU %** | 51 (262) |
49-
| **Memory RSS MB** | 276 (1,017) |
50-
| **Network KBps** | 338 (18,033) |
51-
52-
## FAQs
53-
54-
Get answers to common questions.
55-
56-
### How much data is sent per agent?
57-
58-
The amount of data sent per agent depends on:
59-
60-
- The solutions you enabled
61-
- The number of logs and performance counters that are collected
62-
- The volume of data in the logs
63-
64-
For more information, see [Analyze usage in a Log Analytics workspace](../logs/analyze-usage.md).
65-
66-
For computers that are able to run the WireData Agent, use the following query to see how much data is sent:
67-
68-
```kusto
69-
WireData
70-
| where ProcessName == "C:\\Program Files\\Microsoft Monitoring Agent\\Agent\\MonitoringHost.exe"
71-
| where Direction == "Outbound"
72-
| summarize sum(TotalBytes) by Computer
73-
```
74-
75-
### How much network bandwidth is used by the Microsoft Monitoring Agent when it sends data to Azure Monitor?
76-
77-
Bandwidth is a function of the amount of data that's sent. Data is compressed as it's sent over the network.
78-
79-
## Related content
80-
81-
- [Use the Log Analytics gateway in Azure Monitor](gateway.md) to connect computers without internet access.
82-
- [Install the Azure Monitor Agent](azure-monitor-agent-manage.md) on Windows and Linux virtual machines.
83-
- [Create a data collection rule](azure-monitor-agent-data-collection.md) to collect data from the agent and send it to Azure Monitor.
1+
---
2+
title: Performance in Forwarding to Azure Monitor Agent
3+
description: Learn about performance benchmark data for the Azure Monitor Agent running in a gateway event forwarding scenario.
4+
ms.topic: conceptual
5+
author: guywi-ms
6+
ms.author: guywild
7+
ms.date: 11/14/2024
8+
ms.reviewer: jeffwo
9+
10+
# Customer intent: As a deployment engineer, I can scope the resources required to scale my gateway data colletors the use the Azure Monitor Agent.
11+
12+
---
13+
# Azure Monitor Agent performance benchmark
14+
15+
The Azure Monitor Agent can handle many thousands of events per second (EPS) in the gateway event forwarding scenario. The exact throughput rate depends on various factors like the size of each event, the specific data type, and physical hardware resources.
16+
17+
This article describes the Microsoft internal benchmark that's used for testing the agent throughput of 10,000 (10K) syslog events in the gateway forwarder scenario. The benchmark results should provide a guide to size the resources that you need in your environment.
18+
19+
20+
> [!NOTE]
21+
> The results in this article are only informational related to the performance of Azure Monitor Agent in a gateway forwarding scenario. The results and the information in the article don't constitute any service agreement on the part of Microsoft.
22+
23+
## Best practices for agent as a forwarder
24+
25+
- The Linux Azure Monitor Agent should target 10K EPS. A 20K EPS warning might occur, but it doesn't mean that data is lost. The Azure Monitor Agent doesn't guarantee a lossless connection. Loss is more likely when EPS is over 10K.
26+
- The forwarder should be on a dedicated system to eliminate potential interference from other workloads.
27+
- The forwarder system should be monitored for CPU, memory, and disk utilization to prevent overloads from causing data loss.
28+
- The load balancer and redundant forwarder systems should be used to improve reliability and scalability. For other considerations for forwarders, see the Log Analytics gateway documentation.
29+
30+
## Agent performance
31+
32+
The benchmark is run in a controlled environment to get repeatable, accurate, and statistically significant results. The resources consumed by the agent are measured under a load of 10,000 simulated syslog events per second. The simulated load is run on the same physical hardware that the agent being tested is on. Test trials run for seven days. For each trial, performance metrics are sampled every second to collect CPU, memory, and network maximum and average usage. This approach provides the right information to help you estimate the resources needed for your environment.
33+
34+
> [!NOTE]
35+
> Performance testing results don't measure the end-to-end throughput ingested by a Log Analytics workspace (or other telemetry sinks). End-to-end variability might occur due to network and back-end pipeline performance.
36+
37+
The benchmarks are run on an Azure virtual machine Standard_F8s_v2 system using Azure Monitor Agent Linux version 1.25.2 and 10 GB of disk space for the event cache.
38+
39+
- vCPUs: Eight with Hyper-Threading (800% CPU is possible)
40+
- Memory: 16 GiB
41+
- Temp storage: 64 GiB
42+
- Max disk IOPS: 6,400
43+
- Network: 12,500 Mbps max on all four physical NICs
44+
45+
## Results
46+
47+
| Perf Metric | Ave (Max) Med |
48+
|:---|:---:|
49+
| **CPU %** | 51 (262) |
50+
| **Memory RSS MB** | 276 (1,017) |
51+
| **Network KBps** | 338 (18,033) |
52+
53+
## FAQs
54+
55+
Get answers to common questions.
56+
57+
### How much data is sent per agent?
58+
59+
The amount of data sent per agent depends on:
60+
61+
- The solutions you enabled
62+
- The number of logs and performance counters that are collected
63+
- The volume of data in the logs
64+
65+
For more information, see [Analyze usage in a Log Analytics workspace](../logs/analyze-usage.md).
66+
67+
For computers that are able to run the WireData Agent, use the following query to see how much data is sent:
68+
69+
```kusto
70+
WireData
71+
| where ProcessName == "C:\\Program Files\\Microsoft Monitoring Agent\\Agent\\MonitoringHost.exe"
72+
| where Direction == "Outbound"
73+
| summarize sum(TotalBytes) by Computer
74+
```
75+
76+
### How much network bandwidth is used by the Microsoft Monitoring Agent when it sends data to Azure Monitor?
77+
78+
Bandwidth is a function of the amount of data that's sent. Data is compressed as it's sent over the network.
79+
80+
## Related content
81+
82+
- [Use the Log Analytics gateway in Azure Monitor](gateway.md) to connect computers without internet access.
83+
- [Install the Azure Monitor Agent../vm/data-collection.mdndows and Linux virtual machines.
84+
- [Create a data collection rule](azure-monitor-agent-data-collection.md) to collect data from the agent and send it to Azure Monitor.

articles/azure-monitor/agents/azure-monitor-agent-private-link.md

+1-1
Original file line numberDiff line numberDiff line change
@@ -35,7 +35,7 @@ By default, Azure Monitor Agent connects to a public endpoint to connect to your
3535

3636
## Associate DCEs to target resources
3737

38-
Associate the DCEs to the target resources by editing the DCR in the Azure portal. On the **Resources** tab, select **Enable Data Collection Endpoints**. Select a DCE for each virtual machine. For more information, see [Configure data collection for the Azure Monitor Agent](../agents/azure-monitor-agent-data-collection.md).
38+
Associate the DCEs to the target resources by editing the DCR in the Azure portal. On the **Resources** tab, select **Enable Data Collection Endpoints**. Select a DCE for each virtual machine. For more information, see [Configure data collection for the Azure Monitor Agent](../vm/data-collection.md).
3939

4040
:::image type="content" source="media/azure-monitor-agent-dce/data-collection-rule-virtual-machines-with-endpoint.png" lightbox="media/azure-monitor-agent-dce/data-collection-rule-virtual-machines-with-endpoint.png" alt-text="Screenshot that shows configuring data collection endpoints for an agent.":::
4141

articles/azure-monitor/agents/azure-monitor-agent-requirements.md

+1-1
Original file line numberDiff line numberDiff line change
@@ -79,4 +79,4 @@ The Azure Monitor Agent does not work on Linux virtual machines when the systemw
7979

8080
## Related content
8181

82-
- [Create a data collection rule](azure-monitor-agent-data-collection.md) to collect data from the agent and send it to Azure Monitor.
82+
- [Create a data collection rule](../vm/data-collection.md) to collect data from the agent and send it to Azure Monitor.

articles/azure-monitor/agents/azure-monitor-agent-send-data-to-event-hubs-and-storage.md

+1-1
Original file line numberDiff line numberDiff line change
@@ -750,4 +750,4 @@ WAD and LAD will only be getting security/patches going forward. Most engineerin
750750

751751
## See also
752752

753-
- For more information on creating a data collection rule, see [Collect data from virtual machines using Azure Monitor Agent](./azure-monitor-agent-data-collection.md).
753+
- For more information on creating a data collection rule, see [Collect data from virtual machines using Azure Monitor Agent](../vm/data-collection.md).

articles/azure-monitor/agents/azure-monitor-agent-troubleshoot-windows-arc.md

+2-2
Original file line numberDiff line numberDiff line change
@@ -53,7 +53,7 @@ Follow the steps below to troubleshoot the latest version of the Azure Monitor a
5353
1. Check if you see the latest DCR downloaded at this location `C:\Resources\Directory\AMADataStore\mcs\configchunks`
5454
5555
## Issues collecting Performance counters
56-
1. Check that your DCR JSON contains a section for 'performanceCounters'. If not, fix your DCR. See [how to create DCR](./azure-monitor-agent-data-collection.md) or [sample DCR](./data-collection-rule-sample-agent.md).
56+
1. Check that your DCR JSON contains a section for 'performanceCounters'. If not, fix your DCR. See [how to create DCR](../vm/data-collection.md) or [sample DCR](./data-collection-rule-sample-agent.md).
5757
2. Check that the file `C:\Resources\Directory\AMADataStore\mcs\mcsconfig.lkg.xml` exists.
5858
3. Open the file and check if it contains `CounterSet` nodes as shown in the example below:
5959
```xml
@@ -92,7 +92,7 @@ Follow the steps below to troubleshoot the latest version of the Azure Monitor a
9292
2. Open it and look for any Level 2 errors and try to fix them.
9393
9494
## Issues collecting Windows event logs
95-
1. Check that your DCR JSON contains a section for 'windowsEventLogs'. If not, fix your DCR. See [how to create DCR](./azure-monitor-agent-data-collection.md) or [sample DCR](./data-collection-rule-sample-agent.md).
95+
1. Check that your DCR JSON contains a section for 'windowsEventLogs'. If not, fix your DCR. See [how to create DCR](../vm/data-collection.md) or [sample DCR](./data-collection-rule-sample-agent.md).
9696
2. Check that the file `C:\Resources\Directory\AMADataStore\mcs\mcsconfig.lkg.xml` exists.
9797
3. Open the file and check if it contains `Subscription` nodes as shown in the example below:
9898
```xml

articles/azure-monitor/agents/azure-monitor-agent-troubleshoot-windows-vm.md

+2-2
Original file line numberDiff line numberDiff line change
@@ -47,7 +47,7 @@ Follow the steps below to troubleshoot the latest version of the Azure Monitor a
4747
1. Check if you see the latest DCR downloaded at this location `C:\WindowsAzure\Resources\AMADataStore.<virtual-machine-name>\mcs\configchunks`
4848
4949
## Issues collecting Performance counters
50-
1. Check that your DCR JSON contains a section for 'performanceCounters'. If not, fix your DCR. See [how to create DCR](./azure-monitor-agent-data-collection.md) or [sample DCR](./data-collection-rule-sample-agent.md).
50+
1. Check that your DCR JSON contains a section for 'performanceCounters'. If not, fix your DCR. See [how to create DCR](../vm/data-collection.md) or [sample DCR](./data-collection-rule-sample-agent.md).
5151
2. Check that the file `C:\WindowsAzure\Resources\AMADataStore.<virtual-machine-name>\mcs\mcsconfig.lkg.xml` exists.
5252
3. Open the file and check if it contains `CounterSet` nodes as shown in the example below:
5353
```xml
@@ -84,7 +84,7 @@ Follow the steps below to troubleshoot the latest version of the Azure Monitor a
8484
2. Open it and look for any Level 2 errors and try to fix them.
8585
8686
## Issues collecting Windows event logs
87-
1. Check that your DCR JSON contains a section for 'windowsEventLogs'. If not, fix your DCR. See [how to create DCR](./azure-monitor-agent-data-collection.md) or [sample DCR](./data-collection-rule-sample-agent.md).
87+
1. Check that your DCR JSON contains a section for 'windowsEventLogs'. If not, fix your DCR. See [how to create DCR](../vm/data-collection.md) or [sample DCR](./data-collection-rule-sample-agent.md).
8888
2. Check that the file `C:\WindowsAzure\Resources\AMADataStore.<virtual-machine-name>\mcs\mcsconfig.lkg.xml` exists.
8989
3. Open the file and check if it contains `Subscription` nodes as shown in the example below:
9090
```xml

articles/azure-monitor/agents/azure-monitor-agent-windows-client.md

+1-1
Original file line numberDiff line numberDiff line change
@@ -63,7 +63,7 @@ Here's a comparison between using the client installer and using the virtual mac
6363
(Example: `12345a01-b1cd-1234-e1f2-1234567g8h99.ods.opinsights.azure.com`)
6464

6565
If you use private links on the agent, you must also add the [data collection endpoints](../essentials/data-collection-endpoint-overview.md#components-of-a-dce).
66-
- A DCR that you want to associate with the devices. If it doesn't exist already, [create a data collection rule](./azure-monitor-agent-data-collection.md). *Don't associate the rule to any resources yet*.
66+
- A DCR that you want to associate with the devices. If it doesn't exist already, [create a data collection rule](../vm/data-collection.md). *Don't associate the rule to any resources yet*.
6767
- Before you use any PowerShell cmdlet, ensure that the cmdlet-related PowerShell module is installed and imported.
6868

6969
## Limitations

0 commit comments

Comments
 (0)