-
Notifications
You must be signed in to change notification settings - Fork 3k
Loadtestservice - SDK initial review #25264
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from 120 commits
507ae4d
9b6061c
74eb518
2d8eafd
33c26d6
a91c728
9af9160
081fd12
fb9cb76
11d1531
2be648f
1d98c14
6881429
b42ef95
3dea6da
0d1dbf3
07abc8e
e43a9fb
4e2c523
5e87691
7a6a769
10fe0aa
5ac02da
806badd
3e3b655
015cf2e
1204f67
8e27ba7
d577901
5e08cc1
df3e5b1
b95aa58
ed84eb0
5959144
8703e03
0b6b1a5
592718d
deb0b9c
6c6110e
2028ca3
742cc82
62eb9f1
ea5e975
3edb300
e8b0cff
85a7208
d1d4d9e
4047427
08b2b8a
2649f5c
c1696e7
6d18d56
b341212
b49be9b
2cd4310
277b22d
29b3cb1
ff45aee
70ddd05
3f39ea4
05e96ef
08e481a
be1061f
80e09ca
f68724d
a384829
50cfcf0
c8563b0
58e4964
924ccc7
37abf4e
714f5e7
9ffe28b
d2254d7
568cc65
deced9b
f29ccdd
7dc92f0
2207eae
26758c4
b4c17c9
78cadd6
e6aa88a
f8f7cf1
573bf9e
e1e1153
e053006
e723a75
4c398d9
4a6d627
d989897
faddc05
c558817
fe9a681
3307408
eac412d
ce66a8a
cf979da
f44a41b
2e6b81f
f3039cb
bbdef79
7743fb3
c26cd14
20e0cc2
a162887
7b66a7f
0c9ff64
a4564ac
919e95f
2cc19b9
e509d21
d39dc05
e1e6b5d
1ef6f32
3819643
ea024ac
5ae8a12
59dfeb8
0493ca9
1c9c4e2
9c2de2f
084f491
23d98fe
032e780
884c6ac
4f5a9a1
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,5 @@ | ||
# Release History | ||
|
||
## 1.0.0b1 (2022-07-28) | ||
|
||
- Initial version |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,21 @@ | ||
Copyright (c) Microsoft Corporation. | ||
|
||
MIT License | ||
|
||
Permission is hereby granted, free of charge, to any person obtaining a copy | ||
of this software and associated documentation files (the "Software"), to deal | ||
in the Software without restriction, including without limitation the rights | ||
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell | ||
copies of the Software, and to permit persons to whom the Software is | ||
furnished to do so, subject to the following conditions: | ||
|
||
The above copyright notice and this permission notice shall be included in all | ||
copies or substantial portions of the Software. | ||
|
||
THE SOFTWARE IS PROVIDED *AS IS*, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | ||
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | ||
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE | ||
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER | ||
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, | ||
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE | ||
SOFTWARE. |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,7 @@ | ||
include *.md | ||
include LICENSE | ||
include azure/developer/loadtesting/py.typed | ||
recursive-include tests *.py | ||
recursive-include samples *.py *.md | ||
include azure/__init__.py | ||
include azure/developer/__init__.py |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,180 @@ | ||
|
||
# Azure Developer LoadTesting client library for Python | ||
Azure Developer LoadTesting provides client library in python to the user by which they can interact natively with Load Test Service.The service is for performing the load test to optimize application performance, scalability or capacity. The user can get the client-side and server-side metrices to see the details reported by the test engine and information about your Azure application components. | ||
|
||
|
||
## Getting started | ||
|
||
### Installating the package | ||
|
||
```bash | ||
python -m pip install azure-developer-loadtesting | ||
``` | ||
|
||
#### Prequisites | ||
|
||
- Python 3.7 or later is required to use this package. | ||
- You need an [Azure subscription][azure_sub] to use this package. | ||
- An existing Azure Developer LoadTesting instance. | ||
#### Create with an Azure Active Directory Credential | ||
To use an [Azure Active Directory (AAD) token credential][authenticate_with_token], | ||
provide an instance of the desired credential type obtained from the | ||
[azure-identity][azure_identity_credentials] library. | ||
|
||
To authenticate with AAD, you must first [pip][pip] install [`azure-identity`][azure_identity_pip] | ||
|
||
After setup, you can choose which type of [credential][azure_identity_credentials] from azure.identity to use. | ||
As an example, [DefaultAzureCredential][default_azure_credential] can be used to authenticate the client: | ||
|
||
Set the values of the client ID, tenant ID, and client secret of the AAD application as environment variables: | ||
`AZURE_CLIENT_ID`, `AZURE_TENANT_ID`, `AZURE_CLIENT_SECRET` | ||
|
||
Use the returned token credential to authenticate the client: | ||
|
||
```python | ||
from azure.developer.loadtesting import LoadTestClient | ||
from azure.identity import DefaultAzureCredential | ||
client = LoadTestClient(endpoint='<endpoint>', credential=DefaultAzureCredential()) | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Should be LoadTestingClient There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. fixed in 084f491 |
||
``` | ||
|
||
## Examples | ||
|
||
### Creating a load test | ||
```python | ||
from azure.developer.loadtesting import LoadTestClient | ||
from azure.identity import DefaultAzureCredential | ||
from azure.core.exceptions import HttpResponseError | ||
|
||
TEST_ID = "some-test-id" | ||
DISPLAY_NAME = "new_namespace-new-namespace" | ||
SUBSCRIPTION_ID = os.environ["SUBSCRIPTION_ID"] | ||
|
||
client = LoadTestClient(endpoint='<endpoint>', credential=DefaultAzureCredential()) | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Should be LoadTestingClient There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. fixed in 084f491 |
||
try: | ||
result = client.load_test_administration.create_or_update_test( | ||
TEST_ID, | ||
{ | ||
"resourceId": f"/subscriptions/{SUBSCRIPTION_ID}/resourceGroups/yashika-rg/providers/Microsoft.LoadTestService/loadtests/loadtestsdk", | ||
"description": "", | ||
"displayName": DISPLAY_NAME, | ||
"loadTestConfig": { | ||
"engineSize": "m", | ||
"engineInstances": 1, | ||
"splitAllCSVs": False, | ||
}, | ||
"secrets": {}, | ||
"environmentVariables": {}, | ||
"passFailCriteria": {"passFailMetrics": {}}, | ||
"keyvaultReferenceIdentityType": "SystemAssigned", | ||
"keyvaultReferenceIdentityId": None, | ||
}, | ||
) | ||
print(result) | ||
except HttpResponseError as e: | ||
print('service responds error: {}'.format(e.response.json())) | ||
|
||
``` | ||
|
||
### Uploading .jmx file to a Test | ||
```python | ||
from azure.developer.loadtesting import LoadTestClient | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Should be LoadTestingClient There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. fixed in 084f491 |
||
from azure.identity import DefaultAzureCredential | ||
from azure.core.exceptions import HttpResponseError | ||
|
||
TEST_ID = "some-test-id" | ||
FILE_ID = "some-file-id" | ||
|
||
try: | ||
# opening .jmx file | ||
body = {} | ||
body["file"] = open("sample.jmx", "rb") | ||
|
||
result = client.load_test_administration.upload_test_file(TEST_ID, FILE_ID, body) | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. client has not been create in that sample There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. fixed in, 084f491 |
||
print(result) | ||
except HttpResponseError as e: | ||
print("Failed to send JSON message: {}".format(e.response.json())) | ||
``` | ||
|
||
### Running a Test | ||
```python | ||
from azure.developer.loadtesting import LoadTestClient | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Should be LoadTestingClient There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. fixed in 084f491 |
||
from azure.identity import DefaultAzureCredential | ||
from azure.core.exceptions import HttpResponseError | ||
|
||
TEST_ID = "some-test-id" | ||
TEST_RUN_ID = "some-testrun-id" | ||
DISPLAY_NAME = "new_namespace-new-namespace" | ||
|
||
try: | ||
result = client.load_test_runs.create_and_update_test( | ||
lmazuel marked this conversation as resolved.
Show resolved
Hide resolved
|
||
TEST_RUN_ID, | ||
{ | ||
"testId": TEST_ID, | ||
"displayName": DISPLAY_NAME, | ||
"requestSamplers": [], | ||
"errors": [], | ||
"percentiles": ["90"], | ||
"groupByInterval": "5s", | ||
}, | ||
) | ||
print(result) | ||
except HttpResponseError as e: | ||
print("Failed to send JSON message: {}".format(e.response.json())) | ||
``` | ||
## Key concepts | ||
The following components make up the Azure Load Testing Service. The Azure Load Test client library for Python allows you to interact with each of these components through the use of a dedicated client object. | ||
|
||
### Load testing resource | ||
The Load testing resource is the top-level resource for your load-testing activities. This resource provides a centralized place to view and manage load tests, test results, and related artifacts. A load testing resource contains zero or more load tests. | ||
|
||
### Test | ||
A test specifies the test script, and configuration settings for running a load test. You can create one or more tests in an Azure Load Testing resource. | ||
|
||
### Test Engine | ||
A test engine is computing infrastructure that runs the Apache JMeter test script. You can scale out your load test by configuring the number of test engines. The test script runs in parallel across the specified number of test engines. | ||
|
||
### Test Run | ||
A test run represents one execution of a load test. It collects the logs associated with running the Apache JMeter script, the load test YAML configuration, the list of app components to monitor, and the results of the test. | ||
|
||
### App Component | ||
When you run a load test for an Azure-hosted application, you can monitor resource metrics for the different Azure application components (server-side metrics). While the load test runs, and after completion of the test, you can monitor and analyze the resource metrics in the Azure Load Testing dashboard. | ||
|
||
### Metrics | ||
During a load test, Azure Load Testing collects metrics about the test execution. There are two types of metrics: | ||
|
||
1. Client-side metrics give you details reported by the test engine. These metrics include the number of virtual users, the request response time, the number of failed requests, or the number of requests per second. | ||
|
||
2. Server-side metrics are available for Azure-hosted applications and provide information about your Azure application components. Metrics can be for the number of database reads, the type of HTTP responses, or container resource consumption. | ||
|
||
## Troubleshooting | ||
More about it is coming soon... | ||
|
||
## Next steps | ||
|
||
More examples are coming soon... | ||
|
||
## Contributing | ||
|
||
This project welcomes contributions and suggestions. Most contributions require | ||
you to agree to a Contributor License Agreement (CLA) declaring that you have | ||
the right to, and actually do, grant us the rights to use your contribution. | ||
For details, visit https://cla.microsoft.com. | ||
|
||
When you submit a pull request, a CLA-bot will automatically determine whether | ||
you need to provide a CLA and decorate the PR appropriately (e.g., label, | ||
comment). Simply follow the instructions provided by the bot. You will only | ||
need to do this once across all repos using our CLA. | ||
|
||
This project has adopted the | ||
[Microsoft Open Source Code of Conduct][code_of_conduct]. For more information, | ||
see the Code of Conduct FAQ or contact [email protected] with any | ||
additional questions or comments. | ||
|
||
<!-- LINKS --> | ||
[code_of_conduct]: https://opensource.microsoft.com/codeofconduct/ | ||
[authenticate_with_token]: https://docs.microsoft.com/azure/cognitive-services/authentication?tabs=powershell#authenticate-with-an-authentication-token | ||
[azure_identity_credentials]: https://github.com/Azure/azure-sdk-for-python/tree/main/sdk/identity/azure-identity#credentials | ||
[azure_identity_pip]: https://pypi.org/project/azure-identity/ | ||
[default_azure_credential]: https://github.com/Azure/azure-sdk-for-python/tree/main/sdk/identity/azure-identity#defaultazurecredential | ||
[pip]: https://pypi.org/project/pip/ | ||
[azure_sub]: https://azure.microsoft.com/free/ |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1 @@ | ||
__path__ = __import__("pkgutil").extend_path(__path__, __name__) # type: ignore |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1 @@ | ||
__path__ = __import__("pkgutil").extend_path(__path__, __name__) # type: ignore |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,24 @@ | ||
# coding=utf-8 | ||
# -------------------------------------------------------------------------- | ||
# Copyright (c) Microsoft Corporation. All rights reserved. | ||
# Licensed under the MIT License. See License.txt in the project root for license information. | ||
# Code generated by Microsoft (R) AutoRest Code Generator. | ||
# Changes may cause incorrect behavior and will be lost if the code is regenerated. | ||
# -------------------------------------------------------------------------- | ||
|
||
from ._client import LoadTestingClient | ||
from ._version import VERSION | ||
|
||
__version__ = VERSION | ||
|
||
try: | ||
from ._patch import __all__ as _patch_all | ||
from ._patch import * # type: ignore # pylint: disable=unused-wildcard-import | ||
except ImportError: | ||
_patch_all = [] | ||
from ._patch import patch_sdk as _patch_sdk | ||
|
||
__all__ = ["LoadTestingClient"] | ||
__all__.extend([p for p in _patch_all if p not in __all__]) | ||
|
||
_patch_sdk() |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,98 @@ | ||
# coding=utf-8 | ||
# -------------------------------------------------------------------------- | ||
# Copyright (c) Microsoft Corporation. All rights reserved. | ||
# Licensed under the MIT License. See License.txt in the project root for license information. | ||
# Code generated by Microsoft (R) AutoRest Code Generator. | ||
# Changes may cause incorrect behavior and will be lost if the code is regenerated. | ||
# -------------------------------------------------------------------------- | ||
|
||
from copy import deepcopy | ||
from typing import Any, TYPE_CHECKING | ||
|
||
from azure.core import PipelineClient | ||
from azure.core.rest import HttpRequest, HttpResponse | ||
|
||
from ._configuration import LoadTestingClientConfiguration | ||
from ._serialization import Deserializer, Serializer | ||
from .operations import AppComponentOperations, ServerMetricsOperations, TestOperations, TestRunOperations | ||
|
||
if TYPE_CHECKING: | ||
# pylint: disable=unused-import,ungrouped-imports | ||
from typing import Dict | ||
|
||
from azure.core.credentials import TokenCredential | ||
|
||
|
||
class LoadTestingClient: # pylint: disable=client-accepts-api-version-keyword | ||
"""These APIs allow end users to create, view and run load tests using Azure Load Test Service. | ||
|
||
:ivar app_component: AppComponentOperations operations | ||
:vartype app_component: azure.developer.loadtesting.operations.AppComponentOperations | ||
:ivar server_metrics: ServerMetricsOperations operations | ||
:vartype server_metrics: azure.developer.loadtesting.operations.ServerMetricsOperations | ||
:ivar test: TestOperations operations | ||
:vartype test: azure.developer.loadtesting.operations.TestOperations | ||
:ivar test_run: TestRunOperations operations | ||
:vartype test_run: azure.developer.loadtesting.operations.TestRunOperations | ||
:param endpoint: URL to perform data plane API operations on the resource. Required. | ||
:type endpoint: str | ||
:param credential: Credential needed for the client to connect to Azure. Required. | ||
:type credential: ~azure.core.credentials.TokenCredential | ||
:keyword api_version: Api Version. Default value is "2022-06-01-preview". Note that overriding | ||
this default value may result in unsupported behavior. | ||
:paramtype api_version: str | ||
""" | ||
|
||
def __init__(self, endpoint: str, credential: "TokenCredential", **kwargs: Any) -> None: | ||
_endpoint = "https://{Endpoint}" | ||
self._config = LoadTestingClientConfiguration(endpoint=endpoint, credential=credential, **kwargs) | ||
self._client = PipelineClient(base_url=_endpoint, config=self._config, **kwargs) | ||
|
||
self._serialize = Serializer() | ||
self._deserialize = Deserializer() | ||
self._serialize.client_side_validation = False | ||
self.app_component = AppComponentOperations(self._client, self._config, self._serialize, self._deserialize) | ||
self.server_metrics = ServerMetricsOperations(self._client, self._config, self._serialize, self._deserialize) | ||
self.test = TestOperations( # type: ignore # pylint: disable=abstract-class-instantiated | ||
self._client, self._config, self._serialize, self._deserialize | ||
) | ||
self.test_run = TestRunOperations(self._client, self._config, self._serialize, self._deserialize) | ||
|
||
def send_request(self, request: HttpRequest, **kwargs: Any) -> HttpResponse: | ||
"""Runs the network request through the client's chained policies. | ||
|
||
>>> from azure.core.rest import HttpRequest | ||
>>> request = HttpRequest("GET", "https://www.example.org/") | ||
<HttpRequest [GET], url: 'https://www.example.org/'> | ||
>>> response = client.send_request(request) | ||
<HttpResponse: 200 OK> | ||
|
||
For more information on this code flow, see https://aka.ms/azsdk/dpcodegen/python/send_request | ||
|
||
:param request: The network request you want to make. Required. | ||
:type request: ~azure.core.rest.HttpRequest | ||
:keyword bool stream: Whether the response payload will be streamed. Defaults to False. | ||
:return: The response of your network call. Does not do error handling on your response. | ||
:rtype: ~azure.core.rest.HttpResponse | ||
""" | ||
|
||
request_copy = deepcopy(request) | ||
path_format_arguments = { | ||
"Endpoint": self._serialize.url("self._config.endpoint", self._config.endpoint, "str", skip_quote=True), | ||
} | ||
|
||
request_copy.url = self._client.format_url(request_copy.url, **path_format_arguments) | ||
return self._client.send_request(request_copy, **kwargs) | ||
|
||
def close(self): | ||
# type: () -> None | ||
self._client.close() | ||
|
||
def __enter__(self): | ||
# type: () -> LoadTestingClient | ||
self._client.__enter__() | ||
return self | ||
|
||
def __exit__(self, *exc_details): | ||
# type: (Any) -> None | ||
self._client.__exit__(*exc_details) |
Uh oh!
There was an error while loading. Please reload this page.