Skip to content

Commit 6bf1b90

Browse files
Nivedit JainYashikaj14msyycBigCat20196Nivedit Jain
authored
azure-developer-loadtesting v1.0.0b2 (#25550)
* SDK generated autorest * upload function added * patch.py changed * upload updated * whl file removed * whls deleted * aio updated * spell check added * spell check * Update shared_requirements.txt * Update MANIFEST.in Co-authored-by: Yuchao Yan <[email protected]> * Update setup.py Co-authored-by: Yuchao Yan <[email protected]> * spell added * patch.py updated * linting corrected * Update allowed_pylint_failures.py * readme updated * patch.py updation * Update _patch.py * tests added * async naming * test and result * push the test result * fix * fix * tests added and async built * yaml added * yaml updated * sample and readme updated * fix ci.yml * more tests and samples addded and readme updated * tests * fix * fix2 * type added * type added * Update CHANGELOG.md * chnaged to developer * renamed package * chnaged ci files * linting fixes * changed namespace from analytics to developer * app_component naming cleaned * folded get_app_component * various name changes and added type hinting * lint changes * added wrapper for LoadTestClient * removed msrest dependencies * corrected Serializer method * test_smoke fixed * fixes in aio class * updates samples * namespace chg AppComponent, serverMetrics, test * updated samples * namespace changes done * LoadTestClient -> LoadTestingClient * apnd load_test to test_run & adminstration * added Readme Descirption * few changes in tests * moved SUBSCRIPTION_ID to .env * resloved readme issues by @iscai-msft in #24744 * resolved #discussion_r925248158 * resolving other issue by @iscai-msft #24744 * #25264#discussion_r925284462 * cleaned tests * namechanges * resolved folding issues * fixing dependencies verison * added tests for _patch(es) * removed subs id from recordings * fixing isodate requirements * removed SaaS key from smoke test * folder name chnage + setup.py cleaned * Readme changes suggested by @msyyc * added dependencies in shared_requirements.txt * removed os.environ from tests * liniting * removed async test * fixing tests * ci fix try * trying dev_tools * fix for azure-mgmt-loadtestservice dev_reqs * added async tests * some tests * cleaned setup * fixing async tests * added mgmt to loadtestservice to ci * changed fie mode from json to BinaryIO * reverting merge by @BigCat20196 * removed TEST_ID * solving issues by @lmazuel * update with autorest6.1.0 and update tests * update and format * fix playback failed * Update test_smoke.test_create_or_update_test_run.yaml * Update test_smoke.test_smoke_create_or_update_test.yaml * Update test_smoke_async.test_smoke_create_or_update_test.yaml * updated README * test case of app components create * delete_app_components fixed * complete testing for AppComponent Class * tested create_or_update_test * client.load_test_administration.delete_load_test * tests for get_load_test * tested upload_test_file * complete TestOperations testing * removed sas keys from recordings * corrected name error * corrected sample and added tests for test_run(1/2) * TestRun testing done * completed ServerMetrics testing * remove SAS key from files * removing SAS key * Update README.md * Update README.md * Update README.md * Update README.md * some work in tests * cleaned samples as per @ninallam * updated CHANGELOG.md * updated CHANGELOG.md * trying to fix chnagelog CI * migrate tests to testproxy * added changes suggested by @ninallam & @msyyc * updated change log * Update README.md * fixed issues with sample * fix playback failed * Update sdk/loadtestservice/azure-developer-loadtesting/CHANGELOG.md * Update CHANGELOG.md * fix playback failed * Fix changelog * Update _version.py * Update CHANGELOG.md * added bug explanation in Readme file. * removed samples and test cases from Changelog * added resource creation example in samples for mgmt plane * fixed sample resource creation link * adding ref to DefaultAzureCredential * refactor mgmt sample file * broken samples into files * fixed samples as disucssed with @iscai-msft * added few access to reult dict * update tests and recordings * Update README.md * fixed typo, matrics -> metrics * fixed typo, matrics -> metrics * update test recording * rename non test files * improved samples as per @ninallam * simplified check validation in SDK * resolved comments by @iscai-msft * pushing new design for validation checking * minor changes * removed LRO operations * minor fixes * started async tests * pushing more tests * completed corresponding async tests * removed mgmt plane samples * fixing broken link * ran black * chnaged security scope * Update CHANGELOG.md * Update conftest.py * updated README * added troubleshooting section * minor fixes Co-authored-by: Yashika Jain <[email protected]> Co-authored-by: Yuchao Yan <[email protected]> Co-authored-by: Yashika Jain <[email protected]> Co-authored-by: Jiefeng Chen (WICRESOFT NORTH AMERICA LTD) <[email protected]> Co-authored-by: Nivedit Jain <[email protected]> Co-authored-by: scbedd <[email protected]> Co-authored-by: BigCat20196 <[email protected]> Co-authored-by: Nikita Nallamothu <[email protected]>
1 parent f360b2f commit 6bf1b90

File tree

91 files changed

+25677
-762
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

91 files changed

+25677
-762
lines changed
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,14 @@
11
# Release History
22

3+
## 1.0.0b2 (2022-10-17)
4+
5+
### Bug Fixed
6+
- `delete_app_components` method from `azure.developer.loadtesting.LoadTestingClient.load_test_administration.delete_app_components` was not discoverable in expected location, fixed discoverability.
7+
8+
### Other Changes
9+
- Updated README
10+
311
## 1.0.0b1 (2022-07-28)
412

13+
### Features Added
514
- Initial version
Original file line numberDiff line numberDiff line change
@@ -1,11 +1,17 @@
11

2-
# Azure Developer LoadTesting client library for Python
3-
Azure Developer LoadTesting provides client library in python to the user by which they can interact natively with Load Test Service.The service is for performing the load test to optimize application performance, scalability or capacity. The user can get the client-side and server-side metrices to see the details reported by the test engine and information about your Azure application components.
2+
# Azure Load Testing client library for Python
3+
Azure Load Testing provides client library in python to the user by which they can interact natively with Azure Load Testing service. Azure Load Testing is a fully managed load-testing service that enables you to generate high-scale load. The service simulates traffic for your applications, regardless of where they're hosted. Developers, testers, and quality assurance (QA) engineers can use it to optimize application performance, scalability, or capacity.
44

5+
## Documentation
6+
Various documentation is available to help you get started
7+
8+
<!-- - [Source code][source_code] -->
9+
- [API reference documentation][api_reference_doc]
10+
- [Product Documentation][product_documentation]
511

612
## Getting started
713

8-
### Installating the package
14+
### Installing the package
915

1016
```bash
1117
python -m pip install azure-developer-loadtesting
@@ -16,26 +22,90 @@ python -m pip install azure-developer-loadtesting
1622
- Python 3.7 or later is required to use this package.
1723
- You need an [Azure subscription][azure_sub] to use this package.
1824
- An existing Azure Developer LoadTesting instance.
25+
1926
#### Create with an Azure Active Directory Credential
27+
2028
To use an [Azure Active Directory (AAD) token credential][authenticate_with_token],
2129
provide an instance of the desired credential type obtained from the
2230
[azure-identity][azure_identity_credentials] library.
2331

2432
To authenticate with AAD, you must first [pip][pip] install [`azure-identity`][azure_identity_pip]
2533

2634
After setup, you can choose which type of [credential][azure_identity_credentials] from azure.identity to use.
27-
As an example, [DefaultAzureCredential][default_azure_credential] can be used to authenticate the client:
2835

29-
Set the values of the client ID, tenant ID, and client secret of the AAD application as environment variables:
30-
`AZURE_CLIENT_ID`, `AZURE_TENANT_ID`, `AZURE_CLIENT_SECRET`
36+
As an example, sign in via the Azure CLI `az login` command and [DefaultAzureCredential](https://learn.microsoft.com/python/api/azure-identity/azure.identity.defaultazurecredential?view=azure-python) will authenticate as that user.
37+
38+
Use the returned token credential to authenticate the client.
3139

32-
Use the returned token credential to authenticate the client:
40+
#### Create the client
41+
42+
The data plane URI should be provided as the endpoint to create the client.
3343

3444
```python
3545
from azure.developer.loadtesting import LoadTestingClient
46+
47+
# for managing authentication and authorization
48+
# can be installed from pypi, follow: https://pypi.org/project/azure-identity/
49+
# using DefaultAzureCredentials, read more at: https://learn.microsoft.com/en-us/python/api/azure-identity/azure.identity.defaultazurecredential?view=azure-python
3650
from azure.identity import DefaultAzureCredential
51+
3752
client = LoadTestingClient(endpoint='<endpoint>', credential=DefaultAzureCredential())
3853
```
54+
## Key concepts
55+
56+
The following components make up te Azure Load Testing service. The Azure Load Test client library for python allows you to interact with each of these components through the use of clients. There are two top-level clients which are the main entry points for the library
57+
58+
- `LoadTestingClient` (`azure.developer.loadtesting.LoadTestingClient`)
59+
60+
- Async `LoadTestingClient` (`azure.developer.loadtesting.aio.LoadTestingClient`)
61+
62+
The two clients have similar methods in them except the methods in the async client are async as well.
63+
64+
The top-level clients have two sub-clients
65+
66+
- `load_test_runs` (`azure.developer.loadtesting.LoadTestingClient.load_test_runs`)
67+
68+
- `load_test_administration` (`azure.developer.loadtesting.LoadTestingClient.load_test_administration`)
69+
70+
These sub-clients are used for managing and using different components of the service.
71+
72+
### Load Test Administration Client
73+
74+
The `load_test_administration` sub-clients is used to administer and configure the load tests, app components and metrics.
75+
76+
#### Test
77+
78+
A test specifies the test script, and configuration settings for running a load test. You can create one or more tests in an Azure Load Testing resource.
79+
80+
#### App Component
81+
82+
When you run a load test for an Azure-hosted application, you can monitor resource metrics for the different Azure application components (server-side metrics). While the load test runs, and after completion of the test, you can monitor and analyze the resource metrics in the Azure Load Testing dashboard.
83+
84+
#### Metrics
85+
86+
During a load test, Azure Load Testing collects metrics about the test execution. There are two types of metrics:
87+
88+
1. Client-side metrics give you details reported by the test engine. These metrics include the number of virtual users, the request response time, the number of failed requests, or the number of requests per second.
89+
90+
2. Server-side metrics are available for Azure-hosted applications and provide information about your Azure application components. Metrics can be for the number of database reads, the type of HTTP responses, or container resource consumption.
91+
92+
### Test Run Client
93+
94+
The `load_test_runs` sub-clients is used to start and stop test runs corresponding to a load test. A test run represents one execution of a load test. It collects the logs associated with running the Apache JMeter script, the load test YAML configuration, the list of app components to monitor, and the results of the test.
95+
96+
### Data-Plane Endpoint
97+
98+
Data-plane of Azure Load Testing resources is addressable using the following URL format:
99+
100+
`00000000-0000-0000-0000-000000000000.aaa.cnt-prod.loadtesting.azure.com`
101+
102+
The first GUID `00000000-0000-0000-0000-000000000000` is the unique identifier used for accessing the Azure Load Testing resource. This is followed by `aaa` which is the Azure region of the resource.
103+
104+
The data-plane endpoint is obtained from Control Plane APIs.
105+
106+
**Example:** `1234abcd-12ab-12ab-12ab-123456abcdef.eus.cnt-prod.loadtesting.azure.com`
107+
108+
In the above example, `eus` represents the Azure region `East US`.
39109

40110
## Examples
41111

@@ -46,7 +116,7 @@ from azure.identity import DefaultAzureCredential
46116
from azure.core.exceptions import HttpResponseError
47117

48118
TEST_ID = "some-test-id"
49-
DISPLAY_NAME = "new_namespace-new-namespace"
119+
DISPLAY_NAME = "my-load-test"
50120
SUBSCRIPTION_ID = os.environ["SUBSCRIPTION_ID"]
51121

52122
client = LoadTestingClient(endpoint='<endpoint>', credential=DefaultAzureCredential())
@@ -55,24 +125,20 @@ try:
55125
result = client.load_test_administration.create_or_update_test(
56126
TEST_ID,
57127
{
58-
"resourceId": f"/subscriptions/{SUBSCRIPTION_ID}/resourceGroups/yashika-rg/providers/Microsoft.LoadTestService/loadtests/loadtestsdk",
59128
"description": "",
60129
"displayName": DISPLAY_NAME,
61130
"loadTestConfig": {
62-
"engineSize": "m",
63131
"engineInstances": 1,
64132
"splitAllCSVs": False,
65133
},
66134
"secrets": {},
67135
"environmentVariables": {},
68-
"passFailCriteria": {"passFailMetrics": {}},
69-
"keyvaultReferenceIdentityType": "SystemAssigned",
70-
"keyvaultReferenceIdentityId": None,
136+
"passFailCriteria": {"passFailMetrics": {}}
71137
},
72138
)
73139
print(result)
74140
except HttpResponseError as e:
75-
print('service responds error: {}'.format(e.response.json()))
141+
print('Service responded with error: {}'.format(e.response.json()))
76142

77143
```
78144

@@ -92,7 +158,7 @@ try:
92158
result = client.load_test_administration.upload_test_file(TEST_ID, FILE_ID, open("sample.jmx", "rb"))
93159
print(result)
94160
except HttpResponseError as e:
95-
print("Failed to send JSON message: {}".format(e.response.json()))
161+
print("Failed with error: {}".format(e.response.json()))
96162
```
97163

98164
### Running a Test
@@ -103,57 +169,26 @@ from azure.core.exceptions import HttpResponseError
103169

104170
TEST_ID = "some-test-id"
105171
TEST_RUN_ID = "some-testrun-id"
106-
DISPLAY_NAME = "new_namespace-new-namespace"
172+
DISPLAY_NAME = "my-load-test-run"
107173

108174
client = LoadTestingClient(endpoint='<endpoint>', credential=DefaultAzureCredential())
109175

110176
try:
111-
result = client.load_test_runs.create_and_update_test(
177+
result = client.load_test_runs.create_or_update_test(
112178
TEST_RUN_ID,
113179
{
114180
"testId": TEST_ID,
115181
"displayName": DISPLAY_NAME,
116-
"requestSamplers": [],
117-
"errors": [],
118-
"percentiles": ["90"],
119-
"groupByInterval": "5s",
120182
},
121183
)
122184
print(result)
123185
except HttpResponseError as e:
124-
print("Failed to send JSON message: {}".format(e.response.json()))
186+
print("Failed with error: {}".format(e.response.json()))
125187
```
126-
## Key concepts
127-
The following components make up the Azure Load Testing Service. The Azure Load Test client library for Python allows you to interact with each of these components through the use of a dedicated client object.
128-
129-
### Load testing resource
130-
The Load testing resource is the top-level resource for your load-testing activities. This resource provides a centralized place to view and manage load tests, test results, and related artifacts. A load testing resource contains zero or more load tests.
131-
132-
### Test
133-
A test specifies the test script, and configuration settings for running a load test. You can create one or more tests in an Azure Load Testing resource.
134-
135-
### Test Engine
136-
A test engine is computing infrastructure that runs the Apache JMeter test script. You can scale out your load test by configuring the number of test engines. The test script runs in parallel across the specified number of test engines.
137-
138-
### Test Run
139-
A test run represents one execution of a load test. It collects the logs associated with running the Apache JMeter script, the load test YAML configuration, the list of app components to monitor, and the results of the test.
140-
141-
### App Component
142-
When you run a load test for an Azure-hosted application, you can monitor resource metrics for the different Azure application components (server-side metrics). While the load test runs, and after completion of the test, you can monitor and analyze the resource metrics in the Azure Load Testing dashboard.
143-
144-
### Metrics
145-
During a load test, Azure Load Testing collects metrics about the test execution. There are two types of metrics:
146-
147-
1. Client-side metrics give you details reported by the test engine. These metrics include the number of virtual users, the request response time, the number of failed requests, or the number of requests per second.
148-
149-
2. Server-side metrics are available for Azure-hosted applications and provide information about your Azure application components. Metrics can be for the number of database reads, the type of HTTP responses, or container resource consumption.
150-
151-
## Troubleshooting
152-
More about it is coming soon...
153188

154189
## Next steps
155190

156-
More examples are coming soon...
191+
More samples can be found [here](https://github.com/Azure/azure-sdk-for-python/tree/main/sdk/loadtestservice/azure-developer-loadtesting/samples).
157192

158193
## Contributing
159194

@@ -172,6 +207,9 @@ This project has adopted the
172207
see the Code of Conduct FAQ or contact [email protected] with any
173208
additional questions or comments.
174209

210+
## Troubleshooting
211+
More about it is coming soon...
212+
175213
<!-- LINKS -->
176214
[code_of_conduct]: https://opensource.microsoft.com/codeofconduct/
177215
[authenticate_with_token]: https://docs.microsoft.com/azure/cognitive-services/authentication?tabs=powershell#authenticate-with-an-authentication-token
@@ -180,3 +218,5 @@ additional questions or comments.
180218
[default_azure_credential]: https://github.com/Azure/azure-sdk-for-python/tree/main/sdk/identity/azure-identity#defaultazurecredential
181219
[pip]: https://pypi.org/project/pip/
182220
[azure_sub]: https://azure.microsoft.com/free/
221+
[api_reference_doc]: https://docs.microsoft.com/rest/api/loadtesting/
222+
[product_documentation]: https://azure.microsoft.com/services/load-testing/

sdk/loadtestservice/azure-developer-loadtesting/azure/developer/loadtesting/_configuration.py

+1-1
Original file line numberDiff line numberDiff line change
@@ -45,7 +45,7 @@ def __init__(self, endpoint: str, credential: "TokenCredential", **kwargs: Any)
4545
self.endpoint = endpoint
4646
self.credential = credential
4747
self.api_version = api_version
48-
self.credential_scopes = kwargs.pop("credential_scopes", ["https://loadtest.azure-dev.com/.default"])
48+
self.credential_scopes = kwargs.pop("credential_scopes", ["https://cnt-prod.loadtesting.azure.com/.default"])
4949
kwargs.setdefault("sdk_moniker", "developer-loadtesting/{}".format(VERSION))
5050
self._configure(**kwargs)
5151

sdk/loadtestservice/azure-developer-loadtesting/azure/developer/loadtesting/_version.py

+1-1
Original file line numberDiff line numberDiff line change
@@ -6,4 +6,4 @@
66
# Changes may cause incorrect behavior and will be lost if the code is regenerated.
77
# --------------------------------------------------------------------------
88

9-
VERSION = "1.0.0b1"
9+
VERSION = "1.0.0b2"

sdk/loadtestservice/azure-developer-loadtesting/azure/developer/loadtesting/aio/_configuration.py

+1-1
Original file line numberDiff line numberDiff line change
@@ -45,7 +45,7 @@ def __init__(self, endpoint: str, credential: "AsyncTokenCredential", **kwargs:
4545
self.endpoint = endpoint
4646
self.credential = credential
4747
self.api_version = api_version
48-
self.credential_scopes = kwargs.pop("credential_scopes", ["https://loadtest.azure-dev.com/.default"])
48+
self.credential_scopes = kwargs.pop("credential_scopes", ["https://cnt-prod.loadtesting.azure.com/.default"])
4949
kwargs.setdefault("sdk_moniker", "developer-loadtesting/{}".format(VERSION))
5050
self._configure(**kwargs)
5151

sdk/loadtestservice/azure-developer-loadtesting/azure/developer/loadtesting/aio/operations/_patch.py

+25-25
Original file line numberDiff line numberDiff line change
@@ -6,9 +6,7 @@
66
77
Follow our quickstart for examples: https://aka.ms/azsdk/python/dpcodegen/python/customize
88
"""
9-
10-
from typing import Any, IO, Optional, Union, cast, overload, List, MutableMapping, Coroutine, BinaryIO
11-
9+
from typing import Any, IO, Optional, Union, cast, overload, List, MutableMapping, Coroutine, BinaryIO, Awaitable
1210
from azure.core.exceptions import (
1311
ClientAuthenticationError,
1412
HttpResponseError,
@@ -95,14 +93,14 @@ class AppComponentOperations:
9593
def __init__(self, *args, **kwargs):
9694
self.__app_component_operations_generated = AppComponentOperationsGenerated(*args, **kwargs)
9795

98-
def get_app_components(
96+
async def get_app_components(
9997
self,
10098
*,
10199
test_run_id: Optional[str] = None,
102100
test_id: Optional[str] = None,
103101
name: Optional[str] = None,
104102
**kwargs: Any,
105-
) -> Coroutine[Any, Any, MutableMapping[str, Any]]:
103+
) -> MutableMapping[str, Any]:
106104
"""Get App Components for a test or a test run by its name.
107105
108106
Get App Components for a test or a test run by its name.
@@ -150,14 +148,14 @@ def get_app_components(
150148
"""
151149

152150
if name is not None:
153-
return self.__app_component_operations_generated.get_app_component_by_name(name=name, **kwargs)
151+
return await self.__app_component_operations_generated.get_app_component_by_name(name=name, **kwargs)
154152
else:
155-
return self.__app_component_operations_generated.get_app_component(
153+
return await self.__app_component_operations_generated.get_app_component(
156154
test_run_id=test_run_id, test_id=test_id, **kwargs
157155
)
158156

159157
@overload
160-
def create_or_update_app_components(
158+
async def create_or_update_app_components(
161159
self, name: str, body: JSON, *, content_type: str = "application/merge-patch+json", **kwargs: Any
162160
) -> JSON:
163161
"""Associate an App Component (Azure resource) to a test or test run.
@@ -233,7 +231,7 @@ def create_or_update_app_components(
233231
"""
234232

235233
@overload
236-
def create_or_update_app_components(
234+
async def create_or_update_app_components(
237235
self, name: str, body: IO, *, content_type: str = "application/merge-patch+json", **kwargs: Any
238236
) -> JSON:
239237
"""Associate an App Component (Azure resource) to a test or test run.
@@ -283,7 +281,9 @@ def create_or_update_app_components(
283281
"""
284282

285283
@distributed_trace
286-
def create_or_update_app_components(self, name: str, body: Union[JSON, IO], **kwargs: Any) -> JSON:
284+
async def create_or_update_app_components(
285+
self, name: str, body: Union[JSON, IO], **kwargs: Any
286+
) -> MutableMapping[str, Any]:
287287
"""Associate an App Component (Azure resource) to a test or test run.
288288
289289
Associate an App Component (Azure resource) to a test or test run.
@@ -329,25 +329,25 @@ def create_or_update_app_components(self, name: str, body: Union[JSON, IO], **kw
329329
}
330330
}
331331
"""
332-
return self.__app_component_operations_generated.create_or_update_app_components(name, body, **kwargs)
332+
return await self.__app_component_operations_generated.create_or_update_app_components(name, body, **kwargs)
333333

334-
@distributed_trace
335-
def delete_app_components( # pylint: disable=inconsistent-return-statements
336-
self, name: str, **kwargs: Any
337-
) -> None:
338-
"""Delete an App Component.
334+
@distributed_trace
335+
async def delete_app_components( # pylint: disable=inconsistent-return-statements
336+
self, name: str, **kwargs: Any
337+
) -> None:
338+
"""Delete an App Component.
339339
340-
Delete an App Component.
340+
Delete an App Component.
341341
342-
:param name: Unique name of the App Component, must be a valid URL character ^[a-z0-9_-]*$.
343-
Required.
344-
:type name: str
345-
:return: None
346-
:rtype: None
347-
:raises ~azure.core.exceptions.HttpResponseError:
348-
"""
342+
:param name: Unique name of the App Component, must be a valid URL character ^[a-z0-9_-]*$.
343+
Required.
344+
:type name: str
345+
:return: None
346+
:rtype: None
347+
:raises ~azure.core.exceptions.HttpResponseError:
348+
"""
349349

350-
return self.__app_component_operations_generated.delete_app_components(name, **kwargs)
350+
return await self.__app_component_operations_generated.delete_app_components(name, **kwargs)
351351

352352

353353
__all__: List[str] = ["TestOperations", "AppComponentOperations"]

0 commit comments

Comments
 (0)