Skip to content

Commit 017a88a

Browse files
committed
Merge branch 'master' of https://github.com/Azure/azure-sdk-for-python into enum-meta
* 'master' of https://github.com/Azure/azure-sdk-for-python: (128 commits) Communication identity api redesign (Azure#16420) fix EH samples and docs (Azure#16457) T2 redisenterprise 2021 02 02 (Azure#16472) Update automation_generate.sh (Azure#16470) Sync eng/common directory with azure-sdk-tools for PR 1353 (Azure#16465) Normalize the package name for Doc.Ms readme (Azure#16401) fix changelog and version (Azure#16445) we should always run integration to publish from our artifacts. if the build pipeline crashed too early in the pipeline, this will fail regardless, due to inability to pull the artifacts down (Azure#15058) Fix DateTime bug (Azure#16456) Resolve Regression Failures (Azure#16455) [text analytics] Expose 'string_index_type' parameter in all service client methods where applicable (Azure#16412) adding devtools to the appconfig dev_reqs to solve python core issue (Azure#16381) Copy job matrix functionality (Azure#16450) Add APIView KV variable group to prepare pipelines bot (Azure#16451) [Datalake] Added support for PurePosixPath (Azure#16400) Regenerate baseline because last one break. (Azure#16415) adding step to test for crlf line endings (Azure#16398) [Datalake] Removed list_paths manual paging and deserialization (Azure#16309) Sync eng/common directory with azure-sdk-tools for PR 1351 (Azure#16448) Update auto_codegen.py (Azure#16443) ...
2 parents dbfc49e + 30b917b commit 017a88a

File tree

2,260 files changed

+478807
-76908
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

2,260 files changed

+478807
-76908
lines changed

.github/CODEOWNERS

+7-4
Original file line numberDiff line numberDiff line change
@@ -20,7 +20,7 @@
2020
/sdk/identity/ @chlowell @mccoyp @schaabs
2121

2222
# PRLabel: %Event Hubs
23-
/sdk/eventhub/ @annatisch @yunhaoling @KieranBrantnerMagee
23+
/sdk/eventhub/ @annatisch @yunhaoling @swathipil @rakshith91
2424

2525
# PRLabel: %Storage
2626
/sdk/storage/ @amishra-dev @zezha-msft @annatisch @xiafu-msft @tasherif-msft @kasobol-msft
@@ -36,7 +36,7 @@
3636
/sdk/cognitiveservices/azure-cognitiveservices-vision-customvision/ @areddish
3737

3838
# PRLabel: %Communication
39-
/sdk/communication/ @RezaJooyandeh @turalf @ankitarorabit @memontic-ms @Azure/azure-sdk-communication-code-reviewers
39+
/sdk/communication/ @turalf @ankitarorabit @memontic-ms @Azure/azure-sdk-communication-code-reviewers
4040

4141
# PRLabel: %KeyVault
4242
/sdk/keyvault/ @schaabs @chlowell @mccoyp
@@ -66,7 +66,7 @@
6666
/sdk/datadatamigration/ @vchske
6767

6868
# PRLabel: %Event Grid
69-
/sdk/eventgrid/ @lmazuel @rakshith91 @KieranBrantnerMagee
69+
/sdk/eventgrid/ @lmazuel @yunhaoling @swathipil @rakshith91
7070

7171
# PRLabel: %HDInsight
7272
/sdk/hdinsight/ @idear1203
@@ -90,7 +90,7 @@
9090
/sql/sql/ @jaredmoo
9191

9292
# PRLabel: %Service Bus
93-
/sdk/servicebus/ @annatisch @yunhaoling @KieranBrantnerMagee @rakshith91
93+
/sdk/servicebus/ @annatisch @yunhaoling @swathipil @rakshith91
9494

9595
# PRLabel: %Synapse
9696
/sdk/synapse/ @aim-for-better @idear1203
@@ -110,6 +110,9 @@
110110
# PRLabel: %Tables
111111
/sdk/tables/ @seankane-msft
112112

113+
# PRLabel: %Media
114+
/sdk/media/ @naiteeks @bennage @giakas
115+
113116
# Smoke Tests
114117
/common/smoketest/ @lmazuel @chlowell @annatisch @rakshith91 @shurd @southpolesteve
115118

.gitignore

+1
Original file line numberDiff line numberDiff line change
@@ -33,6 +33,7 @@ build/
3333

3434
# Test results
3535
TestResults/
36+
ENV_DIR/
3637

3738
# tox generated artifacts
3839
test-junit-*.xml

build_package.py

+3-2
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,7 @@
99
import argparse
1010
import os
1111
import glob
12+
import sys
1213
from subprocess import check_call
1314

1415

@@ -19,8 +20,8 @@ def create_package(name, dest_folder=DEFAULT_DEST_FOLDER):
1920
absdirs = [os.path.dirname(package) for package in (glob.glob('{}/setup.py'.format(name)) + glob.glob('sdk/*/{}/setup.py'.format(name)))]
2021

2122
absdirpath = os.path.abspath(absdirs[0])
22-
check_call(['python', 'setup.py', 'bdist_wheel', '-d', dest_folder], cwd=absdirpath)
23-
check_call(['python', 'setup.py', "sdist", "--format", "zip", '-d', dest_folder], cwd=absdirpath)
23+
check_call([sys.executable, 'setup.py', 'bdist_wheel', '-d', dest_folder], cwd=absdirpath)
24+
check_call([sys.executable, 'setup.py', "sdist", "--format", "zip", '-d', dest_folder], cwd=absdirpath)
2425

2526

2627
if __name__ == '__main__':

ci_template.yml

+33
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,33 @@
1+
# DO NOT EDIT THIS FILE
2+
# This file is generated automatically and any changes will be lost.
3+
4+
trigger:
5+
branches:
6+
include:
7+
- master
8+
- hotfix/*
9+
- release/*
10+
- restapi*
11+
paths:
12+
include:
13+
- sdk/MyService/
14+
15+
pr:
16+
branches:
17+
include:
18+
- master
19+
- feature/*
20+
- hotfix/*
21+
- release/*
22+
- restapi*
23+
paths:
24+
include:
25+
- sdk/MyService/
26+
27+
extends:
28+
template: ../../eng/pipelines/templates/stages/archetype-sdk-client.yml
29+
parameters:
30+
ServiceDirectory: MyService
31+
Artifacts:
32+
- name: azure_mgmt_MyService
33+
safeName: azuremgmtMyService
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
azure-core>=0.0.0b1
2+
azure-identity>=0.0.0b1
3+
azure-cosmos>=4.0.0b5
4+
azure-eventhub>=0.0.0b1
5+
azure-keyvault-certificates>=0.0.0b1
6+
azure-keyvault-keys>=0.0.0b1
7+
azure-keyvault-secrets>=0.0.0b1
8+
azure-storage-blob>=0.0.0b1

common/smoketest/requirements.txt

+1-1
Original file line numberDiff line numberDiff line change
@@ -5,4 +5,4 @@ azure-eventhub
55
azure-keyvault-certificates
66
azure-keyvault-keys
77
azure-keyvault-secrets
8-
azure-storage-blob
8+
azure-storage-blob

common/smoketest/smoke-test.yml

+3-122
Original file line numberDiff line numberDiff line change
@@ -1,123 +1,4 @@
1-
variables:
2-
InstallAsyncRequirements: true
3-
41
jobs:
5-
- job:
6-
strategy:
7-
matrix:
8-
Python_27_Linux (AzureCloud):
9-
PythonVersion: '2.7'
10-
InstallAsyncRequirements: false
11-
OSVmImage: ubuntu-18.04
12-
SubscriptionConfiguration: $(sub-config-azure-cloud-test-resources)
13-
ArmTemplateParameters: $(azureCloudArmParameters)
14-
Python_37_Linux (AzureCloud):
15-
PythonVersion: '3.7'
16-
OSVmImage: ubuntu-18.04
17-
SubscriptionConfiguration: $(sub-config-azure-cloud-test-resources)
18-
ArmTemplateParameters: $(azureCloudArmParameters)
19-
Python_38_Linux (AzureCloud):
20-
PythonVersion: '3.8'
21-
OSVmImage: ubuntu-18.04
22-
SubscriptionConfiguration: $(sub-config-azure-cloud-test-resources)
23-
ArmTemplateParameters: $(azureCloudArmParameters)
24-
Python_38_Linux (AzureCloud Canary):
25-
PythonVersion: '3.8'
26-
OSVmImage: ubuntu-18.04
27-
SubscriptionConfiguration: $(sub-config-azure-cloud-test-resources-preview)
28-
ArmTemplateParameters: $(azureCloudArmParameters)
29-
Location: 'eastus2euap'
30-
Python_37_Windows (AzureCloud):
31-
PythonVersion: '3.7'
32-
OSVmImage: windows-2019
33-
SubscriptionConfiguration: $(sub-config-azure-cloud-test-resources)
34-
ArmTemplateParameters: $(azureCloudArmParameters)
35-
Python_38_Windows (AzureCloud):
36-
PythonVersion: '3.8'
37-
OSVmImage: windows-2019
38-
SubscriptionConfiguration: $(sub-config-azure-cloud-test-resources)
39-
ArmTemplateParameters: $(azureCloudArmParameters)
40-
Python_37_Mac (AzureCloud):
41-
PythonVersion: '3.7'
42-
OSVmImage: macOS-10.15
43-
SubscriptionConfiguration: $(sub-config-azure-cloud-test-resources)
44-
ArmTemplateParameters: $(azureCloudArmParameters)
45-
Python_38_Mac (AzureCloud):
46-
PythonVersion: '3.8'
47-
OSVmImage: macOS-10.15
48-
SubscriptionConfiguration: $(sub-config-azure-cloud-test-resources)
49-
ArmTemplateParameters: $(azureCloudArmParameters)
50-
Python_38_Linux (AzureUSGovernment):
51-
PythonVersion: '3.8'
52-
OSVmImage: ubuntu-18.04
53-
SubscriptionConfiguration: $(sub-config-gov-test-resources)
54-
ArmTemplateParameters: $(azureUSGovernmentArmParameters)
55-
Python_37_Windows (AzureUSGovernment):
56-
PythonVersion: '3.7'
57-
OSVmImage: windows-2019
58-
SubscriptionConfiguration: $(sub-config-gov-test-resources)
59-
ArmTemplateParameters: $(azureUSGovernmentArmParameters)
60-
Python_38_Linux (AzureChinaCloud):
61-
PythonVersion: '3.8'
62-
OSVmImage: ubuntu-18.04
63-
SubscriptionConfiguration: $(sub-config-cn-test-resources)
64-
Location: 'chinanorth'
65-
ArmTemplateParameters: $(azureChinaCloudArmParameters)
66-
Python_37_Windows (AzureChinaCloud):
67-
PythonVersion: '3.7'
68-
OSVmImage: windows-2019
69-
SubscriptionConfiguration: $(sub-config-cn-test-resources)
70-
Location: 'chinanorth'
71-
ArmTemplateParameters: $(azureChinaCloudArmParameters)
72-
73-
pool:
74-
vmImage: $(OSVmImage)
75-
76-
variables:
77-
Location: ''
78-
azureCloudArmParameters: "@{ storageEndpointSuffix = 'core.windows.net'; azureCloud = 'AzureCloud'; }"
79-
azureUSGovernmentArmParameters: "@{ storageEndpointSuffix = 'core.usgovcloudapi.net'; azureCloud = 'AzureUSGovernment'; }"
80-
azureChinaCloudArmParameters: "@{ storageEndpointSuffix = 'core.chinacloudapi.cn'; azureCloud = 'AzureChinaCloud'; }"
81-
82-
steps:
83-
- task: UsePythonVersion@0
84-
displayName: "Use Python $(PythonVersion)"
85-
inputs:
86-
versionSpec: $(PythonVersion)
87-
88-
- script: |
89-
python -m pip install pip==20.0.2
90-
pip --version
91-
displayName: pip --version
92-
93-
- script: pip install -r ./common/smoketest/requirements.txt --pre --no-deps --index-url https://pkgs.dev.azure.com/azure-sdk/public/_packaging/azure-sdk-for-python/pypi/simple
94-
displayName: Install requirements from dev feed without dependencies
95-
96-
- script: pip install -r ./common/smoketest/requirements_async.txt
97-
displayName: "Install requirements_async.txt"
98-
condition: and(succeeded(), eq(variables['InstallAsyncRequirements'], 'true'))
99-
100-
- script: python ./common/smoketest/dependencies.py -r ./common/smoketest/requirements.txt | tee ./common/smoketest/requirements_dependencies.txt
101-
displayName: Create dependency list from installed dev packages
102-
103-
- script: pip install -r ./common/smoketest/requirements_dependencies.txt
104-
displayName: Install dev package dependencies from PyPI
105-
106-
- script: pip freeze
107-
displayName: Show installed packages (pip freeze)
108-
109-
- template: /eng/common/TestResources/deploy-test-resources.yml
110-
parameters:
111-
ServiceDirectory: '$(Build.SourcesDirectory)/common/smoketest/'
112-
SubscriptionConfiguration: $(SubscriptionConfiguration)
113-
Location: $(Location)
114-
ArmTemplateParameters: $(ArmTemplateParameters)
115-
116-
- script: python ./common/smoketest/program.py
117-
displayName: Run Smoke Test
118-
119-
- template: /eng/common/TestResources/remove-test-resources.yml
120-
parameters:
121-
ServiceDirectory: '$(Build.SourcesDirectory)/common/smoketest/'
122-
SubscriptionConfiguration: $(SubscriptionConfiguration)
123-
2+
- template: /eng/pipelines/templates/jobs/smoke-test.yml
3+
parameters:
4+
Daily: true

doc/dev/mgmt/swagger/multi_api/readme.md

+1-1
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,7 @@ This is the AutoRest configuration file for Network.
88

99
## Getting Started
1010

11-
To build the SDK for Network, simply [Install AutoRest](https://github.com/Azure/autorest/blob/master/docs/installing-autorest.md) and in this folder, run:
11+
To build the SDK for Network, simply [Install AutoRest](https://github.com/Azure/autorest/blob/master/docs/install/readme.md) and in this folder, run:
1212

1313
> `autorest`
1414

doc/dev/mgmt/swagger/single_api/readme.md

+1-1
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,7 @@ This is the AutoRest configuration file for Cdn.
88

99
---
1010
## Getting Started
11-
To build the SDK for Cdn, simply [Install AutoRest](https://github.com/Azure/autorest/blob/master/docs/installing-autorest.md) and in this folder, run:
11+
To build the SDK for Cdn, simply [Install AutoRest](https://github.com/Azure/autorest/blob/master/docs/install/readme.md) and in this folder, run:
1212

1313
> `autorest`
1414

doc/dev/mgmt/tests.md

+6-5
Original file line numberDiff line numberDiff line change
@@ -35,13 +35,13 @@ sdk/storage/azure-mgmt-storage> pytest
3535

3636
You can provide directories or individual files as positional arguments to specify particular tests to run rather than running the entire test suite. For example:
3737
```Shell
38-
sdk/storage/azure-mgmt-storage> pytest sdk/storage/azure-mgmt-storage/
39-
sdk/storage/azure-mgmt-storage> pytest sdk/storage/azure-mgmt-storage/tests/test_mgmt_storage.py
38+
sdk/storage/azure-mgmt-storage> pytest
39+
sdk/storage/azure-mgmt-storage> pytest tests/test_mgmt_storage.py
4040
```
4141

4242
If you have print statements in your tests for debugging you can add the `-s` flag to send those print statements to standard output:
4343
```Shell
44-
sdk/storage/azure-mgmt-storage> pytest sdk/storage/azure-mgmt-storage/ -s
44+
sdk/storage/azure-mgmt-storage> pytest -s
4545
```
4646

4747
## Getting Azure credentials
@@ -106,6 +106,7 @@ def get_credentials(**kwargs):
106106
```python
107107
def get_azure_core_credentials(**kwargs):
108108
from azure.identity import ClientSecretCredential
109+
import os
109110
return ClientSecretCredential(
110111
client_id = os.environ['AZURE_CLIENT_ID'],
111112
client_secret = os.environ['AZURE_CLIENT_SECRET'],
@@ -309,11 +310,11 @@ class ExampleSqlServerTestCase(AzureMgmtTestCase):
309310
```
310311

311312
<!-- LINKS -->
312-
[arm_apis]: https://docs.microsoft.com/en-us/rest/api/resources/
313+
[arm_apis]: https://docs.microsoft.com/rest/api/resources/
313314
[azure_devtools]: https://pypi.org/project/azure-devtools/
314315
[azure_portal]: https://portal.azure.com/
315316
[decorators]: https://www.python.org/dev/peps/pep-0318/
316-
[dev_setup]: ../dev_setup.md
317+
[dev_setup]: https://github.com/Azure/azure-sdk-for-python/blob/master/doc/dev/dev_setup.md
317318
[devtools_testutils]: https://github.com/Azure/azure-sdk-for-python/tree/master/tools/azure-sdk-tools/devtools_testutils
318319
[mgmt_settings_fake]: https://github.com/Azure/azure-sdk-for-python/blob/master/tools/azure-sdk-tools/devtools_testutils/mgmt_settings_fake.py
319320
[pytest]: https://docs.pytest.org/en/latest/

doc/dev/tests.md

+9-3
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@ In this document we will provide the introduction to the testing framework by:
55
- [Integrating with pytest](#integrate-with-the-pytest-test-framework)
66
- [Using Tox](#tox)
77
- [The `devtools_testutils` package](#devtools_testutils-package)
8-
- [Writing New Tests](#Writing-new-tests)
8+
- [Writing New Tests](#writing-new-tests)
99
- [Define our credentials and settings](#define-credentials)
1010
- [Create live test resources](#create-live-test-resources)
1111
- [Write our test](#writing-your-test)
@@ -84,6 +84,12 @@ azure-sdk-for-python\sdk\my-directory\my-library> pytest
8484
azure-sdk-for-python\sdk\my-directory\my-library> pytest <test_file.py>
8585
```
8686

87+
If your tests are broken up into multiple folders for organization, you can run specific folders:
88+
```cmd
89+
azure-sdk-for-python\sdk\my-directory\my-library> pytest .\tests\async_tests\
90+
azure-sdk-for-python\sdk\my-directory\my-library> pytest .\tests\async_tests\<test_file.py>
91+
```
92+
8793
In addition you can provide keywords to run specific tests within the suite or within a specific file
8894
```cmd
8995
azure-sdk-for-python\sdk\my-directory\my-library> pytest -k <keyword>
@@ -289,8 +295,8 @@ For more information, refer to the [advanced tests notes][advanced_tests_notes]
289295

290296

291297
<!-- Links -->
292-
[advanced_tests_notes]: ./tests-advanced.md
298+
[advanced_tests_notes]: https://github.com/Azure/azure-sdk-for-python/blob/master/doc/dev/tests-advanced.md
293299
[azure_devtools]: https://pypi.org/project/azure-devtools/
294300
[engsys_wiki]: https://dev.azure.com/azure-sdk/internal/_wiki/wikis/internal.wiki/48/Create-a-new-Live-Test-pipeline?anchor=test-resources.json
295301
[mgmt_settings_fake]: https://github.com/Azure/azure-sdk-for-python/blob/master/tools/azure-sdk-tools/devtools_testutils/mgmt_settings_fake.py
296-
[packaging]: ./packaging.md
302+
[packaging]: https://github.com/Azure/azure-sdk-for-python/blob/master/doc/dev/packaging.md

eng/.docsettings.yml

+2
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,7 @@ omitted_paths:
88
- doc/*
99
- sdk/**/samples/*
1010
- sdk/identity/azure-identity/tests/*
11+
- sdk/**/tests/perfstress_tests/*
1112

1213
language: python
1314
root_check_enabled: True
@@ -90,6 +91,7 @@ known_content_issues:
9091
- ['sdk/schemaregistry/azure-schemaregistry/swagger/README.md', '#4554']
9192
- ['sdk/servicebus/azure-servicebus/README.md', '#4554']
9293
- ['sdk/servicebus/azure-servicebus/swagger/README.md', '#4554']
94+
- ['sdk/servicebus/azure-servicebus/tests/perf_tests/README.md', '#4554']
9395
- ['sdk/servicefabric/azure-servicefabric/README.md', '#4554']
9496
- ['sdk/storage/azure-storage-nspkg/README.md', '#4554']
9597
- ['sdk/storage/azure-storage-blob/swagger/README.md', '#4554']

eng/CredScanSuppression.json

+4
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
{
2+
"tool": "Credential Scanner",
3+
"suppressions": []
4+
}

eng/common/TestResources/README.md

+7-1
Original file line numberDiff line numberDiff line change
@@ -30,7 +30,7 @@ Connect-AzAccount -Subscription 'YOUR SUBSCRIPTION ID'
3030
eng\common\TestResources\New-TestResources.ps1 -ServiceDirectory 'search'
3131
```
3232

33-
The `OutFile` switch would be set if you are running this for a .NET project on Windows. This will save test environment settings
33+
The `OutFile` switch will be set by default if you are running this for a .NET project on Windows. This will save test environment settings
3434
into a test-resources.json.env file next to test-resources.json. The file is protected via DPAPI.
3535
The environment file would be scoped to the current repository directory and avoids the need to
3636
set environment variables or restart your IDE to recognize them.
@@ -89,6 +89,12 @@ Remove-AzADServicePrincipal -ApplicationId $sp.ApplicationId -Force
8989

9090
If you persisted environment variables, you should also remove those as well.
9191

92+
Some test-resources.json templates utilize the `AdditionalParameters` parameter to control additional resource configuration options. For example:
93+
94+
```powershell
95+
New-TestResources.ps1 keyvault -AdditionalParameters @{enableHsm = $true}
96+
```
97+
9298
## In CI
9399

94100
Test pipelines should include deploy-test-resources.yml and

0 commit comments

Comments
 (0)