merge master

Author: yunhaoling

.github/CODEOWNERS

@@ -23,13 +23,13 @@
 /sdk/eventhub/                                                       @annatisch @yunhaoling @YijunXieMS
 
 # PRLabel: %Storage
-/sdk/storage/                                                        @amishra-dev @zezha-msft @annatisch @rakshith91 @xiafu-msft @kasobol-msft
+/sdk/storage/                                                        @amishra-dev @zezha-msft @annatisch @rakshith91 @xiafu-msft @tasherif-msft @kasobol-msft
 
-/sdk/applicationinsights/                                            @alexeldeib      
+/sdk/applicationinsights/                                            @alexeldeib
 
 # PRLabel: %Batch
 /sdk/batch/                                                          @bgklein @xingwu1
-/sdk/cognitiveservices/azure-cognitiveservices-vision-customvision/  @areddish          
+/sdk/cognitiveservices/azure-cognitiveservices-vision-customvision/  @areddish
 
 # PRLabel: %KeyVault
 /sdk/keyvault/                                                       @schaabs @chlowell @iscai-msft
@@ -52,7 +52,7 @@
 
 # PRLabel: %Data Factory
 /sdk/datafactory/                                                    @hvermis
-/sdk/datalake/                                                       @ro-joowan                                                     
+/sdk/datalake/                                                       @ro-joowan
 /sdk/datadatamigration/                                              @vchske
 
 # PRLabel: %Event Grid
@@ -88,6 +88,9 @@
 # PRLabel: %Cognitive - Form Recognizer
 /sdk/formrecognizer/                                                 @kristapratico @iscai-msft @rakshith91
 
+# PRLabel: %Tables
+/sdk/tables/                                                         @seankane-msft
+
 # Smoke Tests
 /common/smoketest/                                                   @lmazuel @chlowell @annatisch @rakshith91 @shurd @southpolesteve
 

.gitignore

@@ -84,7 +84,7 @@ src/build
 
 # [begoldsm] ignore virtual env if it exists.
 adlEnv/
-
+venv/
 code_reports
 
 # Azure Storage test credentials

README.md

@@ -2,7 +2,7 @@
 
 [![Packages](https://img.shields.io/badge/packages-latest-blue.svg)](https://azure.github.io/azure-sdk/releases/latest/python.html) [![Dependencies](https://img.shields.io/badge/dependency-report-blue.svg)](https://azuresdkartifacts.blob.core.windows.net/azure-sdk-for-python/dependencies/dependencies.html) [![DepGraph](https://img.shields.io/badge/dependency-graph-blue.svg)](https://azuresdkartifacts.blob.core.windows.net/azure-sdk-for-python/dependencies/InterdependencyGraph.html) [![Python](https://img.shields.io/pypi/pyversions/azure-core.svg?maxAge=2592000)](https://pypi.python.org/pypi/azure/) [![Build Status](https://dev.azure.com/azure-sdk/public/_apis/build/status/python/python%20-%20core%20-%20ci?branchName=master)](https://dev.azure.com/azure-sdk/public/_build/latest?definitionId=458&branchName=master)
 
-This repository is for active development of the Azure SDK for Python. For consumers of the SDK we recommend visiting our [public developer docs](https://docs.microsoft.com/en-us/python/azure/) or our versioned [developer docs](https://azure.github.io/azure-sdk-for-python).
+This repository is for active development of the Azure SDK for Python. For consumers of the SDK we recommend visiting our [public developer docs](https://docs.microsoft.com/python/azure/) or our versioned [developer docs](https://azure.github.io/azure-sdk-for-python).
 
 ## Getting started
 

SECURITY.md

@@ -4,15 +4,15 @@
 
 Microsoft takes the security of our software products and services seriously, which includes all source code repositories managed through our GitHub organizations, which include [Microsoft](https://github.com/Microsoft), [Azure](https://github.com/Azure), [DotNet](https://github.com/dotnet), [AspNet](https://github.com/aspnet), [Xamarin](https://github.com/xamarin), and [our GitHub organizations](https://opensource.microsoft.com/).
 
-If you believe you have found a security vulnerability in any Microsoft-owned repository that meets [Microsoft's definition of a security vulnerability](https://docs.microsoft.com/en-us/previous-versions/tn-archive/cc751383(v=technet.10)), please report it to us as described below.
+If you believe you have found a security vulnerability in any Microsoft-owned repository that meets [Microsoft's definition of a security vulnerability](https://docs.microsoft.com/previous-versions/tn-archive/cc751383(v=technet.10)), please report it to us as described below.
 
 ## Reporting Security Issues
 
 **Please do not report security vulnerabilities through public GitHub issues.**
 
 Instead, please report them to the Microsoft Security Response Center (MSRC) at [https://msrc.microsoft.com/create-report](https://msrc.microsoft.com/create-report).
 
-If you prefer to submit without logging in, send email to [[email protected]](mailto:[email protected]).  If possible, encrypt your message with our PGP key; please download it from the [Microsoft Security Response Center PGP Key page](https://www.microsoft.com/en-us/msrc/pgp-key-msrc).
+If you prefer to submit without logging in, send email to [[email protected]](mailto:[email protected]).  If possible, encrypt your message with our PGP key; please download it from the [Microsoft Security Response Center PGP Key page](https://www.microsoft.com/msrc/pgp-key-msrc).
 
 You should receive a response within 24 hours. If for some reason you do not, please follow up via email to ensure we received your original message. Additional information can be found at [microsoft.com/msrc](https://www.microsoft.com/msrc). 
 
@@ -36,6 +36,6 @@ We prefer all communications to be in English.
 
 ## Policy
 
-Microsoft follows the principle of [Coordinated Vulnerability Disclosure](https://www.microsoft.com/en-us/msrc/cvd).
+Microsoft follows the principle of [Coordinated Vulnerability Disclosure](https://www.microsoft.com/msrc/cvd).
 
 <!-- END MICROSOFT SECURITY.MD BLOCK -->

common/smoketest/key_vault_base.py

@@ -1,17 +1,22 @@
+# ------------------------------------
+# Copyright (c) Microsoft Corporation.
+# Licensed under the MIT License.
+# ------------------------------------
 import os
-from azure.identity import DefaultAzureCredential, KnownAuthorities
+from azure.identity import AzureAuthorityHosts, DefaultAzureCredential
+
 
 class KeyVaultBase:
     credential_type = DefaultAzureCredential
     host_alias_map = {
-        'AzureChinaCloud': KnownAuthorities.AZURE_CHINA,
-        'AzureGermanCloud': KnownAuthorities.AZURE_GERMANY,
-        'AzureUSGovernment': KnownAuthorities.AZURE_GOVERNMENT,
-        'AzureCloud': KnownAuthorities.AZURE_PUBLIC_CLOUD,
+        "AzureChinaCloud": (AzureAuthorityHosts.AZURE_CHINA, "2016-10-01"),
+        "AzureGermanCloud": (AzureAuthorityHosts.AZURE_GERMANY, "2016-10-01"),
+        "AzureUSGovernment": (AzureAuthorityHosts.AZURE_GOVERNMENT, "2016-10-01"),
+        "AzureCloud": (AzureAuthorityHosts.AZURE_PUBLIC_CLOUD, "7.1"),
     }
 
-    # Instantiate a default credential based on the credential_type
-    def get_default_credential(self, authority_host_alias=None):
-        alias = authority_host_alias or os.environ.get("AZURE_CLOUD")
-        authority_host = self.host_alias_map.get(alias, KnownAuthorities.AZURE_PUBLIC_CLOUD)
-        return self.credential_type(authority=authority_host)
+    def get_client_args(self, authority_host_alias=None):
+        alias = authority_host_alias or os.environ.get("AZURE_CLOUD", "AzureCloud")
+        authority_host, api_version = self.host_alias_map[alias]
+        credential = self.credential_type(authority=authority_host)
+        return {"api_version": api_version, "credential": credential, "vault_url": os.environ["AZURE_PROJECT_URL"]}

common/smoketest/key_vault_base_async.py

@@ -1,5 +1,10 @@
+# ------------------------------------
+# Copyright (c) Microsoft Corporation.
+# Licensed under the MIT License.
+# ------------------------------------
 from key_vault_base import KeyVaultBase
 from azure.identity.aio import DefaultAzureCredential
 
+
 class KeyVaultBaseAsync(KeyVaultBase):
     credential_type = DefaultAzureCredential

common/smoketest/key_vault_certificates.py

@@ -2,19 +2,15 @@
 # Copyright (c) Microsoft Corporation.
 # Licensed under the MIT License.
 # ------------------------------------
-import os
 import uuid
 from azure.keyvault.certificates import CertificateClient, CertificatePolicy
 from key_vault_base import KeyVaultBase
 
+
 class KeyVaultCertificates(KeyVaultBase):
     def __init__(self):
-
-        credential = self.get_default_credential()
-        self.certificate_client = CertificateClient(
-            vault_url=os.environ["AZURE_PROJECT_URL"], credential=credential
-        )
-
+        args = self.get_client_args()
+        self.certificate_client = CertificateClient(**args)
         self.certificate_name = "cert-name-" + uuid.uuid1().hex
 
     def create_certificate(self):

common/smoketest/key_vault_certificates_async.py

@@ -2,19 +2,16 @@
 # Copyright (c) Microsoft Corporation.
 # Licensed under the MIT License.
 # ------------------------------------
-import os
 import uuid
 from azure.keyvault.certificates import CertificatePolicy
 from azure.keyvault.certificates.aio import CertificateClient
 from key_vault_base_async import KeyVaultBaseAsync
 
+
 class KeyVaultCertificates(KeyVaultBaseAsync):
     def __init__(self):
-        credential = self.get_default_credential()
-        self.certificate_client = CertificateClient(
-            vault_url=os.environ["AZURE_PROJECT_URL"], credential=credential
-        )
-
+        args = self.get_client_args()
+        self.certificate_client = CertificateClient(**args)
         self.certificate_name = "cert-name-" + uuid.uuid1().hex
 
     async def create_certificate(self):

common/smoketest/key_vault_keys.py

@@ -2,19 +2,15 @@
 # Copyright (c) Microsoft Corporation.
 # Licensed under the MIT License.
 # ------------------------------------
-import os
 import uuid
 from azure.keyvault.keys import KeyClient
 from key_vault_base import KeyVaultBase
 
 
 class KeyVaultKeys(KeyVaultBase):
     def __init__(self):
-        credential = self.get_default_credential()
-        self.key_client = KeyClient(
-            vault_url=os.environ["AZURE_PROJECT_URL"], credential=credential
-        )
-
+        args = self.get_client_args()
+        self.key_client = KeyClient(**args)
         self.key_name = "key-name-" + uuid.uuid1().hex
 
     def create_rsa_key(self):

common/smoketest/key_vault_keys_async.py

@@ -2,20 +2,15 @@
 # Copyright (c) Microsoft Corporation.
 # Licensed under the MIT License.
 # ------------------------------------
-import os
 import uuid
 from azure.keyvault.keys.aio import KeyClient
 from key_vault_base_async import KeyVaultBaseAsync
 
 
 class KeyVaultKeys(KeyVaultBaseAsync):
     def __init__(self):
-
-        credential = self.get_default_credential()
-        self.key_client = KeyClient(
-            vault_url=os.environ["AZURE_PROJECT_URL"], credential=credential
-        )
-
+        args = self.get_client_args()
+        self.key_client = KeyClient(**args)
         self.key_name = "key-name-" + uuid.uuid1().hex
 
     async def create_rsa_key(self):

common/smoketest/key_vault_secrets.py

@@ -2,18 +2,15 @@
 # Copyright (c) Microsoft Corporation.
 # Licensed under the MIT License.
 # ------------------------------------
-import os
 import uuid
 from azure.keyvault.secrets import SecretClient
 from key_vault_base import KeyVaultBase
 
+
 class KeyVaultSecrets(KeyVaultBase):
     def __init__(self):
-        credential = self.get_default_credential()
-        self.secret_client = SecretClient(
-            vault_url=os.environ["AZURE_PROJECT_URL"], credential=credential
-        )
-
+        args = self.get_client_args()
+        self.secret_client = SecretClient(**args)
         self.secret_name = "secret-name-" + uuid.uuid1().hex
         self.secret_Value = "secret-value"
 

common/smoketest/key_vault_secrets_async.py

@@ -2,17 +2,15 @@
 # Copyright (c) Microsoft Corporation.
 # Licensed under the MIT License.
 # ------------------------------------
-import os
 import uuid
 from azure.keyvault.secrets.aio import SecretClient
 from key_vault_base_async import KeyVaultBaseAsync
 
+
 class KeyVaultSecrets(KeyVaultBaseAsync):
     def __init__(self):
-        credential = self.get_default_credential()
-        self.secret_client = SecretClient(
-            vault_url=os.environ["AZURE_PROJECT_URL"], credential=credential
-        )
+        args = self.get_client_args()
+        self.secret_client = SecretClient(**args)
         self.secret_name = "secret-name-" + uuid.uuid1().hex
         self.secret_value = "secret-value"
 

doc/dev/mgmt/generating-integration-test.md

@@ -123,7 +123,7 @@ now you can run live integration test:
 
     pytest -s sdk/attestation/azure-mgmt-attestation
 
->NOTE: To create service principal, follow instructions here: https://docs.microsoft.com/en-us/azure/active-directory/develop/howto-create-service-principal-portal
+>NOTE: To create service principal, follow instructions here: https://docs.microsoft.com/azure/active-directory/develop/howto-create-service-principal-portal
 
 ## Fixing Test
 

doc/dev/mgmt/tests.md

@@ -66,10 +66,10 @@ you must use an OAuth authentication method which gives you a token:
 
 Certificate authentication does not allow you to record HTTP queries for testing.
 
-### Get a token with Azure Active Directory user/password. This is considered deprecated and should not be used anymore (https://docs.microsoft.com/en-us/python/azure/python-sdk-azure-authenticate?view=azure-python#mgmt-auth-legacy).
+### Get a token with Azure Active Directory user/password. This is considered deprecated and should not be used anymore (https://docs.microsoft.com/python/azure/python-sdk-azure-authenticate?view=azure-python#mgmt-auth-legacy).
 
 1.  Connect to the [Azure Classic Portal](https://manage.windowsazure.com/) with your admin account.
-2.  Create a user in your default AAD https://azure.microsoft.com/en-us/documentation/articles/active-directory-create-users/
+2.  Create a user in your default AAD https://azure.microsoft.com/documentation/articles/active-directory-create-users/
    **You must NOT activate Multi-Factor Authentication!**
 3.  Go to Settings - Administrators.
 4.  Click on *Add* and enter the email of the new user.
@@ -89,13 +89,13 @@ credentials = UserPassCredentials(
 ### Get a token with Active Directory application and service principal
 
 Follow this detailed tutorial to set up an Active Directory application and service principal:
-https://azure.microsoft.com/en-us/documentation/articles/resource-group-create-service-principal-portal/
+https://azure.microsoft.com/documentation/articles/resource-group-create-service-principal-portal/
 
 To use the credentials from Python,
 you need the application ID (a.k.a. client ID),
 authentication key (a.k.a. client secret),
 tenant ID and subscription ID from the Azure portal for use in the next step.
-[This section of the above tutorial](https://docs.microsoft.com/en-us/azure/azure-resource-manager/resource-group-create-service-principal-portal#get-application-id-and-authentication-key)
+[This section of the above tutorial](https://docs.microsoft.com/azure/azure-resource-manager/resource-group-create-service-principal-portal#get-application-id-and-authentication-key)
 describes where to find them
 (besides the subscription ID,
 which is in the "Overview" section of the "Subscriptions" blade.)

doc/dev/release.md

@@ -16,11 +16,11 @@ Python packages are uploaded to [PyPI](https://pypi.org/). Once you've uploaded
 
 ### Production - Deploy with Azure Dev Ops
 
-To avoid "accidental" pushes to our target repositories, [approval](https://docs.microsoft.com/en-us/azure/devops/pipelines/release/approvals/approvals?view=azure-devops) will be requested directly prior to the final PyPI publish. Reference this [wiki page](https://aka.ms/python-approval-groups) and click on `Release to PyPI Approvers` to add yourself to the group for PyPI publishing. 
+To avoid "accidental" pushes to our target repositories, [approval](https://docs.microsoft.com/azure/devops/pipelines/release/approvals/approvals?view=azure-devops) will be requested directly prior to the final PyPI publish. Reference this [wiki page](https://aka.ms/python-approval-groups) and click on `Release to PyPI Approvers` to add yourself to the group for PyPI publishing. 
 
 Instead of a single central pipeline, the python SDK has moved to `service directory` associated build pipelines. These are driven by yml templates at the root of each service folder. [Example for storage service folder.](https://github.com/Azure/azure-sdk-for-python/blob/master/sdk/storage/ci.yml#L44)
 
-As an aside, please note that the preview feature `multi-stage pipelines` must be enabled to properly interact with unified pipelines. If you aren't aware, find out how to enable by visiting [this link.](https://docs.microsoft.com/en-us/azure/devops/project/navigation/preview-features?view=azure-devops)
+As an aside, please note that the preview feature `multi-stage pipelines` must be enabled to properly interact with unified pipelines. If you aren't aware, find out how to enable by visiting [this link.](https://docs.microsoft.com/azure/devops/project/navigation/preview-features?view=azure-devops)
 
 #### Releasing Through Unified Pipelines
 

eng/.docsettings.yml

@@ -48,6 +48,7 @@ known_content_issues:
   - ['sdk/synapse/azure-synapse-spark/README.md', '#4554']
   - ['sdk/synapse/azure-synapse-artifacts/README.md', '#4554']
   - ['sdk/synapse/azure-synapse-nspkg/README.md', '#4554']
+  - ['sdk/anomalydetector/azure-ai-anomalydetector/README.md', '#4554']
   - ['sdk/applicationinsights/azure-applicationinsights/README.md', '#4554']
   - ['sdk/batch/azure-batch/README.md', '#4554']
   - ['sdk/cognitiveservices/azure-cognitiveservices-anomalydetector/README.md', '#4554']

eng/common/README.md

@@ -4,9 +4,20 @@ The `eng/common` directory contains engineering files that are common across the
 It should remain relatively small and only contain textual based files like scripts, configs, or templates. It
 should not contain binary files as they don't play well with git.
 
-# Updating
+## Updating
 
 Any updates to files in the `eng/common` directory should be made in the [azure-sdk-tools](https://github.com/azure/azure-sdk-tools) repo.
 All changes made will cause a PR to created in all subscribed azure-sdk language repos which will blindly replace all contents of
 the `eng/common` directory in that repo. For that reason do **NOT** make changes to files in this directory in the individual azure-sdk
-languages repos as they will be overwritten the next time an update is taken from the common azure-sdk-tools repo.
+languages repos as they will be overwritten the next time an update is taken from the common azure-sdk-tools repo.
+
+### Workflow
+
+The 'Sync eng/common directory' PRs will be created in the language repositories once a pull request that touches the eng/common directory is submitted against the master branch. This will make it easier for changes to be tested in each individual language repo before merging the changes in the azure-sdk-tools repo. The workflow is explained below:
+
+1. Create a PR against Azure/azure-sdk-tools:master. This is the **Tools PR**.
+2. `azure-sdk-tools - sync - eng-common` is run automatically. It creates **Sync PRs** in each of the connected language repositories using the format `Sync eng/common directory with azure-sdk-tools for PR {Tools PR Number}`. Each **Sync PR** will contain a link back to the **Tools PR** that triggered it.
+3. More changes pushed to the **Tools PR**, will automatically triggered new pipeline runs in the respective **Sync PRs**. The **Sync PRs** are used to make sure the changes would not break any of the connected pipelines.
+4. Once satisfied with the changes;
+    - First merge all the **Sync PRs**. The **Tools PR** contains links to all the **Sync PRs**
+    - Finally merge the **Tools PR**. Each **Sync PR** contains the link to the corresponding **Tools PR**.