Merge branch 'master' of https://github.com/Azure/azure-sdk-for-python into assertions

* 'master' of https://github.com/Azure/azure-sdk-for-python:
  [core] add HttpRequest and HttpResponse reprs (Azure#16972)
  [text analytics] add actual normalized_text tests (Azure#17123)
  update samples to use actual role names (Azure#17124)
  Sync eng/common directory with azure-sdk-tools for PR 1448 (Azure#17085)
  Enable APIView status check (Azure#17107)
  Fix PackageName typo (Azure#17109)
  Move SetTestPipelineVersion.ps1 to eng/common (Azure#17103)
  Fix paths for non-windows agents (Azure#17096)
  [Communication] - Identity - Update README (Azure#17091)
  Rename CertificateCredential's certificate_bytes -> certificate_data (Azure#17090)
  fix shared reqs (Azure#17095)
  [translation] initial library (Azure#16837)
  EG - more docs imrpovement (Azure#17079)
  [EventHubs] add logging.info to warn the usage of partition key of non-string type (Azure#17057)
  march release (Azure#16966)
  Release of Device Update for IoT Hub SDK for Python. (Azure#17005)
  Add Get-AllPackageInfoFromRepo (Azure#16947)
  Track1 package is incorrectly set as track2 (Azure#17075)

Author: iscai-msft

eng/common/pipelines/templates/steps/create-apireview.yml

@@ -19,4 +19,3 @@ steps:
         workingDirectory: $(Pipeline.Workspace)
       displayName: Create API Review for ${{ artifact.name}}
       condition: and(succeededOrFailed(), ne(variables['Skip.CreateApiReview'], 'true') , ne(variables['Build.Reason'],'PullRequest'), eq(variables['System.TeamProject'], 'internal'))
-      continueOnError: true

eng/common/scripts/Create-APIReview.ps1

@@ -41,11 +41,12 @@ function Submit-APIReview($packagename, $filePath, $uri, $apiKey, $apiLabel)
     try
     {
         $Response = Invoke-WebRequest -Method 'POST' -Uri $uri -Body $multipartContent -Headers $headers
-        Write-Host "API Review: $($Response)"
+        Write-Host "API Review URL: $($Response.Content)"
         $StatusCode = $Response.StatusCode
     }
     catch
     {
+        Write-Host "Exception details: $($_.Exception.Response)"
         $StatusCode = $_.Exception.Response.StatusCode
     }
 
@@ -101,6 +102,9 @@ foreach ($pkgName in $responses.Keys)
         {
             $pkgInfo = Get-Content $pkgPropPath | ConvertFrom-Json
             $version = [AzureEngSemanticVersion]::ParseVersionString($pkgInfo.Version)
+            Write-Host "Package name: $($PackageName)"
+            Write-Host "Version: $($version)"
+            Write-Host "SDK Type: $($pkgInfo.SdkType)"
             if ($version.IsPrerelease)
             {
                 Write-Host "Package version is not GA. Ignoring API view approval status"
@@ -110,12 +114,16 @@ foreach ($pkgName in $responses.Keys)
                 $FoundFailure = $True
                 if ($respCode -eq '201')
                 {
-                    Write-Error "Automatic API Review approval is pending for package $($PackageName)"
+                    Write-Host "Package version $($version) is GA and automatic API Review is not yet approved for package $($PackageName)."
+                    Write-Host "Build and release is not allowed for GA package without API review approval."
+                    Write-Host "You will need to queue another build to proceed further after API review is approved"
+                    Write-Host "You can check http://aka.ms/azsdk/engsys/apireview/faq for more details on API Approval."
                 }
                 else
                 {
-                    Write-Error "Failed to create API Review for package $($PackageName)"
-                }                
+                    Write-Host "Failed to create API Review for package $($PackageName). Please reach out to Azure SDK engineering systems on teams channel and share this build details."
+                }
+                exit 1
             }
             else
             {
@@ -124,8 +132,3 @@ foreach ($pkgName in $responses.Keys)
         }
     }
 }
-if ($FoundFailure)
-{
-    Write-Error "Automatic API review is not yet approved for package $($PackageName)"
-    exit 1
-}

eng/common/scripts/SetTestPipelineVersion.ps1

@@ -0,0 +1,33 @@
+# Sets a valid version for a package using the buildID
+
+param (
+  [Parameter(mandatory = $true)]
+  $BuildID,
+  [Parameter(mandatory = $true)]
+  $PackageName,
+  [Parameter(mandatory = $true)]
+  $ServiceDirectory
+)
+
+. common.ps1
+
+$latestTags = git tag -l "${PackageName}_*"
+$semVars = @()
+
+Foreach ($tags in $latestTags)
+{
+  $semVars += $tags.Replace("${PackageName}_", "")
+}
+
+$semVarsSorted = [AzureEngSemanticVersion]::SortVersionStrings($semVars)
+LogDebug "Last Published Version $($semVarsSorted[0])"
+
+$newVersion = [AzureEngSemanticVersion]::new($semVarsSorted[0])
+$newVersion.PrereleaseLabel = $newVersion.DefaultPrereleaseLabel
+$newVersion.PrereleaseNumber = $BuildID
+
+LogDebug "Version to publish [ $($newVersion.ToString()) ]"
+
+SetPackageVersion -PackageName $PackagName `
+  -Version $newVersion `
+  -ServiceDirectory $ServiceDirectory `

eng/scripts/Language-Settings.ps1

@@ -5,6 +5,58 @@ $packagePattern = "*.zip"
 $MetadataUri = "https://raw.githubusercontent.com/Azure/azure-sdk/master/_data/releases/latest/python-packages.csv"
 $BlobStorageUrl = "https://azuresdkdocs.blob.core.windows.net/%24web?restype=container&comp=list&prefix=python%2F&delimiter=%2F"
 
+function Get-AllPackageInfoFromRepo ($serviceDirectory)
+{
+  $allPackageProps = @()
+  $searchPath = Join-Path sdk * * setup.py
+  if ($serviceDirectory)
+  {
+    $searchPath = Join-Path sdk ${serviceDirectory} * setup.py
+  }
+
+  $allPkgPropLines = $null
+  try
+  {
+    Push-Location $RepoRoot
+    pip install packaging==20.4 -q -I
+    $allPkgPropLines = python (Join-path eng scripts get_package_properties.py) -s $searchPath
+  }
+  catch
+  {
+    # This is soft error and failure is expected for python metapackages
+    LogError "Failed to get all package properties"
+  }
+  finally
+  {
+    Pop-Location
+  }
+
+  foreach ($line in $allPkgPropLines)
+  {
+    $pkgInfo = ($line -Split ",").Trim("()' ")
+    $packageName = $pkgInfo[0]
+    $packageVersion = $pkgInfo[1]
+    $isNewSdk = ($pkgInfo[2] -eq "True")
+    $setupPyDir = $pkgInfo[3]
+    $pkgDirectoryPath = Resolve-Path (Join-Path -Path $RepoRoot $setupPyDir)
+    $serviceDirectoryName = Split-Path (Split-Path -Path $pkgDirectoryPath -Parent) -Leaf
+    if ($packageName -match "mgmt")
+    {
+      $sdkType = "mgmt"
+    }
+    else
+    {
+      $sdkType = "client"
+    }
+    $pkgProp = [PackageProps]::new($packageName, $packageVersion, $pkgDirectoryPath, $serviceDirectoryName)
+    $pkgProp.IsNewSdk = $isNewSdk
+    $pkgProp.SdkType = $sdkType
+    $pkgProp.ArtifactName = $packageName
+    $allPackageProps += $pkgProp
+  }
+  return $allPackageProps
+}
+
 function Get-python-PackageInfoFromRepo  ($pkgPath, $serviceDirectory, $pkgName)
 {  
   $packageName = $pkgName.Replace('_', '-')
@@ -41,7 +93,7 @@ function Get-python-PackageInfoFromRepo  ($pkgPath, $serviceDirectory, $pkgName)
       {
         $pkgProp.SdkType = "client"
       }
-      $pkgProp.IsNewSdk = $setupProps[2]
+      $pkgProp.IsNewSdk = ($setupProps[2] -eq "True")
       $pkgProp.ArtifactName = $pkgName
       return $pkgProp
     }

eng/scripts/get_package_properties.py

@@ -0,0 +1,16 @@
+import argparse
+import sys
+import glob
+import os
+
+sys.path.append(os.path.join('scripts', 'devops_tasks'))
+from common_tasks import get_package_properties
+
+if __name__ == '__main__':
+    parser = argparse.ArgumentParser(description='Get package version details from the repo')
+    parser.add_argument('-s', '--search_path', required=True, help='The scope of the search')
+    args = parser.parse_args()
+
+    for p in glob.glob(args.search_path, recursive=True):
+        if os.path.basename(os.path.dirname(p)) != 'azure-mgmt' and os.path.basename(os.path.dirname(p)) != 'azure' and os.path.basename(os.path.dirname(p)) != 'azure-storage':
+            print(get_package_properties(os.path.dirname(p)))

sdk/communication/azure-communication-identity/README.md

@@ -2,7 +2,7 @@
 
 # Azure Communication Identity Package client library for Python
 
-Azure Communication Identity client package is intended to be used to setup the basics for opening a way to use Azure Communication Service offerings. This package helps to create identities user tokens to be used by other client packages such as chat, calling, sms. 
+Azure Communication Identity client package is intended to be used to setup the basics for opening a way to use Azure Communication Service offerings. This package helps to create identity user tokens to be used by other client packages such as chat, calling, sms.
 
 [Source code](https://github.com/Azure/azure-sdk-for-python/blob/master/sdk/communication/azure-communication-identity) | [Package (Pypi)](https://pypi.org/project/azure-communication-identity/) | [API reference documentation](https://github.com/Azure/azure-sdk-for-python/blob/master/sdk/communication/azure-communication-identity) | [Product documentation](https://docs.microsoft.com/azure/communication-services/quickstarts/access-tokens?pivots=programming-language-python)
 
@@ -51,21 +51,12 @@ The following section provides several code snippets covering some of the most c
 
 ### Creating a new user
 
-Use the `create_user` method to create a new user. 
+Use the `create_user` method to create a new user.
 ```python
 user = identity_client.create_user()
 print("User created with id:" + user.identifier)
 ```
 
-Alternatively, use the `create_user_and_token` method to create a new user and issue a token for it.\
-For this option, a list of `CommunicationTokenScope` must be defined (see "Issuing an access token" for more information)
-
-```python
-user, tokenresponse = identity_client.create_user_and_token(scopes=[CommunicationTokenScope.CHAT])
-print("User id:" + user.identifier)
-print("Token issued with value: " + tokenresponse.token)
-```
-
 ### Issuing or Refreshing an access token for a user
 
 Use the `get_token` method to issue or refresh a scoped access token for the user. \
@@ -77,6 +68,14 @@ Pass in the user object as a parameter, and a list of `CommunicationTokenScope`.
 tokenresponse = identity_client.get_token(user, scopes=[CommunicationTokenScope.CHAT])
 print("Token issued with value: " + tokenresponse.token)
 ```
+### Creating a user and a token in a single request
+For convenience, use `create_user_and_token` to create a new user and issue a token with one function call. This translates into a single web request as opposed to creating a user first and then issuing a token.
+
+```python
+user, tokenresponse = identity_client.create_user_and_token(scopes=[CommunicationTokenScope.CHAT])
+print("User id:" + user.identifier)
+print("Token issued with value: " + tokenresponse.token)
+```
 
 ### Revoking a user's access tokens
 

sdk/core/azure-core/CHANGELOG.md

@@ -1,11 +1,15 @@
 # Release History
 
-## 1.12.0 (Unreleased)
+## 1.12.0 (2021-03-08)
 
 ### Features
 
 - Added `azure.core.messaging.CloudEvent` model that follows the cloud event spec.
-- Added `azure.core.serialization.NULL` sentinel value 
+- Added `azure.core.serialization.NULL` sentinel value
+
+### Bug Fixes
+
+- Improve `repr`s for `HttpRequest` and `HttpResponse`s  #16972
 
 ## 1.11.0 (2021-02-08)
 

sdk/core/azure-core/azure/core/pipeline/transport/_base.py

@@ -234,7 +234,9 @@ def __init__(self, method, url, headers=None, files=None, data=None):
         self.multipart_mixed_info = None  # type: Optional[Tuple]
 
     def __repr__(self):
-        return "<HttpRequest [%s]>" % (self.method)
+        return "<HttpRequest [{}], url: '{}'>".format(
+            self.method, self.url
+        )
 
     def __deepcopy__(self, memo=None):
         try:
@@ -592,6 +594,15 @@ def raise_for_status(self):
         if self.status_code >= 400:
             raise HttpResponseError(response=self)
 
+    def __repr__(self):
+        # there doesn't have to be a content type
+        content_type_str = (
+            ", Content-Type: {}".format(self.content_type) if self.content_type else ""
+        )
+        return "<{}: {} {}{}>".format(
+            type(self).__name__, self.status_code, self.reason, content_type_str
+        )
+
 
 class HttpResponse(_HttpResponseBase):  # pylint: disable=abstract-method
     def stream_download(self, pipeline):

sdk/core/azure-core/tests/async_tests/test_universal_http_async.py

@@ -98,6 +98,8 @@ def __init__(self, body_bytes, headers=None):
             self._body = body_bytes
             self._headers = headers
             self._cache = {}
+            self.status = 200
+            self.reason = "OK"
 
     req_response = MockAiohttpClientResponse(body_bytes, headers)
 
@@ -120,3 +122,12 @@ async def test_aiohttp_response_text():
             {'Content-Type': 'text/plain'}
         )
         assert res.text(encoding) == '56', "Encoding {} didn't work".format(encoding)
+
+def test_repr():
+    res = _create_aiohttp_response(
+        b'\xef\xbb\xbf56',
+        {}
+    )
+    res.content_type = "text/plain"
+
+    assert repr(res) == "<AioHttpTransportResponse: 200 OK, Content-Type: text/plain>"

sdk/core/azure-core/tests/test_pipeline.py

@@ -285,7 +285,7 @@ def test_request_url_with_params(self):
         request.format_parameters({"g": "h"})
 
         self.assertIn(request.url, ["a/b/c?g=h&t=y", "a/b/c?t=y&g=h"])
-    
+
     def test_request_url_with_params_as_list(self):
 
         request = HttpRequest("GET", "/")
@@ -300,7 +300,7 @@ def test_request_url_with_params_with_none_in_list(self):
         request.url = "a/b/c?t=y"
         with pytest.raises(ValueError):
             request.format_parameters({"g": ["h",None]})
-    
+
     def test_request_url_with_params_with_none(self):
 
         request = HttpRequest("GET", "/")
@@ -328,6 +328,10 @@ def test_request_text(self):
         # We want a direct string
         assert request.data == "foo"
 
+    def test_repr(self):
+        request = HttpRequest("GET", "hello.com")
+        assert repr(request) == "<HttpRequest [GET], url: 'hello.com'>"
+
 
 if __name__ == "__main__":
     unittest.main()

sdk/core/azure-core/tests/test_requests_universal.py

@@ -82,3 +82,10 @@ def test_requests_response_text():
             {'Content-Type': 'text/plain'}
         )
         assert res.text(encoding) == '56', "Encoding {} didn't work".format(encoding)
+
+def test_repr():
+    res = _create_requests_response(
+        b'\xef\xbb\xbf56',
+        {'Content-Type': 'text/plain'}
+    )
+    assert repr(res) == "<RequestsTransportResponse: 200 OK, Content-Type: text/plain>"

sdk/deviceupdate/azure-iot-deviceupdate/CHANGELOG.md

@@ -0,0 +1,4 @@
+# Release History
+
+## 1.0.0b1 (2021-03-03)
+* This is the initial release of Azure Device Update for IoT Hub library. 

sdk/deviceupdate/azure-iot-deviceupdate/MANIFEST.in

@@ -0,0 +1,5 @@
+recursive-include tests *.py
+recursive-include samples *.py *.md
+include *.md
+include azure/__init__.py
+include azure/iot/__init__.py

sdk/deviceupdate/azure-iot-deviceupdate/README.md

@@ -0,0 +1,51 @@
+# Azure Device Update for IoT Hub client library for Python
+
+The library provides access to the Device Update for IoT Hub service that enables customers to publish updates for their IoT devices to the cloud, and then deploy these updates to their devices (approve updates to groups of devices managed and provisioned in IoT Hub).
+
+[Source code](https://github.com/Azure/azure-sdk-for-python/tree/master/sdk/deviceupdate/azure-iot-deviceupdate) | [Package (PyPI)](https://aka.ms/azsdk/python/deviceupdate-pypi) | [Product documentation](https://docs.microsoft.com/azure/iot-hub-device-update/understand-device-update)
+
+## Getting started
+
+### Prerequisites
+
+- Microsoft Azure Subscription: To call Microsoft Azure services, you need to create an [Azure subscription](https://azure.microsoft.com/free/)
+- Device Update for IoT Hub instance
+- Azure IoT Hub instance
+- Python 2.7, or 3.6 or later is required to use this package.
+
+### Install the package
+
+Install the Device Update for IoT Hub client library for Python with [pip](https://pypi.org/project/pip/):
+
+```bash
+pip install azure-iot-deviceupdate --pre
+```
+
+## Key concepts
+
+Device Update for IoT Hub is a managed service that enables you to deploy over-the-air updates for your IoT devices. The client library has one main component named **AzureDeviceUpdateServiceDataPlane**. The component allows you to access all three main client services:
+
+- **UpdatesOperations**: update management (import, enumerate, delete, etc.)
+- **DevicesOperations**: device management (enumerate devices and retrieve device properties)
+- **DeploymentsOperations**: deployment management (start and monitor update deployments to a set of devices)
+
+You can learn more about Device Update for IoT Hub by visiting [Device Update for IoT Hub](https://github.com/azure/iot-hub-device-update).
+
+## Examples
+
+You can familiarize yourself with different APIs using [Samples](https://github.com/Azure/azure-sdk-for-python/tree/master/sdk/deviceupdate/azure-iot-deviceupdate/samples).
+
+## Troubleshooting
+
+The Device Update for IoT Hub client will raise exceptions defined in [Azure Core][https://github.com/azure/azure-sdk-for-python/blob/master/sdk/core/azure-core/readme.md].
+
+## Next steps
+
+Get started with our [Device Update for IoT Hub samples](https://github.com/Azure/azure-sdk-for-python/tree/master/sdk/deviceupdate/azure-iot-deviceupdate/samples)
+
+## Contributing
+
+If you encounter any bugs or have suggestions, please file an issue in the [Issues](https://github.com/Azure/azure-sdk-for-python/issues) section of the project.
+
+<!-- LINKS -->
+[azure_core]: https://github.com/Azure/azure-sdk-for-python/blob/master/sdk/core/azure-core/README.md

sdk/deviceupdate/azure-iot-deviceupdate/azure/__init__.py

@@ -0,0 +1 @@
+__path__ = __import__('pkgutil').extend_path(__path__, __name__)  # type: ignore