Skip to content

Commit 217dabc

Browse files
allenkim0129allenkim0129
committed
Merged main
1 parent c6e2f1b commit 217dabc

File tree

649 files changed

+25945
-21517
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

649 files changed

+25945
-21517
lines changed

‎doc/analyze_check_versions.md

+2-2
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,7 @@ This table is to clarify the currently pinned version of tools we run in CI and
66
| Tool | Current Version | Next Version | Next Version Merge Date |
77
|------|-----------------|--------------|-------------------------|
88
Pylint | 3.2.5 | 3.2.7 | 2025-01-13 |
9-
MyPy | 1.10.1 | TBD | 2025-01-13 |
10-
Pyright | 1.1.371 | TBD | 2025-01-13 |
9+
MyPy | 1.10.1 | 1.13.0 | 2025-01-13 |
10+
Pyright | 1.1.371 | 1.1.389 | 2025-01-13 |
1111
Sphinx | 7.3.7 | N/A | N/A |
1212
Black | 24.4.0 | N/A | N/A |

‎doc/eng_sys_checks.md

+10-8
Original file line numberDiff line numberDiff line change
@@ -112,6 +112,8 @@ This is the most useful skip, but the following skip variables are also supporte
112112
- Omit checking that a package's dependencies are on PyPI before releasing.
113113
- `Skip.KeywordCheck`
114114
- Omit checking that a package's keywords are correctly formulated before releasing.
115+
- `Skip.Black`
116+
- Omit checking `black` in the `analyze` job.
115117

116118
## The pyproject.toml
117119

@@ -172,7 +174,7 @@ You can enable test logging in a pipeline by setting the queue time variable `PY
172174

173175
`PYTEST_LOG_LEVEL=INFO`
174176

175-
This also works locally with tox by setting the `PYTEST_LOG_LEVEL` environment variable.
177+
This also works locally with tox by setting the `PYTEST_LOG_LEVEL` environment variable.
176178

177179
Note that if you want DEBUG level logging with sensitive information unredacted in the test logs, then you still must pass `logging_enable=True` into the client(s) being used in tests.
178180

@@ -237,17 +239,17 @@ fail if docstring are invalid, helping to ensure the resulting documentation wil
237239

238240
#### Opt-in to formatting validation
239241

240-
Make the following change to your projects `ci.yml`:
242+
Ensure that `black = true` is present within your `pyproject.toml`:
241243

242244
```yml
243-
extends:
244-
template: ../../eng/pipelines/templates/stages/archetype-sdk-client.yml
245-
parameters:
246-
...
247-
ValidateFormatting: true
248-
...
245+
[tool.azure-sdk-build]
246+
...other checks enabled/disabled
247+
black = true
248+
...other checks enabled/disabled
249249
```
250250

251+
to opt into the black invocation.
252+
251253
#### Running locally
252254

253255
1. Go to package root directory.

‎eng/common/scripts/Cadl-Project-Generate.ps1

+1-1
Original file line numberDiff line numberDiff line change
@@ -12,7 +12,7 @@ param (
1212
$ErrorActionPreference = "Stop"
1313
. $PSScriptRoot/Helpers/PSModule-Helpers.ps1
1414
. $PSScriptRoot/common.ps1
15-
Install-ModuleIfNotInstalled "powershell-yaml" "0.4.1" | Import-Module
15+
Install-ModuleIfNotInstalled "powershell-yaml" "0.4.7" | Import-Module
1616

1717
function NpmInstallForProject([string]$workingDirectory) {
1818
Push-Location $workingDirectory

‎eng/common/scripts/Cadl-Project-Sync.ps1

+1-1
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,7 @@ param (
99

1010
$ErrorActionPreference = "Stop"
1111
. $PSScriptRoot/Helpers/PSModule-Helpers.ps1
12-
Install-ModuleIfNotInstalled "powershell-yaml" "0.4.1" | Import-Module
12+
Install-ModuleIfNotInstalled "powershell-yaml" "0.4.7" | Import-Module
1313
$sparseCheckoutFile = ".git/info/sparse-checkout"
1414

1515
function AddSparseCheckoutPath([string]$subDirectory) {

‎eng/common/scripts/Helpers/ApiView-Helpers.ps1

+2-1
Original file line numberDiff line numberDiff line change
@@ -74,6 +74,7 @@ function Process-ReviewStatusCode($statusCode, $packageName, $apiApprovalStatus,
7474
{
7575
$apiApproved = $false
7676
$apiApprovalDetails = "API Review is not approved for package $($packageName). Release pipeline will fail if API review is not approved for a GA version release. You can check http://aka.ms/azsdk/engsys/apireview/faq for more details on API Approval."
77+
$apiApprovalDetails += " Once your API is approved, re-trigger the release pipeline again."
7778

7879
$packageNameApproved = $false
7980
$packageNameApprovalDetails = ""
@@ -115,4 +116,4 @@ function Process-ReviewStatusCode($statusCode, $packageName, $apiApprovalStatus,
115116

116117
$packageNameStatus.IsApproved = $packageNameApproved
117118
$packageNameStatus.Details = $packageNameApprovalDetails
118-
}
119+
}

‎eng/common/scripts/Helpers/Package-Helpers.ps1

+1-1
Original file line numberDiff line numberDiff line change
@@ -80,7 +80,7 @@ function CompatibleConvertFrom-Yaml {
8080
$yqPresent = Get-Command 'yq' -ErrorAction SilentlyContinue
8181
if (-not $yqPresent) {
8282
. (Join-Path $PSScriptRoot PSModule-Helpers.ps1)
83-
Install-ModuleIfNotInstalled -WhatIf:$false "powershell-yaml" "0.4.1" | Import-Module
83+
Install-ModuleIfNotInstalled -WhatIf:$false "powershell-yaml" "0.4.7" | Import-Module
8484
}
8585

8686
# Process the content (for example, you could convert from YAML here)

‎eng/common/scripts/TypeSpec-Project-Generate.ps1

+1-1
Original file line numberDiff line numberDiff line change
@@ -13,7 +13,7 @@ $ErrorActionPreference = "Stop"
1313
. $PSScriptRoot/Helpers/PSModule-Helpers.ps1
1414
. $PSScriptRoot/Helpers/CommandInvocation-Helpers.ps1
1515
. $PSScriptRoot/common.ps1
16-
Install-ModuleIfNotInstalled "powershell-yaml" "0.4.1" | Import-Module
16+
Install-ModuleIfNotInstalled "powershell-yaml" "0.4.7" | Import-Module
1717

1818
function NpmInstallForProject([string]$workingDirectory) {
1919
Push-Location $workingDirectory

‎eng/common/scripts/TypeSpec-Project-Process.ps1

+2-2
Original file line numberDiff line numberDiff line change
@@ -14,7 +14,7 @@ param (
1414

1515
. $PSScriptRoot/common.ps1
1616
. $PSScriptRoot/Helpers/PSModule-Helpers.ps1
17-
Install-ModuleIfNotInstalled "powershell-yaml" "0.4.1" | Import-Module
17+
Install-ModuleIfNotInstalled "powershell-yaml" "0.4.7" | Import-Module
1818

1919
function CreateUpdate-TspLocation([System.Object]$tspConfig, [string]$TypeSpecProjectDirectory, [string]$CommitHash, [string]$repo, [string]$repoRoot, [ref]$isNewSdkProject) {
2020
$additionalDirs = @()
@@ -225,4 +225,4 @@ if ($SkipSyncAndGenerate -and !$isNewSdkProject) {
225225
}
226226
}
227227

228-
return $sdkProjectFolder
228+
return $sdkProjectFolder

‎eng/common/scripts/TypeSpec-Project-Sync.ps1

+1-1
Original file line numberDiff line numberDiff line change
@@ -11,7 +11,7 @@ param (
1111

1212
$ErrorActionPreference = "Stop"
1313
. $PSScriptRoot/Helpers/PSModule-Helpers.ps1
14-
Install-ModuleIfNotInstalled "powershell-yaml" "0.4.1" | Import-Module
14+
Install-ModuleIfNotInstalled "powershell-yaml" "0.4.7" | Import-Module
1515
$sparseCheckoutFile = ".git/info/sparse-checkout"
1616

1717
function AddSparseCheckoutPath([string]$subDirectory) {

‎eng/common/scripts/Update-DocsMsToc.ps1

+1-1
Original file line numberDiff line numberDiff line change
@@ -55,7 +55,7 @@ param(
5555
. $PSScriptRoot/common.ps1
5656
. $PSScriptRoot/Helpers/PSModule-Helpers.ps1
5757

58-
Install-ModuleIfNotInstalled "powershell-yaml" "0.4.1" | Import-Module
58+
Install-ModuleIfNotInstalled "powershell-yaml" "0.4.7" | Import-Module
5959

6060
Set-StrictMode -Version 3
6161

‎eng/common/scripts/Verify-ChangeLogs.ps1

+10-23
Original file line numberDiff line numberDiff line change
@@ -7,30 +7,17 @@ Set-StrictMode -Version 3
77

88
. (Join-Path $PSScriptRoot common.ps1)
99

10-
function ShouldVerifyChangeLog ($ServiceDirectory, $PackageName) {
11-
$jsonCiYmlPath = Join-Path $ServiceDirectory "ci.yml"
12-
13-
if (Test-Path $jsonCiYmlPath)
14-
{
15-
$ciYml = Get-Content $jsonCiYmlPath -Raw | yq -o=json | ConvertFrom-Json -AsHashTable
16-
17-
if ($ciYml.extends -and $ciYml.extends.parameters -and $ciYml.extends.parameters.Artifacts) {
18-
$packagesCheckingChangeLog = $ciYml.extends.parameters.Artifacts `
19-
| Where-Object { -not ($_["skipVerifyChangelog"] -eq $true) } `
20-
| Select-Object -ExpandProperty name
21-
if ($packagesCheckingChangeLog -contains $PackageName)
22-
{
23-
return $true
24-
} else {
25-
return $false
26-
}
27-
}
10+
11+
function ShouldVerifyChangeLog ($PkgArtifactDetails) {
12+
if ($PkgArtifactDetails) {
13+
if ($PkgArtifactDetails.PSObject.Properties["skipVerifyChangeLog"] -eq $true) {
14+
return $false
2815
}
29-
}
3016

31-
if (-not (Get-Command 'yq' -ErrorAction SilentlyContinue)) {
32-
Write-Host "Error: 'yq' is not installed or not found in PATH. Please remedy this before running this script."
33-
exit 1
17+
return $true
18+
}
19+
20+
return $false
3421
}
3522

3623
# find which packages we need to confirm the changelog for
@@ -41,7 +28,7 @@ $allPassing = $true
4128
foreach($propertiesFile in $packageProperties) {
4229
$PackageProp = Get-Content -Path $propertiesFile | ConvertFrom-Json
4330

44-
if (-not (ShouldVerifyChangeLog -ServiceDirectory (Join-Path $RepoRoot "sdk" $PackageProp.ServiceDirectory) -PackageName $PackageProp.Name)) {
31+
if (-not (ShouldVerifyChangeLog $PackageProp.ArtifactDetails)) {
4532
Write-Host "Skipping changelog verification for $($PackageProp.Name)"
4633
continue
4734
}

‎eng/common/scripts/Verify-Resource-Ref.ps1

+2-2
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
. (Join-Path $PSScriptRoot common.ps1)
2-
Install-Module -Name powershell-yaml -RequiredVersion 0.4.1 -Force -Scope CurrentUser
2+
Install-Module -Name powershell-yaml -RequiredVersion 0.4.7 -Force -Scope CurrentUser
33
$ymlfiles = Get-ChildItem $RepoRoot -recurse | Where-Object {$_ -like '*.yml'}
44
$affectedRepos = [System.Collections.ArrayList]::new()
55

@@ -48,4 +48,4 @@ if ($affectedRepos.Count -gt 0)
4848
exit 1
4949
}
5050

51-
Write-Output "All repository resources in yaml files reference a valid tag"
51+
Write-Output "All repository resources in yaml files reference a valid tag"

‎eng/common/scripts/Verify-RestApiSpecLocation.ps1

+1-1
Original file line numberDiff line numberDiff line change
@@ -50,7 +50,7 @@ if (-not $GitHubPat) {
5050

5151
Write-Host "The spec used to release SDK should be from the main branch of Azure/azure-rest-api-specs repository."
5252
Write-Host "ServiceDir:$ServiceDirectory, PackageName:$PackageName, ArtifactLocation:$ArtifactLocation, PackageInfoDirectory:$PackageInfoDirectory."
53-
Install-ModuleIfNotInstalled "powershell-yaml" "0.4.1" | Import-Module
53+
Install-ModuleIfNotInstalled "powershell-yaml" "0.4.7" | Import-Module
5454

5555
# This function is used to verify the 'require' and 'input-file' settings in autorest.md point to the main branch of Azure/azure-rest-api-specs repository
5656
# input-file may be:

‎eng/common/scripts/job-matrix/job-matrix-functions.ps1

+1-1
Original file line numberDiff line numberDiff line change
@@ -213,7 +213,7 @@ function GetMatrixConfigFromFile([String] $config) {
213213
}
214214

215215
function GetMatrixConfigFromYaml([String] $yamlConfig) {
216-
Install-ModuleIfNotInstalled "powershell-yaml" "0.4.1" | Import-Module
216+
Install-ModuleIfNotInstalled "powershell-yaml" "0.4.7" | Import-Module
217217
# ConvertTo then from json is to make sure the nested values are in PSCustomObject
218218
[MatrixConfig]$config = ConvertFrom-Yaml $yamlConfig -Ordered | ConvertTo-Json -Depth 100 | ConvertFrom-Json
219219
return GetMatrixConfig $config

‎eng/common/scripts/stress-testing/stress-test-deployment-lib.ps1

+1-1
Original file line numberDiff line numberDiff line change
@@ -414,7 +414,7 @@ function CheckDependencies()
414414
}
415415
)
416416

417-
Install-ModuleIfNotInstalled "powershell-yaml" "0.4.1" | Import-Module
417+
Install-ModuleIfNotInstalled "powershell-yaml" "0.4.7" | Import-Module
418418

419419
$shouldError = $false
420420
foreach ($dep in $deps) {

‎eng/emitter-package-lock.json

+11-11
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎eng/emitter-package.json

+1-1
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
{
22
"main": "dist/src/index.js",
33
"dependencies": {
4-
"@azure-tools/typespec-python": "0.36.4"
4+
"@azure-tools/typespec-python": "0.36.5"
55
},
66
"devDependencies": {
77
"@azure-tools/typespec-autorest": "~0.48.0",

‎eng/pipelines/templates/jobs/ci.yml

+20-18
Original file line numberDiff line numberDiff line change
@@ -49,9 +49,6 @@ parameters:
4949
- name: VerifyAutorest
5050
type: boolean
5151
default: false
52-
- name: ValidateFormatting
53-
type: boolean
54-
default: false
5552
- name: UnsupportedToxEnvironments
5653
type: string
5754
default: ''
@@ -229,7 +226,6 @@ jobs:
229226
TestPipeline: ${{ parameters.TestPipeline }}
230227
Artifacts: ${{ parameters.Artifacts }}
231228
VerifyAutorest: ${{ parameters.VerifyAutorest }}
232-
ValidateFormatting: ${{ parameters.ValidateFormatting }}
233229
GenerateApiReviewForManualOnly: ${{ parameters.GenerateApiReviewForManualOnly }}
234230

235231
- template: /eng/common/pipelines/templates/jobs/generate-job-matrix.yml
@@ -244,20 +240,26 @@ jobs:
244240
MatrixConfigs: ${{ parameters.MatrixConfigs }}
245241
MatrixFilters: ${{ parameters.MatrixFilters }}
246242
MatrixReplace: ${{ parameters.MatrixReplace }}
247-
PreGenerationSteps:
248-
249-
- template: /eng/pipelines/templates/steps/download-package-artifacts.yml
250-
251-
- task: Powershell@2
252-
inputs:
253-
pwsh: true
254-
filePath: eng/scripts/distribute-packages-to-matrix.ps1
255-
arguments: >-
256-
-PackageInfoFolder "$(Build.ArtifactStagingDirectory)/PackageInfo"
257-
-PlatformMatrix "${{ parameters.MatrixConfigs[0].Path }}"
258-
displayName: 'Distribute Packages to Matrix'
259-
condition: and(eq(variables['Build.Reason'], 'PullRequest'), eq('${{ parameters.ServiceDirectory }}','auto'))
260-
243+
${{ if eq(parameters.ServiceDirectory, 'auto') }}:
244+
SparseCheckoutPaths:
245+
- "**/setup.py"
246+
- "**/pyproject.toml"
247+
- "tools/"
248+
- "**/ci*.yml"
249+
- "**/_version.py"
250+
- "**/version.py"
251+
- "**/README.md"
252+
- "**/CHANGELOG.md"
253+
- "**/*matrix*.json"
254+
EnablePRGeneration: true
255+
PRMatrixSetting: "TargetingString"
256+
PreGenerationSteps:
257+
- task: UsePythonVersion@0
258+
inputs:
259+
versionSpec: '3.12'
260+
- template: /eng/common/pipelines/templates/steps/save-package-properties.yml
261+
parameters:
262+
ServiceDirectory: ${{parameters.ServiceDirectory}}
261263
CloudConfig:
262264
Cloud: Public
263265
AdditionalParameters:

‎eng/pipelines/templates/stages/archetype-sdk-client.yml

-4
Original file line numberDiff line numberDiff line change
@@ -63,9 +63,6 @@ parameters:
6363
- name: VerifyAutorest
6464
type: boolean
6565
default: false
66-
- name: ValidateFormatting
67-
type: boolean
68-
default: false
6966
- name: TestProxy
7067
type: boolean
7168
default: true
@@ -107,7 +104,6 @@ extends:
107104
MatrixFilters: ${{ parameters.MatrixFilters }}
108105
MatrixReplace: ${{ parameters.MatrixReplace }}
109106
VerifyAutorest: ${{ parameters.VerifyAutorest }}
110-
ValidateFormatting: ${{ parameters.ValidateFormatting }}
111107
TestProxy: ${{ parameters.TestProxy }}
112108
GenerateApiReviewForManualOnly: ${{ parameters.GenerateApiReviewForManualOnly }}
113109

‎eng/pipelines/templates/stages/platform-matrix.json

+2-2
Original file line numberDiff line numberDiff line change
@@ -17,8 +17,8 @@
1717
{
1818
"MacTestConfig": {
1919
"macos311": {
20-
"OSVmImage": "env:LINUXVMIMAGE",
21-
"Pool": "env:LINUXPOOL",
20+
"OSVmImage": "env:MACVMIMAGE",
21+
"Pool": "env:MACPOOL",
2222
"PythonVersion": "3.11",
2323
"CoverageArg": "--disablecov",
2424
"TestSamples": "false"

‎eng/pipelines/templates/steps/analyze.yml

+3-6
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,6 @@ parameters:
55
Artifacts: []
66
TestPipeline: false
77
VerifyAutorest: false
8-
ValidateFormatting: false
98
GenerateApiReviewForManualOnly: false
109

1110
# Please use `$(TargetingString)` to refer to the python packages glob string. This variable is set from resolve-package-targeting.yml.
@@ -110,11 +109,9 @@ steps:
110109
ServiceDirectory: ${{ parameters.ServiceDirectory }}
111110
AdditionalTestArgs: ${{ parameters.AdditionalTestArgs }}
112111

113-
- ${{ if parameters.ValidateFormatting }}:
114-
- template: run_black.yml
115-
parameters:
116-
ServiceDirectory: ${{ parameters.ServiceDirectory }}
117-
ValidateFormatting: ${{ parameters.ValidateFormatting }}
112+
- template: run_black.yml
113+
parameters:
114+
ServiceDirectory: ${{ parameters.ServiceDirectory }}
118115

119116
- task: PythonScript@0
120117
displayName: 'Run Keyword Validation Check'

0 commit comments

Comments
 (0)