Using with .NET 8 SDK ArtifactsPath

[tompazourek]

I'm trying to use the ArtifactsPath approach that's new in .NET SDK 8 (https://learn.microsoft.com/en-us/dotnet/core/sdk/artifacts-output) by adding Directory.Build.props:

<Project>
  <PropertyGroup>
    <ArtifactsPath>$(MSBuildThisFileDirectory)artifacts</ArtifactsPath>
  </PropertyGroup>
</Project>

I'm trying this with the sample Md5VsSha256 job and getting this output:

// Validating benchmarks:
// ***** BenchmarkRunner: Start   *****
// ***** Found 2 benchmark(s) in total *****
// ***** Building 1 exe(s) in Parallel: Start   *****
// start dotnet  restore /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 /p:Deterministic=true /p:Optimize=true in C:\YYY\XXX\artifacts\bin\XXX.Benchmarks\release_net8.0\e641c047-f8ab-4f6b-b62a-8ae4d3e3ed6b
// command took 2.36 sec and exited with 0
// start dotnet  build -c Release --no-restore /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 /p:Deterministic=true /p:Optimize=true in C:\YYY\XXX\artifacts\bin\XXX.Benchmarks\release_net8.0\e641c047-f8ab-4f6b-b62a-8ae4d3e3ed6b
// command took 18.12 sec and exited with 1
// start dotnet  build -c Release --no-restore --no-dependencies /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 /p:Deterministic=true /p:Optimize=true in C:\YYY\XXX\artifacts\bin\XXX.Benchmarks\release_net8.0\e641c047-f8ab-4f6b-b62a-8ae4d3e3ed6b
// command took 4.08 sec and exited with 0
// ***** Done, took 00:00:24 (24.79 sec)   *****
// Found 2 benchmarks:
//   Md5VsSha256.Sha256: DefaultJob
//   Md5VsSha256.Md5: DefaultJob

Setup power plan (GUID: 8c5e7fda-e8bf-4a96-9a85-a6e23a8c635c FriendlyName: High performance)
// **************************
// Benchmark: Md5VsSha256.Sha256: DefaultJob
// *** Execute ***
// Launch: 1 / 1
Did not find C:\YYY\XXX\artifacts\bin\XXX.Benchmarks\release_net8.0\e641c047-f8ab-4f6b-b62a-8ae4d3e3ed6b\bin\Release\net8.0\e641c047-f8ab-4f6b-b62a-8ae4d3e3ed6b.dll, but the folder contained:
e641c047-f8ab-4f6b-b62a-8ae4d3e3ed6b.dll.config
Executable C:\YYY\XXX\artifacts\bin\XXX.Benchmarks\release_net8.0\e641c047-f8ab-4f6b-b62a-8ae4d3e3ed6b\bin\Release\net8.0\e641c047-f8ab-4f6b-b62a-8ae4d3e3ed6b.dll not found
ExitCode != 0 and no results reported
No Workload Results were obtained from the run.

// ** Remained 1 (50.0%) benchmark(s) to run. Estimated finish 2024-02-26 9:41 (0h 0m from now) **
Setup power plan (GUID: 8c5e7fda-e8bf-4a96-9a85-a6e23a8c635c FriendlyName: High performance)
// **************************
// Benchmark: Md5VsSha256.Md5: DefaultJob
// *** Execute ***
// Launch: 1 / 1
Did not find C:\YYY\XXX\artifacts\bin\XXX.Benchmarks\release_net8.0\e641c047-f8ab-4f6b-b62a-8ae4d3e3ed6b\bin\Release\net8.0\e641c047-f8ab-4f6b-b62a-8ae4d3e3ed6b.dll, but the folder contained:
e641c047-f8ab-4f6b-b62a-8ae4d3e3ed6b.dll.config
Executable C:\YYY\XXX\artifacts\bin\XXX.Benchmarks\release_net8.0\e641c047-f8ab-4f6b-b62a-8ae4d3e3ed6b\bin\Release\net8.0\e641c047-f8ab-4f6b-b62a-8ae4d3e3ed6b.dll not found
ExitCode != 0 and no results reported
No Workload Results were obtained from the run.

// ** Remained 0 (0.0%) benchmark(s) to run. Estimated finish 2024-02-26 9:41 (0h 0m from now) **
Successfully reverted power plan (GUID: 8c5e7fda-e8bf-4a96-9a85-a6e23a8c635c FriendlyName: High performance)
// ***** BenchmarkRunner: Finish  *****

// * Export *
  BenchmarkDotNet.Artifacts\results\MyBenchmarks.Md5VsSha256-report.csv
  BenchmarkDotNet.Artifacts\results\MyBenchmarks.Md5VsSha256-report-github.md
  BenchmarkDotNet.Artifacts\results\MyBenchmarks.Md5VsSha256-report.html

// * Detailed results *
Md5VsSha256.Sha256: DefaultJob
Runtime = ; GC =
There are not any results runs

Md5VsSha256.Md5: DefaultJob
Runtime = ; GC =
There are not any results runs

// * Summary *

BenchmarkDotNet v0.13.12, Windows 11 (10.0.22631.3155/23H2/2023Update/SunValley3)
Intel Core i7-8550U CPU 1.80GHz (Kaby Lake R), 1 CPU, 8 logical and 4 physical cores
.NET SDK 8.0.200
  [Host] : .NET 8.0.2 (8.0.224.6711), X64 RyuJIT AVX2 [AttachedDebugger]


| Method | Mean | Error |
|------- |-----:|------:|
| Sha256 |   NA |    NA |
| Md5    |   NA |    NA |

Benchmarks with issues:
  Md5VsSha256.Sha256: DefaultJob
  Md5VsSha256.Md5: DefaultJob

// * Warnings *
Environment
  Summary -> Benchmark was executed with attached debugger
  Summary -> Detected error exit code from one of the benchmarks. It might be caused by following antivirus software:
        - Windows Defender (windowsdefender://)
Use InProcessEmitToolchain or InProcessNoEmitToolchain to avoid new process creation.


// * Legends *
  Mean  : Arithmetic mean of all measurements
  Error : Half of 99.9% confidence interval
  1 ns  : 1 Nanosecond (0.000000001 sec)

// ***** BenchmarkRunner: End *****
Run time: 00:00:00 (0.13 sec), executed benchmarks: 0

Global total time: 00:00:25 (25.29 sec), executed benchmarks: 0
// * Artifacts cleanup *
Artifacts cleanup is finished

C:\YYY\XXX\artifacts\bin\XXX.Benchmarks\release_net8.0\XXX.Benchmarks.exe (process 26424) exited with code 0.
To automatically close the console when debugging stops, enable Tools->Options->Debugging->Automatically close the console when debugging stops.
Press any key to close this window . . .

The error is:

Did not find C:\YYY\XXX\artifacts\bin\XXX.Benchmarks\release_net8.0\e641c047-f8ab-4f6b-b62a-8ae4d3e3ed6b\bin\Release\net8.0\e641c047-f8ab-4f6b-b62a-8ae4d3e3ed6b.dll, but the folder contained:
e641c047-f8ab-4f6b-b62a-8ae4d3e3ed6b.dll.config

Anyone knows how to resolve this?

My only option currently is to stop using the ArtifactsPath approach. But I really like it otherwise.

[timcassell]

#2466

[tompazourek]

Thank you for this @timcassell. I was searching for ArtifactsPath instead of UseArtifactsOutput.