Skip to content

Commit a2082b3

Browse files
MarcoRossignoliMarcoRossignoli
authored
Add deterministic report support (#1113)
Add deterministic report support
1 parent 9716c45 commit a2082b3

38 files changed

+336
-187
lines changed

‎Documentation/DeterministicBuild.md

+17
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,23 @@ As explained above, to improve the level of security of generated artifacts (for
1010

1111
Finally, thanks to deterministic CI builds (with the `ContinuousIntegrationBuild` property set to `true`) plus signature we can validate artifacts and be sure that the binary was built from specific sources (because there is no hard-coded variable metadata, like paths from different build machines).
1212

13+
# Deterministic report
14+
15+
Coverlet supports also deterministic reports(for now only for cobertura coverage format).
16+
If you include `DeterministicReport` parameters for `msbuild` and `collectors` integrations resulting report will be like:
17+
```xml
18+
<?xml version="1.0" encoding="utf-8"?>
19+
<coverage line-rate="0.8571" branch-rate="0.5" version="1.9" timestamp="1612702997" lines-covered="6" lines-valid="7" branches-covered="1" branches-valid="2">
20+
<sources />
21+
<packages>
22+
<package name="MyLibrary" line-rate="0.8571" branch-rate="0.5" complexity="3">
23+
<classes>
24+
<class name="MyLibrary.Hello" filename="/_/MyLibrary/Hello.cs" line-rate="0.8571" branch-rate="0.5" complexity="3">
25+
<methods>
26+
...
27+
```
28+
As you can see we have empty `<sources />` element and the `filename` start with well known deterministic fragment `/_/...`
29+
1330
**Deterministic build is supported without any workaround since version 3.1.100 of .NET Core SDK**
1431

1532
## Workaround only for .NET Core SDK < 3.1.100

‎Documentation/MSBuildIntegration.md

+8
Original file line numberDiff line numberDiff line change
@@ -206,3 +206,11 @@ The workaround is to use the .NET Core `dotnet msbuild` command instead of using
206206
## SourceLink
207207

208208
Coverlet supports [SourceLink](https://github.com/dotnet/sourcelink) custom debug information contained in PDBs. When you specify the `/p:UseSourceLink=true` property, Coverlet will generate results that contain the URL to the source files in your source control instead of local file paths.
209+
210+
## Deterministic build
211+
212+
Take a look at [documentation](Documentation/DeterministicBuild.md) for further informations.
213+
To generate deterministc report the parameter is:
214+
```
215+
/p:DeterministicReport=true
216+
```

‎Documentation/VSTestIntegration.md

+14-12
Original file line numberDiff line numberDiff line change
@@ -85,18 +85,19 @@ We're working to fill the gaps.
8585

8686
These are a list of options that are supported by coverlet. These can be specified as datacollector configurations in the runsettings.
8787

88-
| Option | Summary |
89-
|:-------------------------|:----------------------------------------------------------------------------------------------------------------------------------|
90-
| Format | Coverage output format. These are either cobertura, json, lcov, opencover or teamcity as well as combinations of these formats. |
91-
| Exclude | Exclude from code coverage analysing using filter expressions. |
92-
| ExcludeByFile | Ignore specific source files from code coverage. |
93-
| Include | Explicitly set what to include in code coverage analysis using filter expressions. |
94-
| IncludeDirectory | Explicitly set which directories to include in code coverage analysis. |
95-
| SingleHit | Specifies whether to limit code coverage hit reporting to a single hit for each location. |
96-
| UseSourceLink | Specifies whether to use SourceLink URIs in place of file system paths. |
97-
| IncludeTestAssembly | Include coverage of the test assembly. |
98-
| SkipAutoProps | Neither track nor record auto-implemented properties. |
99-
| DoesNotReturnAttribute | Methods marked with these attributes are known not to return, statements following them will be excluded from coverage |
88+
| Option | Summary |
89+
|:-------------------------|:----------------------------------------------------------------------------------------------------------------------------------------------------------------|
90+
| Format | Coverage output format. These are either cobertura, json, lcov, opencover or teamcity as well as combinations of these formats. |
91+
| Exclude | Exclude from code coverage analysing using filter expressions. |
92+
| ExcludeByFile | Ignore specific source files from code coverage. |
93+
| Include | Explicitly set what to include in code coverage analysis using filter expressions. |
94+
| IncludeDirectory | Explicitly set which directories to include in code coverage analysis. |
95+
| SingleHit | Specifies whether to limit code coverage hit reporting to a single hit for each location. |
96+
| UseSourceLink | Specifies whether to use SourceLink URIs in place of file system paths. |
97+
| IncludeTestAssembly | Include coverage of the test assembly. |
98+
| SkipAutoProps | Neither track nor record auto-implemented properties. |
99+
| DoesNotReturnAttribute | Methods marked with these attributes are known not to return, statements following them will be excluded from coverage |
100+
| DeterministicReport | Generates deterministic report in context of deterministic build. Take a look at [documentation](Documentation/DeterministicBuild.md) for further informations. |
100101

101102
How to specify these options via runsettings?
102103

@@ -117,6 +118,7 @@ How to specify these options via runsettings?
117118
<UseSourceLink>true</UseSourceLink>
118119
<IncludeTestAssembly>true</IncludeTestAssembly>
119120
<SkipAutoProps>true</SkipAutoProps>
121+
<DeterministicReport>false</DeterministicReport>
120122
</Configuration>
121123
</DataCollector>
122124
</DataCollectors>

‎src/coverlet.collector/DataCollection/CoverageManager.cs

+4-5
Original file line numberDiff line numberDiff line change
@@ -17,9 +17,8 @@ namespace Coverlet.Collector.DataCollection
1717
internal class CoverageManager
1818
{
1919
private readonly Coverage _coverage;
20-
21-
private ICoverageWrapper _coverageWrapper;
22-
20+
private readonly ICoverageWrapper _coverageWrapper;
21+
private readonly ISourceRootTranslator _sourceRootTranslator;
2322
public IReporter[] Reporters { get; }
2423

2524
public CoverageManager(CoverletSettings settings, TestPlatformEqtTrace eqtTrace, TestPlatformLogger logger, ICoverageWrapper coverageWrapper,
@@ -49,7 +48,7 @@ public CoverageManager(CoverletSettings settings, IReporter[] reporters, ILogger
4948
// Store input vars
5049
Reporters = reporters;
5150
_coverageWrapper = coverageWrapper;
52-
51+
_sourceRootTranslator = sourceRootTranslator;
5352
// Coverage object
5453
_coverage = _coverageWrapper.CreateCoverage(settings, logger, instrumentationHelper, fileSystem, sourceRootTranslator, cecilSymbolHelper);
5554
}
@@ -108,7 +107,7 @@ private CoverageResult GetCoverageResult()
108107
{
109108
try
110109
{
111-
return Reporters.Select(reporter => (reporter.Report(coverageResult), Path.ChangeExtension(CoverletConstants.DefaultFileName, reporter.Extension)));
110+
return Reporters.Select(reporter => (reporter.Report(coverageResult, _sourceRootTranslator), Path.ChangeExtension(CoverletConstants.DefaultFileName, reporter.Extension)));
112111
}
113112
catch (Exception ex)
114113
{

‎src/coverlet.collector/DataCollection/CoverageWrapper.cs

+3-2
Original file line numberDiff line numberDiff line change
@@ -17,7 +17,7 @@ internal class CoverageWrapper : ICoverageWrapper
1717
/// <returns>Coverage object</returns>
1818
public Coverage CreateCoverage(CoverletSettings settings, ILogger coverletLogger, IInstrumentationHelper instrumentationHelper, IFileSystem fileSystem, ISourceRootTranslator sourceRootTranslator, ICecilSymbolHelper cecilSymbolHelper)
1919
{
20-
CoverageParameters parameters = new CoverageParameters
20+
CoverageParameters parameters = new()
2121
{
2222
IncludeFilters = settings.IncludeFilters,
2323
IncludeDirectories = settings.IncludeDirectories,
@@ -29,7 +29,8 @@ public Coverage CreateCoverage(CoverletSettings settings, ILogger coverletLogger
2929
MergeWith = settings.MergeWith,
3030
UseSourceLink = settings.UseSourceLink,
3131
SkipAutoProps = settings.SkipAutoProps,
32-
DoesNotReturnAttributes = settings.DoesNotReturnAttributes
32+
DoesNotReturnAttributes = settings.DoesNotReturnAttributes,
33+
DeterministicReport = settings.DeterministicReport
3334
};
3435

3536
return new Coverage(

‎src/coverlet.collector/DataCollection/CoverletSettings.cs

+6
Original file line numberDiff line numberDiff line change
@@ -73,6 +73,11 @@ internal class CoverletSettings
7373
/// </summary>
7474
public string[] DoesNotReturnAttributes { get; set; }
7575

76+
/// <summary>
77+
/// DeterministicReport flag
78+
/// </summary>
79+
public bool DeterministicReport { get; set; }
80+
7681
public override string ToString()
7782
{
7883
var builder = new StringBuilder();
@@ -89,6 +94,7 @@ public override string ToString()
8994
builder.AppendFormat("IncludeTestAssembly: '{0}'", IncludeTestAssembly);
9095
builder.AppendFormat("SkipAutoProps: '{0}'", SkipAutoProps);
9196
builder.AppendFormat("DoesNotReturnAttributes: '{0}'", string.Join(",", DoesNotReturnAttributes ?? Enumerable.Empty<string>()));
97+
builder.AppendFormat("DeterministicReport: '{0}'", DeterministicReport);
9298

9399
return builder.ToString();
94100
}

‎src/coverlet.collector/DataCollection/CoverletSettingsParser.cs

+13
Original file line numberDiff line numberDiff line change
@@ -44,6 +44,7 @@ public CoverletSettings Parse(XmlElement configurationElement, IEnumerable<strin
4444
coverletSettings.IncludeTestAssembly = ParseIncludeTestAssembly(configurationElement);
4545
coverletSettings.SkipAutoProps = ParseSkipAutoProps(configurationElement);
4646
coverletSettings.DoesNotReturnAttributes = ParseDoesNotReturnAttributes(configurationElement);
47+
coverletSettings.DeterministicReport = ParseDeterministicReport(configurationElement);
4748
}
4849

4950
coverletSettings.ReportFormats = ParseReportFormats(configurationElement);
@@ -195,6 +196,18 @@ private bool ParseSingleHit(XmlElement configurationElement)
195196
return singleHit;
196197
}
197198

199+
/// <summary>
200+
/// Parse ParseDeterministicReport flag
201+
/// </summary>
202+
/// <param name="configurationElement">Configuration element</param>
203+
/// <returns>ParseDeterministicReport flag</returns>
204+
private bool ParseDeterministicReport(XmlElement configurationElement)
205+
{
206+
XmlElement deterministicReportElement = configurationElement[CoverletConstants.DeterministicReport];
207+
bool.TryParse(deterministicReportElement?.InnerText, out bool deterministicReport);
208+
return deterministicReport;
209+
}
210+
198211
/// <summary>
199212
/// Parse include test assembly flag
200213
/// </summary>

‎src/coverlet.collector/Utilities/CoverletConstants.cs

+1
Original file line numberDiff line numberDiff line change
@@ -22,5 +22,6 @@ internal static class CoverletConstants
2222
public const string InProcDataCollectorName = "CoverletInProcDataCollector";
2323
public const string SkipAutoProps = "SkipAutoProps";
2424
public const string DoesNotReturnAttributesElementName = "DoesNotReturnAttribute";
25+
public const string DeterministicReport = "DeterministicReport";
2526
}
2627
}

‎src/coverlet.console/Program.cs

+14-10
Original file line numberDiff line numberDiff line change
@@ -37,9 +37,11 @@ static int Main(string[] args)
3737
var logger = (ConsoleLogger)serviceProvider.GetService<ILogger>();
3838
var fileSystem = serviceProvider.GetService<IFileSystem>();
3939

40-
var app = new CommandLineApplication();
41-
app.Name = "coverlet";
42-
app.FullName = "Cross platform .NET Core code coverage tool";
40+
var app = new CommandLineApplication
41+
{
42+
Name = "coverlet",
43+
FullName = "Cross platform .NET Core code coverage tool"
44+
};
4345
app.HelpOption("-h|--help");
4446
app.VersionOption("-v|--version", GetAssemblyVersion());
4547
int exitCode = (int)CommandExitCodes.Success;
@@ -79,7 +81,7 @@ static int Main(string[] args)
7981
logger.Level = verbosity.ParsedValue;
8082
}
8183

82-
CoverageParameters parameters = new CoverageParameters
84+
CoverageParameters parameters = new()
8385
{
8486
IncludeFilters = includeFilters.Values.ToArray(),
8587
IncludeDirectories = includeDirectories.Values.ToArray(),
@@ -94,16 +96,18 @@ static int Main(string[] args)
9496
DoesNotReturnAttributes = doesNotReturnAttributes.Values.ToArray()
9597
};
9698

97-
Coverage coverage = new Coverage(moduleOrAppDirectory.Value,
99+
ISourceRootTranslator sourceRootTranslator = serviceProvider.GetRequiredService<ISourceRootTranslator>();
100+
101+
Coverage coverage = new(moduleOrAppDirectory.Value,
98102
parameters,
99103
logger,
100104
serviceProvider.GetRequiredService<IInstrumentationHelper>(),
101105
fileSystem,
102-
serviceProvider.GetRequiredService<ISourceRootTranslator>(),
106+
sourceRootTranslator,
103107
serviceProvider.GetRequiredService<ICecilSymbolHelper>());
104108
coverage.PrepareModules();
105109

106-
Process process = new Process();
110+
Process process = new();
107111
process.StartInfo.FileName = target.Value();
108112
process.StartInfo.Arguments = targs.HasValue() ? targs.Value() : string.Empty;
109113
process.StartInfo.CreateNoWindow = true;
@@ -146,7 +150,7 @@ static int Main(string[] args)
146150
Directory.CreateDirectory(directory);
147151
}
148152

149-
foreach (var format in (formats.HasValue() ? formats.Values : new List<string>(new string[] { "json" })))
153+
foreach (var format in formats.HasValue() ? formats.Values : new List<string>(new string[] { "json" }))
150154
{
151155
var reporter = new ReporterFactory(format).CreateReporter();
152156
if (reporter == null)
@@ -158,7 +162,7 @@ static int Main(string[] args)
158162
{
159163
// Output to console
160164
logger.LogInformation(" Outputting results to console", important: true);
161-
logger.LogInformation(reporter.Report(result), important: true);
165+
logger.LogInformation(reporter.Report(result, sourceRootTranslator), important: true);
162166
}
163167
else
164168
{
@@ -169,7 +173,7 @@ static int Main(string[] args)
169173

170174
var report = Path.Combine(directory, filename);
171175
logger.LogInformation($" Generating report '{report}'", important: true);
172-
fileSystem.WriteAllText(report, reporter.Report(result));
176+
fileSystem.WriteAllText(report, reporter.Report(result, sourceRootTranslator));
173177
}
174178
}
175179

‎src/coverlet.core/Reporters/IReporter.cs renamed to ‎src/coverlet.core/Abstractions/IReporter.cs

+2-2
Original file line numberDiff line numberDiff line change
@@ -1,11 +1,11 @@
1-
namespace Coverlet.Core.Reporters
1+
namespace Coverlet.Core.Abstractions
22
{
33
internal interface IReporter
44
{
55
ReporterOutputType OutputType { get; }
66
string Format { get; }
77
string Extension { get; }
8-
string Report(CoverageResult result);
8+
string Report(CoverageResult result, ISourceRootTranslator sourceRootTranslator);
99
}
1010

1111
internal enum ReporterOutputType

‎src/coverlet.core/Abstractions/ISourceRootTranslator.cs

+1
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,7 @@ namespace Coverlet.Core.Abstractions
66
internal interface ISourceRootTranslator
77
{
88
string ResolveFilePath(string originalFileName);
9+
string ResolveDeterministicPath(string originalFileName);
910
IReadOnlyList<SourceRootMapping> ResolvePathRoot(string pathRoot);
1011
}
1112
}

0 commit comments

Comments
 (0)