"ng test" - no code coverage report if --code-coverage and --reporters options used together

[alistairmgreen]

Bug Report or Feature Request (mark with an x)

- [X] bug report -> please search issues before submitting
- [ ] feature request

Versions.

@angular/cli: 1.0.0 (Problem was first discovered using version 1.0.0.rc1)
node: 6.9.4
os: win32 x64 (Windows 7 Enterprise)

Repro steps.

1. ng new coverage-test
2. npm install --save-dev karma-teamcity-reporter
3. Add require('karma-teamcity-reporter') to list of plugins in karma.conf.js
4. ng test --single-run --code-coverage --reporters teamcity - the tests run with TeamCity reporting as expected.
5. ls coverage - the coverage folder is missing.
6. Try again with a different reporter: ng test --single-run --code-coverage --reporters progress - same result, tests pass but there is no coverage report.
7. Try again with no reporter: ng test --single-run --code-coverage - works as expected and generates a coverage report.

The log given by the failure.

No error messages displayed. It just silently fails to produce a coverage report.

Desired functionality.

When running the unit tests on a TeamCity build server, I want to get both a test results summary in TeamCity and a code coverage report.

At present I can only do this by running the tests twice: first with --reporters teamcity to get the results summary, and then again with --code-coverage but no TeamCity reporting.

Mention any other details that might be useful.

I asked a question about this on Stack Overflow but got no replies.

[filipesilva]

Does ng test --code-coverage --reporters=teamcity,coverage-istanbul work?

The --reporters option doesn't add to the existing list, but rather replaces it. If you only add the teamcity reporter, coverage-instanbul is not there.

[alistairmgreen]

Yes, that does work, thanks!

It would be helpful if the documentation could be updated to indicate explicitly that the list of reporters is comma-separated, and that the --code-coverage switch depends upon a reporter being present.

[alistairmgreen]

Would it be possible for ng test to produce a warning if --code-coverage is supplied without a suitable reporter?

Because it fails silently, and the code coverage report is left behind from the previous build, it took me several days to realize that I had broken the coverage report by turning on TeamCity reporting.

[filipesilva]

Fixed in #6160, we now add the reporter dynamically on top of the other reporters.

Ok great, now that I found something that works from within the last year, it's incomplete. What does a proper karma.conf look like? Why do I need to specify karma.conf in angular-cli.json?

[filipesilva]

@megamindbrian Karma is the test runner that runs your unit tests. Angular CLI is, overall, a collection of tools that are configured to give you a good out-of-the-box experience.

It doesn't aim at completely replacing existing tools now providing full abstraction. The Karma ecosystem can still be used to extend functionality.

If you are unsure what Karma configuration ships now with ng new, the easiest way is to make a new project without installing dependencies: ng new new-proj --skip-install.

[playground]

ng test is not showing code coverage report. What do I need to do so code coverage report will show locally, travis-ci and in coveralls?

@playground I ended up adding to webpack:

module: {
        loaders: [
            .... all other loaders for angular
            {
                test: /\.ts/,
                include: helpers.root('src', 'app'),
                loader: 'istanbul-instrumenter-loader',
                enforce: 'post',
                exclude: /(testing|node_modules|\.spec\.ts$)/
            },
    }
}

And adding coverage to karma.conf.js:

coverageReporter: {
            dir: './coverage/',
            exclude: [
                "node_modules",
                "**/*.spec.ts"
            ],
            includeAllSources: true,
            reporters: [
                {type: 'text-summary'},
                {type: 'text'},
           ],
},

What tripped me up for a day was if you use plugins: ['karma-coverage'], you have to specify all the plugins. Removing plugins: entirely and the plugins starting with karma- are loaded automatically. Including karma-coverage-istanbul-reporter and karma-coverage.

I've submitted a PR for improving Angular 2 documentation but I have to remove some parts of the commit because they don't want to accept in karma-spec-reporter as a default provided reporter.

Still working on feedback but my instructions are here:
https://github.com/megamindbrian/karma/blob/81b498d4795949a2dc45089e8abc602f53d54f58/docs/plus/06-angular.md

And the PR:
karma-runner/karma#2742

Does that help?

@filipesilva I asked what a valid Karma config is. After digging in to other people's code, I found this: https://github.com/angular/angular-cli/blob/master/packages/%40angular/cli/tasks/test.ts

Which would lead to a 1 to 1 relationship between the input settings and these karma based settings;
https://github.com/angular/angular-cli/blob/master/packages/%40angular/cli/tasks/test.ts#L32

Not sure what the point of packing this all in to a CLI is. Might be better to just direct people to learning how karma-runner works?

[playground]

@megamindbrian thanks, I currently do not have a webpack config in my project. What does your webpack look like?

Angular cli is supposed to do all of this automatically, unless of course you want to do something different than what angular cli does. That is why I was asking what angular cli is imitating. I will get the full webpack config when I'm back at my desk but there should be a sample for angular on the web or in the tour of hero's starter project.

…

On Jul 11, 2017 11:46 AM, "playground" ***@***.***> wrote: @megamindbrian <https://github.com/megamindbrian> thanks, I currently do not have a webpack config in my project. What does your webpack look like? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#5685 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AX5XbhS_piBJ8mLSUZ0DspRlR1ioHWJJks5sM8MOgaJpZM4MqDN7> .

[playground]

What about the existing section?
coverageIstanbulReporter: {
reports: [ 'html', 'lcovonly' ],
fixWebpackSourcePaths: true,
thresholds: {
statements: 50,
lines: 50,
branches: 50,
functions: 50
}
},

Does coverageReporter replace coverageIstanbulReporter?

Here a good example https://github.com/AngularClass/angular-starter/blob/master/config/webpack.common.js

…

On Jul 11, 2017 12:03 PM, "Brian Cullinan" ***@***.***> wrote: Angular cli is supposed to do all of this automatically, unless of course you want to do something different than what angular cli does. That is why I was asking what angular cli is imitating. I will get the full webpack config when I'm back at my desk but there should be a sample for angular on the web or in the tour of hero's starter project. On Jul 11, 2017 11:46 AM, "playground" ***@***.***> wrote: > @megamindbrian <https://github.com/megamindbrian> thanks, I currently do > not have a webpack config in my project. What does your webpack look like? > > — > You are receiving this because you were mentioned. > Reply to this email directly, view it on GitHub > <#5685 (comment)>, > or mute the thread > <https://github.com/notifications/unsubscribe-auth/AX5XbhS_piBJ8mLSUZ0DspRlR1ioHWJJks5sM8MOgaJpZM4MqDN7> > . >