Skip to content

[WIP] A way to submit tests that should fail. #831

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 11 commits into from
May 16, 2016
Merged

[WIP] A way to submit tests that should fail. #831

Changes from 1 commit
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
e107a3c
add failing metadata
cgcgbcbc May 13, 2016
e8ab228
flip Test._result when failing=true
cgcgbcbc May 14, 2016
2413594
add tests for verifying failing worked for callback
cgcgbcbc May 14, 2016
fa32209
add tests verifying failing test fail if passed
cgcgbcbc May 14, 2016
25b4979
add tests for failing test returns a resolved promise
cgcgbcbc May 14, 2016
7d10392
add tests for failing test returns a rejected promise
cgcgbcbc May 14, 2016
98e43e4
add tests for failing test with t.throws
cgcgbcbc May 16, 2016
a12227e
fix test
cgcgbcbc May 16, 2016
2378a44
fix test title
cgcgbcbc May 16, 2016
c4d1306
update readme about failing modifier
cgcgbcbc May 16, 2016
4ac5ebe
update readme
cgcgbcbc May 16, 2016
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions readme.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -415,6 +415,16 @@ You can use the `.todo` modifier when you're planning to write a test. Like skip
test.todo('will think about writing this later');
```

### Failing tests

You can use the `.failing` modifier for submitting tests that are intended to be failures without killing CI process. And if the failing test does pass, it will fail, and it will remind you remove the failing modifier.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would like to expand on this a bit:

You can use the .failing modifier to document issues with your code that need to be fixed. Failing tests are run just like normal ones, but they are expected to fail, and will not break your build when they do. If a test marked as failing actually passes, it will be reported as an error and fail the build with a helpful message instructing you to remove the .failing modifier.

This allows you to merge .failing tests before a fix is implemented without breaking CI. This is a great way to recognize good bug report PR's with a commit credit, even if the reporter is unable to actually fix the problem.


```js
test.failing('this test prove something wrong', t => {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

People should link to the relevant issue. We should model that behavior in the docs.

Also, a bit of language/style cleanup.

// See: github.com/user/repo/issues/1234
test.failing('demonstrate some bug', t => {
  t.fail(); // test will count as passed
});

t.fail();
}); // will count as passed test
```

### Before & after hooks

AVA lets you register hooks that are run before and after your tests. This allows you to run setup and/or teardown code.
Expand Down