Skip to content

Refactor assertions #1302

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 10 commits into from
Mar 14, 2017
Merged

Refactor assertions #1302

merged 10 commits into from
Mar 14, 2017

Conversation

novemberborn
Copy link
Member

@novemberborn novemberborn commented Mar 9, 2017

See commits. Behavior should be the same, aside from these changes:

  • Fixes t.notThrows(promise) should return an "empty" promise #1227
  • Fixes t.notThrows() gives no info on thrown error in case an error is thrown #1150
  • Assertions now return undefined rather than null
  • Removes the undocumented t.assertCount getter
  • All test failures (exception thrown, rejected promise returned, t.end(error)) are now wrapped in an assertion error, but we'll still print the actual error
  • Do the same for when t.throws() / t.notThrows() are called with an improper argument. We can now even print the actual argument passed!
  • In the TAP reporter, the error YAML block now has varying properties (e.g. do not assume it has operator or even assertion)
  • Test serialization no longer overwrites non-standard error properties

@novemberborn
Copy link
Member Author

Tests are red across the board, but it works on my machine…

I'll look into that tomorrow. Please hit me with feedback regardless 😄

@novemberborn novemberborn force-pushed the refactor-assertions branch 2 times, most recently from f2639b9 to 4cda666 Compare March 10, 2017 16:10
@novemberborn
Copy link
Member Author

Tests are red across the board, but it works on my machine…

I'll look into that tomorrow.

Clearing CI caches helped. Pushed more changes too.

@novemberborn novemberborn force-pushed the refactor-assertions branch 2 times, most recently from 8dbeda6 to f90a9ef Compare March 10, 2017 18:03
@sindresorhus sindresorhus changed the base branch from improve-tap-reporter to master March 11, 2017 08:50
This matches the terminology used in the README.
* Use our own AssertionError class. Now we can track the assertion
rather than abusing the operator field. We can also stop depending on
`core-assert` once we implement `t.throws()` ourselves. Reserve the
`statements` property for power-assert output

* Directly create `AssertionError`s, making it easier to specify the
varying options

* Default the assertion message to an empty string, removing workarounds
in the mini and verbose reporters

* Improve `t.plan()` assertion errors. Specify 'plan' as the assertion,
and use `===` as the operator

* Refactor `t.throws()` and `t.notThrows()`, improving readability and
fixing #1227

* Move knowledge of when to show output into `serialize-error.js`. It
sat kind of awkwardly in `assert.js`

* Always try to format errors in mini and verbose reporters, check
if actual / expected values are available when formatting

* Wrap assertions in `assert.js` for use with the `ExecutionContext`

* Only enhance the power-assert assertions, don't use `empower-core`
to wrap the others

* Remove superfluous `empower-core` 'originalMessage' handling. Our
assertions create `AssertionError`s with the correct message

* Refactor overloaded `Test#_assert()` method and assertion tracking
internals. Note that only `t.throws()` and `notThrows()` are (possibly)
asynchronous
Avoid having to normalize errors in `run-status.js`.
* Retain non-standard error properties in a cleaned `object` property

* Make AssertionError serialization more explicit

* Flag whether the serialized error came from AVA's AssertionError
This is especially useful for `t.notThrows()`, which includes the
thrown exception as the `actual` value for the assertion error.

Fixes #1150.
Wrap callback errors, rejected promises or synchronously thrown values
in an `AssertionError`. Rely on the `actual` value being printed by the
reporters, which means it's still shown to the user.

Remove now obsolete `Error: ` prefix added to non-assertion errors in
`run-status.js`. Instead more helpful messages from `test.js` can be
shown.

Try to reuse the stack trace from any original error, so magic assert
can show context.

Fail `t.throws()` / `t.notThrows()` with an AssertionError if called
with an improper argument, rather than throwing a TypeError.
Copy link
Member

@sindresorhus sindresorhus left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is fantastic, Mark! It took me a while to read through and understand it all. I agree with pretty much all the changes. Lots of great cleanups too 👌😄.

I ran this on a bunch of projects and tried failing tests in different ways, and it seems to work just fine.

lib/assert.js Outdated
fail(this, new AssertionError({
actual: fn,
message: '`t.throws()` must be called with a function, Promise, or Observable'
}));
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should it really wait until all tests are finished before warning about this? I would think would call for an immediate error even if the user didn't pass --fail-fast.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

--fail-fast implies all tests stop. This still immediately fails the test. The behavior should be similar to before this refactor, except that the error can no longer be caught by calling code.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just tried and you're right. It is the existing behavior. I still think user mistakes like this should fail the test right away no matter what flags the user used though, but that's a separate discussion.

@sindresorhus sindresorhus merged commit 827cc72 into master Mar 14, 2017
@sindresorhus sindresorhus deleted the refactor-assertions branch March 14, 2017 12:50
@sindresorhus
Copy link
Member

🎉🎉🎉

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants