Skip to content

Commit 00f32f6

Browse files
cgeweckecgewecke
committed
Docs for beta release (#415)
1 parent fab8aaa commit 00f32f6

File tree

7 files changed

+359
-231
lines changed

7 files changed

+359
-231
lines changed

‎.gitignore

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -5,6 +5,5 @@ node_modules/
55
.DS_Store
66
test/artifacts
77
test/cache
8-
yarn.lock
98
temp
109
.nyc_output/

‎.npmignore

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,4 @@
11
test/
22
.circleci/
3-
docs/
3+
docs/
4+
.nyc_output/

‎README.md

Lines changed: 74 additions & 52 deletions
Original file line numberDiff line numberDiff line change
@@ -1,33 +1,25 @@
11
# solidity-coverage
22

3-
[![Join the chat at https://gitter.im/sc-forks/solidity-coverage](https://badges.gitter.im/sc-forks/solidity-coverage.svg)](https://gitter.im/sc-forks/solidity-coverage?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge)
4-
[![npm version](https://badge.fury.io/js/solidity-coverage.svg)](https://badge.fury.io/js/solidity-coverage)
5-
[![CircleCI](https://circleci.com/gh/sc-forks/solidity-coverage.svg?style=svg)](https://circleci.com/gh/sc-forks/solidity-coverage)
6-
[![codecov](https://codecov.io/gh/sc-forks/solidity-coverage/branch/master/graph/badge.svg)](https://codecov.io/gh/sc-forks/solidity-coverage)
7-
8-
### Code coverage for Solidity testing
9-
![coverage example](https://cdn-images-1.medium.com/max/800/1*uum8t-31bUaa6dTRVVhj6w.png)
10-
11-
+ For more details about what this is, how it works and potential limitations, see
12-
[the accompanying article](https://blog.colony.io/code-coverage-for-solidity-eecfa88668c2).
13-
+ `solidity-coverage` is in development and **its accuracy is unknown.** If you
14-
find discrepancies between the coverage report and your suite's behavior, please open an
15-
[issue](https://github.com/sc-forks/solidity-coverage/issues).
16-
+ `solidity-coverage` is [Solcover](https://github.com/JoinColony/solcover)
17-
18-
### Install
3+
[![Gitter chat](https://badges.gitter.im/sc-forks/solidity-coverage.svg)][18]
4+
![npm (tag)](https://img.shields.io/npm/v/solidity-coverage/beta)
5+
[![CircleCI](https://circleci.com/gh/sc-forks/solidity-coverage.svg?style=svg)][20]
6+
[![codecov](https://codecov.io/gh/sc-forks/solidity-coverage/branch/beta/graph/badge.svg)][21]
7+
8+
## Code coverage for Solidity testing
9+
![coverage example][22]
10+
11+
+ For more details about what this is, how it works and potential limitations,
12+
see [the accompanying article][16].
13+
+ `solidity-coverage` is [Solcover][17]
14+
15+
## Install
1916
```
2017
$ npm install --save-dev solidity-coverage@beta
2118
```
22-
### Usage notes:
23-
+ Coverage runs tests a little more slowly.
24-
+ Coverage distorts gas consumption. Tests that check exact gas consumption should be skipped.
25-
+ Coverage launches its own in-process ganache server.
26-
+ You can set [ganache options](https://github.com/trufflesuite/ganache-core#options) using the `providerOptions` key in your `.solcover.js` config.
2719

28-
### Truffle V5
20+
## Truffle V5
2921

30-
**Add** "solidity-coverage" to your plugins array in `truffle-config.js`
22+
**Add** this package to your plugins array in `truffle-config.js`
3123
```javascript
3224
module.exports = {
3325
networks: {...},
@@ -36,24 +28,31 @@ module.exports = {
3628
```
3729
**Run**
3830
```
39-
truffle run coverage [options]
31+
truffle run coverage [command-options]
4032
```
4133

34+
## Usage notes:
35+
+ Coverage runs tests a little more slowly.
36+
+ Coverage launches its own in-process ganache server.
37+
+ You can set [ganache options][1] using the `providerOptions` key in your `.solcover.js` [config][15].
38+
+ Coverage [distorts gas consumption][13]. Tests that check exact gas consumption should be skipped.
4239

43-
### Command Options
44-
| Option <img width=200/> | Example <img width=700/>| Description <img width=1000/> |
40+
## Command Options
41+
| Option <img width=200/> | Example <img width=750/>| Description <img width=1000/> |
4542
|--------------|------------------------------------|--------------------------------|
4643
| file | `--file="test/registry/*.js"` | Filename or glob describing a subset of JS tests to run. (Globs must be enclosed by quotes.)|
4744
| solcoverjs | `--solcoverjs ./../.solcover.js` | Relative path from working directory to config. Useful for monorepo packages that share settings. (Path must be "./" prefixed) |
4845
| network | `--network development` | Use network settings defined in the Truffle config |
49-
| temp | `--temp build` | :warning: *Use with caution* :warning:. Disposable folder to store compilation artifacts in. Useful when your test setup scripts include hard-coded paths to a build directory. |
46+
| temp[<sup>*</sup>][14] | `--temp build` | :warning: **Caution** :warning: Path to a *disposable* folder to store compilation artifacts in. Useful when your test setup scripts include hard-coded paths to a build directory. [More...][14] |
5047
| version | | Version info |
5148
| help | | Usage notes |
5249

53-
### Config Options
50+
[<sup>*</sup> Advanced use][14]
51+
52+
## Config Options
5453

55-
Additional options can be specified in a `.solcover.js` config file located in
56-
the root directory of your project.
54+
Additional options can be specified in a `.solcover.js` config file located in the root directory
55+
of your project.
5756

5857
**Example:**
5958
```javascript
@@ -67,45 +66,45 @@ module.exports = {
6766
| ------ | ---- | ------- | ----------- |
6867
| silent | *Boolean* | false | Suppress logging output |
6968
| client | *Object* | `require("ganache-core")` | Useful if you need a specific ganache version. |
70-
| providerOptions | *Object* | `{ }` | [ganache-core options](https://github.com/trufflesuite/ganache-core#options) |
69+
| providerOptions | *Object* | `{ }` | [ganache-core options][1] |
7170
| skipFiles | *Array* | `['Migrations.sol']` | Array of contracts or folders (with paths expressed relative to the `contracts` directory) that should be skipped when doing instrumentation. |
72-
| istanbulReporter | *Array* | `['html', 'lcov', 'text']` | [Istanbul coverage reporters](https://istanbul.js.org/docs/advanced/alternative-reporters/) |
73-
| mocha | *Object* | `{ }` | [Mocha options](https://mochajs.org/api/mocha) to merge into existing mocha config. `grep` and `invert` are useful for skipping certain tests under coverage using tags in the test descriptions.|
74-
| onServerReady | *Function* | | Hook run *after* server is launched, *before* the tests execute. Useful if you need to use the Oraclize bridge or have setup scripts which rely on the server's availability |
75-
| onTestsComplete | *Function* | | Hook run *after* the tests complete, *before* Istanbul reports are generated. |
76-
| onCompileComplete | *Function* | | Hook run *after* compilation completes, *before* tests are run. Useful if you have secondary compilation steps or need to modify built artifacts. |
77-
| onIstanbulComplete | *Function* | | Hook run *after* the Istanbul reports are generated, *before* the ganache server is shut down. Useful if you need to clean resources up. |
71+
| istanbulReporter | *Array* | `['html', 'lcov', 'text']` | [Istanbul coverage reporters][2] |
72+
| mocha | *Object* | `{ }` | [Mocha options][3] to merge into existing mocha config. `grep` and `invert` are useful for skipping certain tests under coverage using tags in the test descriptions.|
73+
| onServerReady[<sup>*</sup>][14] | *Function* | | Hook run *after* server is launched, *before* the tests execute. Useful if you need to use the Oraclize bridge or have setup scripts which rely on the server's availability. [More...][14] |
74+
| onCompileComplete[<sup>*</sup>][14] | *Function* | | Hook run *after* compilation completes, *before* tests are run. Useful if you have secondary compilation steps or need to modify built artifacts. [More...][14]|
75+
| onTestsComplete[<sup>*</sup>][14] | *Function* | | Hook run *after* the tests complete, *before* Istanbul reports are generated.|
76+
| onIstanbulComplete[<sup>*</sup>][14] | *Function* | | Hook run *after* the Istanbul reports are generated, *before* the ganache server is shut down. Useful if you need to clean resources up.|
7877

78+
[<sup>*</sup> Advanced use][14]
7979

80-
81-
### FAQ
80+
## FAQ
8281

8382
Common problems & questions:
8483

85-
+ [Running out of gas](https://github.com/sc-forks/solidity-coverage/blob/master/docs/faq.md#running-out-of-gas)
86-
+ [Running out of memory (locally and in CI)](https://github.com/sc-forks/solidity-coverage/blob/master/docs/faq.md#running-out-of-memory-locally-and-in-ci)
87-
+ [Running out of time (in mocha)](https://github.com/sc-forks/solidity-coverage/blob/master/docs/faq.md#running-out-of-time-in-mocha)
88-
+ [Running in CI](https://github.com/sc-forks/solidity-coverage/blob/master/docs/faq.md#continuous-integration-installing-metacoin-on-travisci-with-coveralls)
89-
+ [Why are asserts and requires highlighted as branch points?](https://github.com/sc-forks/solidity-coverage/blob/master/docs/faq.md#why-has-my-branch-coverage-decreased-why-is-assert-being-shown-as-a-branch-point)
84+
+ [Running in CI][7]
85+
+ [Running out of gas][4]
86+
+ [Running out of time][6]
87+
+ [Running out of memory][5]
88+
+ [Why are `require` statements highlighted as branch points?][8]
9089

9190

92-
### Example reports
93-
+ [metacoin](https://sc-forks.github.io/metacoin/) (Istanbul HTML)
94-
+ [openzeppelin-solidity](https://coveralls.io/github/OpenZeppelin/openzeppelin-solidity?branch=master) (Coveralls)
91+
## Example reports
92+
+ [metacoin][9] (Istanbul HTML)
93+
+ [openzeppelin-solidity][10](Coveralls)
9594

96-
### Contribution Guidelines
95+
## Contribution Guidelines
9796

98-
Contributions are welcome! If you're opening a PR that adds features please consider writing some
99-
[unit tests](https://github.com/sc-forks/solidity-coverage/tree/master/test) for them. Bugs can be
100-
reported in the [issues](https://github.com/sc-forks/solidity-coverage/issues).
97+
Contributions are welcome! If you're opening a PR that adds features or options *please consider
98+
writing full [unit tests][11] for them*. (We've built simple fixtures for almost everything
99+
and are happy to add some for your case if necessary).
101100

102101
Set up the development environment with:
103102
```
104103
$ git clone https://github.com/sc-forks/solidity-coverage.git
105104
$ yarn
106105
```
107106

108-
### Contributors
107+
## Contributors
109108
+ [@area](https://github.com/area)
110109
+ [@cgewecke](https://github.com/cgewecke)
111110
+ [@adriamb](https://github.com/adriamb)
@@ -126,3 +125,26 @@ $ yarn
126125
+ [@angus-hamill](https://github.com/angus-hamill)
127126
+ [@kandrianov](https://github.com/kandrianov)
128127
+ [@yxliang01](https://github.com/yxliang01)
128+
129+
[1]: https://github.com/trufflesuite/ganache-core#options
130+
[2]: https://istanbul.js.org/docs/advanced/alternative-reporters/
131+
[3]: https://mochajs.org/api/mocha
132+
[4]: https://github.com/sc-forks/solidity-coverage/blob/beta/docs/faq.md#running-out-of-gas
133+
[5]: https://github.com/sc-forks/solidity-coverage/blob/beta/docs/faq.md#running-out-of-memory
134+
[6]: https://github.com/sc-forks/solidity-coverage/blob/beta/docs/faq.md#running-out-of-time
135+
[7]: https://github.com/sc-forks/solidity-coverage/blob/beta/docs/faq.md#continuous-integration
136+
[8]: https://github.com/sc-forks/solidity-coverage/blob/beta/docs/faq.md#notes-on-branch-coverage
137+
[9]: https://sc-forks.github.io/metacoin/
138+
[10]: https://coveralls.io/github/OpenZeppelin/openzeppelin-solidity?branch=master
139+
[11]: https://github.com/sc-forks/solidity-coverage/tree/beta/test/units
140+
[12]: https://github.com/sc-forks/solidity-coverage/issues
141+
[13]: https://github.com/sc-forks/solidity-coverage/blob/beta/docs/faq.md#notes-on-gas-distortion
142+
[14]: https://github.com/sc-forks/solidity-coverage/blob/beta/docs/advanced.md
143+
[15]: #config-options
144+
[16]: https://blog.colony.io/code-coverage-for-solidity-eecfa88668c2
145+
[17]: https://github.com/JoinColony/solcover
146+
[18]: https://gitter.im/sc-forks/solidity-coverage?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge
147+
[19]: https://badge.fury.io/js/solidity-coverage
148+
[20]: https://circleci.com/gh/sc-forks/solidity-coverage
149+
[21]: https://codecov.io/gh/sc-forks/solidity-coverage
150+
[22]: https://cdn-images-1.medium.com/max/800/1*uum8t-31bUaa6dTRVVhj6w.png

‎docs/advanced.md

Lines changed: 88 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,88 @@
1+
# Advanced Use
2+
3+
## Skipping tests
4+
5+
Sometimes it's convenient to skip specific tests when running coverage. You can do this by
6+
tagging your test descriptions and setting appropriate filters in the `.solcover.js` mocha options.
7+
8+
**Example**
9+
```javascript
10+
// Mocha test to skip
11+
it("is a gas usage simulation [ @skip-on-coverage ]", async function(){
12+
...
13+
})
14+
```
15+
16+
```javascript
17+
//.solcover.js
18+
module.exports = {
19+
mocha: {
20+
grep: "@skip-on-coverage", // Find everything with this tag
21+
invert: true // Run the grep's inverse set.
22+
}
23+
}
24+
```
25+
26+
## Workflow hooks
27+
28+
The plugin exposes a set of workflow hooks that let you run arbitrary async logic between the main
29+
stages of the coverage generation process. These are useful for tasks like launching secondary
30+
services which need to connect to a running ganache instance (ex: the Oraclize/Provable bridge),
31+
or reading data from the compilation artifacts to run special preparatory steps for your tests.
32+
33+
The stages/hooks are (in order of execution):
34+
35+
| Stage | Post-stage hook |
36+
|----------------------------------------|--------------------|
37+
| Launch server | onServerReady |
38+
| Instrument and compile contracts | onCompileComplete |
39+
| Run tests using instrumented artifacts | onTestsComplete |
40+
| Generate istanbul coverage reports | onIstanbulComplete |
41+
42+
The tool's general workflow is:
43+
44+
+ Launch an ethereum client, attaching special listeners that monitor each opcode execution step
45+
+ Read Solidity contract sources from a standard contracts directory
46+
+ Rewrite the sources so the code execution path can be tracked by the opcode monitors.
47+
+ Compile the modified sources, without optimization
48+
+ Save the compilation artifacts to a temporary folder
49+
+ Tell the testing framework to use the instrumented artifacts & run tests to completion.
50+
+ Transfer collected data to a coverage reporter & report.
51+
52+
Each hook is passed a `config` object provided by your plugin's dev platform which will contain
53+
relevant source/artifact paths and network info for that stage.
54+
55+
**Example**
56+
57+
```javascript
58+
// .solcover.js
59+
const { provableBridge } = require('./helpers');
60+
61+
async function serverReadyHandler(config){
62+
await provableBridge(config.port);
63+
}
64+
65+
module.exports = {
66+
onServerReady: serverReadyHandler,
67+
}
68+
```
69+
70+
## Setting the temporary artifacts directory
71+
72+
The `temp` command line option lets you to specify the name of a disposable folder to
73+
stage the compilation artifacts of instrumented contracts in before the tests are run.
74+
75+
**Example**
76+
```
77+
$ truffle run coverage --temp build
78+
```
79+
80+
By default this folder is called `.coverage_artifacts`. If you already have
81+
preparatory scripts which run between compilation and the tests, you'll probably
82+
find it inconvenient to modify them to handle an alternate path.
83+
84+
This option allows you to avoid that but it's important to realise that the temp
85+
folder is **automatically deleted** when coverage completes. You shouldn't use it if your preferred
86+
build target contains information you want to preserve between test runs.
87+
88+

0 commit comments

Comments
 (0)