Skip to content

Commit 64a501b

Browse files
authored
Merge pull request #372 from sc-forks/truffle-plugin
0.7.0 Phase 1: opcode trace instrumentation & truffle plugin app architecture
2 parents 15fa08b + 00f32f6 commit 64a501b

File tree

222 files changed

+502424
-4031
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

222 files changed

+502424
-4031
lines changed

.circleci/config.yml

Lines changed: 23 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -14,22 +14,30 @@ step_install_nvm: &step_install_nvm
1414
echo "[ -s \"$NVM_DIR/nvm.sh\" ] && . \"$NVM_DIR/nvm.sh\"" >> $BASH_ENV
1515
jobs:
1616
unit-test:
17-
docker:
18-
- image: circleci/node:8.15.0
17+
machine: true
1918
steps:
2019
- checkout
20+
- <<: *step_install_nvm
2121
- run:
2222
name: Delete any old node_modules
2323
command: |
2424
rm -rf node_modules/
25+
- run:
26+
name: Install yarn
27+
command: |
28+
npm install -g yarn
29+
- run:
30+
name: Install truffle (globally)
31+
command: |
32+
npm install -g truffle
2533
- run:
2634
name: Install dependencies
2735
command: |
2836
yarn
2937
- run:
3038
name: Run tests
3139
command: |
32-
npm run test-cov
40+
npm run test:ci
3341
- run:
3442
name: Upload coverage
3543
command: |
@@ -56,7 +64,6 @@ jobs:
5664
name: Zeppelin E2E
5765
command: |
5866
./scripts/run-zeppelin.sh
59-
6067
e2e-metacoin:
6168
machine: true
6269
steps:
@@ -73,14 +80,16 @@ workflows:
7380
- unit-test
7481
- e2e-zeppelin
7582
- e2e-metacoin
76-
nightly:
77-
triggers:
78-
- schedule:
79-
cron: "0 1 * * *" # 1am UTC
80-
filters:
81-
branches:
82-
only:
83-
- master
84-
jobs:
85-
- e2e-zeppelin
83+
# TODO: re-enable.
84+
# At the moment we're using forks so this is pointless
85+
#nightly:
86+
# triggers:
87+
# - schedule:
88+
# cron: "0 1 * * *" # 1am UTC
89+
# filters:
90+
# branches:
91+
# only:
92+
# - master
93+
# jobs:
94+
#- e2e-zeppelin
8695
#- e2e-colony

.gitignore

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,9 @@
1-
allFiredEvents
2-
scTopics
3-
scDebugLog
41
coverage.json
52
coverage/
63
node_modules/
74
.changelog
85
.DS_Store
6+
test/artifacts
7+
test/cache
8+
temp
9+
.nyc_output/

.npmignore

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
test/
2+
.circleci/
3+
docs/
4+
.nyc_output/

README.md

Lines changed: 104 additions & 102 deletions
Original file line numberDiff line numberDiff line change
@@ -1,133 +1,110 @@
11
# solidity-coverage
22

3-
[![Join the chat at https://gitter.im/sc-forks/solidity-coverage](https://badges.gitter.im/sc-forks/solidity-coverage.svg)](https://gitter.im/sc-forks/solidity-coverage?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge)
4-
[![npm version](https://badge.fury.io/js/solidity-coverage.svg)](https://badge.fury.io/js/solidity-coverage)
5-
[![CircleCI](https://circleci.com/gh/sc-forks/solidity-coverage.svg?style=svg)](https://circleci.com/gh/sc-forks/solidity-coverage)
6-
[![codecov](https://codecov.io/gh/sc-forks/solidity-coverage/branch/master/graph/badge.svg)](https://codecov.io/gh/sc-forks/solidity-coverage)
7-
8-
### Code coverage for Solidity testing
9-
![coverage example](https://cdn-images-1.medium.com/max/800/1*uum8t-31bUaa6dTRVVhj6w.png)
10-
11-
+ For more details about what this is, how it works and potential limitations, see
12-
[the accompanying article](https://blog.colony.io/code-coverage-for-solidity-eecfa88668c2).
13-
+ `solidity-coverage` is in development and **its accuracy is unknown.** If you
14-
find discrepancies between the coverage report and your suite's behavior, please open an
15-
[issue](https://github.com/sc-forks/solidity-coverage/issues).
16-
+ `solidity-coverage` is [Solcover](https://github.com/JoinColony/solcover)
17-
18-
### Install
3+
[![Gitter chat](https://badges.gitter.im/sc-forks/solidity-coverage.svg)][18]
4+
![npm (tag)](https://img.shields.io/npm/v/solidity-coverage/beta)
5+
[![CircleCI](https://circleci.com/gh/sc-forks/solidity-coverage.svg?style=svg)][20]
6+
[![codecov](https://codecov.io/gh/sc-forks/solidity-coverage/branch/beta/graph/badge.svg)][21]
7+
8+
## Code coverage for Solidity testing
9+
![coverage example][22]
10+
11+
+ For more details about what this is, how it works and potential limitations,
12+
see [the accompanying article][16].
13+
+ `solidity-coverage` is [Solcover][17]
14+
15+
## Install
1916
```
20-
$ npm install --save-dev solidity-coverage
17+
$ npm install --save-dev solidity-coverage@beta
2118
```
2219

23-
### Run
24-
Set a `coverage` network in truffle-config.js (see [Network Configuration](#network-configuration)) and then run:
20+
## Truffle V5
21+
22+
**Add** this package to your plugins array in `truffle-config.js`
23+
```javascript
24+
module.exports = {
25+
networks: {...},
26+
plugins: ["solidity-coverage"]
27+
}
2528
```
26-
$ npx solidity-coverage
29+
**Run**
30+
```
31+
truffle run coverage [command-options]
2732
```
2833

29-
### Usage notes:
30-
+ For pragma solidity >=0.4.22 <0.6.0 / Petersburg / Truffle v4 and v5
31-
+ Tests run more slowly while coverage is being generated.
32-
+ Your contracts will be double-compiled and a (long) delay between compilation and
33-
the beginning of test execution is possible if your contracts are large.
34-
+ Truffle should be globallly installed in your environment.. If you prefer running truffle as
35-
a local dependency, please see [this section](https://github.com/sc-forks/solidity-coverage/blob/master/docs/faq.md#running-truffle-as-a-local-dependency) of the FAQ.
36-
+ If your suite uses native Solidity testing or accesses contracts via mocks stored in `tests/` (a la Zeppelin), coverage will trigger test errors because it can't control the way truffle compiles that folder. Mocks should be relocated to the root `contracts` directory. More on why this is necessary at issue [146](https://github.com/sc-forks/solidity-coverage/issues/146)
37-
38-
### Network Configuration
34+
## Usage notes:
35+
+ Coverage runs tests a little more slowly.
36+
+ Coverage launches its own in-process ganache server.
37+
+ You can set [ganache options][1] using the `providerOptions` key in your `.solcover.js` [config][15].
38+
+ Coverage [distorts gas consumption][13]. Tests that check exact gas consumption should be skipped.
3939

40-
By default, this tool connects to a coverage-enabled fork of ganache-cli
41-
called **testrpc-sc** on port 8555.
42-
+ it's a dependency - there's nothing extra to download.
43-
+ the solidity-coverage command launches it automatically in the background. (See [this section of the FAQ](https://github.com/sc-forks/solidity-coverage/blob/master/docs/faq.md#running-testrpc-sc-on-its-own) if you need to launch it separately yourself)
40+
## Command Options
41+
| Option <img width=200/> | Example <img width=750/>| Description <img width=1000/> |
42+
|--------------|------------------------------------|--------------------------------|
43+
| file | `--file="test/registry/*.js"` | Filename or glob describing a subset of JS tests to run. (Globs must be enclosed by quotes.)|
44+
| solcoverjs | `--solcoverjs ./../.solcover.js` | Relative path from working directory to config. Useful for monorepo packages that share settings. (Path must be "./" prefixed) |
45+
| network | `--network development` | Use network settings defined in the Truffle config |
46+
| temp[<sup>*</sup>][14] | `--temp build` | :warning: **Caution** :warning: Path to a *disposable* folder to store compilation artifacts in. Useful when your test setup scripts include hard-coded paths to a build directory. [More...][14] |
47+
| version | | Version info |
48+
| help | | Usage notes |
4449

45-
In `truffle-config.js`, add a coverage network following the example below.
50+
[<sup>*</sup> Advanced use][14]
4651

47-
**Example**
48-
```javascript
49-
module.exports = {
50-
networks: {
51-
development: {
52-
host: "localhost",
53-
port: 8545,
54-
network_id: "*"
55-
},
56-
coverage: {
57-
host: "localhost",
58-
network_id: "*",
59-
port: 8555, // <-- If you change this, also set the port option in .solcover.js.
60-
gas: 0xfffffffffff, // <-- Use this high gas value
61-
gasPrice: 0x01 // <-- Use this low gas price
62-
},
63-
...etc...
64-
}
65-
};
66-
```
67-
### Options
52+
## Config Options
6853

69-
Additional options can be specified in a `.solcover.js` config file located in
70-
the root directory of your project.
54+
Additional options can be specified in a `.solcover.js` config file located in the root directory
55+
of your project.
7156

7257
**Example:**
7358
```javascript
7459
module.exports = {
75-
port: 6545,
76-
testrpcOptions: '-p 6545 -u 0x54fd80d6ae7584d8e9a19fe1df43f04e5282cc43',
77-
testCommand: 'mocha --timeout 5000',
78-
norpc: true,
79-
dir: './secretDirectory',
80-
copyPackages: ['openzeppelin-solidity'],
81-
skipFiles: ['Routers/EtherRouter.sol']
60+
skipFiles: ['Routers/EtherRouter.sol']
8261
};
8362
```
8463

8564

86-
| Option | Type | Default | Description |
65+
| Option <img width=200/>| Type <img width=200/> | Default <img width=700/> | Description <img width=1000/> |
8766
| ------ | ---- | ------- | ----------- |
88-
| accounts | *Number* | 35 | Number of accounts to launch testrpc with. |
89-
| port | *Number* | 8555 | Port to run testrpc on / have truffle connect to |
90-
| norpc | *Boolean* | false | Prevent solidity-coverage from launching its own testrpc. Useful if you are managing a complex test suite with a [shell script](https://github.com/OpenZeppelin/openzeppelin-solidity/blob/ed872ca0a11c4926f8bb91dd103bea1378a3384c/scripts/coverage.sh) |
91-
| testCommand | *String* | `truffle test` | Run an arbitrary test command. ex: `mocha --timeout 5000`. NB: Also set the `port` option to whatever your tests require (probably 8545). |
92-
| testrpcOptions | *String* | `--port 8555` | options to append to a command line invocation of testrpc. NB: Using this overwrites the defaults so always specify a port in this string *and* in the `port` option |
93-
| copyNodeModules | *Boolean* | false | :warning: **DEPRECATED** use `copyPackages` instead :warning: Copies `node_modules` into the coverage environment. May significantly increase the time for coverage to complete if enabled. Useful if your contracts import solidity files from an npm installed package (and your node_modules is small). |
94-
| copyPackages | *Array* | `[]` | Copies specific `node_modules` packages into the coverage environment. May significantly reduce the time for coverage to complete compared to `copyNodeModules`. Useful if your contracts import solidity files from an npm installed package. |
95-
| skipFiles | *Array* | `['Migrations.sol']` | Array of contracts or folders (with paths expressed relative to the `contracts` directory) that should be skipped when doing instrumentation. `Migrations.sol` is skipped by default, and does not need to be added to this configuration option if it is used. |
96-
| deepSkip | boolean | false | Use this if instrumentation hangs on large, skipped files (like Oraclize). It's faster. |
97-
| dir | *String* | `.` | Solidity-coverage copies all the assets in your root directory (except `node_modules`) to a special folder where it instruments the contracts and executes the tests. `dir` allows you to define a relative path from the root directory to those assets. Useful if your contracts & tests are within their own folder as part of a larger project.|
98-
| buildDirPath | *String* | `/build/contracts` | Build directory path for compiled smart contracts
99-
100-
### FAQ
101-
102-
Solutions to common problems people run into:
103-
104-
+ [Running out of gas](https://github.com/sc-forks/solidity-coverage/blob/master/docs/faq.md#running-out-of-gas)
105-
+ [Running out of memory (locally and in CI)](https://github.com/sc-forks/solidity-coverage/blob/master/docs/faq.md#running-out-of-memory-locally-and-in-ci)
106-
+ [Running out of time (in mocha)](https://github.com/sc-forks/solidity-coverage/blob/master/docs/faq.md#running-out-of-time-in-mocha)
107-
+ [Running on windows](https://github.com/sc-forks/solidity-coverage/blob/master/docs/faq.md#running-on-windows)
108-
+ [Running testrpc-sc on its own](https://github.com/sc-forks/solidity-coverage/blob/master/docs/faq.md#running-testrpc-sc-on-its-own)
109-
+ [Running truffle as a local dependency](https://github.com/sc-forks/solidity-coverage/blob/master/docs/faq.md#running-truffle-as-a-local-dependency)
110-
+ [Integrating into CI](https://github.com/sc-forks/solidity-coverage/blob/master/docs/faq.md#continuous-integration-installing-metacoin-on-travisci-with-coveralls)
111-
+ [Why are asserts and requires highlighted as branch points?](https://github.com/sc-forks/solidity-coverage/blob/master/docs/faq.md#why-has-my-branch-coverage-decreased-why-is-assert-being-shown-as-a-branch-point)
112-
113-
114-
### Example reports
115-
+ [metacoin](https://sc-forks.github.io/metacoin/) (Istanbul HTML)
116-
+ [openzeppelin-solidity](https://coveralls.io/github/OpenZeppelin/openzeppelin-solidity?branch=master) (Coveralls)
117-
118-
### Contribution Guidelines
119-
120-
Contributions are welcome! If you're opening a PR that adds features please consider writing some
121-
[unit tests](https://github.com/sc-forks/solidity-coverage/tree/master/test) for them. Bugs can be
122-
reported in the [issues](https://github.com/sc-forks/solidity-coverage/issues).
67+
| silent | *Boolean* | false | Suppress logging output |
68+
| client | *Object* | `require("ganache-core")` | Useful if you need a specific ganache version. |
69+
| providerOptions | *Object* | `{ }` | [ganache-core options][1] |
70+
| skipFiles | *Array* | `['Migrations.sol']` | Array of contracts or folders (with paths expressed relative to the `contracts` directory) that should be skipped when doing instrumentation. |
71+
| istanbulReporter | *Array* | `['html', 'lcov', 'text']` | [Istanbul coverage reporters][2] |
72+
| mocha | *Object* | `{ }` | [Mocha options][3] to merge into existing mocha config. `grep` and `invert` are useful for skipping certain tests under coverage using tags in the test descriptions.|
73+
| onServerReady[<sup>*</sup>][14] | *Function* | | Hook run *after* server is launched, *before* the tests execute. Useful if you need to use the Oraclize bridge or have setup scripts which rely on the server's availability. [More...][14] |
74+
| onCompileComplete[<sup>*</sup>][14] | *Function* | | Hook run *after* compilation completes, *before* tests are run. Useful if you have secondary compilation steps or need to modify built artifacts. [More...][14]|
75+
| onTestsComplete[<sup>*</sup>][14] | *Function* | | Hook run *after* the tests complete, *before* Istanbul reports are generated.|
76+
| onIstanbulComplete[<sup>*</sup>][14] | *Function* | | Hook run *after* the Istanbul reports are generated, *before* the ganache server is shut down. Useful if you need to clean resources up.|
77+
78+
[<sup>*</sup> Advanced use][14]
79+
80+
## FAQ
81+
82+
Common problems & questions:
83+
84+
+ [Running in CI][7]
85+
+ [Running out of gas][4]
86+
+ [Running out of time][6]
87+
+ [Running out of memory][5]
88+
+ [Why are `require` statements highlighted as branch points?][8]
89+
90+
91+
## Example reports
92+
+ [metacoin][9] (Istanbul HTML)
93+
+ [openzeppelin-solidity][10](Coveralls)
94+
95+
## Contribution Guidelines
96+
97+
Contributions are welcome! If you're opening a PR that adds features or options *please consider
98+
writing full [unit tests][11] for them*. (We've built simple fixtures for almost everything
99+
and are happy to add some for your case if necessary).
123100

124101
Set up the development environment with:
125102
```
126103
$ git clone https://github.com/sc-forks/solidity-coverage.git
127104
$ yarn
128105
```
129106

130-
### Contributors
107+
## Contributors
131108
+ [@area](https://github.com/area)
132109
+ [@cgewecke](https://github.com/cgewecke)
133110
+ [@adriamb](https://github.com/adriamb)
@@ -146,3 +123,28 @@ $ yarn
146123
+ [@pinkiebell](https://github.com/pinkiebell)
147124
+ [@obernardovieira](https://github.com/obernardovieira)
148125
+ [@angus-hamill](https://github.com/angus-hamill)
126+
+ [@kandrianov](https://github.com/kandrianov)
127+
+ [@yxliang01](https://github.com/yxliang01)
128+
129+
[1]: https://github.com/trufflesuite/ganache-core#options
130+
[2]: https://istanbul.js.org/docs/advanced/alternative-reporters/
131+
[3]: https://mochajs.org/api/mocha
132+
[4]: https://github.com/sc-forks/solidity-coverage/blob/beta/docs/faq.md#running-out-of-gas
133+
[5]: https://github.com/sc-forks/solidity-coverage/blob/beta/docs/faq.md#running-out-of-memory
134+
[6]: https://github.com/sc-forks/solidity-coverage/blob/beta/docs/faq.md#running-out-of-time
135+
[7]: https://github.com/sc-forks/solidity-coverage/blob/beta/docs/faq.md#continuous-integration
136+
[8]: https://github.com/sc-forks/solidity-coverage/blob/beta/docs/faq.md#notes-on-branch-coverage
137+
[9]: https://sc-forks.github.io/metacoin/
138+
[10]: https://coveralls.io/github/OpenZeppelin/openzeppelin-solidity?branch=master
139+
[11]: https://github.com/sc-forks/solidity-coverage/tree/beta/test/units
140+
[12]: https://github.com/sc-forks/solidity-coverage/issues
141+
[13]: https://github.com/sc-forks/solidity-coverage/blob/beta/docs/faq.md#notes-on-gas-distortion
142+
[14]: https://github.com/sc-forks/solidity-coverage/blob/beta/docs/advanced.md
143+
[15]: #config-options
144+
[16]: https://blog.colony.io/code-coverage-for-solidity-eecfa88668c2
145+
[17]: https://github.com/JoinColony/solcover
146+
[18]: https://gitter.im/sc-forks/solidity-coverage?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge
147+
[19]: https://badge.fury.io/js/solidity-coverage
148+
[20]: https://circleci.com/gh/sc-forks/solidity-coverage
149+
[21]: https://codecov.io/gh/sc-forks/solidity-coverage
150+
[22]: https://cdn-images-1.medium.com/max/800/1*uum8t-31bUaa6dTRVVhj6w.png

bin/exec.js

Lines changed: 0 additions & 23 deletions
This file was deleted.

buidler.config.js

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,18 @@
1+
2+
// Mute compiler warnings - this will need to be addressed properly in
3+
// the Buidler plugin by overloading TASK_COMPILE_COMPILE.
4+
const originalLog = console.log;
5+
console.warn = () => {};
6+
console.log = val => val === '\n' ? null : originalLog(val);
7+
8+
module.exports = {
9+
solc: {
10+
version: "0.5.8"
11+
},
12+
paths: {
13+
artifacts: "./test/artifacts",
14+
cache: "./test/cache",
15+
test: "./test/units",
16+
sources: "./test/sources/contracts",
17+
}
18+
}

dist/bin.js

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
/*
2+
Logs a warning / informational message when user tries to
3+
invoke 'solidity-coverage' as a shell command. This file
4+
is listed as the package.json "bin".
5+
*/
6+
const AppUI = require('../lib/ui').AppUI;
7+
8+
(new AppUI()).report('command');

dist/buidler.plugin.js

Whitespace-only changes.

0 commit comments

Comments
 (0)