Skip to content
This repository was archived by the owner on Feb 12, 2024. It is now read-only.

feat: new libp2p config #1401

Merged
merged 1 commit into from
Jul 2, 2018
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 13 additions & 7 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -224,22 +224,28 @@ Creates and returns an instance of an IPFS node. Use the `options` argument to s

- `pass` (string): A passphrase to encrypt/decrypt your keys.

- `relay` (object): Configure circuit relay (see the [circuit relay tutorial](https://github.com/ipfs/js-ipfs/tree/master/examples/circuit-relaying) to learn more).
- `enabled` (boolean): Enable circuit relay dialer and listener. (Default: `false`)
- `hop` (object)
- `enabled` (boolean): Make this node a relay (other nodes can connect *through* it). (Default: `false`)
- `active` (boolean): Make this an *active* relay node. Active relay nodes will attempt to dial a destination peer even if that peer is not yet connected to the relay. (Default: `false`)

- `EXPERIMENTAL` (object): Enable and configure experimental features.
- `pubsub` (boolean): Enable libp2p pub-sub. (Default: `false`)
- `sharding` (boolean): Enable directory sharding. Directories that have many child objects will be represented by multiple DAG nodes instead of just one. It can improve lookup performance when a directory has several thousand files or more. (Default: `false`)
- `dht` (boolean): Enable KadDHT. **This is currently not interopable with `go-ipfs`.**
- `relay` (object): Configure circuit relay (see the [circuit relay tutorial](https://github.com/ipfs/js-ipfs/tree/master/examples/circuit-relaying) to learn more).
- `enabled` (boolean): Enable circuit relay dialer and listener. (Default: `false`)
- `hop` (object)
- `enabled` (boolean): Make this node a relay (other nodes can connect *through* it). (Default: `false`)
- `active` (boolean): Make this an *active* relay node. Active relay nodes will attempt to dial a destination peer even if that peer is not yet connected to the relay. (Default: `false`)

- `config` (object) Modify the default IPFS node config. Find the Node.js defaults at [`src/core/runtime/config-nodejs.js`](https://github.com/ipfs/js-ipfs/tree/master/src/core/runtime/config-nodejs.js) and the browser defaults at [`src/core/runtime/config-browser.js`](https://github.com/ipfs/js-ipfs/tree/master/src/core/runtime/config-browser.js). This object will be *merged* with the default config; it will not replace it.

- `libp2p` (object) add custom modules to the libp2p stack of your node
- `modules` (object):
- `transport` (Array<[libp2p.Transport](https://github.com/libp2p/interface-transport)>): An array of additional Libp2p transport instances to use. See [libp2p/interface-transport](https://github.com/libp2p/interface-transport) for details.
- `discovery` (Array<[libp2p.PeerDiscovery](https://github.com/libp2p/interface-peer-discovery)>): An array of additional Libp2p peer discovery instances to use. See [libp2p/peer-discovery](https://github.com/libp2p/interface-peer-discovery) for details.
- `transport` (Array<[libp2p.Transport](https://github.com/libp2p/interface-transport)>): An array of Libp2p transport classes/instances to use _instead_ of the defaults. See [libp2p/interface-transport](https://github.com/libp2p/interface-transport) for details.
- `peerDiscovery` (Array<[libp2p.PeerDiscovery](https://github.com/libp2p/interface-peer-discovery)>): An array of Libp2p peer discovery classes/instances to use _instead_ of the defaults. See [libp2p/peer-discovery](https://github.com/libp2p/interface-peer-discovery) for details. If passing a class, configuration can be passed using the config section below under the key corresponding to you module's unique `tag` (a static property on the class)
- `config` (object):
- `peerDiscovery` (object):
- `[PeerDiscovery.tag]` (object): configuration for a peer discovery module
- `enabled` (boolean): whether this module is enabled or disabled
- `[custom config]` (any): other keys are specific to the module

#### Events

Expand Down
40 changes: 17 additions & 23 deletions examples/circuit-relaying/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ Here is a simple diagram depicting how a typical circuit-relay connection might
+---------------------+
```

`Node A` tries to connect to `Node B` but, UH-OH! There is a firewall in between that's preventing it from happening. If both `Node A` and `Node B` know about a relay, they can use it to establish the connection.
`Node A` tries to connect to `Node B` but, UH-OH! There is a firewall in between that's preventing it from happening. If both `Node A` and `Node B` know about a relay, they can use it to establish the connection.

This is what it looks like, in simplified steps:

Expand Down Expand Up @@ -60,7 +60,7 @@ There are a couple of caveats and features to be aware of:

#### A word on circuit relay addresses

A circuit relay address is a [multiaddress](https://multiformats.io/multiaddr/) that describes how to either connect to a peer over a relay (or relays), or allow a peer to announce it is reachable over a particular relay or any relay it is already connected to.
A circuit relay address is a [multiaddress](https://multiformats.io/multiaddr/) that describes how to either connect to a peer over a relay (or relays), or allow a peer to announce it is reachable over a particular relay or any relay it is already connected to.

Circuit relay addresses are very flexible and can describe many different aspects of how to esablish the relayed connection. In its simplest form, it looks something like this:

Expand All @@ -78,8 +78,8 @@ We can take it a step further and encode the same information for the destinatio

- `/ip4/127.0.0.1/tcp/65000/ipfs/QmRelay/p2p-circuit`

If a node is configured with this address, it will use the specified host (`/ip4/127.0.0.1/tcp/65000/ipfs/QmRelay`) as a relay and it will be reachable over this relay.
- There could multiple addresses of this sort specified in the config, in which case the node will be reachable over all of them.
If a node is configured with this address, it will use the specified host (`/ip4/127.0.0.1/tcp/65000/ipfs/QmRelay`) as a relay and it will be reachable over this relay.
- There could multiple addresses of this sort specified in the config, in which case the node will be reachable over all of them.
- This is useful if, for example, the node is behind a firewall but wants to be reachable from the outside over a specific relay.

Other use-cases are also supported by this scheme, e.g. we can have multiple hops (circuit-relay nodes) encoded in the address, something planed for future releases.
Expand Down Expand Up @@ -153,7 +153,7 @@ In order to enable the relay functionality in `go-ipfs` we need to edit it's con
}
```

The two options we're looking for are `DisableRelay` and `EnableRelayHop`. We want the former (`DisableRelay`) set to `false` and the latter (`EnableRelayHop`) to `true`, just like in the example above. That should set our go node as a relay.
The two options we're looking for are `DisableRelay` and `EnableRelayHop`. We want the former (`DisableRelay`) set to `false` and the latter (`EnableRelayHop`) to `true`, just like in the example above. That should set our go node as a relay.

We also need to make sure our go node can be dialed from the browser. For that, we need to enable a transport that both the browser and the go node can communicate over. We will use the web sockets transport, although there are others that can be used, such as `webrtc-star` and `websocket-star`. To enable the transport and set the interface and port we need to edit the `~/.ipfs/config` one more time. Let's find the `Swarm` array and add our desired address there. I picked `/ip4/0.0.0.0/tcp/4004/ws` because it is a port I know is not being used by anything on my machine, but we can also use port `0` so that the OS chooses a random available port for us — either one should work.

Expand All @@ -173,15 +173,11 @@ We need to go through similar steps to enable circuit relay in `jsipfs`. However

Just as we did with `go-ipfs`, go ahead and edit `js-ipfs` config file located under `~/.jsipfs/config`. Let's add the following config:

(Note that the "EXPERIMENTAL" section might be missing from the config file. In that case, just go ahead and add it)

```js
"EXPERIMENTAL": {
"relay": {
"enabled": true,
"hop": {
"enabled": true
}
"relay": {
"enabled": true,
"hop": {
"enabled": true
}
}
```
Expand Down Expand Up @@ -247,7 +243,7 @@ Gateway (readonly) is listening on: /ip4/127.0.0.1/tcp/9090
Daemon is ready
```

Look out for an address similar to `/ip4/127.0.0.1/tcp/4003/ws/ipfs/Qm...`. Note it down somewhere, and let's move on to the next step.
Look out for an address similar to `/ip4/127.0.0.1/tcp/4003/ws/ipfs/Qm...`. Note it down somewhere, and let's move on to the next step.

### 2. Configure and run the bundled example

Expand All @@ -270,7 +266,7 @@ The bundled example is a simple chat app that uses another cool ipfs feature - [

### 3. Connect the two browser nodes to the circuit relay

In order for our browser nodes to be able to messages each other, we need to get them connected. But to do that, we need to use a relay - browser nodes can't be connected directly because of lack of socket support.
In order for our browser nodes to be able to messages each other, we need to get them connected. But to do that, we need to use a relay - browser nodes can't be connected directly because of lack of socket support.

Remember the caveat above `Currently a Relay will only work if it already has a connection to the STOP node`? This means that we need to connect our browser nodes to the relay node first.

Expand Down Expand Up @@ -304,21 +300,19 @@ Thats it!

### So what just happened?

Good question!
Good question!

- We used [js-ipfs](htpps://github.com/ipfs/js-ipfs) running in the browser with circuit relay enabled:
- _Notice the `EXPERIMENTAL.relay.enabled` below_
- _Notice the `relay.enabled` below_

you can find it in [src/app.js](src/app.js)
```js
const ipfs = new IPFS({
repo: repo(),
EXPERIMENTAL: {
relay: {
enabled: true,
hop: {
enabled: true
}
relay: {
enabled: true,
hop: {
enabled: true
}
},
config: {
Expand Down
21 changes: 11 additions & 10 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -61,7 +61,7 @@
},
"homepage": "https://github.com/ipfs/js-ipfs#readme",
"devDependencies": {
"aegir": "^13.1.0",
"aegir": "^14.0.0",
"buffer-loader": "~0.0.1",
"chai": "^4.1.2",
"delay": "^2.0.0",
Expand All @@ -73,7 +73,7 @@
"expose-loader": "~0.7.5",
"form-data": "^2.3.2",
"hat": "0.0.3",
"interface-ipfs-core": "~0.69.0",
"interface-ipfs-core": "~0.69.2",
"ipfsd-ctl": "~0.37.3",
"mocha": "^5.1.1",
"ncp": "^2.0.0",
Expand All @@ -86,6 +86,7 @@
"transform-loader": "~0.2.4"
},
"dependencies": {
"@nodeutils/defaults-deep": "^1.1.0",
"async": "^2.6.0",
"big.js": "^5.1.2",
"binary-querystring": "~0.1.2",
Expand All @@ -105,12 +106,12 @@
"hapi-set-header": "^1.0.2",
"hoek": "^5.0.3",
"human-to-milliseconds": "^1.0.0",
"interface-datastore": "^0.4.1",
"interface-datastore": "~0.4.1",
"ipfs-api": "^22.1.1",
"ipfs-bitswap": "~0.20.0",
"ipfs-block": "~0.7.1",
"ipfs-block-service": "~0.14.0",
"ipfs-http-response": "^0.1.2",
"ipfs-http-response": "~0.1.2",
"ipfs-multipart": "~0.1.0",
"ipfs-repo": "~0.22.1",
"ipfs-unixfs": "~0.1.15",
Expand All @@ -124,18 +125,18 @@
"joi": "^13.2.0",
"joi-browser": "^13.0.1",
"joi-multiaddr": "^2.0.0",
"libp2p": "~0.20.4",
"libp2p": "~0.22.0",
"libp2p-circuit": "~0.2.0",
"libp2p-floodsub": "~0.15.0",
"libp2p-kad-dht": "~0.10.0",
"libp2p-keychain": "~0.3.1",
"libp2p-mdns": "~0.11.0",
"libp2p-mplex": "~0.7.0",
"libp2p-railing": "~0.8.1",
"libp2p-mdns": "~0.12.0",
"libp2p-mplex": "~0.8.0",
"libp2p-railing": "~0.9.2",
"libp2p-secio": "~0.10.0",
"libp2p-tcp": "~0.12.0",
"libp2p-webrtc-star": "~0.15.0",
"libp2p-websocket-star": "~0.8.0",
"libp2p-webrtc-star": "~0.15.3",
"libp2p-websocket-star": "~0.8.1",
"libp2p-websockets": "~0.12.0",
"lodash": "^4.17.10",
"mafmt": "^6.0.0",
Expand Down
55 changes: 38 additions & 17 deletions src/core/components/libp2p.js
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@
const Node = require('../runtime/libp2p-nodejs')
const promisify = require('promisify-es6')
const get = require('lodash/get')
const defaultsDeep = require('@nodeutils/defaults-deep')

module.exports = function libp2p (self) {
return {
Expand All @@ -15,27 +16,47 @@ module.exports = function libp2p (self) {
return callback(err)
}

const options = {
mdns: get(config, 'Discovery.MDNS.Enabled'),
webRTCStar: get(config, 'Discovery.webRTCStar.Enabled'),
bootstrap: get(config, 'Bootstrap'),
modules: self._libp2pModules,
// EXPERIMENTAL
pubsub: get(self._options, 'EXPERIMENTAL.pubsub', false),
dht: get(self._options, 'EXPERIMENTAL.dht', false),
relay: {
enabled: get(self._options, 'EXPERIMENTAL.relay.enabled',
get(config, 'EXPERIMENTAL.relay.enabled', false)),
hop: {
enabled: get(self._options, 'EXPERIMENTAL.relay.hop.enabled',
get(config, 'EXPERIMENTAL.relay.hop.enabled', false)),
active: get(self._options, 'EXPERIMENTAL.relay.hop.active',
get(config, 'EXPERIMENTAL.relay.hop.active', false))
const libp2pDefaults = {
peerInfo: self._peerInfo,
peerBook: self._peerInfoBook,
config: {
peerDiscovery: {
mdns: {
enabled: get(self._options, 'config.Discovery.MDNS.Enabled',
get(config, 'Discovery.MDNS.Enabled', true))
},
webRTCStar: {
enabled: get(self._options, 'config.Discovery.webRTCStar.Enabled',
get(config, 'Discovery.webRTCStar.Enabled', true))
},
bootstrap: {
list: get(self._options, 'config.Bootstrap',
get(config, 'Bootstrap', []))
}
},
relay: {
enabled: get(self._options, 'relay.enabled',
get(config, 'relay.enabled', false)),
hop: {
enabled: get(self._options, 'relay.hop.enabled',
get(config, 'relay.hop.enabled', false)),
active: get(self._options, 'relay.hop.active',
get(config, 'relay.hop.active', false))
}
},
EXPERIMENTAL: {
dht: get(self._options, 'EXPERIMENTAL.dht', false),
pubsub: get(self._options, 'EXPERIMENTAL.pubsub', false)
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice! For the next turn, we should optimize js-ipfs options just like js-libp2p so that when we do new IPFS(options), it is clear that:

const options = {
  ... // any specifics on spawning the daemon this way on the first level
  config: {
    // everything else comes from the config
  }
}

}
}
}

self._libp2pNode = new Node(self._peerInfo, self._peerInfoBook, options)
const libp2pOptions = defaultsDeep(
get(self._options, 'libp2p', {}),
libp2pDefaults
)

self._libp2pNode = new Node(libp2pOptions)

self._libp2pNode.on('peer:discovery', (peerInfo) => {
const dial = () => {
Expand Down
7 changes: 7 additions & 0 deletions src/core/config.js
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,13 @@ const schema = Joi.object().keys({
).allow(null),
start: Joi.boolean(),
pass: Joi.string().allow(''),
relay: Joi.object().keys({
enabled: Joi.boolean(),
hop: Joi.object().keys({
enabled: Joi.boolean(),
active: Joi.boolean()
}).allow(null)
}).allow(null),
EXPERIMENTAL: Joi.object().keys({
pubsub: Joi.boolean(),
sharding: Joi.boolean(),
Expand Down
4 changes: 0 additions & 4 deletions src/core/index.js
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,6 @@ class IPFS extends EventEmitter {
}

options = config.validate(options || {})
this._libp2pModules = options.libp2p && options.libp2p.modules

extend(this._options, options)

Expand Down Expand Up @@ -120,9 +119,6 @@ class IPFS extends EventEmitter {
if (this._options.EXPERIMENTAL.dht) {
this.log('EXPERIMENTAL Kademlia DHT is enabled')
}
if (this._options.EXPERIMENTAL.relay) {
this.log('EXPERIMENTAL Relay is enabled')
}

this.state = require('./state')(this)

Expand Down
68 changes: 42 additions & 26 deletions src/core/runtime/libp2p-browser.js
Original file line number Diff line number Diff line change
Expand Up @@ -5,38 +5,54 @@ const WebRTCStar = require('libp2p-webrtc-star')
const WebSocketStar = require('libp2p-websocket-star')
const Multiplex = require('libp2p-mplex')
const SECIO = require('libp2p-secio')
const Railing = require('libp2p-railing')
const Bootstrap = require('libp2p-railing')
const libp2p = require('libp2p')
const defaultsDeep = require('@nodeutils/defaults-deep')

class Node extends libp2p {
constructor (peerInfo, peerBook, options) {
options = options || {}
const wrtcstar = new WebRTCStar({id: peerInfo.id})
const wsstar = new WebSocketStar({id: peerInfo.id})

const modules = {
transport: [new WS(), wrtcstar, wsstar],
connection: {
muxer: [Multiplex],
crypto: [SECIO]
constructor (_options) {
const wrtcstar = new WebRTCStar({id: _options.peerInfo.id})
const wsstar = new WebSocketStar({id: _options.peerInfo.id})

const defaults = {
modules: {
transport: [
WS,
wrtcstar,
wsstar
],
streamMuxer: [
Multiplex
],
connEncryption: [
SECIO
],
peerDiscovery: [
wrtcstar.discovery,
wsstar.discovery,
Bootstrap
]
},
discovery: [wrtcstar.discovery, wsstar.discovery]
}

if (options.bootstrap) {
const r = new Railing(options.bootstrap)
modules.discovery.push(r)
}

if (options.modules && options.modules.transport) {
options.modules.transport.forEach((t) => modules.transport.push(t))
}

if (options.modules && options.modules.discovery) {
options.modules.discovery.forEach((d) => modules.discovery.push(d))
config: {
peerDiscovery: {
bootstrap: {
enabled: true
},
webRTCStar: {
enabled: true
},
websocketStar: {
enabled: true
}
},
EXPERIMENTAL: {
dht: false,
pubsub: false
}
}
}

super(modules, peerInfo, peerBook, options)
super(defaultsDeep(_options, defaults))
}
}

Expand Down
Loading