waku2: pending items for the integration of status-go and go-waku

[richard-ramos]

2023-12-14: DO NOT ADD MORE ISSUES! Please use: Status Waku Integration & Chat Reliability (view) for further tracking. Freezing and closing this issue.

---

• Some core contributors have run into trouble finding the bootstrap nodes, due to DNS missconfigurations. Perhaps it makes sense to have a hardcoded list of nodes in status-go with the idea of using these when DNS Discovery fails. Otherwise the node will fail to find peers.

• Static sharding for communities is being introduced in waku2: static shards #3944. This is only implemented with WakuRelay: support for working with Filter must be added -- Client

• DiscoveryV5 with Shards - I'm currently testing if this is working fine with DiscV5

• Discovery with Rendezvous - It's currently being worked on. go-waku does provide a set of functions that can be used by status-go to discover peers. Peer management when using sharding is still necessary as the number of peers per shard should be limited. -- not critical

• Peer management with sharding - This is pending in go-waku and might require some work too in status-go? go-waku should maintain a minimum number of peers in each shard the node is subscribed to. Refer to feat: discovery & peer management for static shards waku-org/go-waku#727

• 1:1 and group chats still use the default topic. Ideally they should use an specific shard so clients see reduced bandwidth usage due to not having to subscribe to the default topic. An 'easy' fix we could implement is to make all these chat types use the same shard for now. (Do note that this is a breaking change)

• Some observations from @cammellos on waku2: static shards #3944 regarding messages arriving out of order and communities possibly being null. this is strictly status-go protocol work -- Client

• Add support for community shards Add support for community shards community-dapp#91

• In status-go there is some code to use peer-exchange, but it is too old by now and does not take into account that sharding is a thing. We need to define whether we're going to use Peer exchange and since we have different discovery mechanism, what's the criteria for choosing discovery methods. Add test for Wakuv2: runPeerExchangeLoop #4343

• Changes introduced in some API methods in waku2: static shards #3944 (wakuext_setCommunityShard, wakuext_requestCommunityInfoFromMailserverWithShard, wakuext_requestCommunityInforFromMailserverAsyncWithShard), these requires work both in mobile and in desktop, so users are able to set the shard for a community, and also to be able to retrieve the description from a community -- Client

• Changes introduced in: feat: add sharding information when importing/exporting a community #3986 with new functions to export and import communities including sharding information. Using these functions will require besides asking for the private key of the community, to also ask for the shard information as well as private key to use pubsub topic protection -- Client

• Curated communities and community tokens are not aware that sharding is a thing. They need to so status-go is able to retrieve the community description from the shard where it was published. The following issues were opened: sharding: featured communities #3961 and sharding: community collectibles #3962 - These tasks require work in the smart contracts, status-go and the clients. -- Smart Contract

• Something that has been mentioned in Message Reliability meetings is that communities should be able to setup their own fleet. Currently we only use the status fleet but there's no way to add specific waku nodes. Communities should probably have a custom message type that describes the fleet for a community Communities should be able to define the list of store nodes #4357

• Message history retrieval works either by using a hardcoded list of store nodes from the status fleet, or using store nodes that were added manually via the advanced settings. Communities should be able to define the list of store nodes, either via a DNS discovery URL or specifying the list of nodes Communities should be able to define the list of store nodes #4357

• Use sharding fleet. Once this fleet is available, it must be added in desktop and mobile clients. Set default fleet to shards.test #4434 chore: migrate to shards.test #4435 chore: use shards.test fleet as default status-desktop#12963

• Use postgresql for store persistence #4399

• status.prod fleet is always used regardless of the fleet being chosen status-desktop#12443

• fix: take into account the ENR seq to determine if a peer is new or not waku-org/go-waku#945

• Implement a mechanism to retrieve shard information for a given community ID #4230

• bug: filter delay errors #4459

• bug: no messages returned from store node when multiple content topics and start/end time are used waku-org/nwaku#2288

cc: @fryorcraken @iurimatias @cammellos @chaitanyaprem

[fryorcraken]

Some core contributors have run into trouble finding the bootstrap nodes, due to DNS missconfigurations. Perhaps it makes sense to have a hardcoded list of nodes in status-go with the idea of using these when DNS Discovery fails. Otherwise the node will fail to find peers.

What process does Ethereum has to ensure this does not happen? I think we should replicate here.

[richard-ramos]

What process does Ethereum has to ensure this does not happen? I think we should replicate here.

Based on what I could see in:

• https://github.com/ethereum/go-ethereum/blob/b85c183ea7417e973dbbccd27b3fb7d7097b87dd/cmd/utils/flags.go#L1350-L1351
• https://github.com/ethereum/go-ethereum/blob/master/cmd/utils/flags.go#L1809-L1832

It will first set for DiscoveryV4 depending on the network one of these enode lists: https://github.com/ethereum/go-ethereum/blob/b85c183ea7417e973dbbccd27b3fb7d7097b87dd/params/bootnodes.go#L23-L65

For DiscoveryV5, these bootnodes ENRs are used: https://github.com/ethereum/go-ethereum/blob/b85c183ea7417e973dbbccd27b3fb7d7097b87dd/params/bootnodes.go#L67-L83

If DNS Discovery is enabled then a dns discovery url like enrtree://AKA3AM6LPBYEUDMVNU3BSVQJ5AD45Y7YPOHJLEF6W26QOE4VTUDPE@.all.mainnet.ethdisco.net is used: https://github.com/ethereum/go-ethereum/blob/master/params/bootnodes.go#L85-L104

Geth by default will use discoveryV4 and dnsDiscovery, meaning that it will support two discovery methods with only dns discovery depending on nameservers, while the discv4/5 bootstrap nodes use IPs and are not affected by nameserver configs; and the user is able to override these values.

Sounds like a good approach to follow both in status-go and waku service nodes.

[richard-ramos]

Another item to track: @chair28980

• status.prod fleet is always used regardless of the fleet being chosen status-desktop#12443

[richard-ramos]

In status-im/status-desktop@08c4bcb I fix the first item. The solution was to also add in the node config the individual nodes ENRs in addition to the DNS discovery URL for the fleet. That way if the DNS Discovery fails for some reason, at least the status-go will have some ENRs to use as bootstrap nodes.

@vitvly @cammellos . It might be a good idea to do this in mobile as well.

[richard-ramos]

Update on this issue: waku-org/nwaku#2162
Based on today's discussion with @jm-clius, I'll do a change in go-waku to not filter out bootnodes if their ENR do not contain the shards the client is interested into

[chair28980]

All further integration tracking captured via project: Status Waku Integration & Chat Reliability (view)