Skip to content

Rewrite with compression support #163

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 56 commits into from
Feb 16, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
56 commits
Select commit Hold shift + click to select a range
8604dee
Increase TestWASM timeout
nhooyr Oct 15, 2019
e55ac18
Document compression API
nhooyr Oct 14, 2019
e142e08
Improve compression docs
nhooyr Nov 12, 2019
53c1aea
Implement compression extension negotiation
nhooyr Nov 12, 2019
2cf6c28
Implement compression writer and reader pooling
nhooyr Nov 12, 2019
a01afea
Support x-webkit-deflate-frame extension for Safari
nhooyr Nov 12, 2019
531d4fa
Improve general compression API and write docs
nhooyr Nov 12, 2019
d0a8049
Rewrite core
nhooyr Nov 19, 2019
dd107dd
Update CI
nhooyr Nov 28, 2019
6c6b8e9
Cleanup wspb and wsjson
nhooyr Nov 28, 2019
6b782a3
Run make fmt
nhooyr Nov 28, 2019
989ba2f
Change websocket to WebSocket in docs/errors
nhooyr Nov 28, 2019
9f15963
Simplify dial.go
nhooyr Nov 28, 2019
120911b
Remove use of math/rand.Init
nhooyr Nov 28, 2019
7ad1514
Update README.md comparison
nhooyr Nov 29, 2019
746140b
Further improve README
nhooyr Nov 29, 2019
43cb01e
Refactor read.go/write.go
nhooyr Nov 29, 2019
e8dfe27
Make CI pass
nhooyr Nov 29, 2019
f6137f3
Add minor improvements
nhooyr Dec 6, 2019
6f6fa43
Refactor autobahn
nhooyr Dec 31, 2019
8c87970
Add slidingWindowReader
nhooyr Jan 4, 2020
aaf4b45
Up test coverage of accept.go to 100%
nhooyr Jan 26, 2020
6b76536
Up dial coverage to 100%
nhooyr Jan 30, 2020
0f115ed
Add Go 1.12 support
nhooyr Jan 31, 2020
b6b56b7
Both modes seem to work :)
nhooyr Feb 5, 2020
9e32354
Fix randString method in tests
nhooyr Feb 6, 2020
78da35e
Get test with multiple messages working
nhooyr Feb 7, 2020
d092686
Autobahn tests fully pass :)
nhooyr Feb 8, 2020
6975801
Fix race in tests
nhooyr Feb 9, 2020
bbaf469
Fix test step
nhooyr Feb 9, 2020
faadcc9
Simplify tests
nhooyr Feb 9, 2020
3f2589f
Remove quite a bit of slog
nhooyr Feb 9, 2020
b53f306
Get Wasm tests working
nhooyr Feb 9, 2020
69ff675
More tests and fixes
nhooyr Feb 9, 2020
085e671
Get coverage to 85%
nhooyr Feb 9, 2020
51769b3
Add wspb test
nhooyr Feb 9, 2020
670be05
Merge in handshake improvements from master
nhooyr Feb 9, 2020
988b8f2
Merge remote-tracking branch 'origin/master' into compress
nhooyr Feb 9, 2020
3a526d8
Fix bug in closeHandshake
nhooyr Feb 9, 2020
999b812
Fix race in msgReader
nhooyr Feb 9, 2020
4b84d25
Fix a race with c.closed
nhooyr Feb 9, 2020
85f249d
Up timeouts
nhooyr Feb 9, 2020
6b38ebb
Test fixes
nhooyr Feb 9, 2020
6770421
Fix goroutine leak from deadlock when closing
nhooyr Feb 12, 2020
c752365
Make flateThreshold work
nhooyr Feb 12, 2020
0ea9466
Cleanup writeMu and flateThreshold
nhooyr Feb 12, 2020
b33d48c
Minor cleanup
nhooyr Feb 12, 2020
9c5bfab
Simplifications of conn_test.go
nhooyr Feb 13, 2020
3673c2c
Use basic test assertions
nhooyr Feb 13, 2020
c5b0a00
Fix badPing test duration
nhooyr Feb 13, 2020
1c7c14e
Pool sliding windows
nhooyr Feb 13, 2020
503b469
Simplify sliding window API
nhooyr Feb 13, 2020
dff4af3
Add conn benchmark
nhooyr Feb 13, 2020
2377cca
Switch to klauspost/compress
nhooyr Feb 15, 2020
d57b253
Report how efficient compression is in BenchmarkConn
nhooyr Feb 16, 2020
1bc100d
Update docs and random little issues
nhooyr Feb 16, 2020
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .github/CODEOWNERS
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
* @nhooyr
45 changes: 0 additions & 45 deletions .github/CONTRIBUTING.md

This file was deleted.

4 changes: 0 additions & 4 deletions .github/ISSUE_TEMPLATE.md

This file was deleted.

4 changes: 0 additions & 4 deletions .github/PULL_REQUEST_TEMPLATE.md

This file was deleted.

40 changes: 33 additions & 7 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,24 +4,50 @@ on: [push, pull_request]
jobs:
fmt:
runs-on: ubuntu-latest
container: nhooyr/websocket-ci@sha256:8a8fd73fdea33585d50a33619c4936adfd016246a2ed6bbfbf06def24b518a6a
steps:
- uses: actions/checkout@v1
- run: make fmt
- uses: actions/cache@v1
with:
path: ~/go/pkg/mod
key: ${{ runner.os }}-go-${{ hashFiles('**/go.sum') }}
restore-keys: |
${{ runner.os }}-go-
- name: Run make fmt
uses: ./ci/image
with:
args: make fmt

lint:
runs-on: ubuntu-latest
container: nhooyr/websocket-ci@sha256:8a8fd73fdea33585d50a33619c4936adfd016246a2ed6bbfbf06def24b518a6a
steps:
- uses: actions/checkout@v1
- run: make lint
- uses: actions/cache@v1
with:
path: ~/go/pkg/mod
key: ${{ runner.os }}-go-${{ hashFiles('**/go.sum') }}
restore-keys: |
${{ runner.os }}-go-
- name: Run make lint
uses: ./ci/image
with:
args: make lint

test:
runs-on: ubuntu-latest
container: nhooyr/websocket-ci@sha256:8a8fd73fdea33585d50a33619c4936adfd016246a2ed6bbfbf06def24b518a6a
steps:
- uses: actions/checkout@v1
- run: make test
- uses: actions/cache@v1
with:
path: ~/go/pkg/mod
key: ${{ runner.os }}-go-${{ hashFiles('**/go.sum') }}
restore-keys: |
${{ runner.os }}-go-
- name: Run make test
uses: ./ci/image
with:
args: make test
env:
COVERALLS_TOKEN: ${{ secrets.github_token }}
COVERALLS_TOKEN: ${{ secrets.COVERALLS_TOKEN }}
- name: Upload coverage.html
uses: actions/upload-artifact@master
with:
Expand Down
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
websocket.test
4 changes: 0 additions & 4 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,3 @@ SHELL = bash
include ci/fmt.mk
include ci/lint.mk
include ci/test.mk

ci-image:
docker build -f ./ci/Dockerfile -t nhooyr/websocket-ci .
docker push nhooyr/websocket-ci
161 changes: 56 additions & 105 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
# websocket

[![GitHub Release](https://img.shields.io/github/v/release/nhooyr/websocket?color=6b9ded&sort=semver)](https://github.com/nhooyr/websocket/releases)
[![GoDoc](https://godoc.org/nhooyr.io/websocket?status.svg)](https://godoc.org/nhooyr.io/websocket)
[![Coveralls](https://img.shields.io/coveralls/github/nhooyr/websocket?color=65d6a4)](https://coveralls.io/github/nhooyr/websocket)
[![CI Status](https://github.com/nhooyr/websocket/workflows/ci/badge.svg)](https://github.com/nhooyr/websocket/actions)
[![release](https://img.shields.io/github/v/release/nhooyr/websocket?color=6b9ded&sort=semver)](https://github.com/nhooyr/websocket/releases)
[![godoc](https://godoc.org/nhooyr.io/websocket?status.svg)](https://godoc.org/nhooyr.io/websocket)
[![coverage](https://img.shields.io/coveralls/github/nhooyr/websocket?color=65d6a4)](https://coveralls.io/github/nhooyr/websocket)
[![ci](https://github.com/nhooyr/websocket/workflows/ci/badge.svg)](https://github.com/nhooyr/websocket/actions)

websocket is a minimal and idiomatic WebSocket library for Go.

Expand All @@ -16,28 +16,25 @@ go get nhooyr.io/websocket
## Features

- Minimal and idiomatic API
- Tiny codebase at 2200 lines
- First class [context.Context](https://blog.golang.org/context) support
- Thorough tests, fully passes the [autobahn-testsuite](https://github.com/crossbario/autobahn-testsuite)
- [Zero dependencies](https://godoc.org/nhooyr.io/websocket?imports)
- JSON and ProtoBuf helpers in the [wsjson](https://godoc.org/nhooyr.io/websocket/wsjson) and [wspb](https://godoc.org/nhooyr.io/websocket/wspb) subpackages
- Highly optimized by default
- Concurrent writes out of the box
- [Complete Wasm](https://godoc.org/nhooyr.io/websocket#hdr-Wasm) support
- Thorough tests, fully passes the WebSocket [autobahn-testsuite](https://github.com/crossbario/autobahn-testsuite)
- [Minimal dependencies](https://godoc.org/nhooyr.io/websocket?imports)
- JSON and protobuf helpers in the [wsjson](https://godoc.org/nhooyr.io/websocket/wsjson) and [wspb](https://godoc.org/nhooyr.io/websocket/wspb) subpackages
- Zero alloc reads and writes
- Concurrent writes
- [Close handshake](https://godoc.org/nhooyr.io/websocket#Conn.Close)
- [net.Conn](https://godoc.org/nhooyr.io/websocket#NetConn) wrapper
- [Ping pong](https://godoc.org/nhooyr.io/websocket#Conn.Ping) API
- [RFC 7692](https://tools.ietf.org/html/rfc7692) permessage-deflate compression
- Compile to [Wasm](https://godoc.org/nhooyr.io/websocket#hdr-Wasm)

## Roadmap

- [ ] Compression Extensions [#163](https://github.com/nhooyr/websocket/pull/163)
- [ ] HTTP/2 [#4](https://github.com/nhooyr/websocket/issues/4)

## Examples

For a production quality example that shows off the full API, see the [echo example on the godoc](https://godoc.org/nhooyr.io/websocket#example-package--Echo). On github, the example is at [example_echo_test.go](./example_echo_test.go).

Use the [errors.As](https://golang.org/pkg/errors/#As) function [new in Go 1.13](https://golang.org/doc/go1.13#error_wrapping) to check for [websocket.CloseError](https://godoc.org/nhooyr.io/websocket#CloseError).
There is also [websocket.CloseStatus](https://godoc.org/nhooyr.io/websocket#CloseStatus) to quickly grab the close status code out of a [websocket.CloseError](https://godoc.org/nhooyr.io/websocket#CloseError).
See the [CloseStatus godoc example](https://godoc.org/nhooyr.io/websocket#example-CloseStatus).
For a production quality example that demonstrates the complete API, see the [echo example](https://godoc.org/nhooyr.io/websocket#example-package--Echo).

### Server

Expand Down Expand Up @@ -84,98 +81,52 @@ if err != nil {
c.Close(websocket.StatusNormalClosure, "")
```

## Design justifications

- A minimal API is easier to maintain due to less docs, tests and bugs
- A minimal API is also easier to use and learn
- Context based cancellation is more ergonomic and robust than setting deadlines
- net.Conn is never exposed as WebSocket over HTTP/2 will not have a net.Conn.
- Using net/http's Client for dialing means we do not have to reinvent dialing hooks
and configurations like other WebSocket libraries

## Comparison

Before the comparison, I want to point out that both gorilla/websocket and gobwas/ws were
extremely useful in implementing the WebSocket protocol correctly so _big thanks_ to the
authors of both. In particular, I made sure to go through the issue tracker of gorilla/websocket
to ensure I implemented details correctly and understood how people were using WebSockets in
production.

### gorilla/websocket

https://github.com/gorilla/websocket

The implementation of gorilla/websocket is 6 years old. As such, it is
widely used and very mature compared to nhooyr.io/websocket.

On the other hand, it has grown organically and now there are too many ways to do
the same thing. Compare the godoc of
[nhooyr/websocket](https://godoc.org/nhooyr.io/websocket) with
[gorilla/websocket](https://godoc.org/github.com/gorilla/websocket) side by side.

The API for nhooyr.io/websocket has been designed such that there is only one way to do things.
This makes it easy to use correctly. Not only is the API simpler, the implementation is
only 2200 lines whereas gorilla/websocket is at 3500 lines. That's more code to maintain,
more code to test, more code to document and more surface area for bugs.

Moreover, nhooyr.io/websocket supports newer Go idioms such as context.Context.
It also uses net/http's Client and ResponseWriter directly for WebSocket handshakes.
gorilla/websocket writes its handshakes to the underlying net.Conn.
Thus it has to reinvent hooks for TLS and proxies and prevents support of HTTP/2.

Some more advantages of nhooyr.io/websocket are that it supports concurrent writes and
makes it very easy to close the connection with a status code and reason. In fact,
nhooyr.io/websocket even implements the complete WebSocket close handshake for you whereas
with gorilla/websocket you have to perform it manually. See [gorilla/websocket#448](https://github.com/gorilla/websocket/issues/448).

The ping API is also nicer. gorilla/websocket requires registering a pong handler on the Conn
which results in awkward control flow. With nhooyr.io/websocket you use the Ping method on the Conn
that sends a ping and also waits for the pong.

Additionally, nhooyr.io/websocket can compile to [Wasm](https://godoc.org/nhooyr.io/websocket#hdr-Wasm) for the browser.

In terms of performance, the differences mostly depend on your application code. nhooyr.io/websocket
reuses message buffers out of the box if you use the wsjson and wspb subpackages.
As mentioned above, nhooyr.io/websocket also supports concurrent writers.

The WebSocket masking algorithm used by this package is also [1.75x](https://github.com/nhooyr/websocket/releases/tag/v1.7.4)
faster than gorilla/websocket or gobwas/ws while using only pure safe Go.
Advantages of [gorilla/websocket](https://github.com/gorilla/websocket):

The only performance con to nhooyr.io/websocket is that it uses one extra goroutine to support
cancellation with context.Context. This costs 2 KB of memory which is cheap compared to
the benefits.
- Mature and widely used
- [Prepared writes](https://godoc.org/github.com/gorilla/websocket#PreparedMessage)
- Configurable [buffer sizes](https://godoc.org/github.com/gorilla/websocket#hdr-Buffers)

### x/net/websocket
Advantages of nhooyr.io/websocket:

https://godoc.org/golang.org/x/net/websocket

Unmaintained and the API does not reflect WebSocket semantics. Should never be used.

See https://github.com/golang/go/issues/18152

### gobwas/ws

https://github.com/gobwas/ws

This library has an extremely flexible API but that comes at the cost of usability
and clarity.

This library is fantastic in terms of performance. The author put in significant
effort to ensure its speed and I have applied as many of its optimizations as
I could into nhooyr.io/websocket. Definitely check out his fantastic [blog post](https://medium.freecodecamp.org/million-websockets-and-go-cc58418460bb)
about performant WebSocket servers.

If you want a library that gives you absolute control over everything, this is the library.
But for 99.9% of use cases, nhooyr.io/websocket will fit better. It's nearly as performant
but much easier to use.

## Contributing

See [.github/CONTRIBUTING.md](.github/CONTRIBUTING.md).

## Users

If your company or project is using this library, feel free to open an issue or PR to amend this list.

- [Coder](https://github.com/cdr)
- [Tatsu Works](https://github.com/tatsuworks) - Ingresses 20 TB in websocket data every month on their Discord bot.
- Minimal and idiomatic API
- Compare godoc of [nhooyr.io/websocket](https://godoc.org/nhooyr.io/websocket) with [gorilla/websocket](https://godoc.org/github.com/gorilla/websocket) side by side.
- [net.Conn](https://godoc.org/nhooyr.io/websocket#NetConn) wrapper
- Zero alloc reads and writes ([gorilla/websocket#535](https://github.com/gorilla/websocket/issues/535))
- Full [context.Context](https://blog.golang.org/context) support
- Dial uses [net/http.Client](https://golang.org/pkg/net/http/#Client)
- Will enable easy HTTP/2 support in the future
- Gorilla writes directly to a net.Conn and so duplicates features of net/http.Client.
- Concurrent writes
- Close handshake ([gorilla/websocket#448](https://github.com/gorilla/websocket/issues/448))
- Idiomatic [ping pong](https://godoc.org/nhooyr.io/websocket#Conn.Ping) API
- Gorilla requires registering a pong callback before sending a Ping
- Can target Wasm ([gorilla/websocket#432](https://github.com/gorilla/websocket/issues/432))
- Transparent message buffer reuse with [wsjson](https://godoc.org/nhooyr.io/websocket/wsjson) and [wspb](https://godoc.org/nhooyr.io/websocket/wspb) subpackages
- [1.75x](https://github.com/nhooyr/websocket/releases/tag/v1.7.4) faster WebSocket masking implementation in pure Go
- Gorilla's implementation is slower and uses [unsafe](https://golang.org/pkg/unsafe/).
- Full [permessage-deflate](https://tools.ietf.org/html/rfc7692) compression extension support
- Gorilla only supports no context takeover mode
- Uses [klauspost/compress](https://github.com/klauspost/compress) for optimized compression
- See [gorilla/websocket#203](https://github.com/gorilla/websocket/issues/203)
- [CloseRead](https://godoc.org/nhooyr.io/websocket#Conn.CloseRead) helper ([gorilla/websocket#492](https://github.com/gorilla/websocket/issues/492))
- Actively maintained ([gorilla/websocket#370](https://github.com/gorilla/websocket/issues/370))

#### golang.org/x/net/websocket

[golang.org/x/net/websocket](https://godoc.org/golang.org/x/net/websocket) is deprecated.
See [golang/go/issues/18152](https://github.com/golang/go/issues/18152).

The [net.Conn](https://godoc.org/nhooyr.io/websocket#NetConn) wrapper will ease in transitioning
to nhooyr.io/websocket.

#### gobwas/ws

[gobwas/ws](https://github.com/gobwas/ws) has an extremely flexible API that allows it to be used
in an event driven style for performance. See the author's [blog post](https://medium.freecodecamp.org/million-websockets-and-go-cc58418460bb).

However when writing idiomatic Go, nhooyr.io/websocket will be faster and easier to use.
Loading