Skip to content

Commit 842c0e2

Browse files
authored
feat: add it-cbor-stream module (#148)
Adds module for reading/writing CBOR messages in an imperative style.
1 parent b01501e commit 842c0e2

File tree

9 files changed

+366
-0
lines changed

9 files changed

+366
-0
lines changed

packages/it-cbor-stream/LICENSE

+4
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
This project is dual licensed under MIT and Apache-2.0.
2+
3+
MIT: https://www.opensource.org/licenses/mit
4+
Apache-2.0: https://www.apache.org/licenses/license-2.0
+5
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at
2+
3+
http://www.apache.org/licenses/LICENSE-2.0
4+
5+
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

packages/it-cbor-stream/LICENSE-MIT

+19
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,19 @@
1+
The MIT License (MIT)
2+
3+
Permission is hereby granted, free of charge, to any person obtaining a copy
4+
of this software and associated documentation files (the "Software"), to deal
5+
in the Software without restriction, including without limitation the rights
6+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
7+
copies of the Software, and to permit persons to whom the Software is
8+
furnished to do so, subject to the following conditions:
9+
10+
The above copyright notice and this permission notice shall be included in
11+
all copies or substantial portions of the Software.
12+
13+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
14+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
15+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
16+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
17+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
18+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
19+
THE SOFTWARE.

packages/it-cbor-stream/README.md

+71
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,71 @@
1+
# it-cbor-stream
2+
3+
[![codecov](https://img.shields.io/codecov/c/github/achingbrain/it.svg?style=flat-square)](https://codecov.io/gh/achingbrain/it)
4+
[![CI](https://img.shields.io/github/actions/workflow/status/achingbrain/it/js-test-and-release.yml?branch=main\&style=flat-square)](https://github.com/achingbrain/it/actions/workflows/js-test-and-release.yml?query=branch%3Amain)
5+
6+
> Read and write CBOR messages over a duplex stream
7+
8+
# About
9+
10+
<!--
11+
12+
!IMPORTANT!
13+
14+
Everything in this README between "# About" and "# Install" is automatically
15+
generated and will be overwritten the next time the doc generator is run.
16+
17+
To make changes to this section, please update the @packageDocumentation section
18+
of src/index.js or src/index.ts
19+
20+
To experiment with formatting, please run "npm run docs" from the root of this
21+
repo and examine the changes made.
22+
23+
-->
24+
25+
This module makes it easy to send and receive length-prefixed CBOR encoded
26+
messages over streams.
27+
28+
## Example
29+
30+
```typescript
31+
import { cborStream } from 'it-cbor-stream'
32+
33+
const stream = cborStream(duplex)
34+
35+
// write a message to the stream
36+
stream.write({
37+
foo: 'bar'
38+
})
39+
40+
// read a message from the stream
41+
const res = await stream.read()
42+
```
43+
44+
# Install
45+
46+
```console
47+
$ npm i it-cbor-stream
48+
```
49+
50+
## Browser `<script>` tag
51+
52+
Loading this module through a script tag will make its exports available as `ItCborStream` in the global namespace.
53+
54+
```html
55+
<script src="https://unpkg.com/it-cbor-stream/dist/index.min.js"></script>
56+
```
57+
58+
# API Docs
59+
60+
- <https://achingbrain.github.io/it/modules/it_cbor_stream.html>
61+
62+
# License
63+
64+
Licensed under either of
65+
66+
- Apache 2.0, ([LICENSE-APACHE](https://github.com/achingbrain/it/blob/main/packages/it-cbor-stream/LICENSE-APACHE) / <http://www.apache.org/licenses/LICENSE-2.0>)
67+
- MIT ([LICENSE-MIT](https://github.com/achingbrain/it/blob/main/packages/it-cbor-stream/LICENSE-MIT) / <http://opensource.org/licenses/MIT>)
68+
69+
# Contribution
70+
71+
Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in the work by you, as defined in the Apache-2.0 license, shall be dual licensed as above, without any additional terms or conditions.

packages/it-cbor-stream/package.json

+68
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,68 @@
1+
{
2+
"name": "it-cbor-stream",
3+
"version": "0.0.0",
4+
"description": "Read and write CBOR messages over a duplex stream",
5+
"author": "Alex Potsides <[email protected]>",
6+
"license": "Apache-2.0 OR MIT",
7+
"homepage": "https://github.com/achingbrain/it/tree/main/packages/it-cbor-stream#readme",
8+
"repository": {
9+
"type": "git",
10+
"url": "git+https://github.com/achingbrain/it.git"
11+
},
12+
"bugs": {
13+
"url": "https://github.com/achingbrain/it/issues"
14+
},
15+
"publishConfig": {
16+
"access": "public",
17+
"provenance": true
18+
},
19+
"type": "module",
20+
"types": "./dist/src/index.d.ts",
21+
"files": [
22+
"src",
23+
"dist",
24+
"!dist/test",
25+
"!**/*.tsbuildinfo"
26+
],
27+
"exports": {
28+
".": {
29+
"types": "./dist/src/index.d.ts",
30+
"import": "./dist/src/index.js"
31+
}
32+
},
33+
"eslintConfig": {
34+
"extends": "ipfs",
35+
"parserOptions": {
36+
"project": true,
37+
"sourceType": "module"
38+
}
39+
},
40+
"scripts": {
41+
"clean": "aegir clean",
42+
"lint": "aegir lint",
43+
"dep-check": "aegir dep-check",
44+
"build": "aegir build",
45+
"test": "aegir test",
46+
"test:chrome": "aegir test -t browser --cov",
47+
"test:chrome-webworker": "aegir test -t webworker",
48+
"test:firefox": "aegir test -t browser -- --browser firefox",
49+
"test:firefox-webworker": "aegir test -t webworker -- --browser firefox",
50+
"test:node": "aegir test -t node --cov",
51+
"test:electron-main": "aegir test -t electron-main",
52+
"docs": "aegir docs",
53+
"release": "aegir release"
54+
},
55+
"dependencies": {
56+
"cborg": "^4.2.7",
57+
"it-length-prefixed-stream": "^1.0.0",
58+
"it-stream-types": "^2.0.1"
59+
},
60+
"devDependencies": {
61+
"aegir": "^45.0.8",
62+
"it-map": "^3.0.0",
63+
"it-pair": "^2.0.6",
64+
"it-to-buffer": "^4.0.0",
65+
"uint8-varint": "^2.0.4",
66+
"uint8arrays": "^5.1.0"
67+
}
68+
}

packages/it-cbor-stream/src/index.ts

+84
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,84 @@
1+
/**
2+
* @packageDocumentation
3+
*
4+
* This module makes it easy to send and receive length-prefixed CBOR encoded
5+
* messages over streams.
6+
*
7+
* @example
8+
*
9+
* ```typescript
10+
* import { cborStream } from 'it-cbor-stream'
11+
* import { MessageType } from './src/my-message-type.js'
12+
*
13+
* const stream = cborStream(duplex)
14+
*
15+
* // write a message to the stream
16+
* stream.write({
17+
* foo: 'bar'
18+
* })
19+
*
20+
* // read a message from the stream
21+
* const res = await stream.read()
22+
* ```
23+
*/
24+
25+
import { encode, decode } from 'cborg'
26+
import { lpStream } from 'it-length-prefixed-stream'
27+
import type { EncodeOptions, DecodeOptions } from 'cborg/interface'
28+
import type { LengthPrefixedStreamOpts } from 'it-length-prefixed-stream'
29+
import type { Duplex } from 'it-stream-types'
30+
31+
export interface AbortOptions {
32+
signal?: AbortSignal
33+
}
34+
35+
/**
36+
* Convenience methods for working with CBOR streams
37+
*/
38+
export interface CBORStream <Stream = unknown> {
39+
/**
40+
* Read the next length-prefixed byte array from the stream and decode it as CBOR
41+
*/
42+
read<T>(options?: AbortOptions): Promise<T>
43+
44+
/**
45+
* Encode the passed object as a CBOR message and write it's length-prefixed bytes to the stream
46+
*/
47+
write<T>(data: T, options?: AbortOptions): Promise<void>
48+
49+
/**
50+
* Encode the passed objects as CBOR messages and write their length-prefixed bytes to the stream as a single write
51+
*/
52+
writeV<T>(input: T[], options?: AbortOptions): Promise<void>
53+
54+
/**
55+
* Returns the underlying stream
56+
*/
57+
unwrap(): Stream
58+
}
59+
60+
export function cborStream <Stream extends Duplex<any, any, any>> (duplex: Stream, opts?: LengthPrefixedStreamOpts): CBORStream<Stream> {
61+
const lp = lpStream(duplex, opts)
62+
63+
const W: CBORStream<Stream> = {
64+
read: async (options?: AbortOptions & DecodeOptions) => {
65+
// readLP, decode
66+
const value = await lp.read(options)
67+
68+
return decode(value.subarray(), options)
69+
},
70+
write: async (message, options?: AbortOptions & EncodeOptions) => {
71+
// encode, writeLP
72+
await lp.write(encode(message, options), options)
73+
},
74+
writeV: async (messages, options?: AbortOptions & EncodeOptions) => {
75+
// encode, writeLP
76+
await lp.writeV(messages.map(message => encode(message, options)), options)
77+
},
78+
unwrap: () => {
79+
return lp.unwrap()
80+
}
81+
}
82+
83+
return W
84+
}
+87
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,87 @@
1+
import { expect } from 'aegir/chai'
2+
import { encode } from 'cborg'
3+
import map from 'it-map'
4+
import { pair } from 'it-pair'
5+
import toBuffer from 'it-to-buffer'
6+
import * as varint from 'uint8-varint'
7+
import { concat as uint8ArrayConcat } from 'uint8arrays/concat'
8+
import { cborStream } from '../src/index.js'
9+
import type { CBORStream } from '../src/index.js'
10+
11+
/* eslint-env mocha */
12+
/* eslint-disable max-nested-callbacks */
13+
14+
describe('it-cbor-stream', () => {
15+
let w: CBORStream<any>
16+
17+
beforeEach(async () => {
18+
w = cborStream(pair<Uint8Array>())
19+
})
20+
21+
it('unwraps underlying stream', () => {
22+
const stream = pair<Uint8Array>()
23+
const w = cborStream(stream)
24+
25+
expect(w.unwrap()).to.equal(stream)
26+
})
27+
28+
it('encode/decode', async () => {
29+
const input = {
30+
foo: 'bar'
31+
}
32+
33+
void w.write(input)
34+
35+
const output = await w.read()
36+
37+
expect(output).to.deep.equal(input)
38+
})
39+
40+
it('reads remaining data from unwrapped stream in one buffer', async () => {
41+
const message = {
42+
foo: 'bar'
43+
}
44+
const messageBuf = encode(message)
45+
const extraData = Uint8Array.from([0, 1, 2, 3, 4, 5])
46+
47+
const w = cborStream({
48+
source: (async function * () {
49+
yield uint8ArrayConcat([
50+
varint.encode(messageBuf.byteLength),
51+
messageBuf,
52+
extraData
53+
])
54+
}()),
55+
sink: async () => {}
56+
})
57+
58+
const read = await w.read()
59+
expect(read).to.deep.equal(message)
60+
61+
const rest = await toBuffer(map(w.unwrap().source, (buf) => buf.subarray()))
62+
expect(rest).to.equalBytes(extraData)
63+
})
64+
65+
it('reads remaining data from unwrapped stream in multiple buffers', async () => {
66+
const message = {
67+
foo: 'bar'
68+
}
69+
const messageBuf = encode(message)
70+
const extraData = Uint8Array.from([0, 1, 2, 3, 4, 5])
71+
72+
const w = cborStream({
73+
source: (async function * () {
74+
yield varint.encode(messageBuf.byteLength)
75+
yield messageBuf
76+
yield extraData
77+
}()),
78+
sink: async () => {}
79+
})
80+
81+
const read = await w.read()
82+
expect(read).to.deep.equal(message)
83+
84+
const rest = await toBuffer(map(w.unwrap().source, (buf) => buf.subarray()))
85+
expect(rest).to.equalBytes(extraData)
86+
})
87+
})

packages/it-cbor-stream/tsconfig.json

+23
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,23 @@
1+
{
2+
"extends": "aegir/src/config/tsconfig.aegir.json",
3+
"compilerOptions": {
4+
"outDir": "dist",
5+
"emitDeclarationOnly": false,
6+
"module": "ES2020"
7+
},
8+
"include": [
9+
"src",
10+
"test"
11+
],
12+
"references": [
13+
{
14+
"path": "../it-length-prefixed-stream"
15+
},
16+
{
17+
"path": "../it-map"
18+
},
19+
{
20+
"path": "../it-to-buffer"
21+
}
22+
]
23+
}

packages/it-cbor-stream/typedoc.json

+5
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
{
2+
"entryPoints": [
3+
"./src/index.ts"
4+
]
5+
}

0 commit comments

Comments
 (0)