From 38f5404d33d4f329a788a4c8fbeb1c21ae5063c2 Mon Sep 17 00:00:00 2001 From: Littlefisher619 Date: Fri, 24 Feb 2023 02:32:15 +0800 Subject: [PATCH 01/14] doc: move examples in README --- README.md | 134 -------------------------------------------- README_zh.md | 93 +------------------------------ examples/README.md | 136 ++++++++++++++++++++++++++++++++++++++++++++- 3 files changed, 136 insertions(+), 227 deletions(-) diff --git a/README.md b/README.md index 0e08139..270d18b 100644 --- a/README.md +++ b/README.md @@ -51,140 +51,6 @@ See the [examples](examples) directory for examples of eBPF programs written in - [sockfilter](examples/sockfilter) `networking example` - [sockops](examples/sockops) `networking example` -### C example: [Bootstrap](examples/bootstrap) - -`bootstrap` is an example of a simple (but realistic) BPF application. It -tracks process starts (`exec()` family of syscalls, to be precise) and exits -and emits data about filename, PID and parent PID, as well as exit status and -duration of the process life. With `-d ` you can specify -minimum duration of the process to log. In such mode process start -(technically, `exec()`) events are not output (see example output below). - -`bootstrap` was created in the similar spirit as -[libbpf-tools](https://github.com/iovisor/bcc/tree/master/libbpf-tools) from -BCC package, but is designed to be more stand-alone and with simpler Makefile -to simplify adoption to user's particular needs. It demonstrates the use of -typical BPF features: - -- cooperating BPF programs (tracepoint handlers for process `exec` and `exit` - events, in this particular case); -- BPF map for maintaining the state; -- BPF ring buffer for sending data to user-space; -- global variables for application behavior parameterization. -- it utilizes BPF CO-RE and vmlinux.h to read extra process information from - kernel's `struct task_struct`. - -Here's an example output: - -```console -$ sudo ./wasm-bpf bootstrap.wasm -h -BPF bootstrap demo application. - -It traces process start and exits and shows associated -information (filename, process duration, PID and PPID, etc). - -USAGE: ./bootstrap [-d ] -v -$ sudo ./wasm-bpf bootstrap.wasm -TIME EVENT COMM PID PPID FILENAME/EXIT CODE -18:57:58 EXEC sed 74911 74910 /usr/bin/sed -18:57:58 EXIT sed 74911 74910 [0] (2ms) -18:57:58 EXIT cat 74912 74910 [0] (0ms) -18:57:58 EXEC cat 74913 74910 /usr/bin/cat -18:57:59 EXIT cat 74913 74910 [0] (0ms) -18:57:59 EXEC cat 74914 74910 /usr/bin/cat -18:57:59 EXIT cat 74914 74910 [0] (0ms) -18:57:59 EXEC cat 74915 74910 /usr/bin/cat -18:57:59 EXIT cat 74915 74910 [0] (1ms) -18:57:59 EXEC sleep 74916 74910 /usr/bin/sleep -``` - -See [examples/bootstrap](examples/bootstrap) for more details. - -### Rust example: [Bootstrap](examples/rust-bootstrap) - -similar to C bootstrap, but written in Rust. - -See [examples/rust-bootstrap](examples/rust-bootstrap) for more details. - -### C example: [runqlat](examples/runqlat) - -This program summarizes scheduler run queue latency as a histogram, showing -how long tasks spent waiting their turn to run on-CPU. - -This program summarizes scheduler run queue latency as a histogram, showing -how long tasks spent waiting their turn to run on-CPU. - -```console -$ sudo ./wasm-bpf runqlat.wasm -h -Summarize run queue (scheduler) latency as a histogram. - -USAGE: runqlat [--help] [interval] [count] - -EXAMPLES: - runqlat # summarize run queue latency as a histogram - runqlat 1 10 # print 1 second summaries, 10 times -$ sudo ./wasm-bpf runqlat.wasm 1 - -Tracing run queue latency... Hit Ctrl-C to end. - - usecs : count distribution - 0 -> 1 : 72 |***************************** | - 2 -> 3 : 93 |************************************* | - 4 -> 7 : 98 |****************************************| - 8 -> 15 : 96 |*************************************** | - 16 -> 31 : 38 |*************** | - 32 -> 63 : 4 |* | - 64 -> 127 : 5 |** | - 128 -> 255 : 6 |** | - 256 -> 511 : 0 | | - 512 -> 1023 : 0 | | - 1024 -> 2047 : 0 | | - 2048 -> 4095 : 1 | | -``` - -`runqlat` is also an example of a simple (but realistic) BPF application. It -would show a more complex example of BPF program, which contains more than -one file, and directly access the kernel maps from the user space instead of -polling the kernel ring buffer. - -The runtime would use shared memory to access the kernel maps, and the kernel -would update the maps in the shared memory, so the wasm code can access the -eBPF maps directly, without any serialization or copy overhead between userspace -host and Wasm runtime. - -You can use the `bpf_map_update_elem` API to update the kernel maps from the user -space, for example: - -```c - cg_map_fd = bpf_map__fd(obj->maps.cgroup_map); - .... - bpf_map_update_elem(cg_map_fd, &idx, &cgfd, BPF_ANY); -``` - -So the kernel eBPF can be config by wasm side or recieve the messages from -userspace wasm runtime when it is running. - -See [examples/runqlat](examples/runqlat) for more details. - -### C example: lsm-rmdir - -`lsm-rmdir` hook in dir remove and check the permission to remove a directory. If dir -name with `can_not_rm` will raise Operation not permitted. - -See [examples/lsm](examples/lsm) for more details. - -### C example: Socket filter - -sockfilter is an example of monitoring packet and dealing with __sk_buff structure. - -See [examples/sockfilter](examples/sockfilter) for more details. - -### C example: Sockops - -`sockops` add the pid int tcp option in syn packet. - -See [examples/sockops](examples/sockops) for more details. - ## build the runtime The dependencies are libbpf and wasm-micro-runtime only, they are diff --git a/README_zh.md b/README_zh.md index 50de7b5..adb7970 100644 --- a/README_zh.md +++ b/README_zh.md @@ -25,103 +25,12 @@ ## 示例 -请转到 [examples](examples) 文件夹去查看使用 C, Rust 编写并编译到 Wasm 的 eBPF-Wasm 程序的示例。 +请转到 [examples](examples) 文件夹去查看使用 C, Rust, Go 编写并编译到 Wasm 的 eBPF-Wasm 程序的示例。 - [bootstrap](examples/bootstrap) and [runqlat](examples/runqlat) `追踪` - [lsm](examples/lsm) `安全` - [sockfilter](examples/sockfilter) `网络` -### C 示例: Bootstrap - -`bootstrap` 是个简单但很现实的 eBPF 程序的示例。 这个示例可以跟踪进程的启动 (更精确地来说，是 `exec()` 那些系统调用) 和退出，然后输出进程的文件名、PID、父进程PID之类的数据，以及进程的退出状态和存活时间。使用 `-d <最小周期（毫秒>` 来限制要输出的进程的最小存活时间。 在这种模式下，进程启动事件不会被输出（科学一点，`exec()`，具体见下面的示例）。 - -`bootstrap` 是使用和来自 BCC 里的 -[libbpf-tools](https://github.com/iovisor/bcc/tree/master/libbpf-tools) 类似的思路来开发的。 但是为了让用户的修改容易一些， `bootstrap` 更独立，并且使用了更简单的 Makefile 。 `bootstrap` 演示了典型的 eBPF 用途: - -- 多个 BPF 程序协同工作 (在这里是进程 `exec（启动）` 和 `exit（退出）` 的事件处理函数)； -- 用 BPF map 来维护状态； -- 用 BPF 环形缓冲区来向用户态发送数据； -- 使用全局变量来修改程序行为。 -- `bootstrap` 使用了 BPF 的 CO-RE 特性以及 `vmlinux.h` 来从内核的 `struct task_struct` 来读取额外的进程信息。 - -来看一个样例输出: - -```console -$ sudo sudo ./wasm-bpf bootstrap.wasm -h -BPF bootstrap demo application. - -It traces process start and exits and shows associated -information (filename, process duration, PID and PPID, etc). - -USAGE: ./bootstrap [-d ] -v -$ sudo ./wasm-bpf bootstrap.wasm -TIME EVENT COMM PID PPID FILENAME/EXIT CODE -18:57:58 EXEC sed 74911 74910 /usr/bin/sed -18:57:58 EXIT sed 74911 74910 [0] (2ms) -18:57:58 EXIT cat 74912 74910 [0] (0ms) -18:57:58 EXEC cat 74913 74910 /usr/bin/cat -18:57:59 EXIT cat 74913 74910 [0] (0ms) -18:57:59 EXEC cat 74914 74910 /usr/bin/cat -18:57:59 EXIT cat 74914 74910 [0] (0ms) -18:57:59 EXEC cat 74915 74910 /usr/bin/cat -18:57:59 EXIT cat 74915 74910 [0] (1ms) -18:57:59 EXEC sleep 74916 74910 /usr/bin/sleep -``` - -原始的 C 代码来自 [libbpf-bootstrap](https://github.com/libbpf/libbpf-bootstrap)。 - -### Rust 示例: [Bootstrap](examples/rust-bootstrap) - -类似 C bootstrap, 但是是 Rust 写的。 - -See [examples/rust-bootstrap](examples/rust-bootstrap) for more details. - - -### C 示例: runqlat - -这个程序通过直方图展示调度器运行队列延迟，给我们展现了任务等了多久才能轮到 CPU 用。 - -```console -$ sudo ./wasm-bpf runqlat.wasm -h -Summarize run queue (scheduler) latency as a histogram. - -USAGE: runqlat [--help] [interval] [count] - -EXAMPLES: - runqlat 1 10 # print 1 second summaries, 10 times -$ sudo ./wasm-bpf runqlat.wasm 1 - -Tracing run queue latency... Hit Ctrl-C to end. - - usecs : count distribution - 0 -> 1 : 72 |***************************** | - 2 -> 3 : 93 |************************************* | - 4 -> 7 : 98 |****************************************| - 8 -> 15 : 96 |*************************************** | - 16 -> 31 : 38 |*************** | - 32 -> 63 : 4 |* | - 64 -> 127 : 5 |** | - 128 -> 255 : 6 |** | - 256 -> 511 : 0 | | - 512 -> 1023 : 0 | | - 1024 -> 2047 : 0 | | - 2048 -> 4095 : 1 | | -``` - -`runqlat` 也是个简单但有实际意义的 BPF 程序的例子。这个例子稍微复杂一些，它有超过一个文件，并且能直接读内核 map 而不是从内核的环形缓冲区获取数据。 - -运行时将会使用共享内存来访问内核 map，同时内核将会更新在共享内存中的 map ，所以 wasm 代码可以直接访问 eBPF map，而不需要面对用户态主机侧程序和 Wasm 运行时之间的额外拷贝开销。 - -可以使用 `bpf_map_update_elem` 在用户态程序内更新内核的 eBPF map，比如: - -```c - cg_map_fd = bpf_map__fd(obj->maps.cgroup_map); - .... - bpf_map_update_elem(cg_map_fd, &idx, &cgfd, BPF_ANY); -``` - -所以内核的 eBPF 程序可以从 Wasm 侧的程序获取配置，或者在运行的时候接收消息。 - ## 构建运行时 依赖只有 git submodule 里面的 libbpf 和 wasm-micro-runtime diff --git a/examples/README.md b/examples/README.md index 5121755..3a7b1d3 100644 --- a/examples/README.md +++ b/examples/README.md @@ -7,4 +7,138 @@ Each directory contains one example. - `rust-**` denotes that the userspace program is written in Rust. - Others means that the userspace program is written in `C/C++` -Go examples requires `tinygo`, rust examples requires `cargo`. \ No newline at end of file +Go examples requires `tinygo`, rust examples requires `cargo`. + +### C example: [Bootstrap](examples/bootstrap) + +`bootstrap` is an example of a simple (but realistic) BPF application. It +tracks process starts (`exec()` family of syscalls, to be precise) and exits +and emits data about filename, PID and parent PID, as well as exit status and +duration of the process life. With `-d ` you can specify +minimum duration of the process to log. In such mode process start +(technically, `exec()`) events are not output (see example output below). + +`bootstrap` was created in the similar spirit as +[libbpf-tools](https://github.com/iovisor/bcc/tree/master/libbpf-tools) from +BCC package, but is designed to be more stand-alone and with simpler Makefile +to simplify adoption to user's particular needs. It demonstrates the use of +typical BPF features: + +- cooperating BPF programs (tracepoint handlers for process `exec` and `exit` + events, in this particular case); +- BPF map for maintaining the state; +- BPF ring buffer for sending data to user-space; +- global variables for application behavior parameterization. +- it utilizes BPF CO-RE and vmlinux.h to read extra process information from + kernel's `struct task_struct`. + +Here's an example output: + +```console +$ sudo ./wasm-bpf bootstrap.wasm -h +BPF bootstrap demo application. + +It traces process start and exits and shows associated +information (filename, process duration, PID and PPID, etc). + +USAGE: ./bootstrap [-d ] -v +$ sudo ./wasm-bpf bootstrap.wasm +TIME EVENT COMM PID PPID FILENAME/EXIT CODE +18:57:58 EXEC sed 74911 74910 /usr/bin/sed +18:57:58 EXIT sed 74911 74910 [0] (2ms) +18:57:58 EXIT cat 74912 74910 [0] (0ms) +18:57:58 EXEC cat 74913 74910 /usr/bin/cat +18:57:59 EXIT cat 74913 74910 [0] (0ms) +18:57:59 EXEC cat 74914 74910 /usr/bin/cat +18:57:59 EXIT cat 74914 74910 [0] (0ms) +18:57:59 EXEC cat 74915 74910 /usr/bin/cat +18:57:59 EXIT cat 74915 74910 [0] (1ms) +18:57:59 EXEC sleep 74916 74910 /usr/bin/sleep +``` + +See [examples/bootstrap](examples/bootstrap) for more details. + +### Rust example: [Bootstrap](examples/rust-bootstrap) + +similar to C bootstrap, but written in Rust. + +See [examples/rust-bootstrap](examples/rust-bootstrap) for more details. + +### C example: [runqlat](examples/runqlat) + +This program summarizes scheduler run queue latency as a histogram, showing +how long tasks spent waiting their turn to run on-CPU. + +This program summarizes scheduler run queue latency as a histogram, showing +how long tasks spent waiting their turn to run on-CPU. + +```console +$ sudo ./wasm-bpf runqlat.wasm -h +Summarize run queue (scheduler) latency as a histogram. + +USAGE: runqlat [--help] [interval] [count] + +EXAMPLES: + runqlat # summarize run queue latency as a histogram + runqlat 1 10 # print 1 second summaries, 10 times +$ sudo ./wasm-bpf runqlat.wasm 1 + +Tracing run queue latency... Hit Ctrl-C to end. + + usecs : count distribution + 0 -> 1 : 72 |***************************** | + 2 -> 3 : 93 |************************************* | + 4 -> 7 : 98 |****************************************| + 8 -> 15 : 96 |*************************************** | + 16 -> 31 : 38 |*************** | + 32 -> 63 : 4 |* | + 64 -> 127 : 5 |** | + 128 -> 255 : 6 |** | + 256 -> 511 : 0 | | + 512 -> 1023 : 0 | | + 1024 -> 2047 : 0 | | + 2048 -> 4095 : 1 | | +``` + +`runqlat` is also an example of a simple (but realistic) BPF application. It +would show a more complex example of BPF program, which contains more than +one file, and directly access the kernel maps from the user space instead of +polling the kernel ring buffer. + +The runtime would use shared memory to access the kernel maps, and the kernel +would update the maps in the shared memory, so the wasm code can access the +eBPF maps directly, without any serialization or copy overhead between userspace +host and Wasm runtime. + +You can use the `bpf_map_update_elem` API to update the kernel maps from the user +space, for example: + +```c + cg_map_fd = bpf_map__fd(obj->maps.cgroup_map); + .... + bpf_map_update_elem(cg_map_fd, &idx, &cgfd, BPF_ANY); +``` + +So the kernel eBPF can be config by wasm side or recieve the messages from +userspace wasm runtime when it is running. + +See [examples/runqlat](examples/runqlat) for more details. + +### C example: lsm-rmdir + +`lsm-rmdir` hook in dir remove and check the permission to remove a directory. If dir +name with `can_not_rm` will raise Operation not permitted. + +See [examples/lsm](examples/lsm) for more details. + +### C example: Socket filter + +sockfilter is an example of monitoring packet and dealing with __sk_buff structure. + +See [examples/sockfilter](examples/sockfilter) for more details. + +### C example: Sockops + +`sockops` add the pid int tcp option in syn packet. + +See [examples/sockops](examples/sockops) for more details. From 08dfa68426288e804a09b1611542a1a95041529d Mon Sep 17 00:00:00 2001 From: Littlefisher619 Date: Fri, 24 Feb 2023 02:44:47 +0800 Subject: [PATCH 02/14] doc: add go example in README --- README.md | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/README.md b/README.md index 270d18b..1812beb 100644 --- a/README.md +++ b/README.md @@ -14,10 +14,10 @@ - **`General purpose`**: provide most abilities from eBPF to Wasm, `polling` from the ring buffer or perf buffer, bidirectional communications between `kernel` eBPF and `userspace` Wasm using `maps`, dynamically `loading`, `attaching` or `detaching`, etc. Supports a large number of eBPF program types and map types. - **`High performance`**: No `serialization` overhead for complex data types, using `shared memory` to avoid copy overhead between host and Wasm. -- **`Easy to use`**: provide a similar developing experience as the [libbpf-bootstrap](https://github.com/libbpf/libbpf-bootstrap), `auto generate` the Wasm-eBPF skeleton headers and type definitions for bindings. +- **`Easy to use`**: provide a similar developing experience as the [libbpf-bootstrap](https://github.com/libbpf/libbpf-bootstrap), `auto generate` the Wasm-eBPF skeleton headers and type definitions for bindings. Write your eBPF programs in `C/C++`, `Rust`, `Go` and compile to Wasm. - **`Ultralightweight`**: the sample runtime has only `1.5 MB` in binary size. Compiled Wasm module would be only `~90K`. With the same toolchain, you can easily build your own Wasm-eBPF runtime in any languages and platforms! -See the [examples](examples) directory for examples of eBPF programs written in C, Rust and compiled to Wasm, covering the use cases from `tracing`, `networking` to `security`. +See the [examples](examples) directory for examples of eBPF programs written in C, Rust, Go and compiled to Wasm, covering the use cases from `tracing`, `networking` to `security`. For tools to distribute Wasm-eBPF programs in [`OCI`](https://opencontainers.org/) images, please refer to [eunomia-bpf](https://github.com/eunomia-bpf/eunomia-bpf) repo. @@ -43,11 +43,12 @@ We have proposed a new WASI issue [wasi-bpf](https://github.com/WebAssembly/WASI ## 🔨 Examples -See the [examples](examples) directory for examples of eBPF programs written in C, Rust and compiled to WASM. +See the [examples](examples) directory for examples of eBPF programs written in C, Rust, Go and compiled to WASM. -- [bootstrap](examples/bootstrap) and [runqlat](examples/runqlat) `tracing examples` -- [rust-bootstrap](examples/rust-bootstrap) `tracing examples` -- [lsm](examples/lsm) `security example` +- [bootstrap](examples/bootstrap) and [rust-bootstrap](examples/rust-bootstrap) `tracing examples` +- [runqlat](examples/runqlat) `tracing examples` +- [go-execve](examples/go-execve) `tracing examples` +- [lsm](examples/lsm) and `security example` - [sockfilter](examples/sockfilter) `networking example` - [sockops](examples/sockops) `networking example` @@ -101,7 +102,6 @@ MIT ## 🔗 Links -- GitHub Repository: https://github.com/eunomia-bpf/wasm-bpf - eunomia-bpf project: simplify and enhance eBPF with CO-RE and WebAssembly https://github.com/eunomia-bpf/eunomia-bpf - documents and blogs: https://eunomia-bpf.github.io/blog/ebpf-wasm.html - CO-RE (Compile Once – Run Everywhere): https://facebookmicrosites.github.io/bpf/blog/2020/02/19/bpf-portability-and-co-re.html From 658f03889134d88d0c37d408e425febadb711189 Mon Sep 17 00:00:00 2001 From: Littlefisher619 Date: Fri, 24 Feb 2023 02:49:26 +0800 Subject: [PATCH 03/14] doc: update the build.md --- README.md | 47 +++++----------------------------- README_zh.md | 32 ++--------------------- docs/build.md | 70 +++++++++++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 78 insertions(+), 71 deletions(-) create mode 100644 docs/build.md diff --git a/README.md b/README.md index 1812beb..5929964 100644 --- a/README.md +++ b/README.md @@ -8,14 +8,14 @@ [中文文档](README_zh.md) -`Wasm-bpf` is a WebAssembly eBPF library, toolchain and runtime powered by [CO-RE](https://facebookmicrosites.github.io/bpf/blog/2020/02/19/bpf-portability-and-co-re.html)(Compile Once – Run Everywhere) [libbpf](https://github.com/libbpf/libbpf) and [WAMR](https://github.com/bytecodealliance/wasm-micro-runtime). It can help you build almost every eBPF programs or usecases to `Wasm`. +`Wasm-bpf` is a WebAssembly eBPF library, toolchain and runtime powered by [CO-RE](https://facebookmicrosites.github.io/bpf/blog/2020/02/19/bpf-portability-and-co-re.html)(Compile Once – Run Everywhere) [libbpf](https://github.com/libbpf/libbpf). It can help you build almost every eBPF programs or usecases to `Wasm` and execute them. ## Features - **`General purpose`**: provide most abilities from eBPF to Wasm, `polling` from the ring buffer or perf buffer, bidirectional communications between `kernel` eBPF and `userspace` Wasm using `maps`, dynamically `loading`, `attaching` or `detaching`, etc. Supports a large number of eBPF program types and map types. - **`High performance`**: No `serialization` overhead for complex data types, using `shared memory` to avoid copy overhead between host and Wasm. - **`Easy to use`**: provide a similar developing experience as the [libbpf-bootstrap](https://github.com/libbpf/libbpf-bootstrap), `auto generate` the Wasm-eBPF skeleton headers and type definitions for bindings. Write your eBPF programs in `C/C++`, `Rust`, `Go` and compile to Wasm. -- **`Ultralightweight`**: the sample runtime has only `1.5 MB` in binary size. Compiled Wasm module would be only `~90K`. With the same toolchain, you can easily build your own Wasm-eBPF runtime in any languages and platforms! +- **`Ultralightweight`**: the miminal runtime has only `1.5 MB` in binary size. Compiled Wasm module would be only `~90K`. With the same toolchain, you can easily build your own Wasm-eBPF runtime in any languages and platforms! See the [examples](examples) directory for examples of eBPF programs written in C, Rust, Go and compiled to Wasm, covering the use cases from `tracing`, `networking` to `security`. @@ -54,47 +54,12 @@ See the [examples](examples) directory for examples of eBPF programs written in ## build the runtime -The dependencies are libbpf and wasm-micro-runtime only, they are -registered as git submodules. +We have two types of runtime samples: -```sh -git submodule update --init --recursive -``` +- A C/C++ runtime example, which is a minimal runtime based on WAMR. see [runtime/cpp](../runtime/cpp) for more details. +- A Rust runtime example, which is a more complex runtime based on Wasmtime. see [runtime/rust](../runtime/rust) for more details. -### Install Dependencies - -You will need `clang`, `libelf` and `zlib` to build the examples, -package names may vary across distros. - -On Ubuntu/Debian, you need: - -```shell -apt install clang libelf1 libelf-dev zlib1g-dev -``` - -On CentOS/Fedora, you need: - -```shell -dnf install clang elfutils-libelf elfutils-libelf-devel zlib-devel -``` - -### Build runtime as a executable tool - -Run `make` to build the runtime, which will be placed in the `build` -directory. `cmake` is required to build the runtime. - -```sh -make build -``` - -### Build runtime as a library - -```sh -make build-lib -``` - -You may refer to [CI](.github/workflows/c-cpp.yml) for more details on how -to build and run the examples. +The runtime can be built as a library or a standalone executable. see [docs/build.md](docs/build.md) to build the runtimes. ## LICENSE diff --git a/README_zh.md b/README_zh.md index adb7970..465e87d 100644 --- a/README_zh.md +++ b/README_zh.md @@ -33,39 +33,11 @@ ## 构建运行时 -依赖只有 git submodule 里面的 libbpf 和 wasm-micro-runtime - -```sh -git submodule update --init --recursive -``` - -## 安装依赖 - -构建示例需要用到 `clang`, `libelf` 和 `zlib` 。包名在不同的发行版间可能不同。 - -在 Ubuntu/Debian, 需要: - -```shell -apt install clang libelf1 libelf-dev zlib1g-dev -``` - -在 CentOS/Fedora, 需要: - -```shell -dnf install clang elfutils-libelf elfutils-libelf-devel zlib-devel -``` - -运行 `make` 来构建这些示例。 构建结果会被放在 `build` 文件夹里。 构建运行时需要用到 `cmake`。 - -```sh -make build -``` - -可以查阅 [CI](.github/workflows/c-cpp.yml) 来详细了解如何编译运行这些示例。 +请参考 [docs/build.md](docs/build.md)。 ## Wasm-bpf 总览 -![wasi-bpf](test/asserts/wasm-bpf-no-bcc.png) +![wasi-bpf](docs/wasm-bpf-no-bcc.png) Wasm 模块可以同时加载和控制多个 eBPF 程序， 并且能够调用或者控制（通过[组件模型](https://github.com/WebAssembly/component-model)）其他语言编写的 Wasm 模块来处理数据。 diff --git a/docs/build.md b/docs/build.md new file mode 100644 index 0000000..846c0e2 --- /dev/null +++ b/docs/build.md @@ -0,0 +1,70 @@ +# build the runtime and examples + +We have two types of runtime examples: + +- A C/C++ runtime example, which is a minimal runtime based on WAMR. see [runtime/cpp](../runtime/cpp) for more details. +- A Rust runtime example, which is a more complex runtime based on Wasmtime. see [runtime/rust](../runtime/rust) for more details. + +A new runtime is easy to implement with only a few hundred lines of code, in any language, using any wasm runtime or any ebpf user space library. + +## build the Cpp minimal runtime based on WAMR + +The dependencies are libbpf and wasm-micro-runtime only, they are +registered as git submodules. + +```sh +git submodule update --init --recursive +cd runtime/cpp +``` + +### Install Dependencies + +You will need `clang`, `libelf` and `zlib` to build the examples, +package names may vary across distros. + +On Ubuntu/Debian, you need: + +```shell +sudo apt install clang libelf1 libelf-dev zlib1g-dev +``` + +On CentOS/Fedora, you need: + +```shell +sudo dnf install clang elfutils-libelf elfutils-libelf-devel zlib-devel +``` + +### Build runtime as a executable tool + +Run `make` in the `runtime/cpp` directory to build the runtime, which will be placed in the `build` +directory. `cmake` is required to build the runtime. + +```sh +make build +``` + +### Build runtime as a library + +```sh +make build-lib +``` + +You may refer to [CI](.github/workflows/c-cpp.yml) for more details on how +to build and run the examples. + +## build the Rust runtime based on Wasmtime + +install rust toolchain + +```shell +curl https://sh.rustup.rs -sSf | sh -s +``` + +### Build runtime as a executable tool + +Run `make` in the `runtime/rust` directory to build the runtime, which will be placed in the `target` +directory. + +```sh +make build +``` From 141c75a3158b51d1b9ed3199b6154938a214c048 Mon Sep 17 00:00:00 2001 From: Littlefisher619 Date: Fri, 24 Feb 2023 02:56:49 +0800 Subject: [PATCH 04/14] doc: fix how it works --- README.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/README.md b/README.md index 5929964..e2394b0 100644 --- a/README.md +++ b/README.md @@ -8,7 +8,7 @@ [中文文档](README_zh.md) -`Wasm-bpf` is a WebAssembly eBPF library, toolchain and runtime powered by [CO-RE](https://facebookmicrosites.github.io/bpf/blog/2020/02/19/bpf-portability-and-co-re.html)(Compile Once – Run Everywhere) [libbpf](https://github.com/libbpf/libbpf). It can help you build almost every eBPF programs or usecases to `Wasm` and execute them. +`Wasm-bpf` is a WebAssembly eBPF library, toolchain and runtime powered by [CO-RE](https://facebookmicrosites.github.io/bpf/blog/2020/02/19/bpf-portability-and-co-re.html)(Compile Once – Run Everywhere) [libbpf](https://github.com/libbpf/libbpf). It can help you build almost every eBPF programs or usecases to `Wasm` with nearly zero modification, and run them cross platforms with Wasm sandbox. ## Features @@ -26,12 +26,12 @@ For tools to distribute Wasm-eBPF programs in [`OCI`](https://opencontainers.org The wasm-bpf runtime require two parts: `the host side`(Outside the Wasm runtime) and the `Wasm guest side`(Inside the Wasm runtime). - host side: A simple runtime implementation example - - see [src](runtime/cpp/src) and [include](runtime/cpp/include) directories, which would be a sample runtime built on the top of [libbpf](https://github.com/libbpf/libbpf) and [WAMR](https://github.com/bytecodealliance/wasm-micro-runtime). + - see [runtime/cpp](runtime/cpp), which would be a sample runtime in `C++` built on the top of [libbpf](https://github.com/libbpf/libbpf) and [WAMR](https://github.com/bytecodealliance/wasm-micro-runtime). Another more complex runtime implement in `Rust` is [runtime/rust](runtime/rust), based on [Wasmtime](https://github.com/bytecodealliance/wasmtime). - You can easily build your own Wasm-eBPF runtime in `any` languages, `any` eBPF libraries and `any` Wasm runtimes with the same System interface. - wasm side: toolchains and libraries - a [`libbpf-wasm`](wasm-sdk/c/libbpf-wasm.h) header only library to provide libbpf APIs for Wasm guest `C/C++` code. - a [`bpftool`](https://github.com/eunomia-bpf/bpftool/tree/wasm-bpftool) tool to generate the Wasm-eBPF `skeleton` headers, and `C struct definitions` for passing data between the host and Wasm guest without serialization. - - More languages support(`Rust`, `Go`, etc) is on the way. + - `Rust`, `Go` and other language support is similar to the `C/C++` support. For details compile process, please refer to the [examples/bootstrap/README.md](examples/bootstrap/README.md). The figure below shows the overall interaction between the eBPF and Wasm runtimes: From 628d0c37d990f0e4e616beb29d4bb37f02353f68 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=E4=BA=91=E5=BE=AE?= <1067852565@qq.com> Date: Fri, 24 Feb 2023 13:24:09 +0800 Subject: [PATCH 05/14] Update README.md Co-authored-by: septs --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index e2394b0..bb216c6 100644 --- a/README.md +++ b/README.md @@ -52,7 +52,7 @@ See the [examples](examples) directory for examples of eBPF programs written in - [sockfilter](examples/sockfilter) `networking example` - [sockops](examples/sockops) `networking example` -## build the runtime +## Build the runtime We have two types of runtime samples: From 93ae005a9b38ea99353b9b5a3711532faeae7c95 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=E4=BA=91=E5=BE=AE?= <1067852565@qq.com> Date: Fri, 24 Feb 2023 13:24:22 +0800 Subject: [PATCH 06/14] Update README.md Co-authored-by: septs --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index bb216c6..a87a68d 100644 --- a/README.md +++ b/README.md @@ -63,7 +63,7 @@ The runtime can be built as a library or a standalone executable. see [docs/buil ## LICENSE -MIT +[MIT LICENSE](LICENSE) ## 🔗 Links From 89d2514dd6f65ffa6f5199e8fe6b08525ff073f6 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=E4=BA=91=E5=BE=AE?= <1067852565@qq.com> Date: Fri, 24 Feb 2023 13:24:43 +0800 Subject: [PATCH 07/14] Update docs/build.md Co-authored-by: septs --- docs/build.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/build.md b/docs/build.md index 846c0e2..70436bd 100644 --- a/docs/build.md +++ b/docs/build.md @@ -1,4 +1,4 @@ -# build the runtime and examples +# Build the runtime and examples We have two types of runtime examples: From 3937425bc603a52ccb7927f6c49f9c1d8af92ce4 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=E4=BA=91=E5=BE=AE?= <1067852565@qq.com> Date: Fri, 24 Feb 2023 13:25:16 +0800 Subject: [PATCH 08/14] Update docs/build.md Co-authored-by: septs --- docs/build.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/build.md b/docs/build.md index 70436bd..0d1f922 100644 --- a/docs/build.md +++ b/docs/build.md @@ -7,7 +7,7 @@ We have two types of runtime examples: A new runtime is easy to implement with only a few hundred lines of code, in any language, using any wasm runtime or any ebpf user space library. -## build the Cpp minimal runtime based on WAMR +## Build the C++ minimal runtime based on WAMR[^wamr] The dependencies are libbpf and wasm-micro-runtime only, they are registered as git submodules. From 7e92aa74cbe09be7e8088438e81eb02e4e8cc1b3 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=E4=BA=91=E5=BE=AE?= <1067852565@qq.com> Date: Fri, 24 Feb 2023 13:25:37 +0800 Subject: [PATCH 09/14] Update docs/build.md Co-authored-by: septs --- docs/build.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/build.md b/docs/build.md index 0d1f922..130bdb8 100644 --- a/docs/build.md +++ b/docs/build.md @@ -28,7 +28,7 @@ On Ubuntu/Debian, you need: sudo apt install clang libelf1 libelf-dev zlib1g-dev ``` -On CentOS/Fedora, you need: +on CentOS / Fedora, you need: ```shell sudo dnf install clang elfutils-libelf elfutils-libelf-devel zlib-devel From 3642292af39d25abdd58add13bcab86226a89800 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=E4=BA=91=E5=BE=AE?= <1067852565@qq.com> Date: Fri, 24 Feb 2023 13:25:43 +0800 Subject: [PATCH 10/14] Update docs/build.md Co-authored-by: septs --- docs/build.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/build.md b/docs/build.md index 130bdb8..09df9b5 100644 --- a/docs/build.md +++ b/docs/build.md @@ -22,7 +22,7 @@ cd runtime/cpp You will need `clang`, `libelf` and `zlib` to build the examples, package names may vary across distros. -On Ubuntu/Debian, you need: +on Ubuntu/Debian, you need: ```shell sudo apt install clang libelf1 libelf-dev zlib1g-dev From b9aa7fbb4b25e7b280128489283676fc89806843 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=E4=BA=91=E5=BE=AE?= <1067852565@qq.com> Date: Fri, 24 Feb 2023 13:25:49 +0800 Subject: [PATCH 11/14] Update docs/build.md Co-authored-by: septs --- docs/build.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/build.md b/docs/build.md index 09df9b5..364fe4c 100644 --- a/docs/build.md +++ b/docs/build.md @@ -52,7 +52,7 @@ make build-lib You may refer to [CI](.github/workflows/c-cpp.yml) for more details on how to build and run the examples. -## build the Rust runtime based on Wasmtime +## Build the Rust runtime based on Wasmtime install rust toolchain From 673f8ea1b51530ef7e7a2a8c499bd79453e9df8b Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=E4=BA=91=E5=BE=AE?= <1067852565@qq.com> Date: Fri, 24 Feb 2023 13:26:18 +0800 Subject: [PATCH 12/14] Update examples/README.md Co-authored-by: ocfox --- examples/README.md | 3 --- 1 file changed, 3 deletions(-) diff --git a/examples/README.md b/examples/README.md index 3a7b1d3..e63d826 100644 --- a/examples/README.md +++ b/examples/README.md @@ -69,9 +69,6 @@ See [examples/rust-bootstrap](examples/rust-bootstrap) for more details. This program summarizes scheduler run queue latency as a histogram, showing how long tasks spent waiting their turn to run on-CPU. -This program summarizes scheduler run queue latency as a histogram, showing -how long tasks spent waiting their turn to run on-CPU. - ```console $ sudo ./wasm-bpf runqlat.wasm -h Summarize run queue (scheduler) latency as a histogram. From 76eede4f4c4dc4b21b8ece5e43e997b5bf53b4f5 Mon Sep 17 00:00:00 2001 From: Littlefisher619 Date: Fri, 24 Feb 2023 16:47:20 +0800 Subject: [PATCH 13/14] update documents for example desc --- README.md | 39 +++++++++++++++++++++++++++++++++------ docs/build.md | 8 ++++++-- 2 files changed, 39 insertions(+), 8 deletions(-) diff --git a/README.md b/README.md index a87a68d..56a3780 100644 --- a/README.md +++ b/README.md @@ -45,12 +45,39 @@ We have proposed a new WASI issue [wasi-bpf](https://github.com/WebAssembly/WASI See the [examples](examples) directory for examples of eBPF programs written in C, Rust, Go and compiled to WASM. -- [bootstrap](examples/bootstrap) and [rust-bootstrap](examples/rust-bootstrap) `tracing examples` -- [runqlat](examples/runqlat) `tracing examples` -- [go-execve](examples/go-execve) `tracing examples` -- [lsm](examples/lsm) and `security example` -- [sockfilter](examples/sockfilter) `networking example` -- [sockops](examples/sockops) `networking example` +`tracing examples` + +- [bootstrap](examples/bootstrap) and [rust-bootstrap](examples/rust-bootstrap): trace process exec and exit +- [runqlat](examples/runqlat): summarizes scheduler run queue latency as a histogram +- [execve](examples/execve) and [go-execve](examples/go-execve): trace execve syscall + +`security example` +- [lsm](examples/lsm) and [go-lsm](examples/go-lsm): check the permission to remove a directory + +`networking example` +- [sockfilter](examples/sockfilter): monitoring packet and dealing with __sk_buff. +- [sockops](examples/sockops): Add the pid int tcp option in syn packet. + +An example output of runqlat: + +```console +$ sudo ./wasm-bpf runqlat.wasm 1 +Tracing run queue latency... Hit Ctrl-C to end. + + usecs : count distribution + 0 -> 1 : 72 |***************************** | + 2 -> 3 : 93 |************************************* | + 4 -> 7 : 98 |****************************************| + 8 -> 15 : 96 |*************************************** | + 16 -> 31 : 38 |*************** | + 32 -> 63 : 4 |* | + 64 -> 127 : 5 |** | + 128 -> 255 : 6 |** | + 256 -> 511 : 0 | | + 512 -> 1023 : 0 | | + 1024 -> 2047 : 0 | | + 2048 -> 4095 : 1 | | +``` ## Build the runtime diff --git a/docs/build.md b/docs/build.md index 364fe4c..dc7a846 100644 --- a/docs/build.md +++ b/docs/build.md @@ -7,7 +7,7 @@ We have two types of runtime examples: A new runtime is easy to implement with only a few hundred lines of code, in any language, using any wasm runtime or any ebpf user space library. -## Build the C++ minimal runtime based on WAMR[^wamr] +## Build the C++ minimal runtime based on WAMR[^1] The dependencies are libbpf and wasm-micro-runtime only, they are registered as git submodules. @@ -52,7 +52,9 @@ make build-lib You may refer to [CI](.github/workflows/c-cpp.yml) for more details on how to build and run the examples. -## Build the Rust runtime based on Wasmtime +[^1]: WAMR (WebAssembly Micro Runtime): https://github.com/bytecodealliance/wasm-micro-runtime + +## Build the Rust runtime based on Wasmtime[^2] install rust toolchain @@ -68,3 +70,5 @@ directory. ```sh make build ``` + +[^2]: wasmtime: https://github.com/bytecodealliance/wasmtime From 2c090cd90c37ed5ad4333811a351ad7ef0a00e5e Mon Sep 17 00:00:00 2001 From: Littlefisher619 Date: Fri, 24 Feb 2023 17:04:04 +0800 Subject: [PATCH 14/14] replace example and how it works --- README.md | 40 ++++++++++++++++++++-------------------- 1 file changed, 20 insertions(+), 20 deletions(-) diff --git a/README.md b/README.md index 56a3780..bbc1687 100644 --- a/README.md +++ b/README.md @@ -21,26 +21,6 @@ See the [examples](examples) directory for examples of eBPF programs written in For tools to distribute Wasm-eBPF programs in [`OCI`](https://opencontainers.org/) images, please refer to [eunomia-bpf](https://github.com/eunomia-bpf/eunomia-bpf) repo. -## How it works - -The wasm-bpf runtime require two parts: `the host side`(Outside the Wasm runtime) and the `Wasm guest side`(Inside the Wasm runtime). - -- host side: A simple runtime implementation example - - see [runtime/cpp](runtime/cpp), which would be a sample runtime in `C++` built on the top of [libbpf](https://github.com/libbpf/libbpf) and [WAMR](https://github.com/bytecodealliance/wasm-micro-runtime). Another more complex runtime implement in `Rust` is [runtime/rust](runtime/rust), based on [Wasmtime](https://github.com/bytecodealliance/wasmtime). - - You can easily build your own Wasm-eBPF runtime in `any` languages, `any` eBPF libraries and `any` Wasm runtimes with the same System interface. -- wasm side: toolchains and libraries - - a [`libbpf-wasm`](wasm-sdk/c/libbpf-wasm.h) header only library to provide libbpf APIs for Wasm guest `C/C++` code. - - a [`bpftool`](https://github.com/eunomia-bpf/bpftool/tree/wasm-bpftool) tool to generate the Wasm-eBPF `skeleton` headers, and `C struct definitions` for passing data between the host and Wasm guest without serialization. - - `Rust`, `Go` and other language support is similar to the `C/C++` support. - -For details compile process, please refer to the [examples/bootstrap/README.md](examples/bootstrap/README.md). The figure below shows the overall interaction between the eBPF and Wasm runtimes: - -![wasi-bpf](docs/wasm-bpf-no-bcc.png) - -A Wasm module could load and control multiple eBPF programs at the same time, and can call another Wasm module written in other languages to process the data or control with [the component model](https://github.com/WebAssembly/component-model). - -We have proposed a new WASI issue [wasi-bpf](https://github.com/WebAssembly/WASI/issues/513). - ## 🔨 Examples See the [examples](examples) directory for examples of eBPF programs written in C, Rust, Go and compiled to WASM. @@ -79,6 +59,26 @@ Tracing run queue latency... Hit Ctrl-C to end. 2048 -> 4095 : 1 | | ``` +## How it works + +The wasm-bpf runtime require two parts: `the host side`(Outside the Wasm runtime) and the `Wasm guest side`(Inside the Wasm runtime). + +- host side: A simple runtime implementation example + - see [runtime/cpp](runtime/cpp), which would be a sample runtime in `C++` built on the top of [libbpf](https://github.com/libbpf/libbpf) and [WAMR](https://github.com/bytecodealliance/wasm-micro-runtime). Another more complex runtime implement in `Rust` is [runtime/rust](runtime/rust), based on [Wasmtime](https://github.com/bytecodealliance/wasmtime). + - You can easily build your own Wasm-eBPF runtime in `any` languages, `any` eBPF libraries and `any` Wasm runtimes with the same System interface. +- wasm side: toolchains and libraries + - a [`libbpf-wasm`](wasm-sdk/c/libbpf-wasm.h) header only library to provide libbpf APIs for Wasm guest `C/C++` code. + - a [`bpftool`](https://github.com/eunomia-bpf/bpftool/tree/wasm-bpftool) tool to generate the Wasm-eBPF `skeleton` headers, and `C struct definitions` for passing data between the host and Wasm guest without serialization. + - `Rust`, `Go` and other language support is similar to the `C/C++` support. + +For details compile process, please refer to the [examples/bootstrap/README.md](examples/bootstrap/README.md). The figure below shows the overall interaction between the eBPF and Wasm runtimes: + +![wasi-bpf](docs/wasm-bpf-no-bcc.png) + +A Wasm module could load and control multiple eBPF programs at the same time, and can call another Wasm module written in other languages to process the data or control with [the component model](https://github.com/WebAssembly/component-model). + +We have proposed a new WASI issue [wasi-bpf](https://github.com/WebAssembly/WASI/issues/513). + ## Build the runtime We have two types of runtime samples: