Skip to content

Commit 3e592a4

Browse files
sd2ksd2k
committed
Split up README.md and DEVELOPING.md
1 parent 276ad78 commit 3e592a4

File tree

2 files changed

+124
-87
lines changed

2 files changed

+124
-87
lines changed

‎DEVELOPING.md

+99
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,99 @@
1+
# DEVELOPING
2+
3+
There are a couple of ways to develop the Materialize plugin: either using Docker and docker-compose (much simpler, but possibly slower if you're on a non-Linux OS/architecture), or outside of Docker (faster, but requires various toolchains to be installed locally).
4+
5+
## Getting started
6+
7+
### Using docker-compose
8+
9+
Running `docker-compose up -d` from the repository root will start up a Grafana instance with a
10+
pre-provisioned instance of this data source pointing to a materialized instance inside Docker. It
11+
will also start up two processes to watch the backend and frontend directory for changes,
12+
rebuild the plugin components, and restart the backend process inside the Grafana container.
13+
14+
### Outside of Docker
15+
16+
#### Plugin frontend
17+
18+
At the repository root:
19+
20+
1. Install dependencies
21+
22+
```bash
23+
yarn install
24+
```
25+
26+
2. Build plugin in development mode or run in watch mode
27+
28+
```bash
29+
yarn dev
30+
```
31+
32+
or
33+
34+
```bash
35+
yarn watch
36+
```
37+
38+
3. Build plugin in production mode
39+
40+
```bash
41+
yarn build
42+
```
43+
44+
#### Plugin backend
45+
46+
Make sure you have a recent version of Rust (run `rustup update stable`), and install [`cargo-watch`].
47+
48+
Then run:
49+
50+
```bash
51+
cargo xtask watch
52+
```
53+
54+
This will run the `watch` task using the [`cargo-xtask`] pattern, which rebuilds the backend component on changes, copies the binary into the correct location, and restarts the plugin process (which Grafana subsequently restarts).
55+
56+
#### Running Grafana
57+
58+
You can run a Grafana instance either by cloning the Grafana repository, or running it inside Docker. See the Grafana docs for more information.
59+
60+
## Cross compiling
61+
62+
Cross compilation is generally run in CI now because it's easier to standardize, but instructions to cross-compile from MacOS to all other Grafana-supported platforms are below.
63+
64+
### From MacOS
65+
66+
1. Install the relevant cross compiler toolchains. Using Homebrew:
67+
68+
```bash
69+
brew tap messense/macos-cross-toolchains
70+
brew install armv7-unknown-linux-musleabihf
71+
brew install aarch64-unknown-linux-musl
72+
brew install x86_64-unknown-linux-musl
73+
brew install mingw-w64
74+
```
75+
76+
2. Install the relevant Rust targets. Using `rustup`:
77+
78+
```bash
79+
rustup target add armv7-unknown-linux-musleabihf
80+
rustup target add aarch64-apple-darwin
81+
rustup target add x86_64-apple-darwin
82+
rustup target add aarch64-unknown-linux-musl
83+
rustup target add x86_64-unknown-linux-musl
84+
rustup target add x86_64-pc-windows-gnu
85+
```
86+
87+
3. Run the following to compile the plugin in release mode for each target:
88+
89+
```bash
90+
CARGO_TARGET_ARMV7_UNKNOWN_LINUX_MUSLEABIHF_LINKER=armv7-unknown-linux-musleabihf-ld cargo build --release --target armv7-unknown-linux-musleabihf --bin grafana-materialize-datasource
91+
cargo build --release --target aarch64-apple-darwin --bin grafana-materialize-datasource
92+
cargo build --release --target x86_64-apple-darwin --bin grafana-materialize-datasource
93+
CARGO_TARGET_X86_64_UNKNOWN_LINUX_MUSL_LINKER=x86_64-unknown-linux-musl-gcc cargo build --release --target x86_64-unknown-linux-musl --bin grafana-materialize-datasource
94+
CARGO_TARGET_AARCH64_UNKNOWN_LINUX_MUSL_LINKER=aarch64-unknown-linux-musl-gcc cargo build --release --target aarch64-unknown-linux-musl --bin grafana-materialize-datasource
95+
CARGO_TARGET_X86_64_PC_WINDOWS_GNU_LINKER=x86_64-w64-mingw32-gcc cargo build --release --target x86_64-pc-windows-gnu --bin grafana-materialize-datasource
96+
```
97+
98+
[`cargo-xtask`]: https://github.com/matklad/cargo-xtask
99+
[`cargo-watch`]: https://github.com/watchexec/cargo-watch/

‎README.md

+25-87
Original file line numberDiff line numberDiff line change
@@ -1,107 +1,45 @@
11
# Grafana Materialize Data Source
22

3-
This is a WIP Grafana data source which can connect to the [Materialize][] streaming SQL database.
3+
This is a Grafana data source which can connect to the [Materialize][] streaming SQL database. It supports the Grafana Live capabilities for streaming data and hence can be used to visualise data in materialized views 'live'.
44

5-
## Screenshots
6-
7-
https://user-images.githubusercontent.com/5464991/166680691-8df200d7-e354-43bf-a924-8ce9fbc8582a.mov
8-
9-
![image](https://raw.githubusercontent.com/sd2k/grafana-materialize-datasource/main/src/img/query.png)
10-
11-
![image](https://raw.githubusercontent.com/sd2k/grafana-materialize-datasource/main/src/img/transforms.png)
12-
13-
## Getting started
14-
15-
### Using docker-compose
16-
17-
Running `docker-compose up -d` from the repository root will start up a Grafana instance with a
18-
pre-provisioned instance of this data source pointing to a materialized instance inside Docker. It
19-
will also start up two processes to watch the backend and frontend directory for changes,
20-
rebuild the plugin components, and restart the backend process inside the Grafana container.
21-
22-
### Outside of Docker
23-
24-
#### Plugin frontend
25-
26-
At the repository root:
5+
## Usage
276

28-
1. Install dependencies
7+
### Configuring the datasource
298

30-
```bash
31-
yarn install
32-
```
9+
Only three parameters are available for the datasource:
3310

34-
2. Build plugin in development mode or run in watch mode
11+
- **Host** - the hostname of the Materialize database
12+
- **Port** - the port on which to connect to the Materialize database
13+
- **Username** - the username as which to connect to the Materialize database
3514

36-
```bash
37-
yarn dev
38-
```
15+
### Querying the datasource
3916

40-
or
17+
When querying the datasource in a new panel you have two options available to you:
4118

42-
```bash
43-
yarn watch
44-
```
19+
- **Relation** - the query builder will populate a list of available relations in the Materialize database. Select one and the relation will be `TAIL`ed to the panel.
20+
- **Select statement** - input a custom statement into the query field and the output of the statement will be `TAIL`ed to the panel.
4521

46-
3. Build plugin in production mode
22+
### Configuring panels
4723

48-
```bash
49-
yarn build
50-
```
24+
The plugin includes the `mz_timestamp` and `mz_diff` columns in the streaming output, which may not be what you want to see. The simplest way to solve this is to use the [Transformations][] functionality of the panels. In the panel editor, click the **Transform** button and add any transformations you like. A good place to start is:
5125

52-
#### Plugin backend
26+
1. Organize fields
27+
Use this to hide the `mz_diff` field by clicking the 'eye' symbol next to the field name.
28+
2. Prepare time series
29+
Select **Multi-frame time series** as the format.
30+
3. Rename by regex
31+
Use this to remove any common unwanted prefix from the time series names. E.g. set **Match** to `avg (.*)` and **Replace** to `$1` to remove the prefix `avg `.
5332

54-
Make sure you have a recent version of Rust (run `rustup update stable`), and install [`cargo-watch`].
33+
See the 'transforms' screenshot for an example.
5534

56-
Then run:
57-
58-
```bash
59-
cargo xtask watch
60-
```
61-
62-
This will run the `watch` task using the [`cargo-xtask`] pattern, which rebuilds the backend component on changes, copies the binary into the correct location, and restarts the plugin process (which Grafana subsequently restarts).
63-
64-
#### Running Grafana
65-
66-
You can run a Grafana instance either by cloning the Grafana repository, or running it inside Docker. See the Grafana docs for more information.
67-
68-
## Cross compiling
69-
70-
### From MacOS
71-
72-
1. Install the relevant cross compiler toolchains. Using Homebrew:
73-
74-
```bash
75-
brew tap messense/macos-cross-toolchains
76-
brew install armv7-unknown-linux-musleabihf
77-
brew install aarch64-unknown-linux-musl
78-
brew install x86_64-unknown-linux-musl
79-
brew install mingw-w64
80-
```
81-
82-
2. Install the relevant Rust targets. Using `rustup`:
35+
## Screenshots
8336

84-
```bash
85-
rustup target add armv7-unknown-linux-musleabihf
86-
rustup target add aarch64-apple-darwin
87-
rustup target add x86_64-apple-darwin
88-
rustup target add aarch64-unknown-linux-musl
89-
rustup target add x86_64-unknown-linux-musl
90-
rustup target add x86_64-pc-windows-gnu
91-
```
37+
https://user-images.githubusercontent.com/5464991/166680691-8df200d7-e354-43bf-a924-8ce9fbc8582a.mov
9238

93-
3. Run the following to compile the plugin in release mode for each target:
39+
![image](https://raw.githubusercontent.com/sd2k/grafana-materialize-datasource/main/src/img/query.png)
9440

95-
```bash
96-
CARGO_TARGET_ARMV7_UNKNOWN_LINUX_MUSLEABIHF_LINKER=armv7-unknown-linux-musleabihf-ld cargo build --release --target armv7-unknown-linux-musleabihf --bin grafana-materialize-datasource
97-
cargo build --release --target aarch64-apple-darwin --bin grafana-materialize-datasource
98-
cargo build --release --target x86_64-apple-darwin --bin grafana-materialize-datasource
99-
CARGO_TARGET_X86_64_UNKNOWN_LINUX_MUSL_LINKER=x86_64-unknown-linux-musl-gcc cargo build --release --target x86_64-unknown-linux-musl --bin grafana-materialize-datasource
100-
CARGO_TARGET_AARCH64_UNKNOWN_LINUX_MUSL_LINKER=aarch64-unknown-linux-musl-gcc cargo build --release --target aarch64-unknown-linux-musl --bin grafana-materialize-datasource
101-
CARGO_TARGET_X86_64_PC_WINDOWS_GNU_LINKER=x86_64-w64-mingw32-gcc cargo build --release --target x86_64-pc-windows-gnu --bin grafana-materialize-datasource
102-
```
41+
![image](https://raw.githubusercontent.com/sd2k/grafana-materialize-datasource/main/src/img/transforms.png)
10342

104-
[`cargo-xtask`]: https://github.com/matklad/cargo-xtask
105-
[`cargo-watch`]: https://github.com/watchexec/cargo-watch/
10643
[Materialize]: https://materialize.com
44+
[Transformations]: https://grafana.com/docs/grafana/latest/panels/transform-data/transformation-functions/
10745

0 commit comments

Comments
 (0)