|
| 1 | +# Developing locally without docker-compose |
| 2 | + |
| 3 | +These are instructions for developing docs.rs locally. For deploying in a production environment, see [Self-hosting a docs.rs instance](self-hosting.html). |
| 4 | + |
| 5 | +While the docker-compose allows for easier setup of the required dependencies and environment, here is a breakdown of how to use the service without an outer docker container. This is useful e.g. for quickly iterating during development. |
| 6 | + |
| 7 | +Note that this does not remove the docker dependency altogether, since docs.rs uses docker at runtime for sandboxing. This just allows you to run commands more quickly since you are building in debug mode instead of release and are also caching more of the build. |
| 8 | + |
| 9 | +## Requirements |
| 10 | + |
| 11 | +The commands and package names on this page will assume a Debian-like machine with `apt` installed, but most dependencies should be relatively easy to find on Linux. Do note however that this requires the host to be `x86_64-unknown-linux-gnu`. |
| 12 | + |
| 13 | +Docs.rs has a few basic requirements: |
| 14 | + |
| 15 | +* Rust |
| 16 | +* Git |
| 17 | +* CMake, GCC, G++, and `pkg-config` (to build dependencies for crates and docs.rs itself) |
| 18 | +* OpenSSL, zlib, curl, and `libmagic` (to link against) |
| 19 | +* PostgreSQL |
| 20 | + |
| 21 | +```console |
| 22 | +# apt install build-essential git curl cmake gcc g++ pkg-config libmagic-dev libssl-dev zlib1g-dev postgresql |
| 23 | +$ sudo -u postgres psql -c "CREATE USER cratesfyi WITH PASSWORD 'password';" |
| 24 | +$ sudo -u postgres psql -c "CREATE DATABASE cratesfyi OWNER cratesfyi;" |
| 25 | +``` |
| 26 | + |
| 27 | +## Building the site |
| 28 | + |
| 29 | +Be warned - this builds over 350 crates! Depending on your computer, this may |
| 30 | +take upwards of 10 minutes. |
| 31 | + |
| 32 | +```console |
| 33 | +$ git clone https://github.com/rust-lang/docs.rs && cd docs.rs |
| 34 | +$ cargo build |
| 35 | +``` |
| 36 | + |
| 37 | +## The "prefix" directory |
| 38 | + |
| 39 | +docs.rs stores several files in a "prefix" directory. This can be anywhere, but if you put it in the doc.rs repo, it should go under the ./ignored/ directory so that it is not seen by git or the docker daemon. |
| 40 | + |
| 41 | +```console |
| 42 | +$ mkdir -p ignored/cratesfyi-prefix |
| 43 | +$ cd ignored/cratesfyi-prefix |
| 44 | +$ mkdir -vp documentations public_html sources |
| 45 | +$ git clone https://github.com/rust-lang/crates.io-index && cd crates.io-index |
| 46 | +$ git branch crates-index-diff_last-seen |
| 47 | +``` |
| 48 | + |
| 49 | +(That last command is used to set up the `crates-index-diff` crate, so we can start monitoring new crate releases.) |
| 50 | + |
| 51 | +## Docker group |
| 52 | + |
| 53 | +docs.rs needs to run docker containers for sandboxing. Therefore, you need to be in the 'docker' group to build crates. If you are already in the docker group, you can skip this step (you can check with `groups`). |
| 54 | + |
| 55 | +```console |
| 56 | +# usermod -a -G docker "$USER" |
| 57 | +$ # now logout and back in to your shell |
| 58 | +$ cd /path/to/docs.rs/ignored/cratesfyi-prefix |
| 59 | +``` |
| 60 | + |
| 61 | +## Environment for the server |
| 62 | + |
| 63 | +To ensure that the docs.rs server is configured properly, we need to set a few environment variables. This is most easily done by making a shell script. |
| 64 | + |
| 65 | +```sh |
| 66 | +$ cd .. |
| 67 | +$ echo ' |
| 68 | +export CRATESFYI_PREFIX=. |
| 69 | +# or add an appropriate username/password as necessary |
| 70 | +export CRATESFYI_DATABASE_URL=postgresql://cratesfyi:password@localhost |
| 71 | +export CRATESFYI_GITHUB_USERNAME= |
| 72 | +export CRATESFYI_GITHUB_ACCESSTOKEN= |
| 73 | +export RUST_LOG=cratesfyi,rustwide=info |
| 74 | +' > env.sh |
| 75 | +``` |
| 76 | + |
| 77 | +This last command assumes you put the shell script in `./env.sh`, |
| 78 | +but you can name it anything as long as it is in the current directory. |
| 79 | + |
| 80 | +```console |
| 81 | +$ . ./env.sh |
| 82 | +$ cargo run database migrate |
| 83 | +$ cargo run database update-search-index |
| 84 | +$ cargo run database update-release-activity |
| 85 | +# This will take between 5 and 30 minutes on the first run, depending on your internet speed. |
| 86 | +# It downloads the rustops/crates-build-env crates which is almost 1 GB. |
| 87 | +# It does not currently display a progress bar, this is https://github.com/rust-lang/rustwide/issues/9 |
| 88 | +# As a workaround, you can run `docker pull rustops/crates-build-env` in a separate terminal. |
| 89 | +$ cargo run build crate rand 0.5.5 |
| 90 | +``` |
| 91 | + |
| 92 | +## Building on platforms other than Linux |
| 93 | + |
| 94 | +This is not currently possible. We assume in several places that the rustup toolchain is x86_64-unknown-linux-gnu. As a result, the only way to build on Mac or Alpine is to use the docker-compose file. |
| 95 | + |
| 96 | +## Resetting the database |
| 97 | + |
| 98 | +Occasionally, if you make changes to the migrations in a PR, those migrations will be saved in the database but will not be in the code when you switch back to the master branch. In this case, there is no way to undo the migration without knowing exactly which version of the code made the change (`cargo migrate` will have no effect). Here is a convenient shell script to reset the database so you don't have to remember how to undo your changes. |
| 99 | + |
| 100 | +NOTE: DO NOT RUN THIS IN PRODUCTION. |
| 101 | + |
| 102 | +```sh |
| 103 | +#!/bin/sh |
| 104 | +set -euv |
| 105 | + |
| 106 | +. ./env.sh |
| 107 | + |
| 108 | +sudo -u postgres dropdb cratesfyi --if-exists |
| 109 | +sudo -u postgres psql -c 'CREATE DATABASE cratesfyi OWNER cratesfyi;' |
| 110 | +cargo run database migrate |
| 111 | +``` |
0 commit comments