Skip to content

Use docker initialization scripts for DB init #9993

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 9 commits into from
12 changes: 7 additions & 5 deletions Makefile
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
COMPOSE_PROJECT_NAME ?= $(notdir $(abspath .))
DB := example
IPYTHON := no

Expand Down Expand Up @@ -82,11 +83,12 @@ translations: .state/docker-build-web
requirements/%.txt: requirements/%.in
docker-compose run --rm web bin/pip-compile --allow-unsafe --generate-hashes --output-file=$@ $<

initdb: .state/docker-build-web
docker-compose run --rm web psql -h db -d postgres -U postgres -c "SELECT pg_terminate_backend(pid) FROM pg_stat_activity WHERE datname ='warehouse';"
docker-compose run --rm web psql -h db -d postgres -U postgres -c "DROP DATABASE IF EXISTS warehouse"
docker-compose run --rm web psql -h db -d postgres -U postgres -c "CREATE DATABASE warehouse ENCODING 'UTF8'"
docker-compose run --rm web bash -c "xz -d -f -k dev/$(DB).sql.xz --stdout | psql -h db -d warehouse -U postgres -v ON_ERROR_STOP=1 -1 -f -"
resetdb: .state/docker-build-web
docker-compose stop db
docker volume rm $(COMPOSE_PROJECT_NAME)_pgdata
docker-compose up -d --recreate db

migrate: .state/docker-build-web
docker-compose run --rm web python -m warehouse db upgrade head
docker-compose run --rm web python -m warehouse sponsors populate-db
$(MAKE) reindex
Expand Down
2 changes: 1 addition & 1 deletion dev/environment
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ AWS_ACCESS_KEY_ID=foo
AWS_SECRET_ACCESS_KEY=foo
BROKER_URL=sqs://localstack:4566/?region=us-east-1&queue_name_prefix=warehouse-dev

DATABASE_URL=postgresql://postgres@db/warehouse
DATABASE_URL=postgresql://postgres:password@db/warehouse

ELASTICSEARCH_URL=http://elasticsearch:9200/development

Expand Down
11 changes: 9 additions & 2 deletions docker-compose.yml
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@ version: '3.9'
volumes:
simple:
packages:
pgdata:
sponsorlogos:
vault:

Expand All @@ -25,12 +26,18 @@ services:
- ./dev/vault:/etc/vault

db:
image: postgres:10.1
image: postgres:10
ports:
# 5432 may already in use by another PostgreSQL on host
- "5433:5432"
healthcheck:
test: ["CMD", "pg_isready", "-U", "postgres", "-d", "postgres"]
test: ["CMD", "pg_isready", "-d", "warehouse"]
environment:
POSTGRES_DB: warehouse
POSTGRES_PASSWORD: password
volumes:
- pgdata:/var/lib/postgresql/data
- ./dev/example.sql.xz:/docker-entrypoint-initdb.d/example.sql.xz:ro

redis:
image: redis:4.0
Expand Down
22 changes: 21 additions & 1 deletion docs/development/getting-started.rst
Original file line number Diff line number Diff line change
Expand Up @@ -238,7 +238,27 @@ or that the ``static`` container has finished compiling the static assets:
static_1 | [20:28:37] Starting 'watch'...
static_1 | [20:28:37] Finished 'watch' after 11 ms

or maybe something else.
After the docker containers are setup in the previous step, run:

.. code-block:: console

make migrate

This command will:

* run migrations on the Postgres database,
* index all the data for the search database, and
* load some example data from `Test PyPI`_.

The Postgres database is created automatically when its docker
container starts for the first time. It also gets populated with
example data. If you will need to reset the database, use:

.. code-block:: console

make resetdb

Once the ``make initdb`` command has finished, you are ready to continue.


Viewing Warehouse in a browser
Expand Down