datadir volume causes container to stop from starting

[purinda]

Issue:

MariaDB docker image consistently fails to start for all versions after "v10.0.28" when trying to mount a host volume as the datadir.

I did test almost all the docker images currently available in https://hub.docker.com/r/library/mariadb/tags/ the only ones I can start successfully are v10.0.27 and .28

Platform:

• Mac (OSX 10.13.6), using APFS case insensitive filesystem
• Docker-machine: boot2docker running on Parallels Desktop Version 14.0.0 Pro (45124)

part of the docker-compose.yml which has the mariadb service mapping.

docker-compose.yml

version: "2.4"
services:
    db:
        image: mariadb:10.0.28
        environment:
            MYSQL_ROOT_PASSWORD: toor
            MYSQL_DATABASE: test
            MYSQL_USER: test
            MYSQL_PASSWORD: test
        volumes:
            - ./store/maria-db:/var/lib/mysql
        command: 'mysqld --innodb-flush-method=fsync'
        ports:
            - 3306:3306

Logs:

Samples of failing versions
10.0.30.log
10.2.17.log
10.3.9.log

Working version:
10.0.28.working.log

[wglambert]

Issue seems to be related to your environment, I can't reproduce using mariadb:10.3

This is your exact issue with solution: docker-library/mysql#371 (comment)

version: '2.4'

services:
    mysql_db:
        image: mariadb:10.3
        environment:
            MYSQL_ROOT_PASSWORD: toor
            MYSQL_DATABASE: test
            MYSQL_USER: test
            MYSQL_PASSWORD: test
        volumes:
            - ./store/maria-db:/var/lib/mysql
        command: 'mysqld --innodb-flush-method=fsync'
        ports:
            - 3306:3306

$ docker-compose up -d
Pulling mysql_db (mariadb:10.3)...
10.3: Pulling from library/mariadb
Digest: sha256:b7894bd08e5752acdd41fea654cb89467c99e67b8293975bb5d787b27e66ce1a
Status: Downloaded newer image for mariadb:10.3
Recreating mariadb-200_mysql_db_1 ... done
$ docker-compose down
Stopping mariadb-200_mysql_db_1 ... 
^CERROR: Aborting.
$ docker-compose down
Removing mariadb-200_mysql_db_1 ... done
Removing network mariadb-200_default
$ docker-compose up -d
Creating network "mariadb-200_default" with the default driver
Creating mariadb-200_mysql_db_1 ... done
$ docker-compose logs | tail -n 3
mysql_db_1  | 2018-09-13  0:53:04 0 [Note] Added new Master_info '' to hash table
mysql_db_1  | 2018-09-13  0:53:04 0 [Note] mysqld: ready for connections.
mysql_db_1  | Version: '10.3.9-MariaDB-1:10.3.9+maria~bionic'  socket: '/var/run/mysqld/mysqld.sock'  port: 3306  mariadb.org binary distribution
$ ls ./store/maria-db/
aria_log.00000001  ibdata1      ibtmp1             performance_schema/
aria_log_control   ib_logfile0  multi-master.info  tc.log
ib_buffer_pool     ib_logfile1  mysql/             test/

[purinda]

@wglambert I followed the solution given in the other issue you mentioned. Here is the output running mysql:latest vs mariadb:latest (for mariadb I had to pass args via a separate config file mount)

1. Ran the solution with mysql docker image (as per docker-library/mysql#371 (comment))
docker run --user 1000:50 -e MYSQL_ROOT_PASSWORD=password -e MYSQL_DATABASE=dbname -d -v ~/.tmp:/var/lib/mysql -p 32775:3308 mysql:latest --innodb-use-native-aio=0 --innodb_flush_method=fsync

Log attached: mysql-latest.log - worked fine accepts connections.

2. Ran the mariadb:latest image with the user id mapped
docker run --user 1000:50 -e MYSQL_ROOT_PASSWORD=password -e MYSQL_DATABASE=dbname -d -v ~/.tmp:/var/lib/mysql -v ~/src/asx/asx-platform-docker/mariadb/innodb-fix.cnf:/etc/mysql/conf.d/innodb-fix.cnf -p 32775:3308 mariadb:latest

Contents of innodb-fix.cnf

[mysqld]
lower_case_table_names=2
innodb_flush_method=fsync
innodb_use_native_aio=OFF

Log attached: mariadb-latest.log - tried init process and failed.

Note: I removed all files in ./tmp shared directory before running each test to make sure data files between two images doesn't play a role in this issue.

If there is any switch I can pass to mariadb container for better logging that would be perfect, currently it just shows [ERROR] Aborting.. and nothing much to go on from there.

My boot2docker machine user info, so user mapping was correct.

docker@docker:~$ id
uid=1000(docker) gid=50(staff) groups=50(staff),100(docker)

[purinda]

@wglambert I get this could be related to my docker-machine setup (running parallels), etc. I still would like to understand the why >mariadb:10.0.28 images are specifically breaking.

Were you running that docker-compose test on virtual box (boot2docker) or on linux box with docker-engine?

[yosifkit]

Related issues: docker-library/percona#42, #95. And maybe this workaround: #38 (comment)

[tianon]

I still would like to understand the why >mariadb:10.0.28 images are specifically breaking.

I imagine there was probably a change made in 10.0.28 that doesn't support the quirks of the shared filesystem Docker has implemented.

Seeing as how there's not really anything we can do here (any changes or fixes would need to happen in MariaDB itself), I'm going to close.