Skip to content

Commit 2827dc3

Browse files
authored
Merge pull request #99 from nwops/docker_0.4.x
Backport docker examples for 2018
2 parents fc67d1e + 141b627 commit 2827dc3

40 files changed

+948
-21
lines changed

.fixtures.yml

+13
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
---
2+
fixtures:
3+
repositories:
4+
facts: 'https://github.com/puppetlabs/puppetlabs-facts.git'
5+
puppet_agent: 'https://github.com/puppetlabs/puppetlabs-puppet_agent.git'
6+
provision: 'https://github.com/puppetlabs/provision.git'
7+
stdlib: 'https://github.com/puppetlabs/puppetlabs-stdlib.git'
8+
node_manager: 'https://github.com/WhatsARanjit/puppet-node_manager'
9+
apply_helpers: 'https://github.com/puppetlabs/puppetlabs-apply_helpers'
10+
bolt_shim: 'https://github.com/puppetlabs/puppetlabs-bolt_shim'
11+
debug: 'https://github.com/nwops/puppet-debug'
12+
symlinks:
13+
"pe_xl": "#{source_dir}"

.gitignore

+2
Original file line numberDiff line numberDiff line change
@@ -25,3 +25,5 @@
2525
.project
2626
.envrc
2727
/inventory.yaml
28+
.rerun.json
29+
*.tar.gz

.ruby-version

+1
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
2.6.5

README.md

+2
Original file line numberDiff line numberDiff line change
@@ -14,3 +14,5 @@ See this README file and any documents in the [documentation](documentation) dir
1414
## Architecture
1515

1616
![architecture](documentation/images/architecture.png)
17+
* [Classification](documentation/classification.md)
18+
* [Docker Based Examples](documentation/docker_examples.md)

documentation/docker_examples.md

+80
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,80 @@
1+
## Docker Based Examples
2+
This module provides docker compose files for the various architectures for experimentation purposes. This gives you the ability to stand up an entire PE stack in order to learn how this module and HA works. If you have docker and docker-compose you can start up a full Puppet architecture with a single command. Please note that Puppet does not support PE on containers in production.
3+
4+
In order to decouple Bolt from a dev system, a special bolt container is created that will run all the bolt commands. This is
5+
required to achieve maximum portability. Should you want to run bolt commands against the PE stack you must
6+
first login to this bolt container via ssh, docker or docker-compose.
7+
8+
Example: `docker-compose run --entrypoint=/bin/bash bolt`
9+
10+
### Requirements
11+
To run the container based examples you will need the following requirements:
12+
13+
2. Docker
14+
3. Docker compose
15+
4. 16GB memory, 24GB+ for XL and XL-HA architectures
16+
5. CPU with many cores (Tested with Core i7 6700)
17+
18+
### Starting the example
19+
We have provided a provision.sh script to help making these examples simple.
20+
To use perform the following:
21+
22+
1. cd spec/docker
23+
2. bash provision.sh
24+
3. select desired architecture when prompted (ie. extra-large-ha )
25+
4. Wait 10-20 minutes for provisioning to complete
26+
27+
```
28+
Please choose a PE architecture to build:
29+
1) extra-large/ 3) large/ 5) standard/
30+
2) extra-large-ha/ 4) large-ha/ 6) standard-ha/
31+
#?
32+
```
33+
34+
### Stopping the example
35+
In order to stop and remove the containers you will need to perform the following.
36+
37+
1. cd spec/docker
38+
2. `cd <chosen architecture>`
39+
3. docker-compose down
40+
41+
### Logging into the console
42+
You can login to the PE Console after successful provision. However, first you will need to
43+
grab the mapped port number of the PE console. The port numbers are mapped dynamically as to not
44+
cause port conflicts on your system. To see how the ports are mapped you can view them via:
45+
46+
1. docker ps
47+
```
48+
80c6f0b5525c pe-base "/sbin/init" 2 hours ago Up 2 hours 0.0.0.0:32774->22/tcp, 0.0.0.0:32773->443/tcp, 0.0.0.0:32772->4433/tcp, 0.0.0.0:32771->8080/tcp, 0.0.0.0:32770->8081/tcp, 0.0.0.0:32769->8140/tcp, 0.0.0.0:32768->8443/tcp pe-lg.puppet.vm
49+
```
50+
2. Note the mapped port for 443, which in this case is 32773
51+
3. Visit https://localhost:32773 in your browser
52+
4. Accept security risk (self signed cert)
53+
5. Login: admin/puppetlabs
54+
55+
### Logging into any of the containers
56+
Ssh is running in all the containers so you can use ssh if you grab the mapped ssh port number. `ssh root@localhost -p 32774`
57+
58+
Login: root/test
59+
60+
You can also bypass ssh and run docker exec or docker-compose exec
61+
62+
1. cd spec/docker/extra-large
63+
2. docker-compose exec pe_xl_core /bin/bash
64+
65+
**Note:** pe_xl_core is the name of the service defined in the respective docker-compose file.
66+
67+
This will run an interactive bash shell in the running container.
68+
69+
### Upgrades
70+
There is also a upgrade.sh script that is similar to the provision.sh script. This script will upgrade an already provisioned PE stack to the version specified in the update_params.json file.
71+
72+
### Other notes
73+
1. The provision plan is not fully idempotent.
74+
2. Some tasks may fail when run due to resource constraints.
75+
3. You can rerun the provision.sh script on the same architecture without destroying the containers. This can sometimes complete the provision process successfully.
76+
4. Rerunning the provision script may result in errors due to idempotency issues with tasks and plans.
77+
5. Please remember you are starting the equilivent of 3-6 VMs on a single system.
78+
6. You can use top to view all the processes being run in the containers.
79+
7. Docker will use the privilege mode option when running these examples (systemd support)
80+
8. Systemd is running inside these containers! The real systemd, not the fake one.

plans/unit/install.pp

+5-4
Original file line numberDiff line numberDiff line change
@@ -301,10 +301,11 @@
301301
apply($master_host) { include pe_xl::setup::master }.pe_xl::print_apply_result
302302
}
303303

304-
run_command(inline_epp(@(HEREDOC/L)), $master_target)
305-
/opt/puppetlabs/bin/puppetserver ca sign --certname \
306-
<%= $agent_installer_targets.map |$target| { $target.name }.join(',') -%>
307-
| HEREDOC
304+
if !empty($agent_installer_targets) {
305+
run_task('pe_xl::sign_csr', $master_target,
306+
certnames => $agent_installer_targets.map |$target| { $target.name },
307+
)
308+
}
308309

309310
run_task('pe_xl::puppet_runonce', $master_target)
310311
run_task('pe_xl::puppet_runonce', $all_targets - $master_target)

spec/docker/.dockerignore

+1
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
*.tar.gz

spec/docker/Dockerfile

+43
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,43 @@
1+
# In order to run this with systemd you must do one of the following
2+
# use --privileged when running the container
3+
# mount the volume when running the container -v /sys/fs/cgroup:/sys/fs/cgroup:ro
4+
# docker run –privileged -ti -v /sys/fs/cgroup:/sys/fs/cgroup:ro -p 2222:22
5+
FROM centos:7
6+
EXPOSE 22
7+
ENV LC_ALL="en_US.UTF-8" LANG="en_US.UTF-8" LANGUAGE="en_US.UTF-8"
8+
RUN echo "LANG=en_US.UTF-8" > /etc/locale.conf
9+
STOPSIGNAL SIGRTMIN+3
10+
ADD live_audit.sh /usr/bin/live_audit.sh
11+
ADD live_audit.service /etc/systemd/system/live_audit.service
12+
RUN chmod 644 /etc/systemd/system/live_audit.service && chmod 755 /usr/bin/live_audit.sh && \
13+
echo "root:test" | chpasswd; \
14+
useradd -m -s /bin/bash centos && echo "centos:test" | chpasswd;
15+
RUN yum -y install epel-release systemd rsync tree vim openssh openssh-server openssh-clients anacron sudo curl openssl
16+
RUN yum -y install inotify-tools && mkdir /root/bolt_scripts && yum clean all
17+
# remove any scripts that don't need to be run
18+
RUN (cd /lib/systemd/system/sysinit.target.wants/; for i in *; do [ $i == systemd-tmpfiles-setup.service ] || rm -f $i; done); \
19+
rm -f /lib/systemd/system/multi-user.target.wants/*; \
20+
rm -f /etc/systemd/system/*.wants/*; \
21+
rm -f /lib/systemd/system/local-fs.target.wants/*; \
22+
rm -f /lib/systemd/system/sockets.target.wants/*udev*; \
23+
rm -f /lib/systemd/system/sockets.target.wants/*initctl*; \
24+
rm -f /lib/systemd/system/basic.target.wants/*; \
25+
rm -f /lib/systemd/system/anaconda.target.wants/*; \
26+
rm -rf /var/cache/yum; \
27+
ln -s '/usr/lib/systemd/system/sshd.service' '/etc/systemd/system/multi-user.target.wants/sshd.service' && \
28+
ln -s '/etc/systemd/system/live_audit.service' '/etc/systemd/system/multi-user.target.wants/live_audit.service'
29+
ARG HOST="localhost"
30+
RUN mkdir -p /root/.puppetlabs/client-tools && \
31+
echo $' \n\
32+
{ \n\
33+
"puppetdb": { \n\
34+
"server_urls": "https://'$HOST$':8081", \n\
35+
"cacert": "/etc/puppetlabs/puppet/ssl/certs/ca.pem", \n\
36+
"cert": "/etc/puppetlabs/puppet/ssl/certs/'$HOST$'.pem", \n\
37+
"key": "/etc/puppetlabs/puppet/ssl/private_keys/'$HOST$'.pem" \n\
38+
} \n\
39+
} \n '\
40+
> /root/.puppetlabs/client-tools/puppetdb.conf
41+
#VOLUME [ “/sys/fs/cgroup” ]
42+
#CMD /bin/bash
43+
ENTRYPOINT [ "/sbin/init" ]

spec/docker/Dockerfile_bolt

+10
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
FROM puppet/puppet-bolt
2+
RUN apt-get update && apt-get install -y locales ssh sudo curl; \
3+
/opt/puppetlabs/bolt/bin/gem install bundler puppet-debugger -N -q
4+
ENV LC_ALL="en_US.UTF-8" LANG="en_US.UTF-8" LANGUAGE="en_US.UTF-8"
5+
RUN echo "LC_ALL=en_US.UTF-8" >> /etc/environment && \
6+
echo "en_US.UTF-8 UTF-8" >> /etc/locale.gen && \
7+
echo "LANG=en_US.UTF-8" > /etc/locale.conf && \
8+
locale-gen en_US.UTF-8
9+
CMD /bin/bash
10+
ENTRYPOINT [ "/opt/puppetlabs/bin/bolt" ]
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,129 @@
1+
version: "3"
2+
services:
3+
bolt:
4+
build:
5+
dockerfile: 'Dockerfile_bolt'
6+
context: ../
7+
image: pe-bolt
8+
hostname: bolter.puppet.vm
9+
container_name: bolter.puppet.vm
10+
volumes:
11+
- .:/app
12+
working_dir: /app
13+
depends_on:
14+
- pe_xl_core_0
15+
compiler1:
16+
restart: always
17+
depends_on:
18+
- pe_xl_core_0
19+
build:
20+
dockerfile: 'Dockerfile'
21+
context: ../
22+
entrypoint: /sbin/init
23+
image: pe-base
24+
privileged: true # required for systemd
25+
ports:
26+
- "22"
27+
- "8140"
28+
hostname: pe-xl-compiler-0.puppet.vm
29+
container_name: pe-xl-compiler-0.puppet.vm
30+
stop_signal: SIGRTMIN+3
31+
tmpfs:
32+
- /run
33+
- /tmp
34+
volumes:
35+
- '/sys/fs/cgroup:/sys/fs/cgroup:ro'
36+
pe_pdb:
37+
restart: always
38+
depends_on:
39+
- pe_xl_core_0
40+
build:
41+
dockerfile: 'Dockerfile'
42+
context: ../
43+
entrypoint: /sbin/init
44+
image: pe-base
45+
privileged: true # required for systemd
46+
ports:
47+
- "22"
48+
hostname: pe-xl-db-0.puppet.vm
49+
container_name: pe-xl-db-0.puppet.vm
50+
stop_signal: SIGRTMIN+3
51+
tmpfs:
52+
- /run
53+
- /tmp
54+
volumes:
55+
- '/sys/fs/cgroup:/sys/fs/cgroup:ro'
56+
pe_pdb-replica:
57+
restart: always
58+
depends_on:
59+
- pe_xl_core_0
60+
build:
61+
dockerfile: 'Dockerfile'
62+
context: ../
63+
entrypoint: /sbin/init
64+
image: pe-base
65+
privileged: true # required for systemd
66+
ports:
67+
- "22"
68+
hostname: pe-xl-db-1.puppet.vm
69+
container_name: pe-xl-db-1.puppet.vm
70+
stop_signal: SIGRTMIN+3
71+
tmpfs:
72+
- /run
73+
- /tmp
74+
volumes:
75+
- '/sys/fs/cgroup:/sys/fs/cgroup:ro'
76+
pe_xl_core_1:
77+
restart: always
78+
depends_on:
79+
- pe_xl_core_0
80+
build:
81+
dockerfile: 'Dockerfile'
82+
context: ../
83+
args:
84+
HOST: 'pe-xl-core-1.puppet.vm'
85+
entrypoint: /sbin/init
86+
image: pe-base
87+
privileged: true # required for systemd
88+
ports:
89+
- "22"
90+
- "8140"
91+
- "8443"
92+
- "8080"
93+
- "8081"
94+
- "4433"
95+
- "443"
96+
container_name: pe-xl-core-1.puppet.vm
97+
hostname: pe-xl-core-1.puppet.vm
98+
stop_signal: SIGRTMIN+3
99+
tmpfs:
100+
- /run
101+
- /tmp
102+
volumes:
103+
- '/sys/fs/cgroup:/sys/fs/cgroup:ro'
104+
pe_xl_core_0:
105+
restart: always
106+
build:
107+
dockerfile: 'Dockerfile'
108+
context: ../
109+
args:
110+
HOST: 'pe-xl-core-0.puppet.vm'
111+
entrypoint: /sbin/init
112+
image: pe-base
113+
privileged: true # required for systemd
114+
ports:
115+
- "22"
116+
- "8140"
117+
- "8443"
118+
- "8080"
119+
- "8081"
120+
- "4433"
121+
- "443"
122+
hostname: pe-xl-core-0.puppet.vm
123+
container_name: pe-xl-core-0.puppet.vm
124+
stop_signal: SIGRTMIN+3
125+
tmpfs:
126+
- /run
127+
- /tmp
128+
volumes:
129+
- '/sys/fs/cgroup:/sys/fs/cgroup:ro'
+17
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
---
2+
groups:
3+
- name: puppet-enterprise-nodes
4+
config:
5+
transport: ssh
6+
ssh:
7+
tmpdir: /root/bolt_scripts
8+
host-key-check: false
9+
user: root
10+
password: test
11+
tty: true
12+
targets:
13+
- pe-xl-compiler-0.puppet.vm
14+
- pe-xl-db-0.puppet.vm
15+
- pe-xl-db-1.puppet.vm
16+
- pe-xl-core-0.puppet.vm
17+
- pe-xl-core-1.puppet.vm
+12
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
1+
{
2+
"master_host": "pe-xl-core-0.puppet.vm",
3+
"puppetdb_database_host": "pe-xl-db-0.puppet.vm",
4+
"puppetdb_database_replica_host": "pe-xl-db-1.puppet.vm",
5+
"master_replica_host": "pe-xl-core-1.puppet.vm",
6+
"compiler_hosts": ["pe-xl-compiler-0.puppet.vm"],
7+
"console_password": "puppetlabs",
8+
"dns_alt_names": [ "puppet", "pe-xl-core-0.puppet.vm", "puppet-xl.vm" ],
9+
"version": "2018.1.13",
10+
"compiler_pool_address": "puppet-xl.vm",
11+
"stagingdir": "/downloads"
12+
}
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
{
2+
"master_host": "pe-xl-core-0.puppet.vm",
3+
"puppetdb_database_host": "pe-xl-db-0.puppet.vm",
4+
"puppetdb_database_replica_host": "pe-xl-db-1.puppet.vm",
5+
"master_replica_host": "pe-xl-core-1.puppet.vm",
6+
"compiler_hosts": ["pe-xl-compiler-0.puppet.vm"],
7+
"version": "2018.1.15",
8+
"stagingdir": "/downloads"
9+
}

0 commit comments

Comments
 (0)