Deploy apps with docker swarm

I received alert email that my website crushed down, after checking, I found mysql container is stoped..

I checked system log, found infos as below:

Jan 22 18:42:39 ip-172-31-28-84 kernel: Out of memory: Kill process 597 (mysqld) score 226 or sacrifice child
Jan 22 18:42:39 ip-172-31-28-84 kernel: Killed process 597 (mysqld) total-vm:1128616kB, anon-rss:228980kB, file-rss:0kB, shmem-rss:0kB

I think the process is killed by kernel for lack of memory, because the server only has 1GB memory ..

[root@ip-172-31-28-84 log]# free -h
              total        used        free      shared  buff/cache   available
Mem:           990M        559M         83M        113M        348M        133M
Swap:            0B          0B          0B

I restarted mysql container, and check containers’s status:

[root@ip-172-31-28-84 log]# docker stats --no-stream
CONTAINER           CPU %               MEM USAGE / LIMIT     MEM %               NET I/O             BLOCK I/O           PIDS
9e5a47485105        0.00%               28.66MiB / 990.8MiB   2.89%               90.9MB / 43.2MB     24.9MB / 0B         2
c9187825cc0c        0.00%               273.8MiB / 990.8MiB   27.63%              3.95GB / 1.02GB     11GB / 2.58MB       11
628e301d00a1        0.04%               217.9MiB / 990.8MiB   21.99%              10.4MB / 136MB      101MB / 363MB       31

there is no limitation on resources, so mysql will occupy more memory which caused being killed.

After thinking about this, I decided deploy by docker swarm which will start container if stoped, and also could restrict resources for every container.

1.init docker swarm on single server

docker swarm init

2.modify blog-compose.yml to support swarm, please follow gist

https://gist.githubusercontent.com/hongmengwang/c5ca0368f5de15a612972c4bb676d409/raw/d8d706bb42769f20506d00f01603f34686b4fac9/blog-compose.yml

3.deploy service

docker stack deploy -c blog-compose.yml blog

4.check container status

[root@ip-172-31-28-84 docker]# docker stack services blog
ID                  NAME                MODE                REPLICAS            IMAGE               PORTS
0l68syg6q1bi        blog_nginx          replicated          1/1                 nginx:1.13.8        *:80->80/tcp,*:443->443/tcp
cx82xalbzdzu        blog_wordpress      replicated          1/1                 wordpress:4.9.1     
xulj5sbkbapb        blog_mysql          replicated          1/1                 mysql:5.7

5.check container stats

[root@ip-172-31-28-84 docker]# docker stats --no-stream
CONTAINER           CPU %               MEM USAGE / LIMIT   MEM %               NET I/O             BLOCK I/O           PIDS
08bc88c00f0c        0.04%               189.7MiB / 250MiB   75.86%              70.5kB / 1.02MB     14MB / 13.9MB       30
64d37b150392        0.00%               29.02MiB / 50MiB    58.05%              12.6kB / 14.7kB     1.24MB / 0B         2
f33ecf2c045e        0.00%               92.32MiB / 300MiB   30.77%              1.03MB / 76.8kB     27.8MB / 0B         9

The memory of each container is restricted, it will not occupy more memory than limitation, I will keep on watching to see if works well.