Docker: How to start a container on a specific node in a Swarm cluster?

If you are using a Docker Swarm cluster and are wondering how you can start a container on a specific node, this post is for you!

The short answer is to custom label the docker node(daemon):

[root@hdp1 ~]# cat /etc/sysconfig/docker 
OPTIONS="-H tcp://0.0.0.0:2375 --label nodename=n1.manfrix.com"
[root@hdp1 ~]# 

NOTE: If you want to know more about the "label" option of docker daemon, you can refer here.

Make sure you restart the docker service after the changes:

systemctl restart docker

Verify that the daemon is now started with a label:

[root@hdp1 ~]# ps -ef | grep docker
root      50961      1  0 20:38 ?        00:00:02 /usr/bin/docker daemon -H fd:// -H tcp://0.0.0.0:2375 --label nodename=n1.manfrix.com

Make sure you remember to re-join the nodes to the swarm cluster after you restarted docker:

docker run -d swarm join --addr=[IP of the node]:2375 etcd://[IP of etcd host]:[port]/[optional path prefix]

After you have labeled all the nodes (daemons), then you can proceed to test:

[root@hdp1 ~]# docker -H tcp://localhost:9999 run -d --name centos-1 -p 80 -e constraint:nodename==n3.manfrix.com centos /bin/bash
82d42f3052da181ebb876d79e2aeeb68787c17045c625367cced067107f3cb08
[root@hdp1 ~]# docker -H tcp://localhost:9999 run -d --name nginx-1 -p 80 -e constraint:nodename==n2.manfrix.com nginx
68664b5046b1dc031b015c9241a2f16f1e663f0b384d395d810d36b46f317839

For more information about Swarm node constraints, you can refer here.

Docker: /etc/default/docker or /etc/sysconfig/docker does not work anymore under systemd?

If you question why all of the sudden your OPTIONS under /etc/default/docker or /etc/sysconfig/docker no longer work, I can assure you that you are not alone!

Let's put it simply, it is due to the /usr/lib/systemd/system/docker.service shipped with newer version of Docker (most probably v1.7 and above).

The file looks like this now =>

[Unit]
Description=Docker Application Container Engine
Documentation=https://docs.docker.com
After=network.target docker.socket
Requires=docker.socket

[Service]
Type=notify
ExecStart=/usr/bin/docker daemon -H fd://
MountFlags=slave<
LimitNOFILE=1048576
LimitNPROC=1048576
LimitCORE=infinity

[Install]
WantedBy=multi-user.target

 You can see clearly that it no longer contains any environment variable for you to customize stuff.

To fix that, we need to create a systemd drop-in file for docker.service.

(1) Create a directory "/etc/systemd/system/docker.service.d" on ALL servers.

mkdir /etc/systemd/system/docker.service.d

(2)  In the directory, create a file - "local.conf" - using your favorite text editor with the following lines:

[Service]
EnvironmentFile=-/etc/sysconfig/docker
EnvironmentFile=-/etc/sysconfig/docker-storage
EnvironmentFile=-/etc/sysconfig/docker-network
ExecStart=
ExecStart=/usr/bin/docker daemon -H fd:// $OPTIONS \
      $DOCKER_STORAGE_OPTIONS \
      $DOCKER_NETWORK_OPTIONS \
      $BLOCK_REGISTRY \
      $INSECURE_REGISTRY

(3) Create or edit the /etc/sysconfig/docker or /etc/sysconfig/docker-storage or /etc/sysconfig/docker-network to specify your customized options.

Eg.
[/etc/sysconfig/docker]
OPTIONS="-H tcp://0.0.0.0:2375 --label nodename=node1.mytestmac.com"

(4) Reload systemd.

systemctl daemon-reload

(5) Verify that docker.service is now aware of its environment files.

systemctl show docker | grep -i env

Eg.
[root@hdp1 ~]# systemctl show docker | grep -i env
EnvironmentFile=/etc/sysconfig/docker (ignore_errors=yes)
EnvironmentFile=/etc/sysconfig/docker-storage (ignore_errors=yes)
EnvironmentFile=/etc/sysconfig/docker-network (ignore_errors=yes)
[root@hdp1 ~]# 

(6) Restart docker.

systemctl restart docker

(7) Verify the changes.

ps -ef | grep docker

Hope that helps to resolve your headaches! :)