Learn OpenShift: Deploy, build, manage, and migrate applications with OpenShift Origin 3.9

This architecture brings a lot of advantages to software development.

Some of the major advantages of containers are as follows:

As we mentioned already, Docker is the most popular container platform. It allows for creating, sharing, and running applications inside Docker containers. Docker separates running applications from the infrastructure. It allows you to speed up the application delivery process drastically. Docker also brings application development to an absolutely new level. In the diagram that follows, you can see a high-level overview of the Docker architecture:

Docker architecture

Docker uses a client-server type of architecture:

Docker uses three main components:

As mentioned in the previous section, Docker containers are secured and isolated from each other. In Linux, Docker containers use several standard features of the Linux kernel. This includes:

The beauty of Docker is that it leverages the aforementioned low-level kernel technologies, but hides all complexity by providing an easy way to manage your containers.

The container filesystem, used for every Docker image, is represented as a list of read-only layers stacked on top of each other. These layers eventually form a base root filesystem for a container. In order to make it happen, different storage drivers are being used. All the changes to the filesystem of a running container are done to the top level image layer of a container. This layer is called a Container layer. What it basically means is that several containers may share access to the same underlying level of a Docker image, but write the changes locally and uniquely to each other. This process is shown in the following diagram:

Docker layers

A Docker storage driver is the main component to enable and manage container images. Two main technologies are used for that—copy-on-write and stackable image layers. The storage driver is designed to handle the details of these layers so that they interact with each other. There are several drivers available. They do pretty much the same job, but each and every one of them does it differently. The most common storage drivers are AUFS, Overlay/Overlay2, Devicemapper, Btrfs, and ZFS. All storage drivers can be categorized into three different types:

As previously mentioned, a Docker image contains a number of layers that are combined into a single filesystem using a storage driver. The layers (also called intermediate images) are generated when commands are executed during the Docker image build process. Usually, Docker images are created using a Dockerfile, the syntax of which will be described later. Each layer represents an instruction in the image's Dockerfile.

Each layer, except the very last one, is read-only:

Docker image layers

A Docker image usually consists of several layers, stacked one on top of the other. The top layer has read-write permissions, and all the remaining layers have read-only permissions. This concept is very similar to the copy-on-write technology. So, when you run a container from the image, all the changes are done to this top writable layer. 

You can start a container from an image stored in a public registry. By default, the Docker daemon looks for and downloads Docker images from Docker Hub, which is a public registry provided by Docker. However, many vendors add their own public registries to the Docker configuration at installation time. For example, Red Hat has its own proven and blessed public Docker registry which you can use to pull Docker images and to build containers.  

Some organization or specific teams don't want to share their custom container images with everyone for a reason. They still need a service to share Docker images, but just for internal usage. In that case, a private registry service can be useful. A private registry can be installed and configured as a service on a dedicated server or a virtual machine inside your network.

You can easily install a private Docker registry by running a Docker container from a public registry image. The private Docker registry installation process is no different from running a regular Docker container with additional options.

A Docker registry is accessed via the Docker daemon service using a Docker client. The Docker command line uses a RESTful API to request process execution from the daemon. Most of these commands are translated into HTTP requests and may be transmitted using curl.

The process of using Docker registries is shown in the following section.

A developer can create a Docker image and put it into a private or public registry. Once the image is uploaded, it can be immediately used to run containers or build other images.

The Docker installation process is dependent on the particular operating system. In most cases, it is well described on the official Docker portal—https://docs.docker.com/install/. As a part of this book, we will be working with Docker software on CentOS 7.x. Docker installation and configuration on other platforms is not part of this book. If you still need to install Docker on another operating system, just visit the official Docker web portal.

Usually, the Docker node installation process looks like this:

Docker CE is available on CentOS 7 with standard repositories. The installation process is focused on the docker package installation:

# yum install docker -y
...
output truncated for brevity
...
Installed:
docker.x86_64 2:1.12.6-71.git3e8e77d.el7.centos.1
Dependency Installed:
...
output truncated for brevity
...

Once the installation is completed, you need to run the Docker daemon to be able to manage your containers and images. On RHEL7 and CentOS 7, this just means starting the Docker service like so:

# systemctl start docker
# systemctl enable docker
Created symlink from /etc/systemd/system/multi-user.target.wants/docker.service to /usr/lib/systemd/system/docker.service.

You can verify that your Docker daemon works properly by showing Docker information provided by the docker info command:

# docker info
Containers: 0
Running: 0
Paused: 0
Stopped: 0
Images: 0
...
output truncated for brevity
...
Registries: docker.io (secure)

Docker daemon configuration is managed by the Docker configuration file (/etc/docker/daemon.json) and Docker daemon startup options are usually controlled by the systemd unit named Docker. On Red Hat-based operating systems, some configuration options are available at /etc/sysconfig/docker and /etc/sysconfig/docker-storage. Modification of the mentioned file will allow you to change Docker parameters such as the UNIX socket path, listen on TCP sockets, registry configuration, storage backends, and so on.

The Docker daemon listens on unix:///var/run/docker.sock but you can bind Docker to another host/port or a Unix socket. The Docker client (the docker utility) uses the Docker API to interact with the Docker daemon.

The Docker client supports dozens of commands, each with numerous options, so an attempt to list them all would just result in a copy of the CLI reference from the official documentation. Instead, we will provide you with the most useful subsets of commands to get you up and running.

You can always check available man pages for all Docker sub-commands using:

$ man -k docker

You will be able to see a list of man pages for Docker and all the sub-commands available:

$ man docker
$ man docker-info
$ man Dockerfile

Another way to get information regarding a command is to use docker COMMAND --help:

# docker info --help
Usage: docker info
Display system-wide information
--help             Print usage

The docker utility allows you to manage container infrastructure. All sub-commands can be grouped as follows:

Now we want to check the images available on our local server. To do this, we can use the docker images command:

$ docker images

The output of the preceding command will be as shown in the following screenshot:

If we downloaded a wrong image, we can always delete it from the local server by using the docker rmi command: ReMove Image (RMI). In our case, we have two versions of the same image, so we can specify a tag for the image we want to delete:

$ docker rmi httpd:2.2.29

The output of the preceding command will be as shown in the following screenshot:

At this point, we have only one image left, which is httpd:latest:

$ docker images

The output of the preceding command will be as shown in the following screenshot:

The Docker CLI allows us to export and import Docker images and container layers using export/import or save/load Docker commands. The difference between save/load and export/import is that the first one works with images including metadata, but the export/import combination uses only container layers and doesn't include any image metadata information such as name, tags, and so on. In most cases, the save/load combination is more relevant and works properly for images without special needs. The docker save command packs the layers and metadata of all the chains required to build the image. You can then load this saved images chain into another Docker instance and create containers from these images.

The docker export will fetch the whole container, like a snapshot of a regular VM. It saves the OS, of course, but also any change a you made and any data file written during the container life. This one is more like a traditional backup:

$ docker save httpd -o httpd.tar

$ ls -l httpd.tar

To load the image back from the file, we can use the docker load command. Before we do that, though, let's remove the httpd image from the local repository first:

$ docker rmi httpd:latest

The output of the preceding command will be as shown in the following screenshot:

We verify that we do not have any images in the local repository:

 $ docker images
 REPOSITORY TAG IMAGE ID CREATED SIZE

Load the image file we previously saved with the docker save command. Like docker export and docker import, this command forms a pair with Docker save and thus is used for loading a saved container archive with all intermediate layers and metadata to the Docker cache:

$ docker load -i httpd.tar

The output of the preceding command will be as shown in the following screenshot:

Check the local docker images with docker image command:

$ docker images

The output of the preceding command will be as shown in the following screenshot:

Now we know how to search, pull, remove, save, load, and list available images. The last piece we are missing is how to push images back to Docker Hub or a private registry.

To upload an image to Docker Hub, we need to do a few tricks and follow these steps:

$ docker login
Username: #Enter your username here
Password: #Enter your password here
Login Succeeded

$ docker tag httpd:latest flashdumper/httpd:latest

$ docker push flashdumper/httpd:latest

The output of the preceding command will be as shown in the following screenshot:

Now the image is pushed to your Docker Hub and available for anyone to download. 

$ docker search flashdumper/*

The output of the preceding command will be as shown in the following screenshot:

You can check the same result using a web browser. If you go to https://hub.docker.com/ you should be able to see this httpd image available under your account:

Docker Hub account images

Run the docker ps command to show that there are no running containers:

$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES

Run docker ps -a to show both running and stopped containers:

$ docker ps -a

The output of the preceding command will be as shown in the following screenshot:

There are a few things to note here. The STATUS field says that container 5e3820a43ffc exited about one minute ago. In order to get container log information, we can use the docker logs command:

$ docker logs 5e3820a43ffc

The output of the preceding command will be as shown in the following screenshot:

The last message says caught SIGTERM, shutting down. It happened after we pressed Ctrl + C. In order to run a container in background mode, we can use the -d option with the docker run command:

$ docker run -d httpd
5d549d4684c8e412baa5e30b20697b72593d87130d383c2273f83b5ceebc4af3

It generates a random ID, the first 12 characters of which are used for the container ID. Along with the generated ID, a random container name is also generated.

Run docker ps to verify the container ID, name, and status:

$ docker ps

The output of the preceding command will be as shown in the following screenshot:

From the output, we can see that the container status is UP. Now we can execute some commands inside the container using the docker exec command with different options:

$ docker exec -i 00f343906df3 ls -l /
total 12
drwxr-xr-x. 2 root root 4096 Feb 15 04:18 bin
drwxr-xr-x. 2 root root 6 Nov 19 15:32 boot
drwxr-xr-x. 5 root root 360 Mar 6 21:17 dev
drwxr-xr-x. 42 root root 4096 Mar 6 21:17 etc
drwxr-xr-x. 2 root root 6 Nov 19 15:32 home
...
Output truncated for brevity
...

Option -i (--interactive) allows you to run a Docker without dropping inside the container. But we can easily override this behavior and enter this container by using -i and -t (--tty) options (or just -it):

$ docker exec -it 00f343906df3 /bin/bash
root@00f343906df3:/usr/local/apache2#

We should fall into container bash CLI. From here, we can execute other general Linux commands. This trick is very useful for troubleshooting. To exit the container console, just type exit or press Ctrl + D.

We can also stop and start running containers by running docker stop and docker start commands:

Enter the following command to stop the container:

$ docker stop 00f343906df3
00f343906df3 

Enter the following command to start the container:

$ docker start 00f343906df3
00f343906df3

In order to actually benefit from the container, we need to make it publicly accessible from the outside. This is where we will need to use the -p option with a few arguments while running the docker run command:

$ docker run -d -p 8080:80 httpd
3b1150b5034329cd9e70f90ee21531b8b1ab1d4a85141fd3a362cd40db80e193

Option -p maps container port 80 to your server port 8080. Verify that you have a httpd container exposed and a web server running:

$ curl localhost:8080
<html><body><h1>It works!</h1></body></html>

While the container is running, we can inspect its parameters by using the docker inspect command. The output is provided in JSON format and it gives us a very comprehensive output:

$ docker inspect 00f343906df3
[
   {
       "Id": "00f343906df3f26c24e02cd61d6a37bbc36106b3b0372073673c2983cb6f",
       ...
       output truncated for brevity
       ...
   }
]

$ docker rm 3b1150b50343
Error response from daemon: You cannot remove a running container 3b1150b5034329cd9e70f90ee21531b8b1ab1d4a85141fd3a362cd40db80e193. Stop the container before attempting removal or force remove

$ docker rm  -f 3b1150b50343

$ docker rm -f $(docker ps -qa)
830a42f2e727
00f343906df3
5e3820a43ffc
419e7ce2567e

$ docker ps  -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES

At runtime, environment variables are exposed to the application inside the container. You can set environment variables in a service's containers with the environment key, just like with docker run -e VARIABLE=VALUE. You can also pass environment variables from your shell straight through to a service's containers with the environment key by not giving them a value, just like with docker run -e VARIABLE.

Environment variables are used to set specific application parameters, like IP addresses, for a server to connect the database server address with login credentials.

Some container startup scripts use environment variables to perform the initial configuration of an application.

For example, a mariadb image is created to use several environment variables to start a container and create users/databases at the start time. This image uses the following important parameters, among others:

First, we can try to pull and start a mariadb container without specifying the password/user/database-related information. It will fail since the image expects the parameters. In this example, we are starting a container in the foreground to be able to see all error messages:

$ docker pull mariadb
latest: Pulling from docker.io/library/mariadb
       ...
       output truncated for brevity
       ...
Digest: sha256:d5f0bc88ba397233677ff75b7b1de693d5e84527ecf2b4f59adebf8d0bcac3c4

Now try to run mariadb container without any options and arguments.

$ docker run mariadb
error: database is uninitialized and password option is not specified
You need to specify one of MYSQL_ROOT_PASSWORD, MYSQL_ALLOW_EMPTY_PASSWORD and MYSQL_RANDOM_ROOT_PASSWORD

The docker run command failed because the MariaDB image initial startup script was not able to find the required variables. This script expects us to have at least the MariaDB root password to start a database server. Let's try to start a database container again by providing all required variables:

$ docker run -d --name mariadb -e MYSQL_ROOT_PASSWORD=password -e MYSQL_DATABASE=example -e MYSQL_USER=example_user -e MYSQL_PASSWORD=password mariadb
721dc752ed0929dbac4d8666741b15e1f371aefa664e497477b417fcafee06ce

Run the docker ps command to verify that the container is up and running:

$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
721dc752ed09 mariadb "docker-entrypoint.sh" 10 seconds ago Up 9 seconds 3306/tcp mariadb

The container was created successfully. Run the verification command to check that example_user has access to the example database:

$ docker exec -it mariadb mysql -uexample_user -ppassword example -e "show databases;"
+--------------------+
| Database           |
+--------------------+
| example            |
| information_schema |
+--------------------+

The startup script created a user named example_user with the password password as we specified in the environment variables. It also configured a password for the root user. The full list of MariaDB image variables you can specify is located at https://hub.docker.com/_/mariadb/.

Environment variables adjust settings for a single container. The same approach can be used to start a multi-tier application where one container or application works alongside the other:

Multi-tier application example

In a multi-tier application, both the application server container and database server container may need to share variables such as database login credentials. Of course, we can pass all database connectivity settings to the application container using environment variables. It is very easy to make a mistake while passing multiple -e options to the docker run command, and it is very time-consuming, not to mention that it is very ineffective. Another option is to use container IP addresses to establish connections. We can gather IP address information using docker inspect but it will be difficult to track this information in a multi-container environment.

This means that using environment variables is just not enough to build multi-tier applications where containers depend on each other.

Docker has a featured called linked containers to solve this problem. It automatically copies all environment variables from one container to another. Additionally, by linking containers, we can define environment variables based on the other container's IP address and exposed ports.

Using linked containers is done by simply adding the --link container:alias option to the docker run command. For example, the following command links to a container named MariaDB using the DB alias:

$ docker run --link mariadb:db --name my_application  httpd

The new my_application container will then get all variables defined from the linked container mariadb. Those variable names are prefixed by DB_ENV_ so as not to conflict with the new container's own environment variables.

Variables providing information about container IP addresses and ports are named according to the following scheme:

Continuing with the MariaDB image example, the application container would get the following variables:

If the linked container exposes multiple ports, each of them generates a set of environment variables.

Let's take an example. We will be creating a WordPress container which needs access to a database server. This integration will require shared database access credentials. The first step in creating this application is to create a database server:

$ docker rm -f $(docker ps -qa)
$ docker run -d --name mariadb -e MYSQL_ROOT_PASSWORD=wordpress -e MYSQL_DATABASE=wordpress -e MYSQL_USER=wordpress -e MYSQL_PASSWORD=password mariadb
221462288bc578511154fe79411de002e05f08642b63a72bc7a8f16f7102e52b

The next step is to run a WordPress container. In that command, we will link the wordpress container with the mariadb container:

$ docker run -d --name wordpress --link mariadb:mysql -p 8080:80 wordpress
Unable to find image 'wordpress:latest' locally
Trying to pull repository docker.io/library/wordpress ...
latest: Pulling from docker.io/library/wordpress
...
output truncated for brevity
...
Digest: sha256:670e4156377063df1a02f036354c52722de0348d46222ba30ef6a925c24cd46a
1f69aec1cb88d273de499ca7ab1f52131a87103d865e4d64a7cf5ab7b430983a

Let's check container environments with the docker exec command:

$ docker exec -it wordpress env|grep -i mysql
MYSQL_PORT=tcp://172.17.0.2:3306
MYSQL_PORT_3306_TCP=tcp://172.17.0.2:3306
MYSQL_PORT_3306_TCP_ADDR=172.17.0.2
MYSQL_PORT_3306_TCP_PORT=3306
MYSQL_PORT_3306_TCP_PROTO=tcp
...
output truncated for brevity
...

The general recommendation is that all Docker images should be built from a Dockerfile to create clean and proper image layers without unwanted temporary and log files, despite the fact that some vendors deliver their Docker images without an available Dockerfile . If there is a need to modify that existing image, you can use the standard docker commit functionality to convert an existing container to a new image.

As an example, we will try to modify our existing httpd container and make an image from it.

First, we need to get the httpd image:

$ docker pull httpd
Using default tag: latest
Trying to pull repository docker.io/library/httpd ...
latest: Pulling from docker.io/library/httpd
...
output truncated for brevity
...
Digest: sha256:6e61d60e4142ea44e8e69b22f1e739d89e1dc8a2764182d7eecc83a5bb31181e

Next, we need a container to be running. That container will be used as a template for a future image

$ docker run -d --name httpd httpd
c725209cf0f89612dba981c9bed1f42ac3281f59e5489d41487938aed1e47641

Now we can connect to the container and modify its layers. As an example, we will update index.html:

$ docker exec -it httpd /bin/sh
# echo "This is a custom image" > htdocs/index.html
# exit

Let's see the changes we made using the docker diff command. This command shows you all files that were modified from the original image. The output looks like this:

$ docker diff httpd
C /usr
C /usr/local
C /usr/local/apache2
C /usr/local/apache2/htdocs
C /usr/local/apache2/htdocs/index.html
...
output truncated for brevity
...

The following table shows the file states of the docker diff command:

In our case, docker diff httpd command shows that index.html was changed.

Create a new image from the running container:

$ docker commit httpd custom_image
sha256:ffd3a523f9848776d65de8302253de9dc78e4948a792569ee46fad5c099312f6

Verify that the new image has been created:

$ docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
custom_image latest ffd3a523f984 3 seconds ago 177.4 MB
docker.io/httpd latest 01154c38b473 2 weeks ago 177.4 MB

The final step is to verify that the image works properly:

$ docker run -d --name custom_httpd -p 80:8080 custom_image
78fc5731d62e5a6377a7de152c0ba25d350603e6d97fa26967e06a82c8257e71

$ curl localhost:8080
This is a custom image

Usually, those who use Docker containers expect to have a high-level of automation, and the docker commit command is difficult to automate. Luckily, Docker can build images automatically by reading instructions from a special file usually called a Dockerfile. A Dockerfile is a text document that contains all the commands a user can call on the command line to assemble an image. Using docker build, users can create an automated build that executes several command-line instructions in succession. On CentOS 7, you can learn a lot more using the Dockerfile built-in documentation page man Dockerfile.

A Dockerfile has a number of instructions that help Docker to build an image according to your requirements. Here is a Dockerfile example, which allows us to achieve the same result as in the previous section:

$ cat Dockerfile
FROM httpd
RUN echo "This is a custom image" > /usr/local/apache2/htdocs/index.html

Once this Dockerfile is created, we can build a custom image using the docker build command:

$ docker build -t custom_image2 .
Sending build context to Docker daemon 2.048 kB
Step 1 : FROM httpd
 ---> 01154c38b473
Step 2 : RUN echo "This is a custom image" > /usr/local/apache2/htdocs/index.html
 ---> Using cache
 ---> 6b9be8efcb3a
Successfully built 6b9be8efcb3a

$ docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
custom_image2 latest 6b9be8efcb3a 2 minutes ago 177.4 MB
custom_image latest ffd3a523f984 19 minutes ago 177.4 MB
docker.io/httpd latest 01154c38b473 2 weeks ago 177.4 MB

We can check the history of image modifications using docker history:

$ docker history custom_image2
IMAGE CREATED CREATED BY SIZE COMMENT
6b9be8efcb3a 21 hours ago /bin/sh -c echo "This is a custom image" > /u 23 B
01154c38b473 2 weeks ago /bin/sh -c #(nop) CMD ["httpd-foreground"] 0 B
...
output truncated for brevity
...

Note that a new layer, 6b9be8efcb3a, is added. This is where we change the content of the index.html file in comparison to the original httpd image.

Some Dockerfile instructions are shown in the table:

When the docker build command is run, Docker reads the provided Dockerfile from top to bottom, creating a separate layer for every instruction and placing it in the internal cache. If an instruction from Dockerfile is updated, it invalidates the respective caching layer and every subsequent one, forcing Docker to rebuild them when the docker build command is run again. Therefore, it's more effective to place the most malleable instructions at the end of Dockerfile, so that the number of invalidated layers is minimized and cache usage is maximized. For example, suppose we have a Dockerfile with the following contents:

$ cat Dockerfile
FROM centos:latest
RUN yum -y update
RUN yum -y install nginx, mariadb, php5, php5-mysql
RUN yum -y install httpd
CMD ["nginx", "-g", "daemon off;"]

In the example, if you choose to use MySQL instead of MariaDB, the layer created by the second RUN command, as well as the third one, will be invalidated, which for complex images means a noticeably longer build process.

Consider the following example. Docker includes images for minimal OSes. These base images can be used to build custom images on top of them. In the example, we will be using a CentOS 7 base image to create a web server container from scratch:

$ mkdir custom_project; cd custom_project

Then, we create a Dockerfile with the following content:

$ cat Dockerfile
FROM centos:7
RUN yum install httpd -y
COPY index.html /var/www/html/index.html
ENTRYPOINT ["/usr/sbin/httpd","-D","FOREGROUND"]

$ echo "A new cool image" > index.html

$ docker build -t new_httpd_image .
Sending build context to Docker daemon 3.072 kB
...
output truncated for brevity
...
Successfully built 4f2f77cd3026

$ docker history new_httpd_image
IMAGE CREATED CREATED BY SIZE COMMENT
4f2f77cd3026 20 hours ago /bin/sh -c #(nop) ENTRYPOINT ["/usr/sbin/htt 0 B
8f6eaacaae3c 20 hours ago /bin/sh -c #(nop) COPY file:318d7f73d4297ec33 17 B
e19d80cc688a 20 hours ago /bin/sh -c yum install httpd -y 129 MB
...
output truncated for brevity
...