Introduction
Table of Contents
We were all wondering by now how to create our own images. All the time we are working with pre-built images. In this article we will cover subjects like how to create your own custom image and multi-tier applications which runs as a service. Talking about multi-tier we will show you some cool features which you can use when building multi-tier apps. So let’s start and create Docker image.
Docker build image
There are two ways of creating custom images: commit operation inside container itself and Dockerfile. First option is not recommended because you rely on parent image and already prebuilt container. In other words you run the container, did some changes and run the commit command. It is not so happy solution if you want clean image which was built from beginning.
Dockerfile
Create Docker image with Dockerfile is preferred option when you want to build your own images. Focus is on quality parent image – maintainability is then very easy especially if parent image is kept clean by its author. Motivation of building custom images include: adding runtime libraries and SSL certifications. Changing the Dockerfile can trigger updates like removal of unwanted libraries. It is not possible to use inline comments and keywords are not word sensitive.
Base or parent image?
Parent image is prepacked image which is source on which other layers are built.
For example, RHEL or Centos have their base images which are used in first line of Dockerfile with word FROM:
#Comment
FROM Centos8:8.1
…..
Base image has no parent image in FROM but uses minimal image that is required to build container. It uses FROM scratch directive:
#Comment
FROM scratch
Scratch base image can be found on Public Docker repository – Docker Hub.
Steps to follow
Create working directory
Create Dockerfile specs
Docker build image with Docker command
Dockerfile format
# Comment
INSTRUCTION arguments
Example (Apache web server running on CentOS)
| #This is a comment (1)
FROM centos:latest (2) LABEL decription “Apache server” (3) RUN yum install -y httpd; yum clean all; systemctl enable httpd.service (4) COPY index.html /var/www/html/ (5) EXPOSE 80 (6) ENV APACHEVERSION “1” (7) ENTRYPOINT [“/usr/sbin/httpd”] (8) CMD [“-D”, “FOREGROUND”] (9) |
- This is comment
- Parent container to build you custom image is latest version of Centos
- This is just metadata describing image
- RUN command executes command in /bin/sh shell adding extra layer above first layer of parent image. Notice that there are three commands separated by semicolon (;). In this way you will avoid creating two additional layers with RUN command. This is recommended way of creating custom images and avoiding extra layers and performance issues.
- COPY command copies local file which is located on host to new image.
- EXPOSE command exposes apache server port 80 to host port 80 for service to be visible to external world.
- Set environment variable – APACHEVERSION.
- ENTRYPOINT defines which command will be executed by default when container starts. By default command /bin/sh –c. You want httpd command to start immediately.
- CMD provides arguments for the ENTRYPOINT instruction
Steps
Create working directory Dockerfile under your home path. Inside directory create Dockerfile with content above. Inside same directory create another file index.html with following content:
index.html
| <!doctype html>
<html> <head> <title>Welcome to apache server!</title> </head> <body> <p>Hi! I am apache server!</p> </body> </html> |
Dockerfile
| FROM centos:latest (2)
LABEL decription “Apache server” (3) RUN yum install -y httpd; yum clean all; systemctl enable httpd.service (4) COPY index.html /var/www/html/ (5) EXPOSE 80 (6) ENV APACHEVERSION “1” (7) ENTRYPOINT [“/usr/sbin/httpd”] (8) CMD [“-D”, “FOREGROUND”] (9) |
Run the Docker build command. Parameter –f indicated which file to use. Parameter –t will name the image because you can’t name image inside Dockerfile. Dot (.) tells Docker to search for file inside current directory:
| [root@swmanager dockerfiles]# docker build -f Dockerfile –t apacheserver .
Sending build context to Docker daemon 3.072kB Step 1/8 : FROM centos:latest Removing intermediate container 094156371bd1 —> f4db33717fe8 Step 4/8 : COPY index.html /var/www/html/ —> 29536db0b52b Step 5/8 : EXPOSE 80 —> Running in cea0bbe2bb13 Removing intermediate container cea0bbe2bb13 —> b52bb72e819d Step 6/8 : ENV APACHEVERSION “1” —> Running in 3b2353124159 Removing intermediate container 3b2353124159 —> a5579f1922d7 Step 7/8 : ENTRYPOINT [“/usr/sbin/httpd”] —> Running in 686bc014a689 Removing intermediate container 686bc014a689 —> c0f10b14cb95 Step 8/8 : CMD [“-D”, “FOREGROUND”] —> Running in ee59e916572f Removing intermediate container ee59e916572f —> dd803595bc27 |
If you have latest CentOS image locally, Docker will use it. Otherwise it will try to fetch it from Docker Hub site.
Check that both images are now on your local system:
| [root@swmanager dockerfiles]# docker images
REPOSITORY TAG IMAGE ID CREATED SIZE apacheserver latest dd803595bc27 17 seconds ago 237MB centos latest 831691599b88 5 weeks ago 215MB |
Run the apache server container from image:
| [root@swmanager dockerfiles]# docker run -dit -p 80:80 apacheserver
fb098d560c26fb97b74268b4e96969c611a4002940f38e6896930ff42397137b |
Check that container is running:
| [root@swmanager dockerfiles]# docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES b921ced4b400 apacheserver “/usr/sbin/httpd -D …” 3 seconds ago Up 1 second 0.0.0.0:8080->80/tcp ecstatic_chaplygin |
Browse the home page (it should display message from custom index.html above):
| [root@swmanager dockerfiles]# curl 127.0.0.1:8080
<!doctype html> <html> <head> <title>Welcome to apache server!</title> </head> <body> <p>Hi! I am apache server!</p> </body> </html> |
Multi-container applications
In multi-container world you can merge two or more containers into the one stack. Imagine classic three layer stack with front-end, business and database layer. You would normally start these services individually, configuring their endpoints and establishing communication between layers. Here you can define which type of containers will be part of multi-container world, network, storage, ports, environment variables and many more. You build multi-tier with docker compose. But you can always start containers individually with Docker run but this is more convenient way because you have dynamic file where you put all configuration.
Due its dynamic nature, multi-containers bring challenge in the form of IP addressing. Each time container is stopped and started there is no guarantee that it will get the same IP address. That’s way it is important to establish DNS services. Remember the network part where containers which use default bridge network do not have DNS services. Only containers with custom user network use built-in DNS server. Docker compose by default will always create custom network.
Docker compose
Compose is a tool for defining and running multi-container Docker applications. Compose is software as well as YAML file to configure your application layers. Dockerfile can be used in production, staging, development, testing, as well as CI workflows.
To build multi-tier container you need:
1) Dockerfile
2) Docker-compose file
3) Docker-compose up command to start building
| version: “3”
services: app: ports: – “80:80” db: image: mysql ports: – “3306:3306” |
Networking
Building services with Docker compose install default network if not specified. Default network is built from scratch and includes project name (directory where Docker compose file is located), underscore and word default: projectname_default. In this way all containers are reachable on that network by hostname defined in file, there is no firewall between them. You can always use pre-existing network.
Docker environment variables
Docker environment variables is just one of the options you can put in Docker compose file. There are various way to set variables but two main options are direct assignment or assignment by file:
Direct assignment:
web:
environment:
– DEBUG=1
Assignment by file:
web:
env_file:
– web-variables.env
Assignment during docker run:
Docker-compose run -e DEBUG=1 web python console.py
Multiple Compose files
If you use different environments and workloads you can use multiple Compose files. By default docker Compose runs two files: docker-compose.yml and docker-compose.override.yml. First one contains base configuration and second one includes specific configured based on environment. By convention, the docker-compose.yml contains your base configuration. -f option to specify the list of files.
Extending services
If your environment has more services that have common set of configuration data, you can use feature which is named extending services. You can set options in one place and use it anywhere. Your shared configuration is some kind of template.
Example
| web:
extends: file: common.yml service: app common.yml: app: build: . ports: – “5000:5000” volumes: – “/var/lib/data” |
This instructs web service to use all configuration defined by common.yml file.
Example
In the following example you will learn how to deploy multi-tier application which consists of two images:
- automation_db (simple web application coded in Python to add,delete,update data)
- mysql:5.7 (mysql database which holds records about data)
Image automation_db was created with following Dockerfile:
| FROM python:alpine3.7
COPY . /app WORKDIR /app RUN pip install -r requirements.txt EXPOSE 5000 ENTRYPOINT [ “python” ] CMD [ “app/__init__.py” ] |
Docker file builds custom image from parent image python: alpine3.7. Copies app directory with source code and set app as working directory. After it installs all requirements which are defined in requirements.txt file. Exposes port 5000 to outside and defines app/__init__.py as starting point when run image.
Docker compose file created service and merged two container together:
| version: “2”
services: app: image: automation:latest links: – db ports: – “5000:5000” db: image: mysql:5.7 ports: – “3306:3306” environment: MYSQL_ROOT_PASSWORD: password MYSQL_DATABASE: automation_db volumes: – /mnt/mysql/mysql57:/var/lib/mysql |
File consists of two containers: app and db. App container uses previously build image which we called automation. It links to db container and exposes port 5000 to outside.
Second container -db uses clean mysql: 5.7 image and exposes port 3306 to outside. There are two environment variables which are set to successfully start mysql:
MYSQL_ROOT_PASSWORD: password (defines root password)
MYSQL_DATABASE: automation_db (defines pre-defined database)
Last entry defines volume path where mysql container reads data. Path already has automation_db database created.
Two things to consider:
What about network?
When you don’t specify network in docker compose file, Docker compose will create custom network. For example if docker-compose.yml file is in /home/john/ directory, custom network john_default will be created. Both containers will join this network and each container can now look up the hostname web or db and get back the appropriate container’s IP address. There is no firewall between containers.
Link keyword in docker compose file
Link keyword has two functions: dependency and extra alias by which service is reachable. In our example, for app to start successfully it is mandatory to have running database. In this case first database is up and then app is connecting to database. It is worth to mention that compose does not wait till container is ready, only running. For mysql full availability you have to define extra logic. Second function is extra alias which is just one way for services to reach itself by names.
Proof of concept
1 Download two containers to your local system: automation
and mysql data files.
2. Upload two files to your test Linux machine with Docker installed
3. Download the mysql: 5.7 container to your local repository:
| docker pull mysql:5.7 |
4. Unzip mysql data to /mnt/mysql directory
| [root@swmanager tmp]# mkdir /mnt/mysql
[root@swmanager tmp]# unzip mysql57.zip -d /mnt/mysql Archive: mysql57.zip creating: /mnt/mysql/mysql57/ extracting: /mnt/mysql/mysql57/auto.cnf creating: /mnt/mysql/mysql57/automation_db/ inflating: /mnt/mysql/mysql57/automation_db/db.opt inflating: /mnt/mysql/mysql57/automation_db/esxi.frm inflating: /mnt/mysql/mysql57/automation_db/esxi.ibd inflating: /mnt/mysql/mysql57/ca-key.pem inflating: /mnt/mysql/mysql57/ca.pem inflating: /mnt/mysql/mysql57/client-cert.pem inflating: /mnt/mysql/mysql57/client-key.pem inflating: /mnt/mysql/mysql57/ibdata1 inflating: /mnt/mysql/mysql57/ib_buffer_pool inflating: /mnt/mysql/mysql57/ib_logfile0 inflating: /mnt/mysql/mysql57/ib_logfile1 creating: /mnt/mysql/mysql57/mysql/ |
Data will be stored in following path: /mnt/mysql/mysql57
5. Load the automation_db image:
| [root@swmanager tmp]# docker load -i automation.tar
629164d914fc: Loading layer [==================================================>] 4.464MB/4.464MB 50f8b07e9421: Loading layer [==================================================>] 838.7kB/838.7kB 9b77965e1d3f: Loading layer [==================================================>] 73.31MB/73.31MB 88e61e328a3c: Loading layer [==================================================>] 4.608kB/4.608kB 5fa31f02caa8: Loading layer [==================================================>] 6.453MB/6.453MB f5b2455b79ed: Loading layer [==================================================>] 96.77kB/96.77kB c5a6e8eabb7a: Loading layer [==================================================>] 28.24MB/28.24MB Loaded image: automation:latest |
6. Load the mysql container
| [root@swmanager tmp]# docker load -i mysql.tar
95ef25a32043: Loading layer [==================================================>] 72.49MB/72.49MB 492e36400248: Loading layer [==================================================>] 338.4kB/338.4kB b279ae737dff: Loading layer [==================================================>] 9.539MB/9.539MB 10263d924b2a: Loading layer [==================================================>] 4.2MB/4.2MB ec9e20671283: Loading layer [==================================================>] 1.536kB/1.536kB 7dbd11e8dc05: Loading layer [==================================================>] 53.75MB/53.75MB 4fb0c46aefac: Loading layer [==================================================>] 6.656kB/6.656kB 49368a7f4157: Loading layer [==================================================>] 3.584kB/3.584kB fb6af29036b4: Loading layer [==================================================>] 313.4MB/313.4MB f803b994f6aa: Loading layer [==================================================>] 16.38kB/16.38kB a52072ac0f07: Loading layer [==================================================>] 1.536kB/1.536kB |
7. Check that images are stored on your local system:
| REPOSITORY TAG IMAGE ID CREATED SIZE
automation latest a24edd32a29c 36 minutes ago 108MB mysql 5.7 8679ced16d20 7 days ago 448MB |
8. Install docker-compose
| [root@swmanager ~]# curl -L “https://github.com/docker/compose/releases/download/1.26.2/docker-compose-$(uname -s)-$(uname -m)” -o /usr/local/bin/docker-compose
% Total % Received % Xferd Average Speed Time Time Current Dload Upload Total Spent Left Speed 100 638 100 638 0 0 2133 0 –:–:– –:–:– –:–:– 2133 100 11.6M 100 11.6M 0 0 570k 0 0:00:20 0:00:20 –:–:– 700k |
Add permissions:
| [root@swmanager ~]# chmod +x /usr/local/bin/docker-compose |
9 . Start docker compose in directory where Docker-compose file is:
| [root@swmanager ~] docker-compose up
Creating network “root_default” with the default driver Creating root_db_1 … done Creating root_app_1 … done Attaching to root_db_1, root_app_1 db_1 | 2020-08-04 19:52:50+00:00 [Note] [Entrypoint]: Entrypoint script for MySQL Server 5.7.31-1debian10 started. db_1 | 2020-08-04 19:52:51+00:00 [Note] [Entrypoint]: Switching to dedicated user ‘mysql’ db_1 | 2020-08-04 19:52:51+00:00 [Note] [Entrypoint]: Entrypoint script for MySQL Server 5.7.31-1debian10 started. db_1 | 2020-08-04 19:52:51+00:00 [Note] [Entrypoint]: Initializing database files db_1 | 2020-08-04T19:52:51.355285Z 0 [Warning] TIMESTAMP with implicit DEFAULT value is deprecated. Please use –explicit_defaults_for_timestamp server option (see documentation for more details). db_1 | 2020-08-04T19:52:52.116427Z 0 [Warning] InnoDB: New log files created, LSN=45790 db_1 | 2020-08-04T19:52:52.311565Z 0 [Warning] InnoDB: Creating foreign key constraint system tables. db_1 | 2020-08-04T19:52:52.461787Z 0 [Warning] No existing UUID has been found, so we assume that this is the first time that this server has been started. Generating a new UUID: 1504b770-d68c-11ea-91b3-0242ac130002. db_1 | 2020-08-04T19:52:52.479514Z 0 [Warning] Gtid table is not ready to be used. Table ‘mysql.gtid_executed’ cannot be opened. db_1 | 2020-08-04T19:52:56.696757Z 0 [Warning] CA certificate ca.pem is self signed. db_1 | 2020-08-04T19:52:57.975162Z 1 [Warning] root@localhost is created with an empty password ! Please consider switching off the –initialize-insecure option. app_1 | * Serving Flask app “__init__” (lazy loading) app_1 | * Environment: production app_1 | WARNING: This is a development server. Do not use it in a production deployment. app_1 | Use a production WSGI server instead. app_1 | * Debug mode: on app_1 | * Running on http://0.0.0.0:5000/ (Press CTRL+C to quit) app_1 | * Restarting with stat app_1 | * Debugger is active! app_1 | * Debugger PIN: 213-953-578 |
10. Verify that both containers are up and running:
| [root@swmanager ~]# docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 0a4845b3b33f automation:latest “python app/__init__…” 5 days ago Up 3 minutes 0.0.0.0:5000->5000/tcp root_app_1 0788f12440c9 mysql:5.7 “docker-entrypoint.s…” 5 days ago Up 3 minutes 0.0.0.0:3306->3306/tcp, 33060/tcp root_db_1 |
11. Start your favorite browser from host machine and enter ip address of virtual machine which is hosting containers: IPADDRESS: 5000. If you have installed docker on host just enter: 127.0.0.1:5000:
Image 1 Web App Frontend
12. Select ESXI and if database connection is successful you should see following screen:
Image 2 Web App Database Connection Test