Binding a host filesystem within containers

Previously, we used a third docker-compose configuration file to specify bindings so that our source code directory would be overlaid within the container (in place of the app's home directory). We will do the same for the latest incarnation of our Docker Compose setup.

We first create a docker-compose-dev.yml file:

version: '3'

services:

  publisher:

    volumes:

      - ./publisher:/home/app

  subscriber:

    volumes:

      - ./subscriber:/home/app

This override file simply maps the publisher and subscriber source code directory over /home/app in the related container. Now, we can freely edit sources on the host and, thanks to nodemon, our changes will take effect almost immediately within the running containers. There is no need to stop, rebuild, or restart any containers.

Unfortunately, docker-compose has no facility to remove options using inheritance; we can only modify existing ones or add new ones. If we could remove options, we would bind the source in our docker-compose.override.yml file and remove them in a docker-compose-production.yml file. This would allow us to use the short docker-compose up form for development and to use a command line with three -f switches for production. This would be handy because we would use development most of the time and rarely use production.

As it is, we must specify the three -f switches:

% docker-compose -f docker-compose.yml -f docker-compose.override.yml -f docker-compose-dev.yml up

There are other uses for volumes, which we will explore.

Optimizing our container size

We can examine our container images using the docker images command:

% docker images | grep pub

chapter4_publisher                latest              15f3a84d348d        24 minutes ago      987MB

As you can see, our publisher image is 987 megabytes! All that for an almost-250-line JavaScript program. We can try to shrink this size by moving our node_modules directory out of the container and into a named volume. This will also speed up the building of our container since node_modules will be persisted in this named volume from build to build, and using the yarn command to install the modules will only install anything that is new.

Note

We renamed the Dockerfile to Dockerfile.chapter3 in the publisher/ directory.  The new Dockerfile has been modified to build a very small image.

A smaller image can be created by optimizing our Dockerfile. What we're going to do is build a base image and our result image. The base image will have node_modules installed.  The base image is only rebuilt when something changes that requires one of its layers to be rebuilt.  

Let's look at an optimized Dockerfile for the publisher:

FROM node:12-alpine

We inherit from the alpine OS node v12 image. This image is much lighter than the Debian flavor default node container:

ENV TZ=America/Los_Angeles

WORKDIR /home/app

# add a user - this user will own the files in /home/app

RUN adduser -S app

ENV HOME=/home/app

COPY . /home/app

The resulting image is built without installing or updating node_modules. We will install the modules in another step. This saves us from having to use yarn install every time we build our container:

CMD  ["yarn", "start"]

We use yarn start to launch our publisher app.

After we run docker-compose build publisher, we can see we have greatly reduced the size of our container!

Before our optimizations, the container was 987 megabytes. After the optimizations, 89.5 megabytes, which is almost a 900-megabyte reduction:

# docker images | grep pub

chapter4_publisher                   latest              080efb97e0d3        About a minute ago   89.5MB

We still need to install our node_modules/ modules, which will be done within a named volume and defined in the docker-compose-overrides.yml file. This is done once, and then again only if you add packages to the packages.json file in the publisher/ directory:

# docker-compose run publisher yarn install

This command installs the node_modules/ packages using yarn install within the publisher container. The named volume is mounted correctly because it is specified within the docker-compose configuration (.yml) files.

Note

We did not optimize the subscriber build.

We can verify that the volume was created and does contain the installed node_modules modules by examining the _data directory of our volume, which on Linux should be in /var/lib/docker/volumes:

# cd /var/lib/docker/volumes/

# ls -1 chapter4_node_modules_publisher/_data/

abbrev

accepts

ajv

ansi-align

ansi-regex

ansi-styles

anymatch

The location of the volumes is significantly different for macOS.  You will need to use the following command to get a shell in the Linux virtual machine that is running Docker:

# screen ~/Library/Containers/com.docker.docker/Data/vms/0/tty

You might have to hit ^C a few times to get a shell prompt. This prompt is a shell running in the virtual machine. Within the virtual machine, the volume for the node_modules/ directory in the container is at /var/lib/docker/volumes, as with Docker on Linux.

We can see the speedup of our build. The initial build of the publisher, after completely removing all of the images from the system, takes around 16 seconds:

# time docker-compose build publisher

Successfully built e50ec5f4d53b

Successfully tagged chapter4_publisher:latest

docker-compose build publisher  0.36s user 0.09s system 2% cpu 16.187 total

A subsequent build without node_modules installed takes around a half a second:

# time docker-compose build publisher

Successfully tagged chapter4_publisher:latest

docker-compose build publisher  0.34s user 0.08s system 74% cpu 0.568 total

After editing index.js and doing a rebuild, it takes less than 1 second:

# time docker-compose build publisher

Successfully tagged chapter4_publisher:latest

docker-compose build publisher  0.34s user 0.08s system 49% cpu 0.842 total

As you can see, we were able to reduce the size and build time of our containers!

Using the build.sh script

There is a build.sh script provided in the chapter4/ directory of the GitHub repository. It just contains a few lines of actual shell commands:

#!/bin/sh

# build.sh

# build publisher and subscriber and install node_modules in each

docker-compose build --force-rm --no-cache

docker-compose run publisher yarn install

docker-compose run subscriber yarn install

The build.sh script builds all five containers and runs yarn install in both the publisher and subscriber containers to install the node_modules modules in their respective named volumes. The command-line switches to the docker-compose build command are as follows:

• --force-rm: Forces Docker to remove all the intermediate container images as it builds
• --no-cache: Forces Docker to use no cached/downloaded/built versions of anything

You can drop these two switches to greatly improve the build speed. They are provided here to demonstrate a way of forcibly rebuilding everything from scratch.

That's a decent overview of Docker Compose. It is one of the first, if not the first, composition tools for describing, building, and running Docker applications. But there are also other alternatives out there.