Coffee Chat: Layers inside Docker Image!

👋 Hi, I’m Moby dock I’m the whale 🐳 that you see on the Docker logo.

Yes, I’ve a name! 😉

Please grab your coffee ☕ and stick with me for another five minutes. I’ll make sure that you receive something meaningful from our conversation!

So, here I’m- (Moby dock 🐳), talking to you about Layers in Docker and here’s our agenda:

Introduction: If you know me and what’s on my dock- please skip this!

Docker is a containerization platform that has become the de-facto standard for containerization in the datacenter.

Docker offers a simple way to package an application and its runtime dependencies into a single container; it also provides a runtime abstraction that enables the container to run across different versions of the Linux kernel.

When you compare VMs (Virtual Machine’s) with Docker, the advantage is you can quickly grab the specific parts you require. It’s like putting together the different pieces you need to create the perfect app to your specifications- no extra weight.

Docker images: do you know, they have their own pieces?

The first time when you pull an image from Docker Hub, you’ll have to wait for each pull to load. As you can see in following example- it has loaded 4 separate layers for Ubuntu image.

~ $ docker pull ubuntu
Using default tag: latest
latest: Pulling from library/ubuntu
d51af753c3d3: Pull completefc878cd0a91c: Pull complete6154df8ff988: Pull completefee5db0ff82f: Pull completeDigest: sha256:747d2dbbaaee995098c9792d99bd333c6783ce56150d1b11e333bbceed5c54d7
Status: Downloaded newer image for ubuntu:latest
docker.io/library/ubuntu:latest

The second time, magic does the trick and it will not load all those layers again. 🤡

~ $ docker pull ubuntu
Using default tag: latest
latest: Pulling from library/ubuntu
Digest: sha256:747d2dbbaaee995098c9792d99bd333c6783ce56150d1b11e333bbceed5c54d7
Status: Image is up to date for ubuntu:latest
docker.io/library/ubuntu:latest

Fun analogy- I’m sure you love pizza!

Docker images contain read-only layers, which means that once an image is created it is never modified.

🚀 No no, it’s not rocket science- it just has some math behind to reuse space.

I love this pizza analogy by Chloe Condon from codefresh.io and would like to use the same:

🍕 Let’s say we had a cheese pizza. We have a base layer of dough, tomato sauce, and cheese. Then I decide I want to have half of my pizza with pepperoni with it.

Would I bake another pizza from scratch, make new layers, and add pepperoni to that one?

Of course not! I’d just add a layer of pepperoni to half of my existing pizza. No need to go through the trouble of making a whole new base pie. As we make more changes, we create more layers.

Following are two figures that should simplify things:

Figure 1 shows an example of a container image- Ubuntu base OS. The image is a composition of four base Ubuntu layers.

docker layers

Figure 2 depicts the difference between an image and a running container. The running container is composed of 4 Ubuntu base layers, plus a web application and custom file layer. Note that each running container can have a different writable layer.

In other words, a running Docker container is an instantiation of an image.

Containers derived from the same image are identical to each other in terms of their application code and runtime dependencies. But unlike images, which are read-only, running containers include a writable layer (the container layer) on top of the read-only content. Runtime changes, including any writes and updates to data and files, are saved in the container layer. Thus, multiple concurrent running containers that share the same underlying image may have container layers that differ substantially.

When a running container is deleted, the writable container layer is also deleted and will not persist. The only way to persist changes is to do an explicit docker commit command prior to deleting the container. When you do a docker commit, the running container content, including the writable layer, is written into a new container image and stored to the disk. This becomes a new image distinct from the image by which the container was instantiated.

Example Dockerfile: learn by doing?

Note: The default docker images will show all top level images, their repository and tags, and their size.

Docker images have intermediate layers that increase reusability, decrease disk usage, and speed up docker build by allowing each step to be cached. These intermediate layers are not shown by default. --all , -a: Should be used to show all images.

We will assume for the sake of example assume that, each image adds one layer (not add confusion with intermediate layers, for now).

📓 I’ve two images (fluentd and nginx) already existing on my machine.

~ $ docker images
REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE
fluentd             <none>              ec77776ad237        9 days ago          44.9MB
nginx               latest              602e111c06b6        10 days ago         127MB

Let’s verify it with docker info as well:

~ $ docker info
Client:
 Debug Mode: false
Server:
 Containers: 4
  Running: 2
  Paused: 0
  Stopped: 2
 Images: 2 Server Version: 19.03.6

Alright, so this is telling us that we have 2 images on my machine and for sake of simplicity we take that as 2 layers. Great stuff!

Here’s a sample dockerfile:

  1 FROM ubuntu
  2 RUN apt-get update
  3 RUN apt-get install -y vim
  4 CMD [“echo”, “Hello, I'm Moby Dock!]

As always, we have our FROM instructions first. We’re gonna be basing this new image on ubuntu. We have got 2 RUNs and one CMD instruction.

Now, we could do something to take care of the number of layers our docker files create and one of them is to write effective RUN instructions.

The RUN instruction is not optimized here although, I’ve listed them in 2 lines for sake of simplicity. More could be read in the last section of this article. link here

We are about to build an image from this dockerfile. How many layers do you think this will add overall, as well as to the layers on top of ubuntu base image?

Let’s do a docker build and see!

$ docker build -t="sample-app-moby" .

Sending build context to Docker daemon  2.048kB
Step 1/4 : FROM ubuntulatest: Pulling from library/ubuntu
d51af753c3d3: Pull complete
fc878cd0a91c: Pull complete
6154df8ff988: Pull complete
fee5db0ff82f: Pull complete
Digest: sha256:747d2dbbaaee995098c9792d99bd333c6783ce56150d1b11e333bbceed5c54d7
Status: Downloaded newer image for ubuntu:latest
---> 1d622ef86b13

Step 2/4 : RUN apt-get update ---> Running in 3e61e5d088a0
Removing intermediate container 3e61e5d088a0
 ---> 8e60ea7a2a85

Step 3/4 : RUN apt-get install -y vim ---> Running in 1def19080dcd
Removing intermediate container 1def19080dcd
 ---> a5d4447b41ed

Step 4/4 : CMD [“echo”, “Hello, I'm Moby Dock!] ---> Running in 54b10ce7afd2
Removing intermediate container 54b10ce7afd2
 ---> 8e35d08eb177
Successfully built 8e35d08eb177
Successfully tagged sample-app-moby:latest

Here, overall 4 layers has been added:

8e35d08eb177 | a5d4447b41ed | 8e60ea7a2a85 | 1d622ef86b13.

$ docker images
REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE
sample-app-moby     latest              8e35d08eb177        2 minutes ago       163MB
fluentd             <none>              ec77776ad237        9 days ago          44.9MB
ubuntu              latest              1d622ef86b13        10 days ago         73.8MB
nginx               latest              602e111c06b6        10 days ago         127MB
$ docker image history 8e35d08eb177
IMAGE               CREATED             CREATED BY                                      SIZE                COMMENT
8e35d08eb177        2 minutes ago       /bin/sh -c #(nop)  CMD ["/bin/sh" "-c" "[“ec…   0Ba5d4447b41ed        2 minutes ago       /bin/sh -c apt-get install -y vim               67.9MB8e60ea7a2a85        2 minutes ago       /bin/sh -c apt-get update                       21.2MB1d622ef86b13        10 days ago         /bin/sh -c #(nop)  CMD ["/bin/bash"]            0B<missing>           10 days ago         /bin/sh -c mkdir -p /run/systemd && echo 'do7B
<missing>           10 days ago         /bin/sh -c set -xe   && echo '#!/bin/sh' > /811B
<missing>           10 days ago         /bin/sh -c [ -z "$(apt-get indextargets)" ]     1.01MB
<missing>           10 days ago         /bin/sh -c #(nop) ADD file:a58c8b447951f9e30…   72.8MB

📝 Note: It shows creation date for Ubuntu layer as 10 days ago, because that’s the official date when the image was updated on. (https://hub.docker.com/_/ubuntu/)

So, we have a total of 6 images now, that is addition of 4 to the already existing 2 images we had.

$ docker info
Client:
 Debug Mode: false
Server:
 Containers: 4
  Running: 2
  Paused: 0
  Stopped: 2
 Images: 6 Server Version: 19.03.6

Reducing the Number of Layers in an Image:

As discussed earlier, one important thing about writing Dockerfile is to take care of structure in which you mention the instructions as well. One of the ways to reduce the layers in the above Dockerfile example, would be to append and re-write the RUN command in an effective way.

RUN apt-get update && apt-get install -y vim

This was a quick read on layering, and hence if you want to read further on optimizing things. I’ve mentioned few links at the end of article.

📓📓 To recap- the layered file system allows multiple images and containers to share the same layers. And gives us following major advantages:

1. Faster image transfer.
2. Faster image build.

Top

HOME

References and motivation: https://docs.docker.com/engine/reference/commandline/images/

https://docs.docker.com/storage/storagedriver/

https://hackernoon.com/tips-to-reduce-docker-image-sizes-876095da3b34

https://blog.logrocket.com/reduce-docker-image-sizes-using-multi-stage-builds/


Written by@[Sachin Jha]
An old-school guy seeking new adventures :) When I’m not building cloud solutions @DigitalOcean, you can find me exploring the mountains. I love spending time outdoors. I’m a photographer and high altitude trekker.

GitHubTwitterFacebookLinkedIn