May 05, 2020
👋 Hi, I’m Moby dock I’m the whale 🐳 that you see on the Docker logo.
Yes, I’ve a name! 😉
Please grab your coffee ☕ and stick with me for another five minutes. I’ll make sure that you receive something meaningful from our conversation!
So, here I’m- (
Moby dock 🐳), talking to you about
Layers in Docker and here’s our agenda:
Docker is a containerization platform that has become the de-facto standard for containerization in the datacenter.
Docker offers a simple way to package an application and its runtime dependencies into a single container; it also provides a runtime abstraction that enables the container to run across different versions of the Linux kernel.
When you compare VMs (Virtual Machine’s) with Docker, the advantage is you can quickly grab the specific parts you require. It’s like putting together the different pieces you need to create the perfect app to your specifications- no extra weight.
The first time when you pull an image from Docker Hub, you’ll have to wait for each pull to load. As you can see in following example- it has loaded 4 separate layers for Ubuntu image.
~ $ docker pull ubuntu Using default tag: latest latest: Pulling from library/ubuntu d51af753c3d3: Pull completefc878cd0a91c: Pull complete6154df8ff988: Pull completefee5db0ff82f: Pull completeDigest: sha256:747d2dbbaaee995098c9792d99bd333c6783ce56150d1b11e333bbceed5c54d7 Status: Downloaded newer image for ubuntu:latest docker.io/library/ubuntu:latest
The second time,
magic does the trick and it will not load all those layers again. 🤡
~ $ docker pull ubuntu Using default tag: latest latest: Pulling from library/ubuntu Digest: sha256:747d2dbbaaee995098c9792d99bd333c6783ce56150d1b11e333bbceed5c54d7 Status: Image is up to date for ubuntu:latest docker.io/library/ubuntu:latest
Docker images contain read-only layers, which means that once an image is created it is never modified.
🚀 No no, it’s not rocket science- it just has some math behind to reuse space.
I love this pizza analogy by Chloe Condon from
codefresh.io and would like to use the same:
🍕 Let’s say we had a cheese pizza. We have a base layer of dough, tomato sauce, and cheese. Then I decide I want to have half of my pizza with pepperoni with it.
Would I bake another pizza from scratch, make new layers, and add pepperoni to that one?
Of course not! I’d just add a layer of pepperoni to half of my existing pizza. No need to go through the trouble of making a whole new base pie. As we make more changes, we create more
Figure 1 shows an example of a container image- Ubuntu base OS. The image is a composition of four base Ubuntu layers.
Figure 2 depicts the difference between an image and a running container. The running container is composed of 4 Ubuntu base layers, plus a web application and custom file layer. Note that each running container can have a different writable layer.
In other words, a running Docker container is an instantiation of an image.
Containers derived from the same image are identical to each other in terms of their application code and runtime dependencies. But unlike images, which are read-only, running containers include a writable layer (the container layer) on top of the read-only content. Runtime changes, including any writes and updates to data and files, are saved in the container layer. Thus, multiple concurrent running containers that share the same underlying image may have container layers that differ substantially.
When a running container is deleted, the writable container layer is also deleted and will not persist. The only way to persist changes is to do an explicit
docker commit command prior to deleting the container. When you do a
docker commit, the running container content, including the writable layer, is written into a new container image and stored to the disk. This becomes a new image distinct from the image by which the container was instantiated.
Note: The default docker images will show all top level images, their repository and tags, and their size.
Docker images have intermediate layers that increase reusability, decrease disk usage, and speed up docker build by allowing each step to be cached. These intermediate layers are not shown by default.
--all , -a: Should be used to show all images.
We will assume for the sake of example assume that, each image adds one layer (not add confusion with intermediate layers, for now).
📓 I’ve two images (fluentd and nginx) already existing on my machine.
~ $ docker images REPOSITORY TAG IMAGE ID CREATED SIZE fluentd <none> ec77776ad237 9 days ago 44.9MB nginx latest 602e111c06b6 10 days ago 127MB
Let’s verify it with
docker info as well:
~ $ docker info Client: Debug Mode: false Server: Containers: 4 Running: 2 Paused: 0 Stopped: 2 Images: 2 Server Version: 19.03.6
Alright, so this is telling us that we have 2 images on my machine and for sake of simplicity we take that as 2 layers. Great stuff!
Here’s a sample dockerfile:
1 FROM ubuntu 2 RUN apt-get update 3 RUN apt-get install -y vim 4 CMD [“echo”, “Hello, I'm Moby Dock!”]
As always, we have our FROM instructions first. We’re gonna be basing this new image on
ubuntu. We have got 2 RUNs and one CMD instruction.
Now, we could do something to take care of the number of layers our docker files create and one of them is to write effective RUN instructions.
The RUN instruction is not optimized here although, I’ve listed them in 2 lines for sake of simplicity. More could be read in the last section of this article. link here
We are about to build an image from this dockerfile. How many layers do you think this will add overall, as well as to the layers on top of ubuntu base image?
Let’s do a docker build and see!
$ docker build -t="sample-app-moby" . Sending build context to Docker daemon 2.048kB Step 1/4 : FROM ubuntulatest: Pulling from library/ubuntu d51af753c3d3: Pull complete fc878cd0a91c: Pull complete 6154df8ff988: Pull complete fee5db0ff82f: Pull complete Digest: sha256:747d2dbbaaee995098c9792d99bd333c6783ce56150d1b11e333bbceed5c54d7 Status: Downloaded newer image for ubuntu:latest ---> 1d622ef86b13 Step 2/4 : RUN apt-get update ---> Running in 3e61e5d088a0 Removing intermediate container 3e61e5d088a0 ---> 8e60ea7a2a85 Step 3/4 : RUN apt-get install -y vim ---> Running in 1def19080dcd Removing intermediate container 1def19080dcd ---> a5d4447b41ed Step 4/4 : CMD [“echo”, “Hello, I'm Moby Dock!”] ---> Running in 54b10ce7afd2 Removing intermediate container 54b10ce7afd2 ---> 8e35d08eb177 Successfully built 8e35d08eb177 Successfully tagged sample-app-moby:latest
Here, overall 4 layers has been added:
8e35d08eb177 | a5d4447b41ed | 8e60ea7a2a85 | 1d622ef86b13.
$ docker images REPOSITORY TAG IMAGE ID CREATED SIZE sample-app-moby latest 8e35d08eb177 2 minutes ago 163MB fluentd <none> ec77776ad237 9 days ago 44.9MB ubuntu latest 1d622ef86b13 10 days ago 73.8MB nginx latest 602e111c06b6 10 days ago 127MB
$ docker image history 8e35d08eb177 IMAGE CREATED CREATED BY SIZE COMMENT 8e35d08eb177 2 minutes ago /bin/sh -c #(nop) CMD ["/bin/sh" "-c" "[“ec… 0Ba5d4447b41ed 2 minutes ago /bin/sh -c apt-get install -y vim 67.9MB8e60ea7a2a85 2 minutes ago /bin/sh -c apt-get update 21.2MB1d622ef86b13 10 days ago /bin/sh -c #(nop) CMD ["/bin/bash"] 0B<missing> 10 days ago /bin/sh -c mkdir -p /run/systemd && echo 'do… 7B <missing> 10 days ago /bin/sh -c set -xe && echo '#!/bin/sh' > /… 811B <missing> 10 days ago /bin/sh -c [ -z "$(apt-get indextargets)" ] 1.01MB <missing> 10 days ago /bin/sh -c #(nop) ADD file:a58c8b447951f9e30… 72.8MB
📝 Note: It shows creation date for Ubuntu layer as
10 days ago, because that’s the official date when the image was updated on. (https://hub.docker.com/_/ubuntu/)
So, we have a total of 6 images now, that is addition of 4 to the already existing 2 images we had.
$ docker info Client: Debug Mode: false Server: Containers: 4 Running: 2 Paused: 0 Stopped: 2 Images: 6 Server Version: 19.03.6
As discussed earlier, one important thing about writing Dockerfile is to take care of structure in which you mention the instructions as well. One of the ways to reduce the layers in the above Dockerfile example, would be to append and re-write the RUN command in an effective way.
RUN apt-get update && apt-get install -y vim
This was a quick read on layering, and hence if you want to read further on optimizing things. I’ve mentioned few links at the end of article.
📓📓 To recap- the layered file system allows multiple images and containers to share the same layers. And gives us following major advantages:
1. Faster image transfer. 2. Faster image build.
References and motivation: https://docs.docker.com/engine/reference/commandline/images/