A Tale of Three Docker Images
June 11, 2021
Over the past days I have been looking further into making Docker images of my own after I had installed Docker Desktop for macOS. Creating an image can be a bumpy ride, especially for me; the last time I looked at Docker was back in 2016. But back then Docker for Mac was too experimental for my taste.
So, why giving Docker for Mac a second chance? The main reason is that I want some scripts plus their requirements grouped together without affecting the main installation of the operating system I am using. I could use virtual environments for that, or run everything in a virtual machine. I currently do a combination of the former in the latter using VirtualBox. But now I want to see if Docker for Mac makes things easier for me, especially since in the near future I might switch to Apple Silicon.
Below I am going to discuss three of the images I have been creating over the past days; the choices I made and the issues I encountered. It is not an exact play back of all the steps; it was a messy journey with a lot of Googling and reading. It's more like how I would do those steps today. They still might not be the best ones, so feedback is welcome.
Edit: see also Timezones in Alpine Docker Containers.
Image One: Python version of tumblelog
The first image I want to discuss is a very easy one; it's for the
Python version of
tumblelog. As I didn't
encounter any issues with it, I give the complete version of the
Dockerfile below:
# syntax=docker/dockerfile:1
FROM python:3-alpine
WORKDIR /app
COPY requirements.txt .
RUN apk add --no-cache --virtual .build-deps gcc musl-dev \
&& pip install --no-cache-dir -r requirements.txt \
&& apk del .build-deps
COPY tumblelog.py .
WORKDIR /data
ENTRYPOINT ["python", "/app/tumblelog.py"]
I decided to use python:3-alpine as this gives a very small image.
The Python version of tumblelog requires only three modules to be
installed. I stored a list of the required modules in an external file
named requirements.txt:
pyyaml
commonmark
regex
The WORKDIR instruction creates a new directory named /apps and the
COPY instruction copies the requirements file into this directory
inside the Docker image for later use.
Next, three commands are a single RUN; it's important that this is a
single RUN and not a RUN for each command as this would create a new
layer each. The first one installs gcc and musl-dev in a virtual
package called .build-deps instead of adding them to world. This
way this command can easily be reverted by deleting the virtual
package: apk del .build-deps. The --no-cache option installs
packages with an index that is updated and avoids local caching (not
needed). This keeps the image leaner. The pip command installs the
required Python packages. Again caching is
disabled to
keep the image lean.
The next step, COPY, copies the file tumblelog.py into the image.
After this a work directory is created which will be used by the entry point; the final step.
Finally, an entry point is created. It specifies that python is used
to run the tumblelog program which was copied into the /app
directory earlier on.
I run this container as follows:
docker run --rm --volume "`pwd`:/data" --user `id -u`:`id -g` tumblelog/perl \
--template-filename plurrrr.html --author 'John Bokma' \
--description "John Bokma's tumblelog" --blog-url https://plurrrr.com/ \
--date-format '%a %d %b %Y' --tags --name 'Plurrrr' \
--output-dir htdocs --quiet -css soothe.css plurrrr.md
While this looks overwhelming, only the first line is the actual
docker run call, the rest are arguments to the tumblelog program
itself.
The --volume option mounts the current working directory of the host
as returned by the pwd command to the /data directory inside the
container. As this is the current working directory for the container
tumblelog.py writes the output to htdocs located inside the
current working directory of the host.
The size of the image is 57.19MB.
Image Two: tweetfile.pl
The second Dockerfile I want to discuss is the one I created for an
updated version of
tweetfile.pl. This program
posts a tweet picked from a file at
random.
Because I wanted to have a lean image I started with Alpine. And
because some compiling was required I decided to give multi-stage builds
a try. Hence the first lines of my Dockerfile became:
# Syntax=docker/dockerfile:1
FROM alpine:latest AS base
WORKDIR /app
FROM base AS builder
Next, I added the following lines:
RUN apk add --no-cache --virtual .build-deps perl-app-cpanminus \
&& apk --no-cache add perl \
&& apk del .build-deps
FROM base AS run
COPY --from=builder /usr/bin/perl /usr/bin
COPY tweetfile.pl .
WORKDIR /data
ENTRYPOINT ["perl", "/app/tweetfile.pl"]
As I certainly have to install Perl modules I install cpanminus.
Don't add perl itself to the virtual package .build-deps as apk del .build-deps will delete it. It took me some time to figure out
why I didn't have a working perl...
I could build the image without any issues using:
docker build -f Dockerfile -t perl/tweetfile .
However, when running the image I got:
Error loading shared library libperl.so: No such file or directory (needed by /u
sr/bin/perl)
Error relocating /usr/bin/perl: Perl_sys_term: symbol not found
Error relocating /usr/bin/perl: perl_parse: symbol not found
Error relocating /usr/bin/perl: Perl_rsignal_state: symbol not found
Error relocating /usr/bin/perl: perl_alloc: symbol not found
...
Oops, I forgot to copy the required shared libraries. Where is libperl.so located? In order to find this out I stopped the building on the image at the builder stage, as the file is missing from the run stage:
docker build -f Dockerfile --target builder -t perl/tweetfile .
Next, I ran the container interactively using a shell as the entrypoint:
docker run -it --entrypoint /bin/sh perl/tweetfile
/app #
And I used find to locate the required libperl.so:
/app # find / -name 'libperl.so'
/usr/lib/perl5/core_perl/CORE/libperl.so
So I added an additional COPY instruction:
COPY --from=builder /usr/lib/ /usr/lib/
Note that I use /usr/lib/ and not /usr/lib/perl5 as other shared
libraries might be required by Perl or other modules that I install.
Building and running the complete image now resulted in another error:
Can't locate strict.pm in @INC (you may need to install the strict module) (@INC
contains: /usr/local/lib/perl5/site_perl /usr/local/share/perl5/site_perl /usr/
lib/perl5/vendor_perl /usr/share/perl5/vendor_perl /usr/lib/perl5/core_perl /usr
/share/perl5/core_perl) at /app/tweetfile.pl line 8.
BEGIN failed--compilation aborted at /app/tweetfile.pl line 8.
Good news: we're running the Perl script. The bad news, something is still missing.
So I repeated the previous steps to locate a missing file.
/app # find / -name strict.pm
/usr/share/perl5/core_perl/strict.pm
And again, to play it safe, I added a COPY for /usr/share instead of
/usr/share/perl5:
COPY --from=builder /usr/share /usr/share
Rinse and repeat and I got Can't locate Try/Tiny.pm in @INC. This is
a module that has to be installed as it's not included with Perl. So I
added the following to RUN: cpanm Try::Tiny:
RUN apk add --no-cache --virtual .build-deps perl-app-cpanminus \
&& apk add --no-cache perl \
&& cpanm Try::Tiny \
&& apk del .build-deps
Next I got a lot of error output from the build process. The line that stood out to me was:
...
#8 2.719 /usr/bin/wget: unrecognized option: retry-connrefused
...
After some Googling I learned that BusyBox comes with a wget that
doesn't support the retry-connrefused option. So I added wget to
the list of packages to be installed into .build-deps:
RUN apk add --no-cache --virtual .build-deps perl-app-cpanminus wget \
... as before
Now I got You probably need to have 'make'. during the build
process, so I added this one after the wget.
Try::Tiny was still missing, so I searched again for its location
using the same method as above.
/app # find / -name 'Tiny.pm'
/usr/local/share/perl5/site_perl/Try/Tiny.pm
/usr/share/perl5/core_perl/HTTP/Tiny.pm
/usr/share/perl5/core_perl/Test2/Tools/Tiny.pm
/root/.cpanm/work/1623403651.10/Try-Tiny-0.30/lib/Try/Tiny.pm
/root/.cpanm/work/1623403651.10/Try-Tiny-0.30/blib/lib/Try/Tiny.pm
I solved this by adding the following to the run stage:
COPY --from=builder /usr/local /usr/local
Running the image resulted in another missing module: Path::Tiny. I added
this one to the cpanm line and rebuild the image once more.
Now I got:
Can't locate Net/Twitter/Lite/WithAPIv1_1.pm in @INC (you may need to install th
e Net::Twitter::Lite::WithAPIv1_1 module) (@INC contains: /usr/local/lib/perl5/s
ite_perl /usr/local/share/perl5/site_perl /usr/lib/perl5/vendor_perl /usr/share/
perl5/vendor_perl /usr/lib/perl5/core_perl /usr/share/perl5/core_perl) at /app/t
weetfile.pl line 14.
As I already knew that this package belongs to Net::Twitter::Lite so
I added this to the cpanm line. I also knew that this was going to
be a problematic package as it relies on Perl modules that use C
libraries. I added the packages gcc and musl-dev.
To make things easier to debug I decided to remove the apk del .build-deps, create an image at stage builder and open a shell
inside this image and to try to get this module manually installed.
The Dockerfile at this point is as follows:
# Syntax=docker/dockerfile:1
FROM alpine:latest AS base
WORKDIR /app
FROM base AS builder
RUN apk add --no-cache --virtual .build-deps \
perl-app-cpanminus wget make gcc musl-dev \
&& apk add perl \
&& cpanm Try::Tiny Path::Tiny
FROM base AS run
COPY --from=builder /usr/bin/perl /usr/bin
COPY --from=builder /usr/lib/ /usr/lib/
COPY --from=builder /usr/share /usr/share
COPY --from=builder /usr/local /usr/local
COPY tweetfile.pl .
WORKDIR /data
ENTRYPOINT ["perl", "/app/tweetfile.pl"]
At the shell I entered cpanm Net::Twitter::Lite, which resulted in
the following error:
! Installing Module::Build::Tiny failed. See /root/.cpanm/work/1623406879.6/buil
d.log for details. Retry with --force to force install it.
! Installing the dependencies failed: Module 'Module::Build::Tiny' is not instal
led
! Bailing out the installation for Net-Twitter-Lite-0.12008.
So I used cat to examine the contents of the build log and read the
following:
lib/Simple.xs:2:10: fatal error: EXTERN.h: No such file or directory
2 | #include "EXTERN.h"
| ^~~~~~~~~~
It turns out that EXTERN.h is part of the perl C API which is needed
for embedding the interpreter and building XS modules. I used apk add perl-dev to solve this issue and ran cpanm Net::Twitter::Lite once
more.
After a long installation process (minutes) the process halted with an error:
...
! Configure failed for Net-SSLeay-1.90. See /root/.cpanm/work/1623407221.336/build.log for details.
! Installing the dependencies failed: Module 'Net::SSLeay' is not installed
! Bailing out the installation for IO-Socket-SSL-2.071.
...
Installing the dependency manually using cpanm Net::SSLeay resulted
in a build log with the following error message:
...
*** Could not find OpenSSL
...
So I installed openSSL using apk add openssl. Installing
Net::SSLeay again resulted in yet another issue reported in the
build log:
...
SSLeay.xs:163:10: fatal error: openssl/err.h: No such file or directory
163 | #include <openssl/err.h>
| ^~~~~~~~~~~~~~~
compilation terminated.
...
To solve this issue I installed openssl-dev as well using apk. And
again an issue in the build log:
...
/usr/lib/gcc/x86_64-alpine-linux-musl/10.2.1/../../../../x86_64-alpine-linux-mus
l/bin/ld: cannot find -lz
...
I guessed that this had to do with a missing compression library. I
guessed that I needed to add package zlib-dev, which I did. And now
cpanm Net::SLLeay finally reported Successfully installed Net-SSLeay-1.90.
The updated Dockerfile is as follows. Important: note that I have
added back the deletion of the .build-deps virtual package.
# Syntax=docker/dockerfile:1
FROM alpine:latest AS base
WORKDIR /app
FROM base AS builder
RUN apk add --no-cache --virtual .build-deps \
perl-app-cpanminus wget make gcc musl-dev perl-dev \
openssl openssl-dev zlib-dev \
&& apk add perl \
&& cpanm Try::Tiny Path::Tiny Net::Twitter::Lite \
&& apk del .build-deps
FROM base AS run
COPY --from=builder /usr/bin/perl /usr/bin
COPY --from=builder /usr/lib/ /usr/lib/
COPY --from=builder /usr/share /usr/share
COPY --from=builder /usr/local /usr/local
COPY tweetfile.pl .
WORKDIR /data
ENTRYPOINT ["perl", "/app/tweetfile.pl"]
Using docker build -f Dockerfile -t perl/tweetfile . I could build
the image successfully this time. Building took 379.0 seconds, though.
Next I tried to actually post a tweet to Twitter using:
docker run --volume "`pwd`:/data:ro" --rm \
perl/tweetfile --conf john_bokma.conf --tweets john_bokma-tweets.txt
This resulted in the following message to be displayed. Yet another Perl module was required. Odd that this is not a dependency for Net::Twitter::Lite...
Install Net::OAuth 0.25 or later for OAuth support at /app/tweetfile.pl line 103
.
Aside: I specify ro (read only) in the volume argument as the
program only reads from two files and writes no data.
The final Dockerfile is as follows:
# Syntax=docker/dockerfile:1
FROM alpine:latest AS base
WORKDIR /app
FROM base AS builder
RUN apk add --no-cache --virtual .build-deps \
perl-app-cpanminus wget make gcc musl-dev perl-dev \
openssl openssl-dev zlib-dev \
&& apk add perl \
&& cpanm Try::Tiny Path::Tiny Net::Twitter::Lite Net::OAuth \
&& apk del .build-deps
FROM base AS run
COPY --from=builder /usr/bin/perl /usr/bin
COPY --from=builder /usr/lib/ /usr/lib/
COPY --from=builder /usr/share /usr/share
COPY --from=builder /usr/local /usr/local
COPY tweetfile.pl .
WORKDIR /data
ENTRYPOINT ["perl", "/app/tweetfile.pl"]
The size of the image is 46.76MB.
Image Three: Perl version of tumblelog
The third and final image I want to discuss is for the Perl version of
tumblelog. Based on what
I had learned from the previous image I started with:
# syntax=docker/dockerfile:1
FROM alpine:latest AS base
WORKDIR /app
FROM base AS builder
RUN apk add --no-cache --virtual .build-deps \
make wget \
perl-app-cpanminus \
&& apk add perl \
&& apk del .build-deps
FROM base AS run
COPY --from=builder /usr/bin/perl /usr/bin
COPY --from=builder /usr/lib/ /usr/lib/
COPY --from=builder /usr/share /usr/share
COPY --from=builder /usr/local /usr/local
COPY tumblelog.pl .
WORKDIR /data
ENTRYPOINT ["perl", "/app/tumblelog.pl"]
Running the container resulted in Can't locate URI.pm so I added a
line with cpanm URI to the RUN section. The next missing module
was JSON::XS. And XS means it uses a C library, so I added gcc,
musl-dev, and perl-dev via apk and of course JSON::XS via
cpanm:
# syntax=docker/dockerfile:1
FROM alpine:latest AS base
WORKDIR /app
FROM base AS builder
RUN apk add --no-cache --virtual .build-deps \
make wget gcc musl-dev \
perl-app-cpanminus \
&& apk add perl \
&& cpanm URI JSON::XS \
&& apk del .build-deps
FROM base AS run
COPY --from=builder /usr/bin/perl /usr/bin
COPY --from=builder /usr/lib/ /usr/lib/
COPY --from=builder /usr/share /usr/share
COPY --from=builder /usr/local /usr/local
COPY tumblelog.pl .
WORKDIR /data
ENTRYPOINT ["perl", "/app/tumblelog.pl"]
Next missing module was YAML::XS so I added it to the cpanm line and
rebuild the Docker image. The next missing module turned out to be
Path::Tiny. And the one after CommonMark. As this one uses a
separate C library I knew this was going to be a bit harder as I used
to built this C library from source on
Ubuntu.
So let's do the same as in the previous section: remove the apk del .build-deps and install the module manually in stage builder.
docker run -it --entrypoint /bin/sh tumblelog/perl
/app # cpanm CommonMark
This installation step failed. Inspecting the build log showed:
...
libcmark 0.21.0 or higher not found at Makefile.PL line 16.
...
Installing cmark-dev (not libcmark) fixed this issue. So I added
those dependencies to the Dockerfile, rebuild the image, and ran it
once more.
However, this time I got:
Can't load '/usr/local/lib/perl5/site_perl/auto/CommonMark/CommonMark.so' for mo
dule CommonMark: Error loading shared library libcmark.so.0.29.0: No such file o
r directory (needed by /usr/local/lib/perl5/site_perl/auto/CommonMark/CommonMark
.so) at /usr/share/perl5/core_perl/XSLoader.pm line 93.
at /usr/local/lib/perl5/site_perl/CommonMark.pm line 11.
BEGIN failed--compilation aborted at /usr/local/lib/perl5/site_perl/CommonMark.p
m line 12.
Compilation failed in require at /app/tumblelog.pl line 11.
BEGIN failed--compilation aborted at /app/tumblelog.pl line 11.
Somehow libcmark.so.0.29.0 had disappeared. Just like perl had in
the previous section. And for exactly the same reason: apk del .build-deps had deleted it. So moving cmark-dev to the same apk
line as perl solved this issue.
Next was another Perl module missing: Try::Tiny. So I added this one
to the cpanm line as well. This time I was greeted by the help of
tumblelog; success!
The complete Dockerfile is as follows:
# syntax=docker/dockerfile:1
FROM alpine:latest AS base
WORKDIR /app
FROM base AS builder
RUN apk add --no-cache --virtual .build-deps \
make wget gcc musl-dev perl-dev \
perl-app-cpanminus \
&& apk add perl cmark-dev \
&& cpanm URI JSON::XS YAML::XS Path::Tiny CommonMark Try::Tiny \
&& apk del .build-deps
FROM base AS run
COPY --from=builder /usr/bin/perl /usr/bin
COPY --from=builder /usr/lib/ /usr/lib/
COPY --from=builder /usr/share /usr/share
COPY --from=builder /usr/local /usr/local
COPY tumblelog.pl .
WORKDIR /data
ENTRYPOINT ["perl", "/app/tumblelog.pl"]
The size of the image is 41.82MB.