Perl programmer for hire: download my resume (PDF).
John Bokma's Hacking & Hiking

A Tale of Three Docker Images

June 11, 2021

Over the past days I have been looking further into making Docker images of my own after I had installed Docker Desktop for macOS. Creating an image can be a bumpy ride, especially for me; the last time I looked at Docker was back in 2016. But back then Docker for Mac was too experimental for my taste.

So, why giving Docker for Mac a second chance? The main reason is that I want some scripts plus their requirements grouped together without affecting the main installation of the operating system I am using. I could use virtual environments for that, or run everything in a virtual machine. I currently do a combination of the former in the latter using VirtualBox. But now I want to see if Docker for Mac makes things easier for me, especially since in the near future I might switch to Apple Silicon.

Below I am going to discuss three of the images I have been creating over the past days; the choices I made and the issues I encountered. It is not an exact play back of all the steps; it was a messy journey with a lot of Googling and reading. It's more like how I would do those steps today. They still might not be the best ones, so feedback is welcome.

Edit: see also Timezones in Alpine Docker Containers.

Image One: Python version of tumblelog

The first image I want to discuss is a very easy one; it's for the Python version of tumblelog. As I didn't encounter any issues with it, I give the complete version below:

# syntax=docker/dockerfile:1
FROM python:3-alpine

WORKDIR /app
COPY requirements.txt .
RUN apk add --no-cache --virtual .build-deps gcc musl-dev \
    && pip install --no-cache-dir -r requirements.txt \
    && apk del .build-deps

COPY tumblelog.py .
WORKDIR /data
ENTRYPOINT ["python", "/app/tumblelog.py"]

I decided to use python:3-alpine as this gives a very small image. The Python version of tumblelog requires only three modules to be installed. I stored a list of the required modules in an external file named requirements.txt:

pyyaml
commonmark
regex

The WORKDIR instruction creates a new directory named /apps and the COPY instruction copies the requirements file into this directory inside the Docker image for later use.

Next, three commands are a single RUN; it's important that this is a single RUN and not a RUN for each command as this would create a new layer each. The first one installs gcc and musl-dev in a virtual package called .build-deps instead of adding them to world. This way this command can easily be reverted by deleting the virtual package: apk del .build-deps. The --no-cache option installs packages with an index that is updated and avoids local caching (not needed). This keeps the image leaner. The pip command installs the required Python packages. Again caching is disabled to keep the image lean.

The next step, COPY, copies the file tumblelog.py into the image.

After this a work directory is created which will be used by the entry point; the final step.

Finally, an entry point is created. It specifies that python is used to run the tumblelog program which was copied into the /app directory earlier on.

I run this container as follows:

docker run --rm --volume "`pwd`:/data" --user `id -u`:`id -g` tumblelog/perl \
  --template-filename plurrrr.html --author 'John Bokma' \
  --description "John Bokma's tumblelog" --blog-url https://plurrrr.com/ \
  --date-format '%a %d %b %Y' --tags --name 'Plurrrr' \
  --output-dir htdocs --quiet -css soothe.css plurrrr.md

While this looks overwhelming, only the first line is the actual docker run call, the rest are arguments to the tumblelog program itself.

The --volume option mounts the current working directory of the host as returned by the pwd command to the /data directory inside the container. As this is the current working directory for the container tumblelog.py writes the output to htdocs located inside the current working directory of the host.

The size of the image is 57.19MB.

Image Two: tweetfile.pl

The second Dockerfile I want to discuss is the one I created for an updated version of tweetfile.pl. This program posts a tweet picked from a file at random. Because I wanted to have a lean image I started with Alpine. And because some compiling was required I decided to give multi-stage builds a try. Hence the first lines of my Dockerfile became:

# Syntax=docker/dockerfile:1
FROM alpine:latest AS base

WORKDIR /app

FROM base AS builder

Next, I added the following lines:

RUN apk add --no-cache --virtual .build-deps perl-app-cpanminus \
    && apk --no-cache add perl \
    && apk del .build-deps

FROM base AS run
COPY --from=builder /usr/bin/perl /usr/bin

COPY tweetfile.pl .
WORKDIR /data
ENTRYPOINT ["perl", "/app/tweetfile.pl"]

As I certainly have to install Perl modules I install cpanminus. Don't add perl itself to the virtual package .build-deps as apk del .build-deps will delete it. It took me some time to figure out why I didn't have a working perl...

I could build the image without any issues using:

docker build -f Dockerfile -t perl/tweetfile .

However, when running the image I got:

Error loading shared library libperl.so: No such file or directory (needed by /u
sr/bin/perl)
Error relocating /usr/bin/perl: Perl_sys_term: symbol not found
Error relocating /usr/bin/perl: perl_parse: symbol not found
Error relocating /usr/bin/perl: Perl_rsignal_state: symbol not found
Error relocating /usr/bin/perl: perl_alloc: symbol not found
...

Oops, I forgot to copy the required shared libraries. Where is libperl.so located? In order to find this out I stopped the building on the image at the builder stage, as the file is missing from the run stage:

docker build -f Dockerfile --target builder -t perl/tweetfile .

Next, I ran the container interactively using a shell as the entrypoint:

docker run -it --entrypoint /bin/sh perl/tweetfile
/app # 

And I used find to locate the required libperl.so:

/app # find / -name 'libperl.so'
/usr/lib/perl5/core_perl/CORE/libperl.so

So I added an additional COPY instruction:

COPY --from=builder /usr/lib/ /usr/lib/

Note that I use /usr/lib/ and not /usr/lib/perl5 as other shared libraries might be required by Perl or other modules that I install.

Building and running the complete image now resulted in another error:

Can't locate strict.pm in @INC (you may need to install the strict module) (@INC
 contains: /usr/local/lib/perl5/site_perl /usr/local/share/perl5/site_perl /usr/
lib/perl5/vendor_perl /usr/share/perl5/vendor_perl /usr/lib/perl5/core_perl /usr
/share/perl5/core_perl) at /app/tweetfile.pl line 8.
BEGIN failed--compilation aborted at /app/tweetfile.pl line 8.

Good news: we're running the Perl script. The bad news, something is still missing.

So I repeated the previous steps to locate a missing file.

/app # find / -name strict.pm
/usr/share/perl5/core_perl/strict.pm

And again, to play it safe, I added a COPY for /usr/share instead of /usr/share/perl5:

COPY --from=builder /usr/share /usr/share

Rinse and repeat and I got Can't locate Try/Tiny.pm in @INC. This is a module that has to be installed as it's not included with Perl. So I added the following to RUN: cpanm Try::Tiny:

RUN apk add --no-cache --virtual .build-deps perl-app-cpanminus \
    && apk add --no-cache perl \
    && cpanm Try::Tiny \
    && apk del .build-deps

Next I got a lot of error output from the build process. The line that stood out to me was:

...
#8 2.719 /usr/bin/wget: unrecognized option: retry-connrefused
...

After some Googling I learned that BusyBox comes with a wget that doesn't support the retry-connrefused option. So I added wget to the list of packages to be installed into .build-deps:

RUN apk add --no-cache --virtual .build-deps perl-app-cpanminus wget \
... as before

Now I got You probably need to have 'make'. during the build process, so I added this one after the wget.

Try::Tiny was still missing, so I searched again for its location using the same method as above.

/app # find / -name 'Tiny.pm'
/usr/local/share/perl5/site_perl/Try/Tiny.pm
/usr/share/perl5/core_perl/HTTP/Tiny.pm
/usr/share/perl5/core_perl/Test2/Tools/Tiny.pm
/root/.cpanm/work/1623403651.10/Try-Tiny-0.30/lib/Try/Tiny.pm
/root/.cpanm/work/1623403651.10/Try-Tiny-0.30/blib/lib/Try/Tiny.pm

I solved this by adding the following to the run stage:

COPY --from=builder /usr/local /usr/local

Running the image resulted in another missing module: Path::Tiny. I added this one to the cpanm line and rebuild the image once more.

Now I got:

Can't locate Net/Twitter/Lite/WithAPIv1_1.pm in @INC (you may need to install th
e Net::Twitter::Lite::WithAPIv1_1 module) (@INC contains: /usr/local/lib/perl5/s
ite_perl /usr/local/share/perl5/site_perl /usr/lib/perl5/vendor_perl /usr/share/
perl5/vendor_perl /usr/lib/perl5/core_perl /usr/share/perl5/core_perl) at /app/t
weetfile.pl line 14.

As I already knew that this package belongs to Net::Twitter::Lite so I added this to the cpanm line. I also knew that this was going to be a problematic package as it relies on Perl modules that use C libraries. I added the packages gcc and musl-dev.

To make things easier to debug I decided to remove the apk del .build-deps, create an image at stage builder and open a shell inside this image and to try to get this module manually installed.

The Dockerfile at this point is as follows:

# Syntax=docker/dockerfile:1
FROM alpine:latest AS base

WORKDIR /app

FROM base AS builder
RUN apk add --no-cache --virtual .build-deps \
        perl-app-cpanminus wget make gcc musl-dev \
    && apk add perl \
    && cpanm Try::Tiny Path::Tiny

FROM base AS run
COPY --from=builder /usr/bin/perl /usr/bin
COPY --from=builder /usr/lib/ /usr/lib/
COPY --from=builder /usr/share /usr/share
COPY --from=builder /usr/local /usr/local

COPY tweetfile.pl .
WORKDIR /data
ENTRYPOINT ["perl", "/app/tweetfile.pl"]

At the shell I entered cpanm Net::Twitter::Lite, which resulted in the following error:

! Installing Module::Build::Tiny failed. See /root/.cpanm/work/1623406879.6/buil
d.log for details. Retry with --force to force install it.
! Installing the dependencies failed: Module 'Module::Build::Tiny' is not instal
led
! Bailing out the installation for Net-Twitter-Lite-0.12008.

So I used cat to examine the contents of the build log and read the following:

lib/Simple.xs:2:10: fatal error: EXTERN.h: No such file or directory
    2 | #include "EXTERN.h"
      |          ^~~~~~~~~~

It turns out that EXTERN.h is part of the perl C API which is needed for embedding the interpreter and building XS modules. I used apk add perl-dev to solve this issue and ran cpanm Net::Twitter::Lite once more.

After a long installation process (minutes) the process halted with an error:

...
! Configure failed for Net-SSLeay-1.90. See /root/.cpanm/work/1623407221.336/build.log for details.
! Installing the dependencies failed: Module 'Net::SSLeay' is not installed
! Bailing out the installation for IO-Socket-SSL-2.071.
...

Installing the dependency manually using cpanm Net::SSLeay resulted in a build log with the following error message:

...
*** Could not find OpenSSL
...

So I installed openSSL using apk add openssl. Installing Net::SSLeay again resulted in yet another issue reported in the build log:

...
SSLeay.xs:163:10: fatal error: openssl/err.h: No such file or directory
  163 | #include <openssl/err.h>
      |          ^~~~~~~~~~~~~~~
compilation terminated.
...

To solve this issue I installed openssl-dev as well using apk. And again an issue in the build log:

...
/usr/lib/gcc/x86_64-alpine-linux-musl/10.2.1/../../../../x86_64-alpine-linux-mus
l/bin/ld: cannot find -lz
...

I guessed that this had to do with a missing compression library. I guessed that I needed to add package zlib-dev, which I did. And now cpanm Net::SLLeay finally reported Successfully installed Net-SSLeay-1.90.

The updated Dockerfile is as follows. Important: note that I have added back the deletion of the .build-deps virtual package.

# Syntax=docker/dockerfile:1
FROM alpine:latest AS base

WORKDIR /app

FROM base AS builder
RUN apk add --no-cache --virtual .build-deps \
        perl-app-cpanminus wget make gcc musl-dev perl-dev \
        openssl openssl-dev zlib-dev \
    && apk add perl \
    && cpanm Try::Tiny Path::Tiny Net::Twitter::Lite \
    && apk del .build-deps

FROM base AS run
COPY --from=builder /usr/bin/perl /usr/bin
COPY --from=builder /usr/lib/ /usr/lib/
COPY --from=builder /usr/share /usr/share
COPY --from=builder /usr/local /usr/local

COPY tweetfile.pl .
WORKDIR /data
ENTRYPOINT ["perl", "/app/tweetfile.pl"]

Using docker build -f Dockerfile -t perl/tweetfile . I could build the image successfully this time. Building took 379.0 seconds, though.

Next I tried to actually post a tweet to Twitter using:

docker run --volume "`pwd`:/data:ro" --rm \
  perl/tweetfile --conf john_bokma.conf --tweets john_bokma-tweets.txt

This resulted in the following message to be displayed. Yet another Perl module was required. Odd that this is not a dependency for Net::Twitter::Lite...

Install Net::OAuth 0.25 or later for OAuth support at /app/tweetfile.pl line 103
.

Aside: I specify ro (read only) in the volume argument as the program only reads from two files and writes no data.

The final Dockerfile is as follows:

# Syntax=docker/dockerfile:1
FROM alpine:latest AS base

WORKDIR /app

FROM base AS builder
RUN apk add --no-cache --virtual .build-deps \
        perl-app-cpanminus wget make gcc musl-dev perl-dev \
        openssl openssl-dev zlib-dev \
    && apk add perl \
    && cpanm Try::Tiny Path::Tiny Net::Twitter::Lite Net::OAuth \
    && apk del .build-deps

FROM base AS run
COPY --from=builder /usr/bin/perl /usr/bin
COPY --from=builder /usr/lib/ /usr/lib/
COPY --from=builder /usr/share /usr/share
COPY --from=builder /usr/local /usr/local

COPY tweetfile.pl .
WORKDIR /data
ENTRYPOINT ["perl", "/app/tweetfile.pl"]

The size of the image is 46.76MB.

Image Three: Perl version of tumblelog

The third and final image I want to discuss is for the Perl version of tumblelog. Based on what I had learned from the previous image I started with:

# syntax=docker/dockerfile:1
FROM alpine:latest AS base

WORKDIR /app

FROM base AS builder

RUN apk add --no-cache --virtual .build-deps \
        make wget \
        perl-app-cpanminus \
    && apk add perl \
    && apk del .build-deps

FROM base AS run
COPY --from=builder /usr/bin/perl /usr/bin
COPY --from=builder /usr/lib/ /usr/lib/
COPY --from=builder /usr/share /usr/share
COPY --from=builder /usr/local /usr/local

COPY tumblelog.pl .
WORKDIR /data
ENTRYPOINT ["perl", "/app/tumblelog.pl"]

Running the container resulted in Can't locate URI.pm so I added a line with cpanm URI to the RUN section. The next missing module was JSON::XS. And XS means it uses a C library, so I added gcc, musl-dev, and perl-dev via apk and of course JSON::XS via cpanm:

# syntax=docker/dockerfile:1
FROM alpine:latest AS base

WORKDIR /app

FROM base AS builder

RUN apk add --no-cache --virtual .build-deps \
        make wget gcc musl-dev \
        perl-app-cpanminus \
    && apk add perl \
    && cpanm URI JSON::XS \
    && apk del .build-deps

FROM base AS run
COPY --from=builder /usr/bin/perl /usr/bin
COPY --from=builder /usr/lib/ /usr/lib/
COPY --from=builder /usr/share /usr/share
COPY --from=builder /usr/local /usr/local

COPY tumblelog.pl .
WORKDIR /data
ENTRYPOINT ["perl", "/app/tumblelog.pl"]

Next missing module was YAML::XS so I added it to the cpanm line and rebuild the Docker image. The next missing module turned out to be Path::Tiny. And the one after CommonMark. As this one uses a separate C library I knew this was going to be a bit harder as I used to built this C library from source on Ubuntu.

So let's do the same as in the previous section: remove the apk del .build-deps and install the module manually in stage builder.

docker run -it --entrypoint /bin/sh tumblelog/perl
/app # cpanm CommonMark

This installation step failed. Inspecting the build log showed:

...
libcmark 0.21.0 or higher not found at Makefile.PL line 16.
...

Installing cmark-dev (not libcmark) fixed this issue. So I added those dependencies to the Dockerfile, rebuild the image, and ran it once more.

However, this time I got:

Can't load '/usr/local/lib/perl5/site_perl/auto/CommonMark/CommonMark.so' for mo
dule CommonMark: Error loading shared library libcmark.so.0.29.0: No such file o
r directory (needed by /usr/local/lib/perl5/site_perl/auto/CommonMark/CommonMark
.so) at /usr/share/perl5/core_perl/XSLoader.pm line 93.
 at /usr/local/lib/perl5/site_perl/CommonMark.pm line 11.
BEGIN failed--compilation aborted at /usr/local/lib/perl5/site_perl/CommonMark.p
m line 12.
Compilation failed in require at /app/tumblelog.pl line 11.
BEGIN failed--compilation aborted at /app/tumblelog.pl line 11.

Somehow libcmark.so.0.29.0 had disappeared. Just like perl had in the previous section. And for exactly the same reason: apk del .build-deps had deleted it. So moving cmark-dev to the same apk line as perl solved this issue.

Next was another Perl module missing: Try::Tiny. So I added this one to the cpanm line as well. This time I was greeted by the help of tumblelog; success!

The complete Dockerfile is as follows:

# syntax=docker/dockerfile:1
FROM alpine:latest AS base

WORKDIR /app

FROM base AS builder

RUN apk add --no-cache --virtual .build-deps \
        make wget gcc musl-dev perl-dev \
        perl-app-cpanminus \
    && apk add perl cmark-dev \
    && cpanm URI JSON::XS YAML::XS Path::Tiny CommonMark Try::Tiny \
    && apk del .build-deps

FROM base AS run
COPY --from=builder /usr/bin/perl /usr/bin
COPY --from=builder /usr/lib/ /usr/lib/
COPY --from=builder /usr/share /usr/share
COPY --from=builder /usr/local /usr/local

COPY tumblelog.pl .
WORKDIR /data
ENTRYPOINT ["perl", "/app/tumblelog.pl"]

The size of the image is 41.82MB.

Related