Dev Environments Within Docker Containers

Posted on 9 mins read

Introduction

You’re a software engineer with a new laptop.
You’re going to be writing code in multiple languages.
You’re going to have projects of varying dependencies.

But you want to avoid the issues you’ve had in the past:

  • Messy installations of lots of different software packages.
  • Clogging up your system with programming language version managers.

You decide to use Docker.
It’s a dependency sure, but you’ve got to install something!
With Docker it allows your environment to stay clean.

So let’s see two simple examples:

  1. Python
  2. Go

Note: I don’t claim this to be ’the way’ to do this.
This is just a setup that works well enough for me.
I’m also a Vim user, so your mileage may vary.

Python

You have a Python project you need to work on.

I won’t explain how Python works,
I’ll just presume you’re a Pythonista

Here’s the folder structure we’re dealing with:

foo-project/
| Dockerfile
| Makefile
| app.py
| requirements.txt

We have a Dockerfile (naturally), along with a Makefile to allow us to more easily build our docker image. We also have an application script + a set of dependencies. Nice and simple.

So here’s what the Dockerfile looks like:

FROM python:3.6.1

WORKDIR /tmp

RUN apt-get update -y
RUN apt-get install -y git ncurses-dev
RUN git clone https://github.com/vim/vim.git && cd vim && ./configure --enable-python3interp=yes && make && make install

ADD ./requirements.txt /app/requirements.txt
RUN pip install -r /app/requirements.txt

COPY .vim /root/.vim
COPY .vimrc /root/.vimrc

WORKDIR /app

As you can see we’re building our Docker image from an official Python base image (at the time of writing it’s the latest Python version).

We jump into a tmp directory so that we can install some dependencies required to get our setup just how we want it. So this means installing git so we can clone down the latest build of Vim and we also install ncurses-dev which is necessary in order to compile Vim.

After that we copy our requirements.txt file into the image and install the packages specified inside that file. We also add in our .vim directory and .vimrc file to the image.

Note: the Makefile has a command for copying .vim/.vimrc into the current project directory, as docker build has a specific context environment that is effectively the location of the Dockerfile

Next we have the Makefile:

copy_vim_files:
		@if [ ! -d "./.vim" ]; then cp -r "$$HOME/.vim" ./.vim; fi
		@if [ ! -f "./.vimrc" ]; then cp "$$HOME/.vimrc" ./.vimrc; fi

remove_vim_files:
		@rm -rf ./.vim
		@rm ./.vimrc

build: copy_vim_files
		@docker build -t python-container-with-vim .

run: build
		@docker run -it -v "$$(pwd)":/app python-container-with-vim /bin/bash

clean: remove_vim_files
		-@docker rmi -f python-container-with-vim &> /dev/null || true

rebuild: clean run

There’s lots going on in there, but the important thing to know is that to start up your docker container (with Python and Vim pre-installed) is to use:

make run

This will copy over the host .vim/.vimrc directory/files, then build a new image and then call docker run ... where it’ll mount the project directory as a volume into the running container.

Once you’re inside the container just execute vim app.py and off you go writing code.

Just for completion, here is our application script:

print('hi')
food = "is a thing"  # if linting is installed properly this will error


def hello(message):
    """
    My summary line starts capitalized and ends with a period.

    my bigger description is going here
    so pay attention to what it says
    """
    print(message)

The above script has some ’linting’ issues, so if our packages (see below) are installed correctly then we should see Vim highlight the issue to us.

flake8==3.2.1
flake8-deprecated==1.1
flake8-docstrings==1.0.3
flake8-mock==0.3
flake8-quotes==0.9.0
mypy==0.501
pep8-naming==0.4.1
pylint==1.6.4
pytest==3.0.5

We can see from the requirements.txt file that we’ve installed a few different linters along with the MyPy static analysis tool.

That’s it really. You can reuse the Dockerfile and Makefile for all your projects as they don’t do anything specific to this project. Just setup the docker image/container so you can execute make run and start developing.

Go

You have a Go project you need to work on.

I won’t explain how Go works,
I’ll just presume you’re a Gopher

Here’s the folder structure we’re dealing with:

bar-project/
| Dockerfile
| Dockerfile-compile
| Godeps
| Makefile
| app.go

So here’s our main Dockerfile:

FROM golang:1.8

RUN apt-get update -y
RUN apt-get install -y wget git ncurses-dev time

WORKDIR /tmp
RUN git clone https://github.com/vim/vim.git && cd vim && make && make install

WORKDIR /go/src
COPY .vim /root/.vim
COPY .vimrc /root/.vimrc
COPY ./Godeps /go/src

RUN wget https://raw.githubusercontent.com/pote/gpm/v1.4.0/bin/gpm && chmod +x gpm && mv gpm /usr/local/bin
RUN gpm install
RUN cp -r ./github.com /github.com  # backup packages to root to prevent volume mount removing it

# Install Go binaries that are utilised by the vim-go plugin:
# https://github.com/fatih/vim-go/blob/master/plugin/go.vim#L9
#
# We don't manually install them, we let vim-go handle that
# We use vim's `execute` command to pipe commands
# This helps avoid "Press ENTER or type command to continue"
RUN time vim -c "execute 'silent GoUpdateBinaries' | execute 'quit'"

Again we’re not doing anything too crazy (not until the end, which I’ll explain). We’re building a new image from an official base image, then we’re installing dependencies that allow us to manually compile the Vim editor.

Next we copy over our vim files and the Godeps dependencies file and we install our dependency manager gpm and install the packages we want to use within our application.

Next we back up the installed depedencies (./github.com) to another directory. The reason we do that is because when we mount our host project directory into the running container we will end up accidentally replacing the installed packages.

Finally we run vim and pass it a script to be executed once Vim has loaded. What this does is allow the statically built image to include the updated set of depedencies that the vim-go plugin requires. I could have installed them manually, but then using the built in command provided by vim-go means I don’t have to ensure my list of go tools still matches up to what vim-go is using.

The downside to this is that when you build the image, you’ll see (for ~20-30 seconds) Vim started and you wont be able to interact with it at all during that time. This is because it’s installing the dependencies it uses. But after that, Vim will close and you’ll be placed at the containers shell prompt. From there you can run Vim and start coding.

Here’s the Go Makefile (it works similarly to the Python one):

bin := "/usr/local/bin/fastly"
vim_dir := "./.vim"
vimrc := "./.vimrc"
container_env := "go-container-with-vim"
container_compiler := "go-compiler"

copy_vim_files:
  @if [ ! -d $(vim_dir) ]; then cp -r "$$HOME/.vim" $(vim_dir); fi
  @if [ ! -f $(vimrc) ]; then cp "$$HOME/.vimrc" $(vimrc); fi

remove_vim_files:
  @rm -rf $(vim_dir) &> /dev/null || true
  @rm $(vimrc) &> /dev/null || true

remove_compiled_files:
  @rm ./fastly.{darwin,linux,windows.exe} &> /dev/null || true

clean: remove_vim_files remove_compiled_files
  @docker rmi -f $(container_env) &> /dev/null || true
  @docker rmi -f $(container_compiler) &> /dev/null || true

uninstall: clean
  @rm $(bin) &> /dev/null || true

build: copy_vim_files
  @docker build -t $(container_env) .

dev: build remove_vim_files
  @docker run -it -v "$$(pwd)":/go/src $(container_env) /bin/bash

rebuild: clean run

compile:
  @docker build -t $(container_compiler) -f ./Dockerfile-compile .
  @docker run -it -v "$$(pwd)":/go/src $(container_compiler) || true

copy_binary:
  cp ./fastly.darwin $(bin)

install: compile copy_binary remove_compiled_files

One thing I did this time was change the make run for make dev as I feel that’s more indicative of what we’re doing (the ‘run’ suggests we’re running our application when we’re really just wanting to drop into a development environment).

There’s a few more steps in the Go Makefile and that’s just for the purposes of having a separate Dockerfile for compiling our application. The following is the other Dockerfile we have in our project:

FROM golang:1.8

WORKDIR /go/src
COPY ./Godeps /go/src
COPY ./compile.sh /go/src

RUN apt-get update && apt-get install wget
RUN wget https://raw.githubusercontent.com/pote/gpm/v1.4.0/bin/gpm && chmod +x gpm && mv gpm /usr/local/bin
RUN gpm install
RUN cp -r ./github.com /github.com  # backup packages to root to prevent volume mount removing it

CMD ["./compile.sh"]

This Dockerfile does much the same thing: get dependency file, install those specified dependencies, then back them up to another directory.

This time though, when the container is run we use a separate script as the CMD value as our script was getting quite long (as you’ll see).

Here is the contents of compile.sh:

Note: make sure you chmod +x ./compile.sh from your host

#!/bin/sh

# copy packages back into our source code directory
cp -fr /github.com ./github.com

# compile application for the major operating systems
gox -osarch='linux/amd64' -osarch='darwin/amd64' -osarch='windows/amd64' -output='fastly.{{.OS}}'

# run the relevant compatible compiled binary for this container's OS
./fastly.linux

The tasks we run are:

  1. copy the backed up dependencies back into the mounted project directory
  2. build the app using the default compiler for the OS †
  3. execute the compiled binary to show it can run correctly inside the container

† this means our compiled binary will be a linux based binary, so you can’t run it on your host machine if it’s not linux based (e.g. I’m using macOS). You’ll see that to allow me to compiled my application for multiple OS’s I’ve installed Gox

Now here is our Go application, it simply uses the logging dependency we’ve installed and that’s it. Nothing too fancy necessary for this example.

package main

import (
  "fmt"

  log "github.com/Sirupsen/logrus"
)

func init() {
  log.SetLevel(log.DebugLevel) // otherwise would not be shown
}

func main() {
  fmt.Println("Hello World!")

  logger := log.WithFields(log.Fields{
    "name": "hello-world-app",
  })
  logger.Debug("this is my debug log message")
  logger.Info("this is my info log message")
  logger.Warn("this is my warn log message")
  logger.Error("this is my error log message")
  logger.Fatal("this is my Fatal log message")
  logger.Panic("this is my Panic log message") // we don't actually reach here
}

Here is our dependency file’s content, where we can see the Logrus dependency we’ve specified (as well as Gox for the purposes of our container responsible for compiling our application for multiple OS’):

github.com/mitchellh/gox c9740af9c6574448fd48eb30a71f964014c7a837
github.com/sirupsen/logrus 10f801ebc38b33738c9d17d50860f484a0988ff5

Mounting Volumes

Just remember that when making changes inside the container, because you’ve mounted your host project directory as a volume, if you make a change or add a new file or compile something inside of the container; then it’ll be available on your host machine.

This is all fine, but you might want to look at setting up a .gitignore file to ensure you don’t accidentally commit any unwanted items into your git repository.

But before we wrap up... time (once again) for some self-promotion 🙊