How to use docker-py

Motivation

The Smarkets architecture is mostly but not exclusively dockerised microservices. These microservices are predominantly Python, as is our internal tooling. However, our tooling to work with docker images and containers has been lacking. To improve this I decided to rewrite it in Python using docker-py. While researching how to use docker-py I struggled to find an introductory tutorial, which this blog hopefully serves as.

This is the dock where we work. This blog is about docker-py

Building the Image

The first stage in the docker process is to build the image, which can be done with the following shell command:

$ cd /service && docker build -f Dockerfile -t image_name .

This would simply build the image specified by the Dockerfile and tag it image_name.

The same can be done using python and docker-py like so (assuming you have run pip install docker-py ):

from docker import Client
client = AutoVersionClient(base_url='unix://var/run/docker.sock')
output = client.build(path='/service', tag='image_name')

The AutoVersionClient ensures the docker server version is determined automatically, rather than having to be specified. The output is a generator that can be iterated over to get the build output.

Running the image

The next stage is to create and run a container based on the image, typically with some port and volume bindings. Assuming we’d have something like

$ docker run --volume=/host:/container --publish=9999:80 image_name

to create and run a container based on the image_name image, mounting /host on the host to /container in the container and publishing the container 80 port to 9999 on the host.

To do the same in Python using docker-py requires a little more work. Using the client from above,

ports = [80]
port_bindings = {80: 9999}
volumes = ['/container']
volume_bindings = {
    '/host': {
        'bind': '/container',
        'mode': 'rw',
    },
}
host_config = client.create_host_config(
    binds=volume_bindings, port_bindings=port_bindings,
)
container = client.create_container(
    image='image_name',
    ports=ports,
    volumes=volumes,
    host_config=host_config,
)
client.start(container)

note that strangely the port bindings are keyed by container whereas the volume bindings are keyed by host.

Bash within the container

While developing we often find it convenient to run a terminal in the container and look around. We can do this with:

$ docker run --interactive --tty image_name /bin/bash

To achieve the same in Python, I recommend using the dockerpty library. Using dockerpty (assuming you have run pip install dockerpty), and using the client, ports, port_bindings, volumes, volume_bindings, and host_config from above allows the following equivalent python:

import dockerpty
container = client.create_container(
    image='image_name',
    ports=ports,
    volumes=volumes,
    host_config=host_config,
    tty=True,
    stdin_open=True,
    command='/bin/bash',
)
dockerpt.start(client, container)

Extra snippets

It is often useful to know inside a container how to find its host. We specify a hostname ‘dockerhost’ with the host’s IP address, using the following command,

$ docker run --add-host=dockerhost:$(DOCKER_HOST_IP)

or in Python it can be added via the host_config

extra_hosts = {
    'dockerhost': docker_host_ip,
}
host_config = client.create_host_config(
    binds=volume_bindings, 
    port_bindings=port_bindings, 
    extra_hosts=extra_hosts,
)

In both cases, determining the host IP is left as an exercise to the reader.