Open research

In the spirit of free and open source software the central topic of this site is to make my research open to the public. My methodology is freely available via the internet, along with any data or results extracted or derived from them. This permits collaboration, which anyone may participate at any level of the project. So feel free to contribute on Github.

Homepage » Docker Container Virtualization

Docker

Docker is a tool that can package an application and its dependencies in a virtual container. This container can then be run on any Linux server that has docker installed.

This has benefits for several groups of people:

Docker includes the libcontainer library as a reference implementation for containers, and builds on top of libvirt, LXC (Linux containers) and systemd-nspawn, which provide interfaces to the facilities provided by the Linux kernel.

How libcontainer works with Linux services

Source (image)

Containers offer quite some pros over virtual machines e.g. :

Organizations that use Docker are e.g. Atlassian, eBay, Gilt, Groupon, RelateIQ, Spotify, Tutum

Docker (Host) support

Docker can be run on any x64 host (64-bit architecture e.g. x86_64 and amd64 only – 32-bit is not supported) running a modern Linux kernel (>= 3.8). The kernel must support an appropriate storage driver e.g. Device Mapper, AUFS, vfs, btfrs (default is usually Device Mapper).

The 3 docker components

Images - Build component

At the base of a container is a boot filesystem, bootfs which resembles the typical Linux/Unix boot filesystem. When a container has booted, it is moved into memory, and the boot filesystem is unmounted to free up the RAM used by the initrd disk image. Docker next layers a root filesystem, rootfs, on top of the boot filesystem. This rootfs can be one or more operating systems (e.g., a Debian or Ubuntu filesystem).

In opposite to traditional Linux distributions rootfs stays in read-only mode and Docker takes advantage of a union mount to add more read only filesystems onto the root filesystem. A union mount is a mount that allows several filesystems to be mounted at one time but appear to be one filesystem. The union mount overlays the filesystems on top of one another so that the resulting filesystem may contain files and subdirectories from any or all of the underlying filesystems.

Docker calls each of these filesystems images. Images can be layered on top of one another. Images can either be defined by Dockerfiles or by committing a container. When trying to run a container, docker will automatically download the image you specified. This will become more clear at the examples.

The Docker filesystem layers look like this…

When Docker first starts a container, the initial read-write layer is empty. As changes occur, they are applied to this layer; for example, if you want to change a file, then that file will be copied from the read-only layer below into the read-write layer. The read-only version of the file will still exist but it is now hidden underneath the copy.
This pattern is traditionally called “copy on write” and is one of the features that makes Docker so powerful.

Local images live on the local Docker host in the /var/lib/docker and var/lib/docker/containers directory. Each image will be inside a directory named for your storage driver; for example, aufs or devicemapper.

Base images

Registries - Distribution component

Images live inside repositories, and repositories live on registries. There are two types of registries: public and private.

Docker, Inc. itself provides a public hub where you can upload and share your images. You get one free private repository and unlimited public repositories. You could also host your own registry, more instructions can be found here.

Once the image is on a registry, you can run the same container on another location.

Containers - Run component

Containers are created from an image. These containers hold everything needed for your apps to run. These containers can be run/started/stopped/removed/…

It is possible to turn a container into an image by using the docker commit command.

What are those containers?

Basic commands

Simple example:

Lifecycle

Info

Import / Export

Entering a Docker Container

Images

Lifecycle

Info

Registry & Repository

A repository is a hosted collection of tagged images that together create the file system for a container.

A registry is a host - a server that stores repositories and provides an HTTP API for managing the uploading and downloading of repositories.

Docker.io hosts its own index to a central registry which contains a large number of repositories.

Shortcuts

Dockerfile

There are two ways to create a Docker image. a) via the docker commit command or b) via the docker build command with a Dockerfile e.g.

If a file named .dockerignore exists in the root of the build context then it is interpreted as a newline-separated list of exclusion patterns. Much like a .gitignore file it excludes the listed files from being upload to the build context.

Using the build cache for templating

FROM guttertec/ubuntu:14.04
MAINTAINER Axel Quack "mail@axelquack.de"

ENV REFRESHED_AT_2014-09-16
RUN apt-get --qq update

The ENV instruction sets environment variables in the image. In this case, the ENV instruction set an environment variable called REFRESHED_AT, showing when the template was last updated. Whenever you want to refresh the build you just change the date in the ENV instruction.

Dockerfile Commands

Working with Docker Hub

Deployments

Automated Builds

  1. Create a Docker Hub account and login.
  2. Link your GitHub or BitBucket account through the “Link Accounts” menu.
  3. Configure an Automated Build.
  4. Pick a GitHub or BitBucket project that has a Dockerfile that you want to build.
  5. Pick the branch you want to build (the default is the master branch).
  6. Give the “Automated Build” a name.
  7. Assign an optional Docker tag to the Build.
  8. Specify where the Dockerfile is located. The default is /.

Once the Automated Build is configured it will automatically trigger a build and, in a few minutes, you should see your new Automated Build on the Docker Hub Registry. It will stay in sync with your GitHub and BitBucket repository until you deactivate the Automated Build.

If you want to see the status of your Automated Builds, you can go to your Automated Builds page on the Docker Hub, and it will show you the status of your builds and their build history.

AWS

Run your own Docker registry

Docker installation (OS X)

If you are running Mac OS X you should install Homebrew for easier handling…

Install Homebrew

ruby -e "$(curl -fsSL https://raw.github.com/Homebrew/homebrew/go/install)"

Install VirtualBox

With Homebrew, it’s trivial to install VirtualBox which is a prerequisite to running docker on OS X:

brew update
brew tap phinze/homebrew-cask
brew install brew-cask
brew cask install virtualbox

Option 1: Install boot2docker

Boot2docker is a small script that helps download and setup a minimal Linux VM that will be in charge of running docker daemon.

brew install boot2docker
boot2docker init
boot2docker up
export DOCKER_HOST=tcp://localhost:4243
# docker will be automatically too, since it is a boot2docker dependency

Share folders with boot2docker

Usually you can not use Shared Folders with boot2docker outside of the VM. Though there are still a few workarounds…

Share between containers is surely possible.

Option 2: Install Kitematic

Option 3: Install CoreOS with Vagrant

Other installation methods

Docker user interfaces

Docker orchestration (untested)

other

Docker for Continuous Integration

Docker security

Docker daemon runs with root privileges, which implies there are some issues that need extra care. Some interesting points include the following:

Some key Docker security features include the following:

  1. Processes, when run as non-privileged users in the containers, maintain a good level of security.
  2. Apparmor, SELinux, GRSEC solutions can be used for an extra layer of security.
  3. There’s a capability to inherit security features from other containerization systems.

Interesting…

… resources

… articles

… courses

Snippets unsorted

Some things I found as interesting but I did not manage to sort right now.

Open questions

These are still some things I have to learn about… (this is more or less a reminder to myself)