Shipping your Environment

Continuous integration… continuous deployment… continuous delivery… call it what you will, but they all have a common idea of continuity to them.  The idea that you can truly find a one-size-fits-all solution to the problems that keep developers from developing, is tremendously exciting.

Continuous Delivery

Continuous delivery is all about making the development cycle shorter and more efficient.  If you notice a feature in your application is broken (hopefully before it goes to production), then you want to be able to fix the fault as quickly as possible, without compromising the integrity of your operation.  Thus, on the one end you have a need for developers to be able to iterate as quickly as possible, and on the other end you want to ensure that developers cannot take malicious code to production (whether accidental or otherwise).

To make this process simpler, and with less room for error, there is a key concept to wrap your mind around: shipping your application environment with your application. Wouldn’t it be amazing to have the exact same environment where your work, test, build and deploy?  Using Docker makes this a breeze.  Docker allows you to contain everything your application needs in a single, neat package.  Your application works better on Ubuntu?  Oh, you need version 2.7.5 of Python? You use some obscure package you found on Github?  You can package all of that with your application with the peace of mind that if it works on your machine, it will work on your build server, on the client’s infrastructure, in the cloud, on Windows, on Mac, everywhere.  As long as you have the Docker Engine running.

Docker

In the words of Docker’s own engineers:

Docker containers wrap a piece of software in a complete filesystem that contains everything needed to run: code, runtime, system tools, system libraries – anything that can be installed on a server. This guarantees that the software will always run the same, regardless of its environment.

So the concept of Docker is quite simple: you create a blueprint of your environment, called a Docker Image, that contains everything your application needs, then you instantiate a Docker Container, which then runs your blueprint on top of a Linux kernel.

You might ask “How is this different from virtual machines?”.  Where a virtual machine contains an entire operation system, a docker container only contains the application and its dependencies and then share the underlying Linux kernel with the host infrastructure.  This naturally means that Docker containers take up around 2GB in size (depending on what is in them), where virtual machines can amount to tens of GBs.  This then further means that Docker containers can much more easily scale up to hundreds of instances, because they use up so much less resources.

The Docker Infrastructure

No more need for developers to maintain old machines sitting somewhere in a data center to make sure they have the latest version of Python.  You simply ship a docker image containing the exact version you need with your application.

The Community

There are various emerging technologies that have been born out of the Docker idea space.  Docker orchestrators, registries and sharing hubs are among them.  Teams can now share their images on various platforms, one of which is Docker Hub (think GitHub for images), where teams can both benefit from the massive market of existing images, or keep a private repository of their work.

Then there are orchestrators, allowing you to automatically deploy your Docker image to any number of hosts, automatically distributed, with load balancing, service discovery and redundancy built in.  The most prominent Docker orchestrators are Docker Swarm and Kubernetes.

Conclusion

We at Praelexis have long thought about the challenge of taking machine learning solutions into production on a global scale.  We don’t want to present a solution to a problem, we want to be the solution, and produce real value to client’s and customers.  We believe Docker is one major step in the right direction and we cannot wait to find out where it will take us.