November 19, 2015

First Look at Docker

  1. Wow, Docker!
  2. Problem
    1. DIY
    2. Virtualization
    3. Automating Virtualization
    4. Still not satisfied
  3. Containers vs Virtual Servers
  4. Underlying Kernel Technologies
  5. First Experiences with Docker

Wow, Docker!

I'm excited to dive deeper into Docker over the next few months. I'm optimistic that it might actually be the holy grail of devops (for me at least).

Problem

With all of these buzzwords around virtualization and dev ops, I think it's important to take a step back and define the problem we're trying to solve.

From my perspective, whether you're using virtual machines, physical servers, the cloud, docker, or whatever; someone, somewhere has to do the following steps:

  1. Buy Hardware
  2. Install and configure an operating system
  3. Install software
  4. Maintain Environment (Add/remove users, manage log files, troubleshooting, updating packages, etc).

As I see it, here are the current options as of 2015.

DIY

You can do all of the steps yourself. You can still go to the store and buy an actual, physical, server, bring it home, install an operating system and set it all up.

I actually think that anyone and everyone who is paid to program or administer a computer should do this at least once (or 10 times!). I think this process is essential for understanding how computers work (but this is getting off topic).

Anyone who has done this whole process themselves knows that eventually, it gets debilitatingly redundant and tedious.

Virtualization

One way to make this process easier is to decouple the software from the hardware by using a virtualization technology such as VMWare or Virtualbox.

This way, you do steps 1-3 once in order to build a re-usable virtual "image". Once you have an image, from then on, it's no big deal to spin it up and start using it.

Virualization enables you to start a ruby on rails/mysql image, for example, use it for a few hours, then spin up another image, this time maybe with clojure/postgres and play around with that.

This is something that would take days if you were to use DIY approach above. Virtualization allows you to spend a few hours setting up these images and now you can switch back and forth in minutes. Pretty sweet.

Automating Virtualization

Building and maintaining virtual images manually also gets redundant and tedious after a while. So, this is where solutions like Chef, Puppet, and Pallet can help. Rather than manually executing commands to configure a server, why not have a program do it for you?

You can write a script, send it to Chef, Puppet, or Pallet and they will build and configure a virtual image for you. Now we're talking.

Still not satisfied

The evolution from DIY to virtualization is really useful in my experience. It's really nice to be able to think of hardware as separate from operating system and software.

The promise of easier automatic management of virtual servers with Puppet/Chef/Pallet sounds great in theory. But, I've found that in practice, it leaves a lot to be desired.

Although I can't really put my finger on it, I think the underlying issue has something to do with the fact that maybe we've outgrown the old concepts of "Operating Systems" and "File Systems"? I feel like these abstractions could use a bit of innovation. I wish I had time to dig in more to be able to elaborate on this "hunch", but luckily, it seems like Docker has some good ideas that really resonated with my intuitions.

Containers vs Virtual Servers

So, rather than virtualize an entire operating system environment, Docker takes a different approach. Docker uses advanced features of the kernel to create miniature virtualized environments.

The Docker Host Operating System must have a Kernel that supports features which Docker uses such as Namespaces, Control Groups, and Union File System. Currently, I think it's possible to run Docker Host on RHEL, Ubuntu, and CentOS.

But, the good news is that it works nicely on other Operating Systems too. In order to use Docker on OSX and Windows, for example, Docker provides tools that spin up a Docker host inside Ubuntu running inside virtualbox. Once you do this initial setup, you can sort of "forget" that Docker Host is running in virtualbox. Everything works as if you're running Docker Host Natively.

Underlying Kernel Technologies

I'm really intrigued by the ideas that Docker is built on and I'm looking forward to learning more about the underlying technology

I really like the idea of changing the way the Kernel works fundamentally. Namespaces, control groups, and union file systems all seem to be heading in the right direction.

First Experiences with Docker

My first experience with Docker has been to setup a minecraft server for my kids. I used this as just a quick proof of concept and so far I'm very impressed and excited at the potential.

I was using Pallet to manage minecraft server before and it worked ok.

My first experience with Docker is that it's feels much easier (and seems more "fun") to manage containers compared to entire virtual servers. I'm happy to report that it only took me 3-4 evenings to learn Docker (the documentation is excellent), configure and manage a minecraft server, and setup a container that does scheduled backups to amazon s3. The old solution wasn't even able to backup.

I'm really excited about the potential of moving all my stuff over to Docker. Next up: try deploying a production clojure/postgres code using Docker!

Tags: devops software tech