Commit 584b8947 authored by Maarten de Waard's avatar Maarten de Waard 🤘🏻

some changes to the docker swarm vs kubernetes post

parent 3dc56d93
---
layout: post
author: Arie Peterson
title: "Platform choice: choosing between Docker Swarm and Kubernetes"
excerpt_separator: <!--more-->
title: "Docker Swarm vs. Kubernetes"
excerpt: Choosing the right platform for OpenAppStack.
---
<!--more-->
The OpenAppStack project involves the installation, configuration and
management of containerised applications and services, on a cluster of connected
......@@ -22,7 +21,7 @@ a few weeks.
## Comparison
### Setting up and updating passwords
__Docker Swarm__ provides mechanisms for storing, distributing and injecting
configuration (Docker Config) and passwords (Docker Secrets). These two are
essentially the same, only Secrets are better protected (only mounted in memory,
......@@ -46,7 +45,7 @@ Passwords can be added to __Kubernetes__ by setting "secrets". Secrets can be
mounted as a volume in a container, or used by a "kubelet" (container manager)
when setting up a "pod" (one or more containers linked together, e.g., an
Etherpad instance with an Etherpad container and a MariaDB container).
Kubernetes secrets need to be created *before* creating a pod that depends on
them. This is the same as in Docker Swarm and can be worked around by
"versioning" your secrets.
......@@ -74,20 +73,20 @@ not have corrupted your data in the meantime.
__Docker Swarm__ will restart your container when it exits unexpectedly. You can
configure how many times it should retry a failed container and how long it
should wait between tries.
__Kubernetes__ also detects various faults. Apart from restarting failed pods, it
is also possible to halt an updating process when the updated pod does not
start. Well defined services will usually be able to "garbage collect" old pods
that did not start correctly. It is possible to check whether a pod (or actually
a deployment) started correctly with customisable commands (`livenessProbe` and
`readynessProbe`).
__Kubernetes__ also detects various faults. Apart from restarting failed pods,
it is possible to halt an updating process when the updated pod does not start
as well. Well defined services will usually be able to "garbage collect" old
pods that did not start correctly. It is possible to check whether a pod (or
actually a deployment) started correctly with customisable commands
(`livenessProbe` and `readynessProbe`).
### Generating custom configuration files
The configuration of OpenAppStack apps will be done as much as possible by the
OpenAppStack developers, using sane and secure settings to provide a good default
installation. However, some configuration will inevitably be have to be provided
by the person setting up the app instance, say the external domain name of the
by the person setting up the app instance, like the external domain name of the
instance.
That means that some templating will have to be done by the process that sets up
......@@ -99,19 +98,17 @@ __Docker Swarm__ provides no facility for this, as far as we could find. We
implemented a small proof-of-concept tool to fill this hole, using Jinja2
templating. This has the advantage that Jinja2 templates are quite common,
being used by Ansible. Still it would be unfortunate that using Docker Swarm,
we'd have to write a substantial amount of configuration management code
we would have to write a substantial amount of configuration management code
ourselves.
Custom configuration files are not built into __Kubernetes__. However, a tool
has been developed to relieve this and other shortcomings, called "Helm". Helm
serves to automate big parts of the boilerplate Kubernetes configuration, using
a templating system based on Go templates.
This templating system can also be used to create application configuration
files, that can be added to the cluster as a so-called ConfigMap, which is then
mounted into the application container as a regular file to be read by the
application.
This templating system can, among other things, be used to create application
configuration files that can be added to the cluster as a so-called ConfigMap.
### Performance impact
Preliminary tests show a substantial difference in memory footprint.
......@@ -120,32 +117,35 @@ A single-node __Docker Swarm__ cluster used only around 100 MB of memory without
any services, rising to 300 MB for a functioning Etherpad setup (including a
MariaDB database).
The same setup using __Kubernetes__ started at 700 MB, with problems to fit
inside the 1 GB VPS when installing and updating the Etherpad service.
The same setup using __Kubernetes__ started at 700 MB. Problems arose inside our
Ubuntu VPS with 1 GB of RAM when installing and updating the Etherpad service.
Additionally, a Kubernetes cluster will typically have at least one master node
which does nothing other than managing the cluster (which applications run on
which worker node, etc.). It is possible though to run a complete cluster on a
single node, which we think should be possible for simple OpenAppStack
installations. __Docker Swarm__ naturally runs on a single node.
which worker node, etc.). It is possible, though, to run a complete cluster on a
single node, which we think should be enough for simple OpenAppStack
installations. __Docker Swarm__ runs on a single node out of the box, because
master nodes are allowed to run application containers by default.
### Developer complexity
The learning curve for the __Docker Swarm__ ecosystem has been gentle, starting
from basic knowledge of Docker images and containers. It's pretty easy to get
started and set up a simple cluster. Manageability and debugging-friendliness
are hard to estimate at this early stage of the project.
started and set up a simple cluster. However, Manageability and
debugging-friendliness are hard to estimate at this stage.
__Kubernetes__ has a comprehensible API, but before being able to configure
Kubernetes for a web application one will have to learn about a lot of concepts.
This makes the learning curve of Kubernetes relatively steep.
This is especially true when trying to set up Kubernetes on VPSs, instead of
using of Amazon's AWS or Google's GCE. This makes the learning curve of
Kubernetes relatively steep.
On the other hand, logging is nice. If your container is configured correctly,
you can get your application logs by typing `kubectl logs <pod-name>`.
Logging is nice though. If your container is configured correctly, you can get
your application logs by typing `kubectl logs <pod-name>`.
By using Helm in combination with Kubernetes, the amount of work a developer
needs to do to deploy different applications reduces, because of the templating
system that is built into it.
needs to do to deploy different applications reduces, because they can use
default templates for certain types of applications.
### Well known / big community
......@@ -158,13 +158,13 @@ thousands of StackOverflow questions about it and its Github page is updated
frequently and starred by more than 40.000 people.
Note however, that if Helm is used in combination with Kubernetes, that
reduces the number of active users. The tool is in use with some companies and
reduces the number of active users. The tool is used by some companies and
advised by others, but (logically) the community is not as big as that of Docker
and Kubernetes yet.
or Kubernetes yet.
### Open source
Both Docker Swarm and Kubernetes are open-source projects.
Both Docker Swarm and Kubernetes are open source projects.
### Ease of porting existing applications
......@@ -186,7 +186,10 @@ MariaDB helm chart to create a Kubernetes deployment of Etherpad.
## Decision
We decided to go with Kubernetes for the OpenAppStack project. Deciding factors
It is always difficult to make a decision between two software packages,
especially because there is never a one best choice.
For OpenAppStack, we decided to go with Kubernetes. Deciding factors
for us are the features provided to help with configuration management
(especially when using Kubernetes with Helm), the community backing and
corresponding expected pace of future development
(especially when using Kubernetes with Helm), the community backing it and
the corresponding expected pace of future development.
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment