Commit 6c33de8e authored by Maarten de Waard's avatar Maarten de Waard 🤘🏻

Merge branch 'master' into production

parents a12277b6 8dabd621
---
BUNDLE_PATH: vendor/bundle
BUNDLE_DISABLE_SHARED_GEMS: '1'
BUNDLE_PATH: "vendor/bundle"
BUNDLE_DISABLE_SHARED_GEMS: "1"
BUNDLE_FROZEN: "true"
[general]
ignore=body-is-missing
---
layout: page
title: Design documentation
tagline: OpenAppStack design and setup
---
## Application build pipeline
The following diagram explains the process to go from an application's source
code to a deployment on OpenAppStack.
![Application build process](assets/diagrams/application-build-process.png)
These are the steps in more detail:
- Build container (this process should be maintained by application developer by
providing a Dockerfile with the application)
1. Get application package (source code, installation package, etc.)
1. If not part of the package: get default configuration for the
application
1. Build container with application package installed
1. Install application dependencies
1. Install application package
1. Setup default configuration
1. Setup pluggable configuration override, can be:
- Reading environment variables
- Extra configuration file mounted into the container elsewhere
- Helm chart
- Deployment configuration to specify:
- The container(s) that should be deployed.
- The port(s) that they expose.
- Volume mounts for configuration files and secrets.
- Live/readyness probes
- Persistent storage locations and methods
- A lot of other things
- Service configuration to specify:
- Ports exposed to the user of the application
- Ingress configuration to specify:
- How to proxy to the application (which hostname or URL)
- Some authentication plugins (http auth, for example)
- Custom files:
- Add file templates for mountable application configuration files
- Files that specify integrations with other services
- Deploy
1. Create `values.yaml` file with the variables for the Helm deployment to
the Kubernetes cluster
1. "Manually" add secrets to the Kubernetes cluster.
1. Run `helm install` to install the customised application.
## Configuration
As can be seen in the images, applications are expected to have two different
types of configuration. Containers should provide a default configuration,
that at least configures things like the port the application runs on, the
locations for log files, etc.
What we call the *external configuration* is provided by the user. This includes
overrides of the default application, as well as variables like the hostname
that the application will run on and listen to and the title of the web
interface.
OpenAppStack will use Helm charts to provide the external configuration for the
"Deploy" step. Helm charts can contain configuration file templates with
default values that can be overridden during the installation or upgrade of a
helm chart.
## Application containers
For inclusion in OpenAppStack, it is required that the application developers
provide Docker containers for their applications. There are several reasons for
this:
- If application developers do not provide a container, chances are they also do
not think about how their application would update itself after a new
container is deployed. This can lead to problems with things like database
migrations.
- Maintaining the containerisation for an application can, in most cases, not be
fully automated.
### Container updates
When an application update is available, these updates need to be rolled out to
OpenAppStack instances. This will be done according the following steps:
1. Application container is built with new application source and tagged for
testing.
4. Helm chart for application is updated to provide new container.
3. Helm chart is deployed to an OpenAppStack test cluster following the steps in
the diagram above.
2. Application is tested with automated tests
3. If tests succeed, new container is tagged for release.
4. OpenAppStack automated update job fetches new Helm chart and upgrades current
instance using Helm.
Most of these steps can be developed by configuring a CI system and configuring
Kubernetes and Helm correctly. The automated update job that will run on
OpenAppStack clusters will be developed by us.
## Persistent data
The basic idea of persistent data is that it should not be present within a
Docker container. It is possible to add "volumes" for data storage. More
research should go into "volume drivers" that make sure data can be shared over
different containers. This is especially important if containers run on
different Kubernetes worker nodes.
It is also possible to encrypt the contents of volumes.
---
layout: page
title: Documentation
tagline: OpenAppStack design and setup
tagline: OpenAppStack documentation overview
---
## Application build pipeline
- ## [Installation tutorial]({% link tutorial.md %})
All you need to know to set up your own OpenAppStack on a VPS!
- ## [Design documentation]({% link design.md %})
The following diagram explains the process to go from an application's source
code to a deployment on OpenAppStack.
Description of the design of OpenAppStack. Explains the application build
pipeline and how applications can be configured in such a pipeline
![Application build process](assets/diagrams/application-build-process.png)
These are the steps in more detail:
- Build container (this process should be maintained by application developer by
providing a Dockerfile with the application)
1. Get application package (source code, installation package, etc.)
1. If not part of the package: get default configuration for the
application
1. Build container with application package installed
1. Install application dependencies
1. Install application package
1. Setup default configuration
1. Setup pluggable configuration override, can be:
- Reading environment variables
- Extra configuration file mounted into the container elsewhere
- Helm chart
- Deployment configuration to specify:
- The container(s) that should be deployed.
- The port(s) that they expose.
- Volume mounts for configuration files and secrets.
- Live/readyness probes
- Persistent storage locations and methods
- A lot of other things
- Service configuration to specify:
- Ports exposed to the user of the application
- Ingress configuration to specify:
- How to proxy to the application (which hostname or URL)
- Some authentication plugins (http auth, for example)
- Custom files:
- Add file templates for mountable application configuration files
- Files that specify integrations with other services
- Deploy
1. Create `values.yaml` file with the variables for the Helm deployment to
the Kubernetes cluster
1. "Manually" add secrets to the Kubernetes cluster.
1. Run `helm install` to install the customised application.
## Configuration
As can be seen in the images, applications are expected to have two different
types of configuration. Containers should provide a default configuration,
that at least configures things like the port the application runs on, the
locations for log files, etc.
What we call the *external configuration* is provided by the user. This includes
overrides of the default application, as well as variables like the hostname
that the application will run on and listen to and the title of the web
interface.
OpenAppStack will use Helm charts to provide the external configuration for the
"Deploy" step. Helm charts can contain configuration file templates with
default values that can be overridden during the installation or upgrade of a
helm chart.
## Application containers
For inclusion in OpenAppStack, it is required that the application developers
provide Docker containers for their applications. There are several reasons for
this:
- If application developers do not provide a container, chances are they also do
not think about how their application would update itself after a new
container is deployed. This can lead to problems with things like database
migrations.
- Maintaining the containerisation for an application can, in most cases, not be
fully automated.
### Container updates
When an application update is available, these updates need to be rolled out to
OpenAppStack instances. This will be done according the following steps:
1. Application container is built with new application source and tagged for
testing.
4. Helm chart for application is updated to provide new container.
3. Helm chart is deployed to an OpenAppStack test cluster following the steps in
the diagram above.
2. Application is tested with automated tests
3. If tests succeed, new container is tagged for release.
4. OpenAppStack automated update job fetches new Helm chart and upgrades current
instance using Helm.
Most of these steps can be developed by configuring a CI system and configuring
Kubernetes and Helm correctly. The automated update job that will run on
OpenAppStack clusters will be developed by us.
## Persistent data
The basic idea of persistent data is that it should not be present within a
Docker container. It is possible to add "volumes" for data storage. More
research should go into "volume drivers" that make sure data can be shared over
different containers. This is especially important if containers run on
different Kubernetes worker nodes.
It is also possible to encrypt the contents of volumes.
{:.post-list}
---
layout: page
title: Tutorial
tagline: OpenAppStack installation instructions
---
# OpenAppStack Tutorial
Here's how you can set up a single-node OpenAppStack cluster. Support for multi-node clusters will come in the future.
## Prerequisites
* A virtual machine or bare metal server with:
* Ubuntu 18.04
* A public IP address
* 10GB of disk space and 4GB of RAM
* Root ssh access
### DNS entries
First, begin with creating DNS records for your cluster. It's important to start with configuring DNS because depending on your DNS setup/provider, it takes a while to propagate.
You need one dedicated subdomain entry and a wildcard entry for everything inside it. For example, create an A record for these domains:
- oas.example.org
- \*.oas.example.org
and make them point to your machine's public IP address. It is very important that you use the first address after the '\*' in the second address.
## Configure your cluster
Clone the OAS bootstrap repo:
git clone https://code.greenhost.net/openappstack/bootstrap.git
Copy `ansible/inventory.yml.example` to `ansible/inventory.yml` and edit it to reflect your cluster.
Also copy `ansible/group_vars/cluster/settings.yml.example` to `ansible/group_vars/cluster/settings.yml` and edit as you see fit.
### Prerequisites
* You need `ansible >= 2.6` installed on your workstation to run the bootstrap
scripts. Please install it using your system package manager.
In the case your system package manager doesn't provide `ansible >= 2.6` you
can install it via the python package manager like this:
```
pip3 install --user -r ansible/requirements.txt
```
Hint: if you have several python projects on your PC, consider using [virtualenv](https://virtualenv.pypa.io/en/stable/)
Hint: if you get a [segmentation fault](https://bitbucket.org/cffi/cffi/issues/272/segfault-while-installing-via-pip) using above command, you can add `--no-use-wheel` to it.
### Installation
The bootstrap process sets up a single-node kubernetes cluster on the machine
and installs the utility tools `helmfile`, `helm`, `kubectl` and `rke`.
To run the bootstrap process, you need to move into the `ansible/` directory,
then run
```
ansible-playbook bootstrap.yml
```
It will take approx. 5-10 mins to setup your cluster.
Please report any installation issue as described under [Troubleshooting](#troubleshooting).
### Usage
Right now, there's two applications pre-installed. One of them is [keycloak](https://www.keycloak.org/), an open source identity and access management application. It's used for [single-sign-on](https://en.wikipedia.org/wiki/Single_sign-on) to other application that can get installed later.
The admin console web interface should be reachable at https://auth.oas.example.org.
The other one is [traefik](https://traefik.io/), which acts as a reverse proxy and load balancer for other applications.
It also fetches [letsencrypt](https://letsencrypt.org/) certificates for securing https access to deployed apps. Its web interface is available at https://traefik.oas.example.org.
## Managing an existing cluster
Login to your cluster with:
ssh USER@oas.example.org
where `USER` the `ansible_user` you configured in `ansible/inventory.yml`.
Some programs that are installed on your cluster:
* `kubectl` is the Kubernetes control program.
For example, run `kubectl get pods -n oas` to see a list of pods that exist
in the `oas` namespace (system applications like traefik),
and `kubectl get pods -n oas-apps` for all other OpenAppStack applications
(currently keycloak).
Run `kubectl --help` for help.
* `helm` is the "Kubernetes package manager". Use `helm ls` to see what apps are
installed in your cluster. You can also use it to perform manual upgrades;
see `helm --help`.
* `helmfile` is a high-level tool to manage your app installations.
It's manual usage is a bit tricky since [current helmfile config depends on environmental variables to be present](https://code.greenhost.net/openappstack/bootstrap/issues/101).
## Troubleshooting
In case something goes wrong or you have any questions or comments please contact [get in contact with us otherwise](/contact.html).
The output of these commands is helpful to debug deployment errors:
kubectl get pods
helm ls
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment