Docker¶

What is Docker¶

Step 1 - setting up the servers¶

Nowadays the servers are usually preinstalled or an installation process can be kicked off via web interface. For the F4A use case we chose Ubuntu 16.04 LTS (Long term support).

First we should ensure that the system is up-to-date and secure. This is done by kicking off the advanced packaging tool (apt). Within this process we can directly install the docker server component. All steps are done by issuing the following command:

apt-get update && apt-get upgrade -y && apt install -y docker.io

As docker is still being developed, certain functionality still changes. This tutorial has been created using the following docker version (you can find our yours by executing `docker version`):

root@nxp100:~# docker version
Client:
 Version:      1.13.1
 API version:  1.26
 Go version:   go1.6.2
 Git commit:   092cba3
 Built:        Thu Nov  2 20:40:23 2017
 OS/Arch:      linux/amd64

Server:
 Version:      1.13.1
 API version:  1.26 (minimum version 1.12)
 Go version:   go1.6.2
 Git commit:   092cba3
 Built:        Thu Nov  2 20:40:23 2017
 OS/Arch:      linux/amd64
 Experimental: false
root@nxp100:~#

Step 2 - initiate a swarm¶

Setting up the docker swarm. A swarm is a group of computers:

docker swarm init --advertise-addr 89.144.27.100

getting the feedback:

Swarm initialized: current node (r3t3pu7rd74njml1afsf2uoev) is now a manager.

To add a worker to this swarm, run the following command:

    docker swarm join \
    --token <some secret token displayed here> \
    89.144.27.100:2377

To add a manager to this swarm, run 'docker swarm join-token manager' and follow the instructions.

If you don’t remember the token etc. - just run:

docker swarm join-token worker

Step 3 - preparing the domain¶

register any domain you like. Just make sure that the domain names are pointing to all the server IPs you have. With that load balancing / failover is possible:

f4a.me.           86400 IN  SOA nsa3.schlundtech.de. mail.tillwitt.de. 2017112808 43200 7200 1209600 86400
f4a.me.           86400 IN  NS  nsa3.schlundtech.de.
f4a.me.           86400 IN  NS  nsb3.schlundtech.de.
f4a.me.           86400 IN  NS  nsc3.schlundtech.de.
f4a.me.           86400 IN  NS  nsd3.schlundtech.de.
f4a.me.           600   IN  MX  10 mail.f4a.me.
*.f4a.me.         600   IN  A   89.144.24.15
*.f4a.me.         600   IN  A   89.144.27.100
*.f4a.me.         600   IN  A   89.144.27.101
*.f4a.me.         600   IN  A   89.144.27.102
*.f4a.me.         600   IN  A   89.144.27.103
nxp100.f4a.me.    600   IN  A   89.144.27.100
nxp101.f4a.me.    600   IN  A   89.144.27.101
nxp102.f4a.me.    600   IN  A   89.144.27.102
nxp103.f4a.me.    600   IN  A   89.144.27.103
nxp104.f4a.me.    600   IN  A   89.144.24.15

Docker commands¶

docker run vs docker compose¶

Advantages of docker run are that the command is easy to issue, just a copy & paste to the servers command line. Downside is, that the commands get quite long and adding line breaks introduces another possible fault. If you want to correct a running service you need to remove it first and then reissue it.

Advantages of using a docker-compose.yml is that they are usually easy to edit. Disadvantage is that you have to create them on the server first then issue the command to start them - so one additional step. But the biggest advantage is that they can be re-executed on existing services which will lead to a service update.

Examples¶

starting a generic web application with docker run

docker service create \
    --name demo \
    --label "traefik.port=80" \
    --network traefik-net \
    kitematic/hello-world-nginx

Thats all - and the service is running.

To create the same via docker-compose.yml

version: "3"

services:
  nginx:
    image: kitematic/hello-world-nginx
    networks:
      - traefik-net
    deploy:
      labels:
        - traefik.port=80
        - "traefik.frontend.rule=Host:demo.f4a.me"
networks:
  traefik-net:
   external: true

Then you need to issue the following command

docker stack deploy --compose-file docker-compose.yml demo

Conclusion¶

To quickly test a service - docker run is nice. But to maintain a production environment docker-compose files are strongly recommended.

Docker registry¶

Running your own registry

docker service create \
   --name backoffice \
   --network traefik-net \
   --label "traefik.port=5000" \
   --label 'traefik.frontend.auth.basic=flex4apps:$apr1$G9e4rgPu$jbn2AAk2F.OeGnRVFnIR/1' \
   --mount type=bind,src=/swarm/volumes/registry,dst=/var/lib/registry \
   registry:2

Pushing to private registry¶

The local image needs to be tagged and then pushed

docker tag phabricator_image registry.f4a.me/phabricator
docker push registry.f4a.me/phabricator

Run that image

docker service create \
  --name demo \
  --label "traefik.port=80" \
  -e "GITURL=https://secret@gogs.tillwitt.de/NXP/homomorphic-encryption-demo.git" \
  flex4apps:GQfgCEsjkHC7LRf3Q9PkW4L6onDLtu@backoffice.f4a.me/homomorphic_img

Query the registry¶

Get the overview of all images:

https://registry.f4a.me/v2/_catalog

Get all tags of an image:

https://registry.f4a.me/v2/henc/tags/list

Private repository viewer¶

In the mean time there are good solutions to provide a secure, private and selfhosted docker registry like.

Applications¶

What are applications¶

On the created system applications can be launched. Templates for applications are called images. Running applications are called containers.

Generic demonstrator¶

Simple demo¶

The following application is already working with the current setup.

To launch an easy demonstrator, lets instantiate a webserver and make it available at demo.f4a.net:

docker service create \
    --name demo \
    --label "traefik.port=80" \
    --network traefik-net \
    kitematic/hello-world-nginx

Generating a new user with password run:

htpasswd -nbm flex4apps password

or go to: http://www.htaccesstools.com/htpasswd-generator/

The output will be something like:

flex4apps:$apr1$XqnUcSgR$39wlPxxyyxPxXZjFb34wo.

Example for traefik label usage below. If single quotes are in the password they would need to be escaped.

To do that close the quoting before it, insert the escaped single quote, and re-open the quoting: `'first part'\''second part'`

How to start the demo service:

docker service create \
    --name demopw \
    --label "traefik.port=80" \
    --label 'traefik.frontend.auth.basic=myName:$apr1$a7R637Ua$TvXp8/lgky5MDLGLacI1e1' \
    --network traefik-net \
    kitematic/hello-world-nginx

Docker compose¶

# PULL UPDATE & LAUNCH DOCKER
#     git pull && docker stack deploy --compose-file docker-compose.yml demo

# REMOVE STACK
#     docker stack rm demo

# ADD BASIC AUTH and ESCAPE FOR docker-compose usage
# htpasswd -bBn user password | sed 's/\$/\$\$/g' #escape for docker-compose usage

version: "3"

services:
  nginx:
    image: kitematic/hello-world-nginx
    networks:
      - traefik-net
    environment:
      - test=noContent
    deploy:
      labels:
        - traefik.port=80
#        - "traefik.frontend.auth.basic=witt:$$2y$$05$$kOFY7071ilbnpiJNDaIO9e1WeuhHnKtp9Adrevz4r8wJ3b3X1XuqW"
#        - "traefik.frontend.auth.basic=ich:$$2y$$05$$jTZv0re2cXmiGrzRxW./8Ofse.6g/AEChvbMGdqYKIMqsr8xW/c"
#        - "traefik.frontend.auth.basic=user:$$2y$$05$$IRrTxLpG7ICzroI8Pb5P4.p2rMXGqyeeZM857BJxTFzP5q9W4RYuS"
        - "traefik.frontend.rule=Host:demo.f4a.me"
networks:
  traefik-net:
   external: true

grafana¶

Grafana is an open source software for time series analytics

create the service like this:

docker service create \
  --name=grafana \
  --network traefik-net \
  --label "traefik.port=3000" \
  --mount type=bind,src=/swarm/volumes/grafana,dst=/var/lib/grafana \
  -e "GF_SECURITY_ADMIN_PASSWORD=someSecretPassword" \
  grafana/grafana

Below you find the docker-compose for grafana

# Flex4Apps documentation

# PULL UPDATE & LAUNCH DOCKER
#     git pull && docker stack deploy --compose-file docker-compose.yml grafana

# REMOVE STACK
#     docker stack rm grafana

# ADD BASIC AUTH and ESCAPE FOR docker-compose usage
# htpasswd -bBn user password | sed 's/\$/\$\$/g' #escape for docker-compose usage

version: "3.1"

services:
  server:
    image:   grafana/grafana
    volumes:
      -  /swarm/volumes/grafana:/var/lib/grafana
    networks:
      - traefik-net
    environment:
      - "GF_SECURITY_ADMIN_PASSWORD=someSecretPassword"
    deploy:
      labels:
        - traefik.port=3000
        - "traefik.frontend.rule=Host:grafana.f4a.me"
      replicas: 1
#      update_config:
#        parallelism: 1
#        delay: 30s
#      placement:
#        constraints: [node.role == manager]

networks:
  traefik-net:
   external: true

Prometheus¶

Prometheus is an open-source software application used for event monitoring and alerting. It records real-time metrics in a time series database (allowing for high dimensionality) built using a HTTP pull model, with flexible queries and real-time alerting. The project is written in Go and licensed under the Apache 2 License, with source code available on GitHub, and is a graduated project of the Cloud Native Computing Foundation, along with Kubernetes and Envoy.

Below you find the reference code from the repository

# Flex4Apps documentation

# PULL UPDATE & LAUNCH DOCKER
#     git pull && docker stack deploy --with-registry-auth --compose-file docker-compose.yml prom

# REMOVE STACK
#     docker stack rm prom

# ADD BASIC AUTH and ESCAPE FOR docker-compose usage
# htpasswd -bBn user password | sed 's/\$/\$\$/g' #escape for docker-compose usage

version: "3.1"

services:
  server:
    image: prom/prometheus
    volumes:
      -  /swarm/volumes/prometheus/data:/prometheus
      -  ./prometheus.yml:/etc/prometheus/prometheus.yml
    networks:
      - traefik-net
    deploy:
      labels:
        - traefik.port=9090
        - "traefik.frontend.auth.basic=witt:$$2y$$05$$kOFY7071ilbnpiJNDaIO9e1WeuhHnKtp9Adrevz4r8wJ3b3X1XuqW"
        - "traefik.frontend.rule=Host:prometheus.f4a.me"
      replicas: 1
      update_config:
        parallelism: 1
        delay: 30s
      placement:
        constraints: [node.role == manager]

  nodeexporter:
    image: prom/node-exporter:latest
    volumes:
      - /proc:/host/proc:ro \
      - /sys:/host/sys:ro \
      - /:/rootfs:ro \
    networks:
      - traefik-net
    ports:
      - 9100:9100
    deploy:
      mode: global
      labels:
        - com.group="prom-monitoring"

    command:
      - '--path.procfs=/host/proc'
      - '--path.sysfs=/host/sys'
      - --collector.filesystem.ignored-mount-points
      - "^/(sys|proc|dev|host|etc|rootfs/var/lib/docker/containers|rootfs/var/lib/docker/overlay2|rootfs/run/docker/netns|rootfs/var/lib/docker/aufs)($$|/)"

networks:
  traefik-net:
   external: true

portainer¶

start portainer as a service we first need to create a data directory:

mkdir -p /docker/portainer

To start the container itself:

docker service create \
--name "portainer" \
--constraint 'node.role == manager' \
--network "traefik-net" --replicas "1" \
--mount type=bind,src=/var/run/docker.sock,dst=/var/run/docker.sock \
--mount type=bind,src=/docker/portainer,dst=/data \
--label "traefik.frontend.rule=Host:portainer.f4a.me" \
--label "traefik.backend=tool-portainer" \
--label "traefik.port=9000" \
--label "traefik.docker.network=traefik-net" \
--reserve-memory "20M" --limit-memory "40M" \
--restart-condition "any" --restart-max-attempts "55" \
--update-delay "5s" --update-parallelism "1" \
portainer/portainer \
-H unix:///var/run/docker.sock

Below you find the reference code from the repository for portainer.

# PULL UPDATE & LAUNCH DOCKER
#     git pull && docker stack deploy --compose-file docker-compose.yml portainer

# REMOVE STACK
#     docker stack rm mariadb

# ADD BASIC AUTH and ESCAPE FOR docker-compose usage
# htpasswd -bBn user password | sed 's/\$/\$\$/g' #escape for docker-compose usage

version: "3"

services:
  server:
    image: portainer/portainer
    networks:
      traefik-net:
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
      - /data/local/portainer:/data
    environment:
      - MYSQL_ROOT_PASSWORD=someSecretPassword
    deploy:
      labels:
        - traefik.port=9000
        - "traefik.frontend.rule=Host:portainer.f4a.me"
      placement:
        constraints: [node.role == manager]

networks:
  default:
  traefik-net:
   external: true

Phabricator¶

The database user needs to be able to create databases

change your password:

sed -i 's/<some secret>/yourPassword/g' Dockerfile

build the image with:
```
docker build -t phabricator_image .
```

tag and push it to the registry:

docker tag phabricator_image registry.f4a.me/phabricator
docker push registry.f4a.me/phabricator

deploy it from the registry by executing the docker-compose:
```
docker-compose up
```

Below you find the reference code from the repository for phabricator. This is not completely working at the end of the project

# PULL UPDATE & LAUNCH DOCKER
#     git pull && docker stack deploy --with-registry-auth --compose-file docker-compose.yml phabricator

# REMOVE STACK
#     docker stack rm phabricator

# ADD BASIC AUTH and ESCAPE FOR docker-compose usage
# htpasswd -bBn user password | sed 's/\$/\$\$/g' #escape for docker-compose usage

version: "3.1"


services:
   web:
     image: registry.f4a.me/phabricator
     networks:
       - traefik-net
#     ports:
#       - "80:80"
     deploy:
      labels:
        - traefik.port=80
        - "traefik.frontend.rule=Host:phabricator.f4a.me"

#   db:
#     image: mysql:5.7
#     volumes:
#       - ./data:/var/lib/mysql
#     restart: always
#     environment:
#       MYSQL_ROOT_PASSWORD: root
#       MYSQL_DATABASE: phabricator
#       MYSQL_USER: phabricator
#       MYSQL_PASSWORD: phabricator

networks:
  traefik-net:
   external: true

MariaDB¶

MariaDB is free and open source relational database system. It was created as a :fork: from MySQL after Oracle started releasing new functionality not as open source anymore and due to the high support cost of MySQL.

As usual make sure that the path for data volume exists:

mkdir -p /swarm/volumes/mariadb

The initiate the docker service:

docker service create \
      --name mariadb \
      --publish 3306:3306 \
      --network traefik-net \
      --mount type=bind,src=/swarm/volumes/mariadb,dst=/var/lib/mysq \
      --label "traefik.port=3306" \
      -e MYSQL_ROOT_PASSWORD=someSecretPassword \
      mariadb:latest

Below you find the reference code from the repository for mariadb

# PULL UPDATE & LAUNCH DOCKER
#     git pull && docker stack deploy --compose-file docker-compose.yml mariadb

# REMOVE STACK
#     docker stack rm mariadb

# ADD BASIC AUTH and ESCAPE FOR docker-compose usage
# htpasswd -bBn user password | sed 's/\$/\$\$/g' #escape for docker-compose usage

version: "3"

services:
  server:
    image: mariadb:latest
    networks:
      traefik-net:
    volumes:
      - /data/shared/mariadb=/var/lib/mysq
    environment:
      - MYSQL_ROOT_PASSWORD=someSecretPassword
    deploy:
      labels:
        - traefik.port=3306
        - "traefik.frontend.rule=Host:mariadb.f4a.me"

  phpmyadmin:
    image: nazarpc/phpmyadmin
    networks:
      traefik-net:
    environment:
      - MYSQL_HOST=mariadb_server:3306
    deploy:
      labels:
        - traefik.port=80
        - "traefik.frontend.rule=Host:phpmyadmin.f4a.me"

networks:
  default:
  traefik-net:
   external: true

PhpMyAdmin¶

phpMyAdmin is a free and open source administration tool for MySQL and MariaDB. As a portable web application written primarily in PHP, it has become one of the most popular MySQL administration tools, especially for web hosting services.

The following command will start up PhpMyAdmin:

docker service create \
    --name phpmyadmin \
    --label "traefik.port=80" \
    --network traefik-net \
    -e ALLOW_ARBITRARY=1 \
    nazarpc/phpmyadmin

Below you find the reference code from the repository for phpmyadmin.

gogs¶

Gogs is a painless self-hosted Git service.

Pull image from Docker Hub.

very strange installation. First need to use –publish 3000:3000 and connect direct for install. Then remove instance and also remove published port. This is certainly something I need to review.

create the data volume for gogs:

mkdir -p /swarm/volumes/gogs

start the service:

docker service create \
    --name gogs \
    --mount type=bind,src=/swarm/volumes/gogs,dst=/data \
    --label "traefik.port=3000" \
    --network traefik-net \
    gogs/gogs

We now would suggest to rather look into Gitea

Gitea is a community managed lightweight code hosting solution written in Go. It published under the MIT license.

Below you find the docker-compose for gogs

Below you find the docker-compose for gitea

# PULL UPDATE & LAUNCH DOCKER
#     git pull && docker stack deploy --compose-file docker-compose.yml gitea

# REMOVE STACK
#     docker stack rm gitea

# ADD BASIC AUTH and ESCAPE FOR docker-compose usage
# htpasswd -bBn user password | sed 's/\$/\$\$/g' #escape for docker-compose usage

version: "3"

services:
  server:
    image: gitea/gitea
    networks:
      - traefik-net
    volumes:
      - /swarm/volumes/gitea:/data
    deploy:
      labels:
        - traefik.port=3000
        - "traefik.frontend.rule=Host:gitea.f4a.me"
      replicas: 1 
      update_config:
        parallelism: 1
        delay: 30s        
networks:
  traefik-net:
   external: true

Drone¶

A continuous integration server which is open source, and tightly integrates with open source git platforms like gogs or services like github.

A good installation procedure is available here at http://docs.drone.io/install-for-gogs/. The corresponding commands for F4A are below:

docker run \
  --name drone \
  --label "traefik.port=8000" \
  --publish 8000:8000 \
  --publish 9000:9000 \
  -e DRONE_OPEN=true \
  -e DRONE_HOST=drone.f4a.me \
  -e DRONE_GOGS=true \
  -e DRONE_GOGS_URL=https://gogs.tillwitt.de \
  -e DRONE_SECRET=<some secret> \
  drone/drone:0.8

mkdir -p /swarm/volumes/drone

docker service create \
  --name drone \
  --label "traefik.port=8000" \
  --label "traefik.docker.network=traefik-net" \
  --network traefik-net \
  --mount type=bind,src=/swarm/volumes/drone,dst=/var/lib/drone/ \
  --publish 8000:8000 \
  --publish 9000:9000 \
  -e DRONE_OPEN=true \
  -e DRONE_HOST=drone.f4a.me \
  -e DRONE_GOGS=true \
  -e DRONE_GOGS_URL=https://gogs.tillwitt.de \
  -e DRONE_SECRET=<some secret> \
  -e DRONE_ADMIN=witt \
  drone/drone:0.8

docker service create \
  --name drone_agent \
  --mount type=bind,src=/var/run/docker.sock,dst=/var/run/docker.sock \
  --network traefik-net \
  -e DRONE_SERVER=drone:9000 \
  -e DRONE_SECRET=<some secret> \
  drone/agent:0.8

Once setup, with in this case gogs, you can log into the web interface. After a short sync all repositories should be visible. Activate drone.io for the corresponding repository.

To tell drone.io what to execute you need to add a .drone.yml to your repository. Examples are below.

example:

image: dockerfile/nginx
script:
  - echo hello world

  publish:
    docker:
      registry: registry.f4a.me
      email: witt@f4a.me
      repo: registry.f4a.me/flex4apps/flex4apps/homomorphic-encryption
      file: homomorphic-encryption/Dockerfile
      context: homomorphic-encryption
      tag: latest
      secrets: [ docker_username, docker_password ]

elasticsearch and kibana¶

elasticsearch is a search engine based on the Lucene library. It provides a distributed, multitenant-capable full-text search engine with an HTTP web interface and schema-free JSON documents. Elasticsearch is developed in Java.

Inspired by https://sematext.com/blog/docker-elasticsearch-swarm/. Issue the following command:

docker service create \
   --name esc24 \
   --label "traefik.port=9200" \
   --label 'traefik.frontend.auth.basic=flex4apps:$apr1$G9e4rgPu$jbn2AAk2F.OeGnRVFnIR/1' \
   --network traefik-net \
   --replicas 3 \
   --endpoint-mode dnsrr \
   --update-parallelism 1 \
   --update-delay 60s \
   --mount type=volume,source=esc24,target=/data \
 elasticsearch:2.4 \
   elasticsearch \
   -Des.discovery.zen.ping.multicast.enabled=false \
   -Des.discovery.zen.ping.unicast.hosts=esc24 \
   -Des.gateway.expected_nodes=3 \
   -Des.discovery.zen.minimum_master_nodes=2 \
   -Des.gateway.recover_after_nodes=2 \
   -Des.network.bind=_eth0:ipv4_

Inspired by https://github.com/elastic/elasticsearch-docker/issues/91 and https://idle.run/elasticsearch-cluster

The host systems have to be prepared to run elasticsearch in a docker:

echo vm.max_map_count=262144 >> /etc/sysctl.conf && sysctl --system && sysctl vm.max_map_count

The issue the following command to start three instances of elasticsearch:

docker service create \
  --replicas 3 \
  --name esc56 \
  --label "traefik.port=9200" \
  --label 'traefik.frontend.auth.basic=flex4apps:$apr1$G9e4rgPu$jbn2AAk2F.OeGnRVFnIR/1' \
  --mount type=volume,source=esc56,target=/data \
  --network traefik-net \
  elasticsearch:5.6.4 bash -c 'ip addr && IP=$(ip addr | awk -F"[ /]*" "/inet .*\/24/{print \$3}") && \
      echo publish_host=$IP && \
      exec /docker-entrypoint.sh -Enetwork.bind_host=0.0.0.0 -Enetwork.publish_host=$IP -Ediscovery.zen.minimum_master_nodes=2 -Ediscovery.zen.ping.unicast.hosts=tasks.esc56'

Below you find the reference code from the repository for the ELK stack version 2.4

# PULL UPDATE & LAUNCH DOCKER
#     git pull && docker stack deploy --compose-file docker-compose.yml drone

# REMOVE STACK
#     docker stack rm drone

# ADD BASIC AUTH and ESCAPE FOR docker-compose usage
# htpasswd -bBn user password | sed 's/\$/\$\$/g' #escape for docker-compose usage

version: "3"

services:
  server:
    image: drone/drone:0.8
    volumes:
      - /swarm/volumes/drone:/var/lib/drone/
    environment:
      - DRONE_OPEN=true
      - DRONE_HOST=drone.f4a.me
      - DRONE_GOGS=true
      - DRONE_GOGS_URL=https://gogs.tillwitt.de
      - DRONE_SECRET=<some secret password>
      - DRONE_ADMIN=witt
    deploy:
      labels:
        - traefik.port=8000
        - "traefik.frontend.rule=Host:drone.f4a.me"

  agent:
    image: drone/drone:0.8
    command: agent
    depends_on: [ server ]
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
    environment:
      - DRONE_SERVER=ws://server:8000/ws/broker
      - DRONE_SECRET=<some secret password>

networks:
  traefik-net:
   external: true

Below you find the reference code from the repository for the ELK stack version 5.6

# PULL UPDATE & LAUNCH DOCKER
#     git pull && docker stack deploy --compose-file docker-compose.yml elk56

# REMOVE STACK
#     docker stack rm drone

# ADD BASIC AUTH and ESCAPE FOR docker-compose usage
# htpasswd -bBn user password | sed 's/\$/\$\$/g' #escape for docker-compose usage

version: "3.0"

services:

  elasticsearch:
    environment:
      ES_JAVA_OPTS: -Xms2g -Xmx2g
    image: elasticsearch:5.6
    ulimits:
      memlock: -1
      nofile:
        hard: 65536
        soft: 65536
      nproc: 65538
    volumes:
      - /usr/share/elasticsearch/data
    networks:
      - traefik-net
    command: >
      bash -c 'ip addr && IP=$$(ip addr | awk -F"[ /]*" "/inet .*\/24/{print \$$3}") && \
          echo publish_host=$$IP && \
            exec /docker-entrypoint.sh -Enetwork.bind_host=0.0.0.0 -Enetwork.publish_host=$$IP -Ediscovery.zen.minimum_master_nodes=3 -Ediscovery.zen.ping.unicast.hosts=elasticsearch'
    deploy:
      mode: global
      labels:
        - traefik.port=9200
        - "traefik.frontend.rule=Host:esc56.f4a.me"
        - "traefik.frontend.auth.basic=witt:$$2y$$05$$kOFY7071ilbnpiJNDaIO9e1WeuhHnKtp9Adrevz4r8wJ3b3X1XuqW"
        - "traefik.frontend.auth.basic="
        - traefik.frontend.auth.basic=witt:$$2y$$05$$kOFY7071ilbnpiJNDaIO9e1WeuhHnKtp9Adrevz4r8wJ3b3X1XuqW,sebastian.schmitz:$$apr1$$cCDTHFrI$$m9QaIkyU8d86N4/r/jljJ/,flex4apps:$$2y$$05$$JvvVJToTDOfTzaNA/oIflecuy5l3pct94/3PfCB/g/xq4qZ49baU.


  kibana:
    image: kibana:5.6
    environment:
      ELASTICSEARCH_URL: http://elasticsearch:9200
    networks:
      - traefik-net
    deploy:
      labels:
        - traefik.port=5601
        - "traefik.frontend.rule=Host:kb56.f4a.me"
        - traefik.frontend.auth.basic=witt:$$2y$$05$$kOFY7071ilbnpiJNDaIO9e1WeuhHnKtp9Adrevz4r8wJ3b3X1XuqW,sebastian.schmitz:$$apr1$$cCDTHFrI$$m9QaIkyU8d86N4/r/jljJ/,flex4apps:$$2y$$05$$JvvVJToTDOfTzaNA/oIflecuy5l3pct94/3PfCB/g/xq4qZ49baU.



networks:
  traefik-net:
   external: true

Below you find the reference code from the repository for the ELK stack version 6.2

# PULL UPDATE & LAUNCH DOCKER
#     git pull && docker stack deploy --compose-file docker-compose.yml elk

# REMOVE STACK
#     docker stack rm drone

# ADD BASIC AUTH and ESCAPE FOR docker-compose usage
# htpasswd -bBn user password | sed 's/\$/\$\$/g' #escape for docker-compose usage

version: "3.0"

services:

  elasticsearch:
    image: docker.elastic.co/elasticsearch/elasticsearch:6.2.2
    environment:
      - cluster.name=docker-cluster
      - bootstrap.memory_lock=false
      - http.cors.enabled=true
      - http.cors.allow-origin=*
      - "ES_JAVA_OPTS=-Xms512m -Xmx512m"
#      - "discovery.zen.ping.unicast.hosts=elasticsearch"
    ulimits:
      memlock:
        soft: -1
        hard: -1
    volumes:
      - /data/local/elasticsearch:/usr/share/elasticsearch/data
    networks:
      - traefik-net
#    command: >
#      bash -c 'yum install -y iproute && ip addr && IP=$$(ip addr | awk -F"[ /]*" "/inet .*\/24/{print \$$3}") && \
#          echo publish_host=$$IP && \
#            exec bin/elasticsearch -Enetwork.bind_host=0.0.0.0 -Enetwork.publish_host=$$IP -Ediscovery.zen.minimum_master_nodes=3 -Ediscovery.zen.ping.unicast.hosts=elasticsearch'
    deploy:
      labels:
        - traefik.port=9200
        - "traefik.frontend.rule=Host:elastic.f4a.me"
        - "traefik.frontend.headers.customResponseHeaders='Access-Control-Allow-Origin: *'"

        - traefik.frontend.auth.basic=witt:$$2y$$05$$kOFY7071ilbnpiJNDaIO9e1WeuhHnKtp9Adrevz4r8wJ3b3X1XuqW,sebastian.schmitz:$$apr1$$cCDTHFrI$$m9QaIkyU8d86N4/r/jljJ/,flex4apps:$$2y$$05$$JvvVJToTDOfTzaNA/oIflecuy5l3pct94/3PfCB/g/xq4qZ49baU.,scai.ndv:$$apr1$$FDUP9rVB$$3.oEyn1LEXDCUhcQ5LQza.,christian.pfeiffer:$$apr1$$7efxqZjC$$RKpeAwtFKXjQEA4MzS/K61,f4a:$$2y$$05$$d01dYZVn8MZnIyu.EsSuB.cPDVBCmSSQOaEpt2P2EZTbAtG594pyG
      placement:
        constraints: [node.role == manager]



  kibana:
    image: docker.elastic.co/kibana/kibana:6.2.2
    environment:
      SERVER_NAME: elasticsearch
      ELASTICSEARCH_URL: http://elasticsearch:9200
    networks:
      - traefik-net
    deploy:
      labels:
        - traefik.port=5601
        - "traefik.frontend.rule=Host:kibana.f4a.me"
        - traefik.frontend.auth.basic=witt:$$2y$$05$$kOFY7071ilbnpiJNDaIO9e1WeuhHnKtp9Adrevz4r8wJ3b3X1XuqW,sebastian.schmitz:$$apr1$$cCDTHFrI$$m9QaIkyU8d86N4/r/jljJ/,flex4apps:$$2y$$05$$JvvVJToTDOfTzaNA/oIflecuy5l3pct94/3PfCB/g/xq4qZ49baU.,scai.ndv:$$apr1$$FDUP9rVB$$3.oEyn1LEXDCUhcQ5LQza.,christian.pfeiffer:$$apr1$$7efxqZjC$$RKpeAwtFKXjQEA4MzS/K61,f4a:$$2y$$05$$d01dYZVn8MZnIyu.EsSuB.cPDVBCmSSQOaEpt2P2EZTbAtG594pyG
      placement:
        constraints: [node.role == manager]

  connector:
    image: ubuntu:17.10
    networks:
      - traefik-net
    tty: true
    stdin_open: true
    ports:
      - 26:22
    deploy:
      labels:
        - traefik.port=80
        - "traefik.frontend.rule=Host:connector.f4a.me"
      placement:
        constraints: [node.role == manager]


networks:
  traefik-net:
   external: true

kibana¶

Kibana is an open source data visualization plugin for Elasticsearch. It provides visualization capabilities on top of the content indexed on an Elasticsearch cluster. Users can create bar, line and scatter plots, or pie charts and maps on top of large volumes of data.

Issue the following command:

docker service create \
   --name kb56 \
   --label "traefik.port=5601" \
   --label 'traefik.frontend.auth.basic=flex4apps:$apr1$G9e4rgPu$jbn2AAk2F.OeGnRVFnIR/1' \
   --network traefik-net \
   -e "ELASTICSEARCH_URL=http://esc56:9200" \
   kibana:5.6

Below you find the reference code from the repository for the ELK stack version 2.4

# PULL UPDATE & LAUNCH DOCKER
#     git pull && docker stack deploy --compose-file docker-compose.yml drone

# REMOVE STACK
#     docker stack rm drone

# ADD BASIC AUTH and ESCAPE FOR docker-compose usage
# htpasswd -bBn user password | sed 's/\$/\$\$/g' #escape for docker-compose usage

version: "3"

services:
  server:
    image: drone/drone:0.8
    volumes:
      - /swarm/volumes/drone:/var/lib/drone/
    environment:
      - DRONE_OPEN=true
      - DRONE_HOST=drone.f4a.me
      - DRONE_GOGS=true
      - DRONE_GOGS_URL=https://gogs.tillwitt.de
      - DRONE_SECRET=<some secret password>
      - DRONE_ADMIN=witt
    deploy:
      labels:
        - traefik.port=8000
        - "traefik.frontend.rule=Host:drone.f4a.me"

  agent:
    image: drone/drone:0.8
    command: agent
    depends_on: [ server ]
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
    environment:
      - DRONE_SERVER=ws://server:8000/ws/broker
      - DRONE_SECRET=<some secret password>

networks:
  traefik-net:
   external: true

Below you find the reference code from the repository for the ELK stack version 5.6

# PULL UPDATE & LAUNCH DOCKER
#     git pull && docker stack deploy --compose-file docker-compose.yml elk56

# REMOVE STACK
#     docker stack rm drone

# ADD BASIC AUTH and ESCAPE FOR docker-compose usage
# htpasswd -bBn user password | sed 's/\$/\$\$/g' #escape for docker-compose usage

version: "3.0"

services:

  elasticsearch:
    environment:
      ES_JAVA_OPTS: -Xms2g -Xmx2g
    image: elasticsearch:5.6
    ulimits:
      memlock: -1
      nofile:
        hard: 65536
        soft: 65536
      nproc: 65538
    volumes:
      - /usr/share/elasticsearch/data
    networks:
      - traefik-net
    command: >
      bash -c 'ip addr && IP=$$(ip addr | awk -F"[ /]*" "/inet .*\/24/{print \$$3}") && \
          echo publish_host=$$IP && \
            exec /docker-entrypoint.sh -Enetwork.bind_host=0.0.0.0 -Enetwork.publish_host=$$IP -Ediscovery.zen.minimum_master_nodes=3 -Ediscovery.zen.ping.unicast.hosts=elasticsearch'
    deploy:
      mode: global
      labels:
        - traefik.port=9200
        - "traefik.frontend.rule=Host:esc56.f4a.me"
        - "traefik.frontend.auth.basic=witt:$$2y$$05$$kOFY7071ilbnpiJNDaIO9e1WeuhHnKtp9Adrevz4r8wJ3b3X1XuqW"
        - "traefik.frontend.auth.basic="
        - traefik.frontend.auth.basic=witt:$$2y$$05$$kOFY7071ilbnpiJNDaIO9e1WeuhHnKtp9Adrevz4r8wJ3b3X1XuqW,sebastian.schmitz:$$apr1$$cCDTHFrI$$m9QaIkyU8d86N4/r/jljJ/,flex4apps:$$2y$$05$$JvvVJToTDOfTzaNA/oIflecuy5l3pct94/3PfCB/g/xq4qZ49baU.


  kibana:
    image: kibana:5.6
    environment:
      ELASTICSEARCH_URL: http://elasticsearch:9200
    networks:
      - traefik-net
    deploy:
      labels:
        - traefik.port=5601
        - "traefik.frontend.rule=Host:kb56.f4a.me"
        - traefik.frontend.auth.basic=witt:$$2y$$05$$kOFY7071ilbnpiJNDaIO9e1WeuhHnKtp9Adrevz4r8wJ3b3X1XuqW,sebastian.schmitz:$$apr1$$cCDTHFrI$$m9QaIkyU8d86N4/r/jljJ/,flex4apps:$$2y$$05$$JvvVJToTDOfTzaNA/oIflecuy5l3pct94/3PfCB/g/xq4qZ49baU.



networks:
  traefik-net:
   external: true

Below you find the reference code from the repository for the ELK stack version 6.2

# PULL UPDATE & LAUNCH DOCKER
#     git pull && docker stack deploy --compose-file docker-compose.yml elk

# REMOVE STACK
#     docker stack rm drone

# ADD BASIC AUTH and ESCAPE FOR docker-compose usage
# htpasswd -bBn user password | sed 's/\$/\$\$/g' #escape for docker-compose usage

version: "3.0"

services:

  elasticsearch:
    image: docker.elastic.co/elasticsearch/elasticsearch:6.2.2
    environment:
      - cluster.name=docker-cluster
      - bootstrap.memory_lock=false
      - http.cors.enabled=true
      - http.cors.allow-origin=*
      - "ES_JAVA_OPTS=-Xms512m -Xmx512m"
#      - "discovery.zen.ping.unicast.hosts=elasticsearch"
    ulimits:
      memlock:
        soft: -1
        hard: -1
    volumes:
      - /data/local/elasticsearch:/usr/share/elasticsearch/data
    networks:
      - traefik-net
#    command: >
#      bash -c 'yum install -y iproute && ip addr && IP=$$(ip addr | awk -F"[ /]*" "/inet .*\/24/{print \$$3}") && \
#          echo publish_host=$$IP && \
#            exec bin/elasticsearch -Enetwork.bind_host=0.0.0.0 -Enetwork.publish_host=$$IP -Ediscovery.zen.minimum_master_nodes=3 -Ediscovery.zen.ping.unicast.hosts=elasticsearch'
    deploy:
      labels:
        - traefik.port=9200
        - "traefik.frontend.rule=Host:elastic.f4a.me"
        - "traefik.frontend.headers.customResponseHeaders='Access-Control-Allow-Origin: *'"

        - traefik.frontend.auth.basic=witt:$$2y$$05$$kOFY7071ilbnpiJNDaIO9e1WeuhHnKtp9Adrevz4r8wJ3b3X1XuqW,sebastian.schmitz:$$apr1$$cCDTHFrI$$m9QaIkyU8d86N4/r/jljJ/,flex4apps:$$2y$$05$$JvvVJToTDOfTzaNA/oIflecuy5l3pct94/3PfCB/g/xq4qZ49baU.,scai.ndv:$$apr1$$FDUP9rVB$$3.oEyn1LEXDCUhcQ5LQza.,christian.pfeiffer:$$apr1$$7efxqZjC$$RKpeAwtFKXjQEA4MzS/K61,f4a:$$2y$$05$$d01dYZVn8MZnIyu.EsSuB.cPDVBCmSSQOaEpt2P2EZTbAtG594pyG
      placement:
        constraints: [node.role == manager]



  kibana:
    image: docker.elastic.co/kibana/kibana:6.2.2
    environment:
      SERVER_NAME: elasticsearch
      ELASTICSEARCH_URL: http://elasticsearch:9200
    networks:
      - traefik-net
    deploy:
      labels:
        - traefik.port=5601
        - "traefik.frontend.rule=Host:kibana.f4a.me"
        - traefik.frontend.auth.basic=witt:$$2y$$05$$kOFY7071ilbnpiJNDaIO9e1WeuhHnKtp9Adrevz4r8wJ3b3X1XuqW,sebastian.schmitz:$$apr1$$cCDTHFrI$$m9QaIkyU8d86N4/r/jljJ/,flex4apps:$$2y$$05$$JvvVJToTDOfTzaNA/oIflecuy5l3pct94/3PfCB/g/xq4qZ49baU.,scai.ndv:$$apr1$$FDUP9rVB$$3.oEyn1LEXDCUhcQ5LQza.,christian.pfeiffer:$$apr1$$7efxqZjC$$RKpeAwtFKXjQEA4MzS/K61,f4a:$$2y$$05$$d01dYZVn8MZnIyu.EsSuB.cPDVBCmSSQOaEpt2P2EZTbAtG594pyG
      placement:
        constraints: [node.role == manager]

  connector:
    image: ubuntu:17.10
    networks:
      - traefik-net
    tty: true
    stdin_open: true
    ports:
      - 26:22
    deploy:
      labels:
        - traefik.port=80
        - "traefik.frontend.rule=Host:connector.f4a.me"
      placement:
        constraints: [node.role == manager]


networks:
  traefik-net:
   external: true

Registry¶

the Registry is a stateless, highly scalable server side application that stores and lets you distribute Docker images. The Registry is open-source, under the permissive Apache license.

Below you find the reference code from the repository

# PULL UPDATE & LAUNCH DOCKER
#     git pull && docker stack deploy --compose-file docker-compose.yml registry

# REMOVE STACK
#     docker stack rm registry

# ADD BASIC AUTH and ESCAPE FOR docker-compose usage
# htpasswd -bBn user password | sed 's/\$/\$\$/g' #escape for docker-compose usage

version: "3"

services:
  registry:
    image: registry:latest
    networks:
      - traefik-net
    environment:
      - "REGISTRY_HTTP_SECRET=myOwnSecret"
    volumes:
        - "/data/local/registry:/var/lib/registry"
    deploy:
      labels:
        - traefik.port=5000
        - "traefik.frontend.rule=Host:registry.f4a.me"
        - "traefik.frontend.auth.basic=witt:$$2y$$05$$kOFY7071ilbnpiJNDaIO9e1WeuhHnKtp9Adrevz4r8wJ3b3X1XuqW"
      placement:
        constraints: [node.role == manager]

networks:
  traefik-net:
   external: true

Sheperd¶

Docker swarm service for automatically updating your services whenever their base image is refreshed

Below you find the reference code from the repository

# PULL UPDATE & LAUNCH DOCKER
#     git pull && docker stack deploy --compose-file docker-compose.yml shepherd

# REMOVE STACK
#     docker stack rm shepherd

# ADD BASIC AUTH and ESCAPE FOR docker-compose usage
# htpasswd -bBn user password | sed 's/\$/\$\$/g' #escape for docker-compose usage

version: "3"

services:
  shepherd:
    image: mazzolino/shepherd
    networks:
      - traefik-net
    environment:
      - SLEEP_TIME="1m"
      - BLACKLIST_SERVICES="traefik" #space seperated names
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
    deploy:
      placement:
        constraints: [node.role == manager]

networks:
  traefik-net:
   external: true

Security¶

Let’s encrypt¶

Let’s Encrypt (German “Lasst uns verschlüsseln”) is a certification authority that went into operation at the end of 2015 and offers free X.509 certificates for Transport Layer Security (TLS). An automated process replaces the previously common complex manual procedures for the creation, validation, signing, setup and renewal of certificates for encrypted websites.

There are certain rate limits which apply and you might run into when experimenting with the swarm.

https://letsencrypt.org/docs/rate-limits/

To search for issued certificates you can look go to:

https://crt.sh/?q=%25f4a.me

Docker swarm networks¶

Overlay networking for Docker Engine swarm mode comes secure out of the box. The swarm nodes exchange overlay network information using a gossip protocol. By default the nodes encrypt and authenticate information they exchange via gossip using the AES algorithm in GCM mode. Manager nodes in the swarm rotate the key used to encrypt gossip data every 12 hours. [*]

[*]	https://docs.docker.com/v17.09/engine/userguide/networking/overlay-security-model/

Docker secrets¶

in terms of Docker Swarm services, a secret is a blob of data, such as a password, SSH private key, SSL certificate, or another piece of data that should not be transmitted over a network or stored unencrypted in a Dockerfile or in your application’s source code. In Docker 1.13 and higher, you can use Docker secrets to centrally manage this data and securely transmit it to only those containers that need access to it. Secrets are encrypted during transit and at rest in a Docker swarm. A given secret is only accessible to those services which have been granted explicit access to it, and only while those service tasks are running.

You can use secrets to manage any sensitive data which a container needs at runtime but you don’t want to store in the image or in source control, such as:

Usernames and passwords SSH keys Other important data such as the name of a database or internal server Generic strings or binary content (up to 500 kb in size) [†]

[†]	https://docs.docker.com/engine/swarm/secrets/

Docker notary¶

Notary is a tool for publishing and managing trusted collections of content. Publishers can digitally sign collections and consumers can verify integrity and origin of content. This ability is built on a straightforward key management and signing interface to create signed collections and configure trusted publishers.

With Notary anyone can provide trust over arbitrary collections of data. Using The Update Framework (TUF) as the underlying security framework, Notary takes care of the operations necessary to create, manage, and distribute the metadata necessary to ensure the integrity and freshness of your content. [‡]

[‡]	https://docs.docker.com/notary/getting_started/

As the virtualisation platform Docker becomes more and more widely adopted, it becomes increasingly important to secure images and containers throughout their entire lifecycle. Docker Notary is a component introduced with Docker Engine 1.8 that aims at securing the “last mile” of this lifecycle (image distribution from the registry to the Docker client), by verifying the image publisher as well as image integrity. The verification is based on digital signatures created with multiple keys. Since an in-depth security analysis of key compromise scenarios in the context of Notary seems to be missing in scientific literature, an extensive attack model is developed in this work. Based on the model, it is argued that particularly devastating attacks can be prevented by storing some of the signing keys in a hardware vault, a Secure Element. It is described how an interface can be integrated into the Notary codebase that makes this possible. It is also shown that Notary’s signing process needs to be extended to prevent an attacker from exploiting one particular Notary component, the Notary server, to create arbitrary signatures on his behalf.[*]_

[§]	Docker Notary’s management of signing keys - An analysis of key compromise and integrability of a Secure Element (Simon Wessling, 2018)