Applications

What are applications

On the created system applications can be launched. Templates for applications are called images. Running applications are called containers.

Generic demonstrator

Docker file

# PULL UPDATE & LAUNCH DOCKER
#     git pull && docker stack deploy --compose-file docker-compose.yml demo

# REMOVE STACK
#     docker stack rm demo

# ADD BASIC AUTH and ESCAPE FOR docker-compose usage
# htpasswd -bBn user password | sed 's/\$/\$\$/g' #escape for docker-compose usage

version: "3"

services:
  nginx:
    image: kitematic/hello-world-nginx
    networks:
      - traefik-net
    environment:
      - test=noContent
    deploy:
      labels:
        - traefik.port=80
#        - "traefik.frontend.auth.basic=witt:$$2y$$05$$kOFY7071ilbnpiJNDaIO9e1WeuhHnKtp9Adrevz4r8wJ3b3X1XuqW"
#        - "traefik.frontend.auth.basic=ich:$$2y$$05$$jTZv0re2cXmiGrzRxW./8Ofse.6g/AEChvbMGdqYKIMqsr8xW/c"
#        - "traefik.frontend.auth.basic=user:$$2y$$05$$IRrTxLpG7ICzroI8Pb5P4.p2rMXGqyeeZM857BJxTFzP5q9W4RYuS"
        - "traefik.frontend.rule=Host:demo.f4a.me"
networks:
  traefik-net:
   external: true

Simple demo

The following application is already working with the current setup.

To launch an easy demonstrator, lets instantiate a webserver and make it available at demo.f4a.net:

docker service create \
    --name demo \
    --label "traefik.port=80" \
    --network traefik-net \
    kitematic/hello-world-nginx

Generating a new user with password run:

htpasswd -nbm flex4apps password

or go to: http://www.htaccesstools.com/htpasswd-generator/

The output will be something like:

flex4apps:$apr1$XqnUcSgR$39wlPxxyyxPxXZjFb34wo.

Example for traefik label usage below. If single quotes are in the password they would need to be escaped.

To do that close the quoting before it, insert the escaped single quote, and re-open the quoting: `'first part'\''second part'` But I dont even know if md5 password contain single quotes.

How to start the demo service:

docker service create \
    --name demopw \
    --label "traefik.port=80" \
    --label 'traefik.frontend.auth.basic=myName:$apr1$a7R637Ua$TvXp8/lgky5MDLGLacI1e1' \
    --network traefik-net \
    kitematic/hello-world-nginx

grafana

What is grafana

Setting it up

create the service like this:

docker service create \
  --name=grafana \
  --network traefik-net \
  --label "traefik.port=3000" \
  --mount type=bind,src=/swarm/volumes/grafana,dst=/var/lib/grafana \
  -e "GF_SECURITY_ADMIN_PASSWORD=someSecretPassword" \
  grafana/grafana

portainer

start portainer as a service we first need to create a data directory:

mkdir -p /docker/portainer

To start the container itself:

docker service create \
--name "portainer" \
--constraint 'node.role == manager' \
--network "traefik-net" --replicas "1" \
--mount type=bind,src=/var/run/docker.sock,dst=/var/run/docker.sock \
--mount type=bind,src=/docker/portainer,dst=/data \
--label "traefik.frontend.rule=Host:portainer.f4a.me" \
--label "traefik.backend=tool-portainer" \
--label "traefik.port=9000" \
--label "traefik.docker.network=traefik-net" \
--reserve-memory "20M" --limit-memory "40M" \
--restart-condition "any" --restart-max-attempts "55" \
--update-delay "5s" --update-parallelism "1" \
portainer/portainer \
-H unix:///var/run/docker.sock

Phabricator

The database user needs to be able to create databases

install

1. change your password:

   sed -i 's/<some secret>/yourPassword/g' Dockerfile
   

1. build the image with:

   docker build -t phabricator_image .
   

2. tag and push it to the registry:

   docker tag phabricator_image registry.f4a.me/phabricator
   docker push registry.f4a.me/phabricator
   

3. deploy it from the registry by executing the docker-compose:

   docker-compose up
   

Docker file

# download base image ubuntu 16.10
FROM php:7.1-apache
RUN apt-get update
RUN apt-get -y install git

WORKDIR /repo
RUN git clone https://github.com/phacility/libphutil.git
RUN git clone https://github.com/phacility/arcanist.git
RUN git clone https://github.com/phacility/phabricator.git

RUN a2enmod rewrite
RUN chown -R www-data /repo
RUN chgrp -R www-data /repo


# mysqli extension
RUN docker-php-ext-install -j$(nproc) mysqli

# GD extension
RUN apt-get -y install libfreetype6-dev libjpeg62-turbo-dev libpng-dev
RUN docker-php-ext-configure gd --with-freetype-dir=/usr/include/ --with-jpeg-dir=/usr/include/

RUN pecl install apcu
RUN apt-get install -y python3-pygments
RUN apt-get install -y sendmail

WORKDIR /repo/phabricator

# apache config
RUN sed -i 's#DocumentRoot /var/www/html#DocumentRoot /repo/phabricator/webroot\nRewriteEngine on\nRewriteRule ^(.*)$          /index.php?__path__=$1  [B,L,QSA]\n<Directory "/repo/phabricator/webroot">\n  Require all granted\n</Directory>#g' /etc/apache2/sites-enabled/000-default.conf

# ssl reverse proxy - preabmle
RUN echo \<?php >> support/preamble.php
RUN echo \$_SERVER[\'REMOTE_ADDR\'] = \$_SERVER[\'HTTP_X_FORWARDED_FOR\']\; >> support/preamble.php
RUN echo \$_SERVER[\'HTTPS\'] = true\; >> support/preamble.php

# phabricator configuration
RUN ./bin/config set mysql.host mariadb.f4a.me
RUN ./bin/config set mysql.user phabricator
RUN ./bin/config set mysql.pass ...
RUN ./bin/config set mysql.port 3306
RUN ./bin/config set phabricator.base-uri https://phabricator.f4a.me

RUN ./bin/config set phpmailer.smtp-host home.tillwitt.de
RUN ./bin/config set phpmailer.smtp-protocol TLS
RUN ./bin/config set phpmailer.smtp-port 587
RUN ./bin/config set phpmailer.smtp-user notify
RUN ./bin/config set phpmailer.smtp-password ...


ADD start.sh .
RUN chmod 755 start.sh

ENTRYPOINT ./start.sh && /bin/bash

Docker compose

# PULL UPDATE & LAUNCH DOCKER
#     git pull && docker stack deploy --with-registry-auth --compose-file docker-compose.yml phabricator

# REMOVE STACK
#     docker stack rm phabricator

# ADD BASIC AUTH and ESCAPE FOR docker-compose usage
# htpasswd -bBn user password | sed 's/\$/\$\$/g' #escape for docker-compose usage

version: "3.1"


services:
   web:
     image: registry.f4a.me/phabricator
     networks:
       - traefik-net
#     ports:
#       - "80:80"
     deploy:
      labels:
        - traefik.port=80
        - "traefik.frontend.rule=Host:phabricator.f4a.me"

#   db:
#     image: mysql:5.7
#     volumes:
#       - ./data:/var/lib/mysql
#     restart: always
#     environment:
#       MYSQL_ROOT_PASSWORD: root
#       MYSQL_DATABASE: phabricator
#       MYSQL_USER: phabricator
#       MYSQL_PASSWORD: phabricator

networks:
  traefik-net:
   external: true

MariaDB

What is MariaDB

MariaDB is free and open source relational database system. It was created as a :fork: from MySQL after Oracle started releasing new functionality not as open source anymore and due to the high support cost of MySQL.

How to set it up

As usual make sure that the path for data volume exists:

mkdir -p /swarm/volumes/mariadb

The initiate the docker service:

docker service create \
      --name mariadb \
      --publish 3306:3306 \
      --network traefik-net \
      --mount type=bind,src=/swarm/volumes/mariadb,dst=/var/lib/mysq \
      --label "traefik.port=3306" \
      -e MYSQL_ROOT_PASSWORD=someSecretPassword \
      mariadb:latest

PhpMyAdmin

What is PhpMyAdmin

How to set it up

The following command will start up PhpMyAdmin:

docker service create \
    --name phpmyadmin \
    --label "traefik.port=80" \
    --network traefik-net \
    -e ALLOW_ARBITRARY=1 \
    nazarpc/phpmyadmin

gogs

What is gogs

todo:

How to set it up

Pull image from Docker Hub.

very strange installation. First need to use –publish 3000:3000 and connect direct for install. Then remove instance and also remove published port. This is certainly something I need to review.

create the data volume for gogs:

mkdir -p /swarm/volumes/gogs

start the service:

docker service create \
    --name gogs \
    --mount type=bind,src=/swarm/volumes/gogs,dst=/data \
    --label "traefik.port=3000" \
    --network traefik-net \
    gogs/gogs

Drone

About the software

A continuous integration server which is open source, and tightly integrates with open source git platforms like gogs or services like github.

Setting it up

A good installation procedure is available here at http://docs.drone.io/install-for-gogs/. The corresponding commands for F4A are below:

docker run \
  --name drone \
  --label "traefik.port=8000" \
  --publish 8000:8000 \
  --publish 9000:9000 \
  -e DRONE_OPEN=true \
  -e DRONE_HOST=drone.f4a.me \
  -e DRONE_GOGS=true \
  -e DRONE_GOGS_URL=https://gogs.tillwitt.de \
  -e DRONE_SECRET=<some secret> \
  drone/drone:0.8

mkdir -p /swarm/volumes/drone

docker service create \
  --name drone \
  --label "traefik.port=8000" \
  --label "traefik.docker.network=traefik-net" \
  --network traefik-net \
  --mount type=bind,src=/swarm/volumes/drone,dst=/var/lib/drone/ \
  --publish 8000:8000 \
  --publish 9000:9000 \
  -e DRONE_OPEN=true \
  -e DRONE_HOST=drone.f4a.me \
  -e DRONE_GOGS=true \
  -e DRONE_GOGS_URL=https://gogs.tillwitt.de \
  -e DRONE_SECRET=<some secret> \
  -e DRONE_ADMIN=witt \
  drone/drone:0.8

docker service create \
  --name drone_agent \
  --mount type=bind,src=/var/run/docker.sock,dst=/var/run/docker.sock \
  --network traefik-net \
  -e DRONE_SERVER=drone:9000 \
  -e DRONE_SECRET=<some secret> \
  drone/agent:0.8

How to:

Once setup, with in this case gogs, you can log into the web interface. After a short sync all repositories should be visible. Activate drone.io for the corresponding repository.

To tell drone.io what to execute you need to add a .drone.yml to your repository. Examples are below.

Examples and configuration

example:

image: dockerfile/nginx
script:
  - echo hello world

  publish:
    docker:
      registry: registry.f4a.me
      email: witt@f4a.me
      repo: registry.f4a.me/flex4apps/flex4apps/homomorphic-encryption
      file: homomorphic-encryption/Dockerfile
      context: homomorphic-encryption
      tag: latest
      secrets: [ docker_username, docker_password ]

elasticsearch and kibana

What is elasticsearch and kibana

todo:

How to set it up - release 2.4

Inspired by https://sematext.com/blog/docker-elasticsearch-swarm/. Issue the following command:

docker service create \
   --name esc24 \
   --label "traefik.port=9200" \
   --label 'traefik.frontend.auth.basic=flex4apps:$apr1$G9e4rgPu$jbn2AAk2F.OeGnRVFnIR/1' \
   --network traefik-net \
   --replicas 3 \
   --endpoint-mode dnsrr \
   --update-parallelism 1 \
   --update-delay 60s \
   --mount type=volume,source=esc24,target=/data \
 elasticsearch:2.4 \
   elasticsearch \
   -Des.discovery.zen.ping.multicast.enabled=false \
   -Des.discovery.zen.ping.unicast.hosts=esc24 \
   -Des.gateway.expected_nodes=3 \
   -Des.discovery.zen.minimum_master_nodes=2 \
   -Des.gateway.recover_after_nodes=2 \
   -Des.network.bind=_eth0:ipv4_

Release 5.6

Inspired by https://github.com/elastic/elasticsearch-docker/issues/91 and https://idle.run/elasticsearch-cluster

The host systems have to be prepared to run elasticsearch in a docker:

echo vm.max_map_count=262144 >> /etc/sysctl.conf && sysctl --system && sysctl vm.max_map_count

The issue the following command to start three instances of elasticsearch:

docker service create \
  --replicas 3 \
  --name esc56 \
  --label "traefik.port=9200" \
  --label 'traefik.frontend.auth.basic=flex4apps:$apr1$G9e4rgPu$jbn2AAk2F.OeGnRVFnIR/1' \
  --mount type=volume,source=esc56,target=/data \
  --network traefik-net \
  elasticsearch:5.6.4 bash -c 'ip addr && IP=$(ip addr | awk -F"[ /]*" "/inet .*\/24/{print \$3}") && \
      echo publish_host=$IP && \
      exec /docker-entrypoint.sh -Enetwork.bind_host=0.0.0.0 -Enetwork.publish_host=$IP -Ediscovery.zen.minimum_master_nodes=2 -Ediscovery.zen.ping.unicast.hosts=tasks.esc56'

kibana

What is kibana

todo:

How to set it up - release 2.4

Issue the following command:

docker service create \
   --name kb56 \
   --label "traefik.port=5601" \
   --label 'traefik.frontend.auth.basic=flex4apps:$apr1$G9e4rgPu$jbn2AAk2F.OeGnRVFnIR/1' \
   --network traefik-net \
   -e "ELASTICSEARCH_URL=http://esc56:9200" \
   kibana:5.6

Next steps

The following applications will be considered for next steps:

• drone.io
• https://codeship.com
• pritunl