Carsten Rieger IT-Services - Alle Anleitungen HAProxy Installationsanleitungen Nextcloud Installationsanleitungen nginx Proxy (Manager)-Anleitungen

Nextcloud Loadbalancing – (Ausfall)Sicherheit erhöhen

von

Mit Hilfe dieser Anleitung, resultierend aus unserem Proxmox-Lab, installieren Sie zwei HAProxy-Server sowie zwei keepalived-Instanzen, zwei Webserver (nginx) für Nextcloud, einen zentralen Datenbank- und Redis-Server sowie einen NFS-Server auf Ubuntu 24 LTS oder Debian 12 (x86_64, kein ARM!).

  1. Server-/IP-Übersicht
  2. Installation HAProxy, hatop und keepalived
  3. Installation Webserver (nginx, php8.3-fpm, MariaDB-client, redis-tools, nfs-common)
  4. Installation lsyncd
  5. Installation NFS für lsyncd
  6. Installation Redis-Server, MariaDB-Server, NFS-Server
  7. Einrichtung Nextcloud

1. Server-IP-Übersicht (Labor)

Server / SoftwareIP
Proxmox LXC: haproxy
haproxy v. 3.1, keepalived, hatop		192.168.0.190
192.168.0.180 (failover-IP)
Proxmox LXC: haproxy1
haproxy v. 3.1, keepalived, hatop		192.168.0.191
192.168.0.180 (failover-IP)
Proxmox VM (wegen NFS-Requirements): nc1
nginx, php8.3-fpm, lsyncd, nfs, mariadb-client, redis-tools		192.168.0.197
Proxmox VM (wegen NFS-Requirements): nc2
nginx, php8.3-fpm, nfs, mariadb-client, redis-tools		192.168.0.198
Proxmox VM (wegen NFS-Requirements): db-redis
MariaDB-/Redis-/NFS-Server		 192.168.0.199

2. Installation haproxy, haproxy1

Installation der Basissoftware (beide Nodes: haproxy, haproxy1)

apt install -y \
apt-transport-https bash-completion bzip2 ca-certificates cron curl dialog \
dirmngr ffmpeg ghostscript git gpg gnupg gnupg2 htop jq libfile-fcntllock-perl \
libfontconfig1 libfuse2 locate lsb-release nodejs npm net-tools rsyslog screen smbclient \
socat software-properties-common ssl-cert tree unzip vim wget zip

» Ubuntu 24 LTS: Installation haproxy (beide Nodes: haproxy, haproxy1)

add-apt-repository ppa:vbernat/haproxy-3.1
apt update
apt-get install haproxy=3.1.*

» Debian 12: Installation haproxy (beide Nodes: haproxy, haproxy1)

curl https://haproxy.debian.net/bernat.debian.org.gpg \
| gpg --dearmor > /usr/share/keyrings/haproxy.debian.net.gpg
echo deb "[signed-by=/usr/share/keyrings/haproxy.debian.net.gpg]" \
http://haproxy.debian.net bookworm-backports-3.1 main \
> /etc/apt/sources.list.d/haproxy.list
apt update
apt install -y haproxy=3.1.*

Installation und Konfiguration (beide Nodes: haproxy, haproxy1)

apt install -y hatop

Erstellen Sie die Datei

nano /etc/sysctl.d/haproxy.conf

und fügen dort diese Zeilen hinzu

net.ipv4.ip_forward = 1
net.ipv4.ip_nonlocal_bind = 1

Starten Sie den HAProxy-Monitor manuell:

sudo hatop -s /run/haproxy/admin.sock

Wir legen ein Skript an, um das Tool jederzeit bequem starten zu können:

nano hat.sh

Fügen Sie dafür diesen Inhalt ein:

#!/bin/bash
hatop -s /run/haproxy/admin.sock
exit 0

Konfiguerieren Sie das Skript als ausführbar

chmod +x hat.sh

und starten es ab sofort wie folgt:

sudo ./hat.sh

Installation der keepalived-Failover IP (beide Nodes: haproxy, haproxy1)

apt install -y keepalived

Standardkonfigurationen (beide Nodes: haproxy, haproxy1)

cp /etc/keepalived/keepalived.conf.sample /etc/keepalived/keepalived.conf
cp /etc/haproxy/haproxy.cfg /etc/haproxy/haproxy.cfg.bak

Erstellen der HAProxy-Konfiguration:

nano /etc/haproxy/haproxy.cfg

Fügen Sie diesen Inhalt ein

global
	log /dev/log	local0
	log /dev/log	local1 notice
	chroot /var/lib/haproxy
	stats socket /run/haproxy/admin.sock mode 660 level admin
	stats timeout 30s
	user haproxy
	group haproxy
	daemon

	# Default SSL material locations
	ca-base /etc/ssl/certs
	crt-base /etc/ssl/private

	# See: https://ssl-config.mozilla.org/#server=haproxy&server-version=2.0.3&config=intermediate
        ssl-default-bind-ciphers ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384
        ssl-default-bind-ciphersuites TLS_AES_128_GCM_SHA256:TLS_AES_256_GCM_SHA384:TLS_CHACHA20_POLY1305_SHA256
        ssl-default-bind-options ssl-min-ver TLSv1.2 no-tls-tickets

defaults
	log	global
	mode	tcp
	option	tcplog
	option	dontlognull
        timeout connect 5000
        timeout client  50000
        timeout server  50000
	errorfile 400 /etc/haproxy/errors/400.http
	errorfile 403 /etc/haproxy/errors/403.http
	errorfile 408 /etc/haproxy/errors/408.http
	errorfile 500 /etc/haproxy/errors/500.http
	errorfile 502 /etc/haproxy/errors/502.http
	errorfile 503 /etc/haproxy/errors/503.http
	errorfile 504 /etc/haproxy/errors/504.http

frontend FE_NEXTCLOUD
        bind *:443
        maxconn 20100
        mode tcp
        option tcplog
        tcp-request inspect-delay 5s
        tcp-request content accept if { req_ssl_hello_type 1 }
	##################################################################
	acl ACL_NEXTCLOUD req.ssl_sni -i nc
        use_backend BE_NEXTCLOUD if ACL_NEXTCLOUD
        ##################################################################        
        default_backend BE_NEXTCLOUD
	##################################################################

backend BE_NEXTCLOUD
        mode tcp
        fullconn 20000
        balance leastconn
        stick-table type ip size 100m expire 12h
        stick on src
        option httpchk GET /login
        http-check expect rstatus [2-3][0-9][0-9]
        server nc0 192.168.0.197:443 weight 1 inter 1s downinter 5s rise 2 fall 1 check check-ssl verify none on-marked-down shutdown-sessions maxconn 10000 send-proxy-v2
        server nc1 192.168.0.198:443 weight 1 inter 1s downinter 5s rise 2 fall 1 check check-ssl verify none on-marked-down shutdown-sessions maxconn 10000 send-proxy-v2
# https://www.haproxy.com/documentation/haproxy-configuration-tutorials/service-reliability/health-checks/#change-the-interval

Erstellen der keepalived-Konfiguration:

nano /etc/keepalived/keepalived.conf

Nur am haproxy-Server (192.168.0.190) 

global_defs {
  router_id nextcloud
  enable_script_security
  script_user haproxy
}

vrrp_script check_haproxy {
  script "/usr/bin/killall -0 haproxy"
  interval 2
  weight 2
  user haproxy
}

vrrp_instance NEXTCLOUD {
  state MASTER
  interface eth0
  virtual_router_id 2
  priority 100
  advert_int 1
  authentication {
     auth_type PASS
     auth_pass Nextcloud
  }
  virtual_ipaddress {
    192.168.0.180
  }
  track_script {
    check_haproxy
  }
}

Nur am haproxy1-Server (192.168.0.191) 

global_defs {
  router_id nextcloud
  enable_script_security
  script_user haproxy
}

vrrp_script check_haproxy {
  script "/usr/bin/killall -0 haproxy"
  interval 2
  weight 2
  user haproxy
}

vrrp_instance NEXTCLOUD {
  state BACKUP
  interface eth0
  virtual_router_id 2
  priority 99
  advert_int 1
  authentication {
     auth_type PASS
     auth_pass Nextcloud
  }
  virtual_ipaddress {
    192.168.0.180
  }
  track_script {
    check_haproxy
  }
}

Standardkonfigurationen (beide Nodes: haproxy, haproxy1)

Starten Sie die Dienste neu

systemctl restart haproxy.service keepalived.service

und überprüfen Sie die virtuelle IP (FailoverIP, ClusterIP) am haproxy-Server:

ip --br a

Stoppen Sie nun bspw. den HAProxy-Service am ersten Server, so wird die virtuelle IP vom ersten Server entfernt und am zweiten HAProxy-Server gebunden.

3. Installation der Webserver nc1, nc2

Installation der Basissoftware (beide Nodes: nc1, nc2)

apt install -y \
apt-transport-https bash-completion bzip2 ca-certificates cron curl dialog \
dirmngr ffmpeg ghostscript git gpg gnupg gnupg2 htop jq libfile-fcntllock-perl \
libfontconfig1 libfuse2 locate lsb-release nfs-common nodejs npm net-tools rsyslog screen smbclient \
socat software-properties-common ssl-cert tree unzip vim wget zip

» Ubuntu 24 LTS (beide Nodes: nc1, nc2)

apt install -y curl gnupg2 ca-certificates lsb-release ubuntu-keyring

curl https://nginx.org/keys/nginx_signing.key | gpg --dearmor \
    | sudo tee /usr/share/keyrings/nginx-archive-keyring.gpg >/dev/null

echo "deb [signed-by=/usr/share/keyrings/nginx-archive-keyring.gpg] \
    http://nginx.org/packages/mainline/ubuntu `lsb_release -cs` nginx" \
    | sudo tee /etc/apt/sources.list.d/nginx.list

echo "deb [signed-by=/usr/share/keyrings/nginx-archive-keyring.gpg] \
    http://nginx.org/packages/mainline/ubuntu `lsb_release -cs` nginx" \
    | sudo tee /etc/apt/sources.list.d/nginx.list

echo -e "Package: *\nPin: origin nginx.org\nPin: release o=nginx\nPin-Priority: 900\n" \
    | sudo tee /etc/apt/preferences.d/99nginx

add-apt-repository -y ppa:ondrej/php

curl "https://keyserver.ubuntu.com/pks/lookup?op=get&search=0xB8DC7E53946656EFBCE4C1DD71DAEAAB4AD4CAB6" \
    | gpg --dearmor | sudo tee /usr/share/keyrings/ondrej-ubuntu-php.gpg >/dev/null

cat <<EOF > /etc/apt/sources.list.d/ondrej-ubuntu-php-noble.sources
Types: deb
URIs: https://ppa.launchpadcontent.net/ondrej/php/ubuntu/
Suites: noble
Components: main
Signed-By: /usr/share/keyrings/ondrej-ubuntu-php.gpg
EOF

curl -o /usr/share/keyrings/mariadb-keyring.pgp 'https://mariadb.org/mariadb_release_signing_key.pgp'

echo "deb [signed-by=/usr/share/keyrings/mariadb-keyring.pgp] https://mirror1.hs-esslingen.de/pub/Mirrors/mariadb/repo/10.11/ubuntu $(lsb_release -cs) main" \
    | sudo tee /etc/apt/sources.list.d/mariadb.list

» Debian 12 (beide Nodes: nc1, nc2)

apt install -y curl gnupg2 ca-certificates lsb-release debian-archive-keyring

curl https://nginx.org/keys/nginx_signing.key | gpg --dearmor \
    | sudo tee /usr/share/keyrings/nginx-archive-keyring.gpg >/dev/null

echo "deb [signed-by=/usr/share/keyrings/nginx-archive-keyring.gpg] \
    http://nginx.org/packages/mainline/debian `lsb_release -cs` nginx" \
    | sudo tee /etc/apt/sources.list.d/nginx.list

echo -e "Package: *\nPin: origin nginx.org\nPin: release o=nginx\nPin-Priority: 900\n" \
    | sudo tee /etc/apt/preferences.d/99nginx

sh -c 'echo "deb [signed-by=/usr/share/keyrings/deb.sury.org-php.gpg] https://packages.sury.org/php/ $(lsb_release -sc) main" > /etc/apt/sources.list.d/php.list'

curl -sSLo /usr/share/keyrings/deb.sury.org-php.gpg https://packages.sury.org/php/apt.gpg

wget https://downloads.mariadb.com/MariaDB/mariadb_repo_setup

chmod +x mariadb_repo_setup

./mariadb_repo_setup --mariadb-server-version="mariadb-10.11"

Installation und Einrichtung (beide Nodes: nc1, nc2)

Aktualisierung der Self-Sign-Certificates:

apt update && make-ssl-cert generate-default-snakeoil -y

Installation des Webservers nginx:

apt install -y nginx

systemctl enable nginx.service
mv /etc/nginx/nginx.conf /etc/nginx/nginx.conf.bak
touch /etc/nginx/nginx.conf && nano /etc/nginx/nginx.conf

Kopieren Sie den gesamten nachfolgenden Inhalt in die Datei:

user www-data;
worker_processes auto;
pid /var/run/nginx.pid;
events {
  worker_connections 2048;
  multi_accept on;
  use epoll;
  }
http {
  log_format criegerde escape=json
  '{'
    '"time_local":"$time_local",'
    '"remote_addr":"$remote_addr",'
    '"remote_user":"$remote_user",'
    '"request":"$request",'
    '"status": "$status",'
    '"body_bytes_sent":"$body_bytes_sent",'
    '"request_time":"$request_time",'
    '"http_referrer":"$http_referer",'
    '"http_user_agent":"$http_user_agent"'
  '}';
  server_names_hash_bucket_size 64;
  access_log /var/log/nginx/access.log criegerde;
  error_log /var/log/nginx/error.log warn;
  set_real_ip_from 192.168.0.180;
  real_ip_header proxy_protocol;
  real_ip_recursive on;
  include /etc/nginx/mime.types;
  default_type application/octet-stream;
  sendfile on;
  send_timeout 3600;
  tcp_nopush on;
  tcp_nodelay on;
  open_file_cache max=500 inactive=10m;
  open_file_cache_errors on;
  keepalive_timeout 65;
  reset_timedout_connection on;
  server_tokens off;
  resolver 127.0.0.53 valid=30s;
  resolver_timeout 5s;
  include /etc/nginx/conf.d/*.conf;
  }

Installation PHP 8.3:

apt update && apt install -y php-common \
php8.3-{fpm,gd,curl,xml,zip,intl,mbstring,bz2,ldap,apcu,bcmath,gmp,imagick,igbinary,mysql,redis,smbclient,sqlite3,cli,common,opcache,readline} \
imagemagick libmagickcore-6.q16-6-extra --allow-change-held-packages

Optimieren von PHP 8.3:

timedatectl set-timezone Europe/Berlin
cp /etc/php/8.3/fpm/pool.d/www.conf /etc/php/8.3/fpm/pool.d/www.conf.bak
cp /etc/php/8.3/fpm/php-fpm.conf /etc/php/8.3/fpm/php-fpm.conf.bak
cp /etc/php/8.3/cli/php.ini /etc/php/8.3/cli/php.ini.bak
cp /etc/php/8.3/fpm/php.ini /etc/php/8.3/fpm/php.ini.bak
cp /etc/php/8.3/mods-available/apcu.ini /etc/php/8.3/mods-available/apcu.ini.bak
cp /etc/php/8.3/mods-available/opcache.ini /etc/php/8.3/mods-available/opcache.ini.bak
cp /etc/ImageMagick-6/policy.xml /etc/ImageMagick-6/policy.xml.bak
sed -i "s/;env\[HOSTNAME\] = /env[HOSTNAME] = /" /etc/php/8.3/fpm/pool.d/www.conf
sed -i "s/;env\[TMP\] = /env[TMP] = /" /etc/php/8.3/fpm/pool.d/www.conf
sed -i "s/;env\[TMPDIR\] = /env[TMPDIR] = /" /etc/php/8.3/fpm/pool.d/www.conf
sed -i "s/;env\[TEMP\] = /env[TEMP] = /" /etc/php/8.3/fpm/pool.d/www.conf
sed -i "s/;env\[PATH\] = /env[PATH] = /" /etc/php/8.3/fpm/pool.d/www.conf
sed -i 's/pm = dynamic/pm = ondemand/' /etc/php/8.3/fpm/pool.d/www.conf
sed -i 's/pm.max_children =.*/pm.max_children = 200/' /etc/php/8.3/fpm/pool.d/www.conf
sed -i 's/pm.start_servers =.*/pm.start_servers = 100/' /etc/php/8.3/fpm/pool.d/www.conf
sed -i 's/pm.min_spare_servers =.*/pm.min_spare_servers = 60/' /etc/php/8.3/fpm/pool.d/www.conf
sed -i 's/pm.max_spare_servers =.*/pm.max_spare_servers = 140/' /etc/php/8.3/fpm/pool.d/www.conf
sed -i "s/;pm.max_requests =.*/pm.max_requests = 1000/" /etc/php/8.3/fpm/pool.d/www.conf
sed -i "s/allow_url_fopen =.*/allow_url_fopen = 1/" /etc/php/8.3/fpm/php.ini
sed -i "s/output_buffering =.*/output_buffering = Off/" /etc/php/8.3/cli/php.ini
sed -i "s/max_execution_time =.*/max_execution_time = 3600/" /etc/php/8.3/cli/php.ini
sed -i "s/max_input_time =.*/max_input_time = 3600/" /etc/php/8.3/cli/php.ini
sed -i "s/post_max_size =.*/post_max_size = 10240M/" /etc/php/8.3/cli/php.ini
sed -i "s/upload_max_filesize =.*/upload_max_filesize = 10240M/" /etc/php/8.3/cli/php.ini
sed -i "s/;date.timezone.*/date.timezone = Europe\/\Berlin/" /etc/php/8.3/cli/php.ini
sed -i "s/;cgi.fix_pathinfo.*/cgi.fix_pathinfo=0/" /etc/php/8.3/cli/php.ini
sed -i "s/memory_limit = 128M/memory_limit = 1G/" /etc/php/8.3/fpm/php.ini
sed -i "s/output_buffering =.*/output_buffering = Off/" /etc/php/8.3/fpm/php.ini
sed -i "s/max_execution_time =.*/max_execution_time = 3600/" /etc/php/8.3/fpm/php.ini
sed -i "s/max_input_time =.*/max_input_time = 3600/" /etc/php/8.3/fpm/php.ini
sed -i "s/post_max_size =.*/post_max_size = 10G/" /etc/php/8.3/fpm/php.ini
sed -i "s/upload_max_filesize =.*/upload_max_filesize = 10G/" /etc/php/8.3/fpm/php.ini
sed -i "s/;date.timezone.*/date.timezone = Europe\/\Berlin/" /etc/php/8.3/fpm/php.ini
sed -i "s/;cgi.fix_pathinfo.*/cgi.fix_pathinfo=0/" /etc/php/8.3/fpm/php.ini
sed -i "s/;session.cookie_secure.*/session.cookie_secure = True/" /etc/php/8.3/fpm/php.ini
sed -i "s/;opcache.enable=.*/opcache.enable=1/" /etc/php/8.3/fpm/php.ini
sed -i "s/;opcache.validate_timestamps=.*/opcache.validate_timestamps=1/" /etc/php/8.3/fpm/php.ini
sed -i "s/;opcache.enable_cli=.*/opcache.enable_cli=1/" /etc/php/8.3/fpm/php.ini
sed -i "s/;opcache.memory_consumption=.*/opcache.memory_consumption=256/" /etc/php/8.3/fpm/php.ini
sed -i "s/;opcache.interned_strings_buffer=.*/opcache.interned_strings_buffer=64/" /etc/php/8.3/fpm/php.ini
sed -i "s/;opcache.max_accelerated_files=.*/opcache.max_accelerated_files=100000/" /etc/php/8.3/fpm/php.ini
sed -i "s/;opcache.revalidate_freq=.*/opcache.revalidate_freq=0/" /etc/php/8.3/fpm/php.ini
sed -i "s/;opcache.save_comments=.*/opcache.save_comments=1/" /etc/php/8.3/fpm/php.ini
sed -i "s/;opcache.huge_code_pages=.*/opcache.huge_code_pages=0/" /etc/php/8.3/fpm/php.ini
sed -i "s|;emergency_restart_threshold.*|emergency_restart_threshold = 10|g" /etc/php/8.3/fpm/php-fpm.conf
sed -i "s|;emergency_restart_interval.*|emergency_restart_interval = 1m|g" /etc/php/8.3/fpm/php-fpm.conf
sed -i "s|;process_control_timeout.*|process_control_timeout = 10|g" /etc/php/8.3/fpm/php-fpm.conf
sed -i '$aapc.enable_cli=1' /etc/php/8.3/mods-available/apcu.ini
sed -i 's/opcache.jit=off/opcache.jit=on/' /etc/php/8.3/mods-available/opcache.ini
sed -i '$aopcache.jit=1255' /etc/php/8.3/mods-available/opcache.ini
sed -i '$aopcache.jit_buffer_size=256M' /etc/php/8.3/mods-available/opcache.ini
sed -i "s/rights=\"none\" pattern=\"PS\"/rights=\"read|write\" pattern=\"PS\"/" /etc/ImageMagick-6/policy.xml
sed -i "s/rights=\"none\" pattern=\"EPS\"/rights=\"read|write\" pattern=\"EPS\"/" /etc/ImageMagick-6/policy.xml
sed -i "s/rights=\"none\" pattern=\"PDF\"/rights=\"read|write\" pattern=\"PDF\"/" /etc/ImageMagick-6/policy.xml
sed -i "s/rights=\"none\" pattern=\"XPS\"/rights=\"read|write\" pattern=\"XPS\"/" /etc/ImageMagick-6/policy.xml
sed -i '$a[mysql]' /etc/php/8.3/mods-available/mysqli.ini
sed -i '$amysql.allow_local_infile=On' /etc/php/8.3/mods-available/mysqli.ini
sed -i '$amysql.allow_persistent=On' /etc/php/8.3/mods-available/mysqli.ini
sed -i '$amysql.cache_size=2000' /etc/php/8.3/mods-available/mysqli.ini
sed -i '$amysql.max_persistent=-1' /etc/php/8.3/mods-available/mysqli.ini
sed -i '$amysql.max_links=-1' /etc/php/8.3/mods-available/mysqli.ini
sed -i '$amysql.default_port=3306' /etc/php/8.3/mods-available/mysqli.ini
sed -i '$amysql.connect_timeout=60' /etc/php/8.3/mods-available/mysqli.ini
sed -i '$amysql.trace_mode=Off' /etc/php/8.3/mods-available/mysqli.ini

Editieren Sie PHP8.3 um die Sessions zentral im Redis zu verwalten.

nano /etc/php/8.3/fpm/php.ini

Kommentieren Sie diese Zeile aus

[...]
; session.save_handler = files
[...]

und fügen diese hinzu:

[...]
session.save_handler = redis
session.save_path = "tcp://192.168.0.199:6379?auth=Redis-Passwort-Bitte-ändern?persistent=1&weight=1&database=10"
redis.session.locking_enabled = 1
redis.session.lock_retries = -1
redis.session.lock_wait_time = 10000

Starten Sie nun beide Dienste, nginx und PHP, neu:

systemctl restart php8.3-fpm.service nginx.service

Installieren Sie nun die MariaDB-Client-Tools und die Redis-Tools:

apt install -y mariadb-client redis-tools

Standardkonfigurationen (beide Nodes: nc1, nc2)

Webserver für Nextcloud:

[ -f /etc/nginx/conf.d/default.conf ] && mv /etc/nginx/conf.d/default.conf /etc/nginx/conf.d/default.conf.bak
touch /etc/nginx/conf.d/default.conf
touch /etc/nginx/conf.d/nextcloud.conf

Erzeugen Sie den DH-Key:

openssl dhparam -dsaparam -out /etc/ssl/certs/dhparam.pem 4096

Öffnen Sie den vHost

nano /etc/nginx/conf.d/nextcloud.conf

und fügen den Inhalt ein:

limit_req_zone $binary_remote_addr zone=NextcloudRateLimit:10m rate=2r/s;
server {
  listen 443 ssl default_server proxy_protocol;
  listen [::]:443 ssl default_server proxy_protocol;
  http2 on;
  server_name nc 192.168.0.180;
  ssl_certificate /etc/ssl/certs/ssl-cert-snakeoil.pem;
  ssl_certificate_key /etc/ssl/private/ssl-cert-snakeoil.key;
  ssl_trusted_certificate /etc/ssl/certs/ssl-cert-snakeoil.pem;
  #ssl_certificate /etc/letsencrypt/rsa-certs/fullchain.pem;
  #ssl_certificate_key /etc/letsencrypt/rsa-certs/privkey.pem;
  #ssl_certificate /etc/letsencrypt/ecc-certs/fullchain.pem;
  #ssl_certificate_key /etc/letsencrypt/ecc-certs/privkey.pem;
  #ssl_trusted_certificate /etc/letsencrypt/ecc-certs/chain.pem;
  ssl_dhparam /etc/ssl/certs/dhparam.pem;
  ssl_session_timeout 1d;
  ssl_session_cache shared:SSL:50m;
  ssl_session_tickets off;
  ssl_protocols TLSv1.3 TLSv1.2;
  ssl_ciphers 'TLS-CHACHA20-POLY1305-SHA256:TLS-AES-256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA512:DHE-RSA-AES256-GCM-SHA512:ECDHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES256-GCM-SHA384';
  ssl_ecdh_curve X448:secp521r1:secp384r1;
  ssl_prefer_server_ciphers on;
  ssl_stapling on;
  ssl_stapling_verify on;
  client_max_body_size 10G;
  client_body_timeout 3600s;
  client_body_buffer_size 512k;
  fastcgi_buffers 64 4K;
  gzip on;
  gzip_vary on;
  gzip_comp_level 4;
  gzip_min_length 256;
  gzip_proxied expired no-cache no-store private no_last_modified no_etag auth;
  gzip_types application/atom+xml text/javascript application/javascript application/json application/ld+json application/manifest+json application/rss+xml application/vnd.geo+json application/vnd.ms-fontobject application/wasm application/x-font-ttf application/x-web-app-manifest+json application/xhtml+xml application/xml font/opentype image/bmp image/svg+xml image/x-icon text/cache-manifest text/css text/plain text/vcard text/vnd.rim.location.xloc text/vtt text/x-component text/x-cross-domain-policy;
  add_header Strict-Transport-Security            "max-age=15768000; includeSubDomains; preload;" always;
  add_header Permissions-Policy                   "interest-cohort=()";
  add_header Referrer-Policy                      "no-referrer"   always;
  add_header X-Content-Type-Options               "nosniff"       always;
  add_header X-Download-Options                   "noopen"        always;
  add_header X-Frame-Options                      "SAMEORIGIN"    always;
  add_header X-Permitted-Cross-Domain-Policies    "none"          always;
  add_header X-Robots-Tag                         "noindex, nofollow" always;
  add_header X-XSS-Protection                     "1; mode=block" always;
  add_header Alt-Svc                              'h3=":$server_port"; ma=86400';
  add_header x-quic                               "h3";
  add_header Alt-Svc                              'h3-29=":$server_port"';
  fastcgi_hide_header X-Powered-By;
  include mime.types;
  types {
  text/javascript mjs;
	  }
  root /var/www/nextcloud;
  index index.php index.html /index.php$request_uri;
  location = / {
    if ( $http_user_agent ~ ^DavClnt ) {
      return 302 /remote.php/webdav/$is_args$args;
      }
  }
  location = /robots.txt {
    allow all;
    log_not_found off;
    access_log off;
    }
  location ^~ /.well-known {
    location = /.well-known/carddav { return 301 /remote.php/dav/; }
    location = /.well-known/caldav  { return 301 /remote.php/dav/; }
    location /.well-known/acme-challenge { try_files $uri $uri/ =404; }
    location /.well-known/pki-validation { try_files $uri $uri/ =404; }
    return 301 /index.php$request_uri;
    }
  location ~ ^/(?:build|tests|config|lib|3rdparty|templates|data)(?:$|/)  { return 404; }
  location ~ ^/(?:\.|autotest|occ|issue|indie|db_|console)  { return 404; }
  location ~ \.php(?:$|/) {
    rewrite ^/(?!index|remote|public|cron|core\/ajax\/update|status|ocs\/v[12]|updater\/.+|ocs-provider\/.+|.+\/richdocumentscode\/proxy) /index.php$request_uri;
    fastcgi_split_path_info ^(.+?\.php)(/.*)$;
    set $path_info $fastcgi_path_info;
    try_files $fastcgi_script_name =404;
    include fastcgi_params;
    fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
    fastcgi_param PATH_INFO $path_info;
    fastcgi_param HTTPS on;
    fastcgi_param modHeadersAvailable true;
    fastcgi_param front_controller_active true;
    fastcgi_pass php-handler;
    fastcgi_intercept_errors on;
    fastcgi_request_buffering off;
    fastcgi_read_timeout 3600;
    fastcgi_send_timeout 3600;
    fastcgi_connect_timeout 3600;
    fastcgi_max_temp_file_size 0;
    }
    location ~ \.(?:css|js|mjs|svg|gif|ico|jpg|png|webp|wasm|tflite|map|ogg|flac)$ {
    try_files $uri /index.php$request_uri;
    add_header Cache-Control                        "public, max-age=15778463, $asset_immutable";
    add_header Permissions-Policy                   "interest-cohort=()";
    add_header Referrer-Policy                      "no-referrer"   always;
    add_header X-Content-Type-Options               "nosniff"       always;
    add_header X-Download-Options                   "noopen"        always;
    add_header X-Frame-Options                      "SAMEORIGIN"    always;
    add_header X-Permitted-Cross-Domain-Policies    "none"          always;
    add_header X-Robots-Tag                         "noindex, nofollow" always;
    add_header X-XSS-Protection                     "1; mode=block" always;
    add_header Alt-Svc                              'h3=":$server_port"; ma=86400';
    add_header x-quic                               "h3";
    add_header Alt-Svc                              'h3-29=":$server_port"';
    expires 6M;
    access_log off;
    location ~ \.wasm$ {
      default_type application/wasm;
      }
    }
  location ~ \.(otf|woff2?)$ {
    try_files $uri /index.php$request_uri;
    expires 7d;
    access_log off;
    }
  location /remote {
    return 301 /remote.php$request_uri;
    }
  location /login {
    limit_req zone=NextcloudRateLimit burst=5 nodelay;
    limit_req_status 429;
    try_files $uri $uri/ /index.php$request_uri;
    }    
  location / {
    try_files $uri $uri/ /index.php$request_uri;
    }
}

Starten Sie die Webserver neu:

systemctl restart nginx.service

Nur Server nc1 (192.168.0.197)

Wir beginnen nun die ‚eigentliche‘ Installation der Nextcloud Software am Server nc1 (192.168.0.197).

cd /usr/local/src

Laden Sie das aktuelle Nextcloud Release herunter:

wget https://download.nextcloud.com/server/releases/latest.tar.bz2
wget https://download.nextcloud.com/server/releases/latest.tar.bz2.md5

Überpüfen Sie die heruntergeladene Datei:

md5sum -c latest.tar.bz2.md5 < latest.tar.bz2

Nur wenn Ihnen der Test mit „OK“ bestätigt wird

latest.tar.bz2: OK

fahren Sie fort! Entpacken Sie die Nextcloud Software in das Webverzeichnis (/var/www), setzen dann die notwendigen Berechtigung und Löschen die heruntergeladenen Dateien:

tar -xjf latest.tar.bz2 -C /var/www && chown -R www-data:www-data /var/www/ && rm -f latest.tar.bz2

4. Installation lsyncd

Nur Server nc1 (192.168.0.197)

Installation lsyncd

apt install -y lsyncd

Erstellen Sie die notwendige lsyncd-Konfigurationsdatei

touch /etc/lsyncd/lsyncd.conf.lua

und fügen den Inhalt ein:

settings{
logfile="/var/log/lsyncd.log",
statusFile="/var/log/lsyncd.stat",
statusInterval=1,
}
sync{
default.rsync,
source="/var/www/nextcloud",
target="/lsyncd",
delay=0,
rsync = {
verbose = true,
owner = true,
group = true,
perms = true,
update = false,
chmod = 750,
chown = "www-data:www-data"
}
}

Erweitern Sie die fstab

nano /etc/fstab

um diese Einträge:

[...]
192.168.0.198:/var/www/nextcloud /lsyncd nfs4 nfsvers=4.2,noatime,nolock,sec=sys 0 0
192.168.0.199:/nc_data /nc_data nfs4 nfsvers=4.2,noatime,nolock,sec=sys 0 0

5. Installation NFS für lsyncd

Nur Server nc2 (192.168.0.198)

Installation des NFS-Servers für lsyncd (Replikation nc1 » nc2)

apt install -y nfs-kernel-server

Ergänzen Sie nachfolgend diese Datei

nano /etc/exports

am Ende um diesen Eintrag

[...]
/var/www/nextcloud 192.168.0.197(rw,sync,anonuid=33,anongid=33,no_subtree_check,no_root_squash)

Veröffentlichen Sie die Freigabe

exportfs -av

Erweitern Sie die fstab

nano /etc/fstab

um diesen Eintrag

[...]
192.168.0.199:/nc_data /nc_data nfs4 nfsvers=4.2,noatime,nolock,sec=sys 0 0

6. Installation Redis-Server, MariaDB-Server, NFS-Server

Installation der Basissoftware (Node: db-redis)

apt install -y \
apt-transport-https bash-completion bzip2 ca-certificates cron curl dialog \
dirmngr ffmpeg ghostscript git gpg gnupg gnupg2 htop jq libfile-fcntllock-perl \
libfontconfig1 libfuse2 locate lsb-release nfs-common nodejs npm net-tools rsyslog screen smbclient \
socat software-properties-common ssl-cert tree unzip vim wget zip

» Ubuntu 24 LTS (Node: db-redis)

apt install -y curl gnupg2 ca-certificates lsb-release ubuntu-keyring

curl -o /usr/share/keyrings/mariadb-keyring.pgp 'https://mariadb.org/mariadb_release_signing_key.pgp'

echo "deb [signed-by=/usr/share/keyrings/mariadb-keyring.pgp] https://mirror1.hs-esslingen.de/pub/Mirrors/mariadb/repo/10.11/ubuntu $(lsb_release -cs) main" \
    | sudo tee /etc/apt/sources.list.d/mariadb.list

» Debian 12 (Node: db-redis)

apt install -y curl gnupg2 ca-certificates lsb-release debian-archive-keyring

wget https://downloads.mariadb.com/MariaDB/mariadb_repo_setup

chmod +x mariadb_repo_setup

./mariadb_repo_setup --mariadb-server-version="mariadb-10.11"

Installation und Einrichtung (Node: db-redis)

apt update
apt install -y nfs-kernel-server redis-server redis-tools mariadb-server

Konfiguration MariaDB

systemctl stop mariadb.service
mkdir -p /var/log/mysql
chown -R mysql:mysql /var/log/mysql
mv /etc/mysql/my.cnf /etc/mysql/my.cnf.bak
nano /etc/mysql/my.cnf

Kopieren Sie alle nachfolgenden Zeilen in die leere Datei:

[client]
default-character-set = utf8mb4
port = 3306
socket = /var/run/mysqld/mysqld.sock
[mysqld_safe]
log_error=/var/log/mysql/mysql_error.log
nice = 0
socket = /var/run/mysqld/mysqld.sock
[mysqld]
# performance_schema=ON
basedir = /usr
bind-address = 192.168.0.199
binlog_format = ROW
character-set-server = utf8mb4
collation-server = utf8mb4_general_ci
datadir = /var/lib/mysql
default_storage_engine = InnoDB
expire_logs_days = 2
general_log_file = /var/log/mysql/mysql.log
innodb_buffer_pool_size = 2G
innodb_log_buffer_size = 32M
innodb_log_file_size = 512M
innodb_read_only_compressed=OFF
join_buffer_size = 2M
key_buffer_size = 512M
lc_messages_dir = /usr/share/mysql
lc_messages = en_US
log_bin = /var/log/mysql/mariadb-bin
log_bin_index = /var/log/mysql/mariadb-bin.index
log_bin_trust_function_creators = true
log_error = /var/log/mysql/mysql_error.log
log_slow_verbosity = query_plan
log_warnings = 2
long_query_time = 1
max_connections = 100
max_heap_table_size = 64M
max_allowed_packet = 512M
myisam_sort_buffer_size = 512M
port = 3306
pid-file = /var/run/mysqld/mysqld.pid
query_cache_limit = 0
query_cache_size = 0 
read_buffer_size = 2M
read_rnd_buffer_size = 2M
skip-name-resolve
socket = /var/run/mysqld/mysqld.sock
sort_buffer_size = 2M
table_open_cache = 400
table_definition_cache = 800
tmp_table_size = 32M
tmpdir = /tmp
transaction_isolation = READ-COMMITTED
user = mysql
wait_timeout = 600
[mariadb-dump]
max_allowed_packet = 512M
quick
quote-names
[isamchk]
key_buffer = 16M

Speichern und schließen Sie die Datei und starten dann den Datenbankserver neu,

systemctl restart mariadb.service

um die Nextcloud-Datenbank, den Nextcloud-Benutzer und das dazugehörige Passwort einzurichten:

mariadb -e "CREATE DATABASE nextcloud CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci; CREATE USER 'nextcloud'@'192.168.0.199' identified by 'nextcloud'; GRANT ALL PRIVILEGES on nextcloud.* to 'nextcloud'@'192.168.0.199'; FLUSH privileges;"
mariadb -e "CREATE USER 'nextcloud'@'192.168.0.197' identified by 'nextcloud'; GRANT ALL PRIVILEGES on nextcloud.* to 'nextcloud'@'192.168.0.197'; FLUSH privileges;"
mariadb -e "CREATE USER 'nextcloud'@'192.168.0.198' identified by 'nextcloud'; GRANT ALL PRIVILEGES on nextcloud.* to 'nextcloud'@'192.168.0.198'; FLUSH privileges;"

Konfiguration Redis-Server

cp /etc/redis/redis.conf /etc/redis/redis.conf.bak
sed -i 's/# requirepass foobared/requirepass Redis-Passwort-Bitte-ändern/' /etc/redis/redis.conf
sed -i 's/# maxclients 10000/maxclients 10240/' /etc/redis/redis.conf
usermod -aG redis www-data
cp /etc/sysctl.conf /etc/sysctl.conf.bak
sed -i '$avm.overcommit_memory = 1' /etc/sysctl.conf

Öffnen Sie die Datei

nano /etc/redis/redis.conf

und ersetzen die Zeile wie folgt:

bind 192.168.0.199 127.0.0.1 -::1

Starten Sie Redis neu

systemctl restart redis-server.service

Installation des NFS-Servers für Nextcloud (Replikation nc1 und nc2 » NFS)

apt install -y nfs-kernel-server
mkdir -p /nc_data
chown -R www-data:www-data /nc_data

Ergänzen Sie nachfolgend diese Datei

nano /etc/exports

am Ende um diesen Eintrag

[...]
/nc_data 192.168.0.197(rw,sync,anonuid=33,anongid=33,no_subtree_check,no_root_squash)
/nc_data 192.168.0.198(rw,sync,anonuid=33,anongid=33,no_subtree_check,no_root_squash)

Veröffentlichen Sie die Freigabe

exportfs -av

7. Installation Nextcloud

Installation der Nextcloud (Node: nc1)

Starten Sie zuerst alle Server in folgender Reihenfolge einmalig neu:

1. haproxy » 2. haproxy1 » 3. db-redis » 4. nc1 » 5. nc2

Verbinden Sie sich per SSH und sudo Berechtigungen erneut mit dem Server „nc1“ und starten dort die Installation

sudo -u www-data php /var/www/nextcloud/occ maintenance:install --database "mysql" --database-host "192.168.0.199" --database-port "3306" --database-name "nextcloud" --database-user "nextcloud" --database-pass "nextcloud" --admin-user "ncAdmin" --admin-pass "NcAdminPWD" --data-dir "/nc_data"

Setzen Sie diese Befehle per occ ab:

» trusted domain

sudo -u www-data php /var/www/nextcloud/occ config:system:set trusted_domains 1 --value=192.168.0.180

» overwritehost/cli.url

sudo -u www-data php /var/www/nextcloud/occ config:system:set overwritehost --value=192.168.0.180
sudo -u www-data php /var/www/nextcloud/occ config:system:set overwrite.cli.url --value=https://192.168.0.180

Führen Sie diese Zeilen aus, um Nextcloud zu optimieren:

sudo -u www-data cp /var/www/nextcloud/config/config.php /var/www/nextcloud/config/config.php.bak 

sed -i '/);/d' /var/www/nextcloud/config/config.php
cat <<EOF >>/var/www/nextcloud/config/config.php
  'activity_expire_days' => 14,
  'allow_local_remote_servers' => true,
  'auth.bruteforce.protection.enabled' => true,
  'forbidden_filenames' =>
  array (
    0 => '.htaccess',
    1 => 'Thumbs.db',
    2 => 'thumbs.db',
    ),
    'cron_log' => true,
    'default_phone_region' => 'DE',
    'enable_previews' => true,
    'enabledPreviewProviders' =>
      array (
      0 => 'OC\\Preview\\PNG',
      1 => 'OC\\Preview\\JPEG',
      2 => 'OC\\Preview\\GIF',
      3 => 'OC\\Preview\\BMP',
      4 => 'OC\\Preview\\XBitmap',
      5 => 'OC\\Preview\\Movie',
      6 => 'OC\\Preview\\PDF',
      7 => 'OC\\Preview\\MP3',
      8 => 'OC\\Preview\\TXT',
      9 => 'OC\\Preview\\MarkDown',
      10 => 'OC\\Preview\\HEIC',
      11 => 'OC\\Preview\\Movie',
      12 => 'OC\\Preview\\MKV',
      13 => 'OC\\Preview\\MP4',
      14 => 'OC\\Preview\\AVI',
      ),
      'filesystem_check_changes' => 0,
      'filelocking.enabled' => 'true',
      'htaccess.RewriteBase' => '/',
      'integrity.check.disabled' => false,
      'knowledgebaseenabled' => false,
      'log_rotate_size' => '104857600',
      'logfile' => '/nc-data/nextcloud.log',
      'loglevel' => 2,
      'logtimezone' => 'Europe/Berlin',
      'memcache.local' => '\\OC\\Memcache\\APCu',
      'memcache.locking' => '\\OC\\Memcache\\Redis',
      'overwriteprotocol' => 'https',
      'preview_max_x' => 1024,
      'preview_max_y' => 768,
      'preview_max_scale_factor' => 1,
      'profile.enabled' => false,
      'redis' =>
      array (
        'host' => '192.168.0.199',
        'port' => 6379,
        'password' => 'Redis-Passwort-Bitte-ändern',
        'timeout' => 0.5,
        'dbindex' => 1,
        ),
        'quota_include_external_storage' => false,
        'share_folder' => '/Freigaben',
        'skeletondirectory' => '',
        'trashbin_retention_obligation' => 'auto, 7',
        'maintenance_window_start' => 1,
        );
EOF

Rufen Sie jetzt erstmalig im Browser die IP der Nextcloud auf (192.168.0.180) und melden sich an der neuen Nextcloud an.

Im Beispiel lautet der Benutzer „ncAdmin“ und das Passwort „NcAdminPWD„

Unter System sehen Sie, dass Sie bspw. mit NC1 verbunden sind.

Stoppen wir am NC1 exemplarisch den Webserver, so werden Sie automatisch auf den Webserver NC2 weitergeleitet. Da PHP in Redis verwaltet wird bleibt Ihre Sitzung gültig und es wird keine erneute Anmeldung benötigt:

Aufgrund der lsyncd-Replikation des Nextcloudverzeichnisses (/var/www/nextcloud) müssen Wartungsarbeiten, Nextcloud-Upgrades und andere Anpassungen an der Nextcloud ausschließlich am Server „nc1“ erfolgen!

Zur Fehleranalyse finden Sie das konsolidierte Nextcloud Logfile unter

/nc_data/nextcloud.log

Wir wünschen Ihnen viel Spaß mit der lastverteilten und ausfallsicheren Nextcloud! Über Ihre Unterstützung (diese wird ordnungsgemäß versteuert!) würden sich meine Frau, meine Zwillinge und ich sehr freuen!

Carsten Rieger IT-Services | Slogan