« Changelog History


xsrv v1.0.0 Release Notes

Release Date: 2021-02-12 // about 3 years ago

  • 🚀 This is a major rewrite of https://github.com/nodiscc/srv01. To upgrade/migrate from previous releases, you must redeploy services to a new instance, and restore user data from backups/exports.

    📚 This releases improves usability, portability, standards compliance, separation of concerns, performance, documentation, security, simplifies installation and usage, and adds new features to all roles/components. A summary of changes is included below. See [README.md](README.md) for more information.

    xsrv command-line tool

    • 👌 improve/simplify command-line usage, see xsrv help
    • 🔨 refactor main script/simplify/cleanup
    • 👉 use pwgen (optional) to generate random passwords during host creation
    • 👉 make installation to $PATH and use of sudo optional
    • ⬆️ use ansible-galaxy collections for role upgrades method

    🔨 example playbook: refactor:

    • add examples for playbook, inventory, group_vars and host_vars (cleartext and vaulted) files
    • 0️⃣ disable all but essential roles by default. Additional roles should be enabled manually by the admin
    • 0️⃣ firewall: by default, allow incoming traffic for netdata dashboard from LAN (monitoring role is enabled by default)
    • 0️⃣ firewall: by default, allow incoming SSH from anywhere (key-based authentication is enabled so this is reasonably secure)
    • 0️⃣ firewall: by default, allow HTTP/HTTPS access from anywhere (required for let's encrypt http-01 challenge, and apache role is enabled by default)
    • 🔧 firewall: change the default policy for the 'global' firehol_network definition to RETURN (changes nothing in the default configuration, makes adding other network definitions easier)
    • 0️⃣ doc: add firewall examples for all services (only from LAN by default)
    • doc: add example .gitlab-ci.yml
    • 0️⃣ ansible/all roles: use ansible-vault as default storage for sensitive values
    • ansible: use .ansible-vault-password as vault password file
    • ansible: speed up ansible SSH operations using controlmaster and pipelining SSH options
    • host_vars: add a netdata check for successful daily backups
    • host_vars: add netdata process checks for ssh, fail2ban, ntp, httpd, sql
    • 0️⃣ host_vars: auto-restart services by default when needrestart detects a restart is required
    • ✂ remove unused directories, cleanup

    🔨 common: refactor role:

    • import from https://gitlab.com/nodiscc/ansible-xsrv-common
    • ⚡️ unattended-upgrades: allow automatic upgrades from stable-updates repository
    • ⬆️ unattended-upgrades: install apt-listchanges (mail with a summary of changes will be sent to the server admin)
    • 👉 add ansible_user_allow_sudo_rsync_nopasswd option (allow ansible user to run sudo rsync without password)
    • 🔧 msmtp: require manual configuration of msmtp host/username/password (if msmtp installation is enabled)
    • 🔧 dns: add ability to configure multiple DNS nameservers in /etc/resolv.conf
    • 📦 packages: enable haveged installation by default
    • 📦 packages: don't install pwgen/secure-delete/autojump by default, add man package
    • 🚚 sshd: remove deprecated UsePrivilegeSeparation option
    • 🔧 sshd: make ssh server log level, PasswordAuthentication, AllowTcpForwarding and PermitRootLogin options configurable
    • sshd: fix accepted environment variables LANG,LC_* accepted from the client
    • sshd: explicitely deny AllowTcpForwarding, AllowAgentForwarding, GatewayPorts and X11Forwarding for the sftponly group
    • sshd: add [email protected] KexAlgorithm
    • 🐳 firewall: add an option to generate firewall rules compatible with docker swarm routing/port forwarding
    • 0️⃣ firewall: allow outgoing mail submission/port 587 by default
    • firewall: make firewall config file only readable by root
    • firewall: use an alias/variable to define LAN IP networks, templatize
    • 🔧 firewall/fail2ban: prevent firehol from overwriting fail2Ban rules, remove interaction/integration between services, split firewall/fail2ban configuration tasks, add ability to disable both
    • 🔧 fail2ban: make more settings configurable (destination e-mail, default findtime/maxretry/bantime)
    • users: simplify management, remove remotebackup options/special remotebackup user/tasks
    • 👉 users: linux_users is now compatible with ansible users module syntax, with added ssh_authorized_keys and sudo_nopasswd_commands parameters
    • users: fix user password generation (use random salt, make task idempotent by setting update_password: on_create by default)
    • 👉 users: ensure ansible user home is not world-readable

    🔨 monitoring: refactor role:

    • import from https://gitlab.com/nodiscc/ansible-xsrv-monitoring
    • 0️⃣ netdata: add ssl/x509 expiration checks, make http check timeout value optional, default to 1s)
    • 📦 netdata: allow installation from deb packages/packagecloud APT repository, make it the default
    • 🌲 netdata: decrease frequency of apache status checks to 10 seconds (decrease log spam)
    • netdata: disable access logs and debug logs by default (performance), add netdata_disable_*_log variables to configure it
    • netdata: disable cloud/SaaS features by default, add netdata_cloud_enabled variable to configure it
    • 🌐 netdata: disable web server gzip compression since we only use ssl
    • 🔧 netdata: install and configure https://gitlab.com/nodiscc/netdata-logcountmodule, disable notifications by default
    • 🔧 netdata: install and configure https://gitlab.com/nodiscc/netdata-modtime module
    • 🔧 netdata: make dbengine disk space size and memory page cache size configurable
    • netdata: monitor mysql server if mariadb role is enabled (add netdata mysql user)
    • 🔧 netdata: add default configuration for health notifications
    • 🚀 netdata: upgrade to latest stable release
    • 0️⃣ rsyslog: aggregate all log messages to /var/log/syslog by default
    • ⬆️ rsyslog: monitor samba, gitea, mumble-server, openldap, nextcloud, unattended-upgrades and rsnapshot log files with imfile module (when corresponding roles are enabled)
    • 🔊 rsyslog: make aggregation of apache access logs to syslog optional, disable by default
    • 🔊 rsyslog: disable aggregation of netdata logs to syslog by default (very noisy, many false-positive ERROR messages)
    • 🔊 rsyslog: discard apache access logs caused by netdata apche monitoring
    • 0️⃣ needrestart: don't auto-restart services by default
    • extend list of command-line monitoring tools (lsof/strace)
    • 📚 various fixes, reorder, cleanup, update documentation, fix role/certificate generation idempotence, make more components optional

    backup role

    • import from https://gitlab.com/nodiscc/ansible-xsrv-backup
    • 🚚 auto-load rsnapshot configuration from /etc/rsnapshot.d/.conf, remove hardcoded xsrv roles integration
    • 🔧 check rsnapshot configuration after copying files
    • restrict access to backups directory to root only
    • 👷 redirect cron job stdout to /dev/null, only send errors by mail
    • write rsnapshot last success time to file (allows monitoring the time since last successful backup)
    • store ssh public key to ansible facts (this will allow generating a human readable document/dashboard with hosts information)

    🔨 lamp role: refactor:

    apache role:

    • 🔨 import/refactor/split role from https://gitlab.com/nodiscc/ansible-xsrv-lamp
    • 🚚 use apache mod-md for Let's Encrypt certificate generation, remove certbot and associated ansible tasks
    • 🚚 switch to php-fpm interpreter, remove mod_php
    • switch to mpm_event, disable mpm_worker
    • switch to HTTP2
    • ✂ remove ability to create custom virtualhosts
    • ✂ remove automatic homepage generation feature (will be split to separate role)
    • enforce fail2ban bans on HTTP basic auth failures
    • 0️⃣ set the default log format to vhost_combined (all vhosts to a single file)
    • rename cert_mode variable to https_mode
    • 0️⃣ don't enable mod-deflate by default
    • 👍 add variable apache_allow_robots (allow/disabllow robots globally, default no)
    • ➕ add hard dependency on common role
    • ⚡️ update doc, cleanup, formatting, add screenshot
    • 🔧 require manual configuration of the letsencrypt admin email address
    • 🔒 disable X-Frame-Options header as Content-Security-Policy frame-ancestors replaces/obsoletes it
    • 🔒 disable setting a default Content-Security-Policy, each application is responsible for setting an appropriate CSP
    • mark HTTP->HTTPS redirects as permanent (HTTP code 301)
    • exclude /server-status from automatic HTTP -> HTTPS redirects
    • 0️⃣ ensure the default/fallback vhost is always the first in load order, raise HTTP error 403 and autoindex:error when accessing the default vhost

    🔨 nextcloud: refactor role:

    • import from https://gitlab.com/nodiscc/ansible-xsrv-nextcloud
    • ⬆️ determine appropriate setup procedure depending on whether nextcloud is already installed or not, installed version and current role version (installation/upgrades are now idempotent)
    • add support for let's encrypt certificates (use mod_md when nextcloud_rss_https_mode: letsencrypt. else generate self-signed certificates)
    • 👉 use ansible local fact file to store nextcloud installed version
    • ensure correct/restrictive permissions are set
    • 👌 support postgresql as database engine, make it the default
    • 🔧 move apache configuration steps to separate file, add full automatic virtualhost configuration for nextcloud
    • reorder setup procedure (setup apache last)
    • enable additional php modules https://docs.nextcloud.com/server/16/admin_manual/installation/source_installation.html#apache-web-server-configuration
    • reload apache instead of restarting when possible
    • 🔧 make basic settings configurable through ansible (FQDN, install directory, full URL, share_folder...)
    • 🔧 require manual configuration of nextcloud FQDN
    • enforce fail2ban bans on nextcloud login failures
    • ⬆️ upgrade nextcloud to latest stable version (https://nextcloud.com/changelog)
    • ⬆️ upgrade all nextcloud apps to latest compatible versions
    • 🔧 make installed/enabled applications configurable
    • enable APCu memcache
    • gallery app replaced with photos app
    • optional integration with backup role, delegate database backups to the respective database role (mariadb/postgresql)
    • ➕ add deck, notes, admin_audit and maps apps
    • ➕ add php-fpm configuration
    • 👷 run background jobs via cron every 5 minutes

    Migrating Nextcloud data to Postgresql from a MySQL-based installation: