Advanced Server Performance Tuning (L3+)

[mikesdeman]

Hi

I am running nextcloud (your standard PHP w MySQL webapp) in a very overbuilt VM but performance isn't as good as I would hope (don't get me wrong it is passable but not "fast").

Currently serving with:
-Centos8 OS
-Nginx webserver
-PHP7.4/PHP-FPM
-MariaDB 10.3 (pretty sure it isn't compatible with 10.4)
-Redis caching

HV is proxmox. Architecture is Haswell for the CPU. VM has 12c/16GB RAM (both of which can be increased but I sense this is not my bottleneck).

For comparison I have an alnmost identical nextcloud instance on a VM on Zen+ architecture which runs way way faster with way less cores and RAM. (I know frequency comparison is off but I am talking a seriously noticable difference).


I have opcache & REDIS for caching and GTMetrix from a geo close server registers the first contentful paint at 650ms and a fully loaded time of 1.6s... for the login page. I have upped the following configs but if I am honest I don't know what I am doing and would be grateful of any advice of tweaks I can make to webserver, php-fpm, mariadb or REDIS (or even perhaps the pve VM) confs that will improve performance. All of the guides I have read have either been easy wins (like enable Redis) or a little lacking in substance (the official nc docs don't really discuss changing away from service defaults):

grep worker_ /etc/nginx/nginx.conf
worker_processes 4;
    worker_connections 2048;

grep 'pm\.' /etc/php-fpm.d/www.conf|grep -v \;
pm.max_children = 256
pm.start_servers = 59
pm.min_spare_servers = 59
pm.max_spare_servers = 177

Memory allocated to PHP and PHP-FPM is 4G.

Appreciate any feedback on service performance tuning from experienced sysadmins!

[AOD_LTCaveman]

Hey theree;
Theres a couple of things we can try (Sadly I've got issues with typing due to a broken hand haha')

If you'd like to hit me up in Discord, please do so!
I work as a Systems Network Admin for a datacenter to give you a bit of my background, I deal with this stuff on almost a weekly basis.
Thanks,

[mikesdeman]

Hey theree;
Theres a couple of things we can try (Sadly I've got issues with typing due to a broken hand haha')

If you'd like to hit me up in Discord, please do so!
I work as a Systems Network Admin for a datacenter to give you a bit of my background, I deal with this stuff on almost a weekly basis.
Thanks,

Just a follow up to this I have found to the best of my knowledge that the issue is a combination of a couple of things. It looks like my own expectations of nextcloud and the performance of the base code may be a little too high. It is likely running as well as it can be given the single core performance of the Xeon chip in the production version. Nginx + PHP-FPM seems to be the best performance - I tested with alt-php and Litespeed and the performance seemed significantly worse. This is likely to do with the raw performance difference of Ngnix vs Litespeed when Litespeed Caching isn't compatible (which it isn't with nextcloud).

Additionally SSL packet throughput is significantly degraded performance wise when running (as it is normally) through a HAProxy instance. I will experiment with nginx for SSL termination instead, but may just resolve via 1:1 NAT on a different IP.

Big big thanks to @AOD_LTCaveman for the amount of time he spent with me going over the separate parts and where bottlenecking could be. Massive massive help!