Devbox Performance Tuning

I had a big performance challenges when making Devbox

Sergei FreeBSD machine

Credentials:

dgvip / datingvip
root / datingvip

Mounting shared drives:

mount_smbfs -I 192.168.0.70 //dell-inspiron/sergei /srv

VirtualBox settings

Chipset ICH9 - this is important!

Podesavanje virtualne masine na linux: Solution for Slow Ubuntu in VirtualBox

This can speed up sometimes Increased Performance In Linux With zRam (Virtual Swap Compressed in RAM)

sudo apt-get install zram-config

Mozda smanjiti CPU na 1 kom.

Windows and CIFS problems on Linux

On Linux side

Timings:

CIFS sa problemima write (1.0 GB) copied, 1043.06 s, 1.0 MB/s read (1.0 GB) copied, 1051.41 s, 997 kB/s
CIFS satro sredjen, sada: citanje: 20.5 MB/s pisanje oko 35 MB/s
All the time: smbread - 47MB!

CHECK parameters in work with:

modinfo cifs

All the parameters: https://www.kernel.org/doc/readme/fs-cifs-README

-o guest,noperm,nocase,noserverino,rw,exec,fsc,nostrictsync,file_mode=0777,dir_mode=0777,rsize=32768,wsize=32768,vers=2.0
-o guest,noperm,nocase,serverino,rw,exec,file_mode=0777,dir_mode=0777,rsize=32768,wsize=32768,vers=2.0

Intereting parameters:

+ nostrictsync, this should be ok
- directio. Not working.
- cache=0. Never ever use. Extremly slow.
+ vers=2.0 Definitly working.
  Check on host with: powershell -C "Get-SmbSession | Select Dialect,ClientComputerName,ClientUserName"

Maybe:

Bug #471512 “Slow cifs ubuntu 9.10” : Bugs : “linux” package : Ubuntu

Using smbclient to mount the remote share and copy the files produces normal speeds. An issue with CIFS in kernel, specifically.

You have created the file /etc/modprobe.d/cifs.conf with the inside option: “options cifs CIFSMaxBufSize=130048”?
You have put in /etc/fstab the following two options in the mount points: rsize=32768, wsize=32768?

rsize=arg default network read size (usually 16K). The client currently can not use rsize larger than CIFSMaxBufSize. CIFSMaxBufSize defaults to 16K and may be changed (from 8K to the maximum kmalloc size allowed by your kernel) at module install time for cifs.ko. Setting CIFSMaxBufSize to a very large value will cause cifs to use more memory and may reduce performance in some cases. To use rsize greater than 127K (the original cifs protocol maximum) also requires that the server support a new Unix Capability flag (for very large read) which some newer servers (e.g. Samba 3.0.26 or later) do. rsize can be set from a minimum of 2048 to a maximum of 130048 (127K or CIFSMaxBufSize, whichever is smaller)

wsize=arg default network write size (default 57344) maximum wsize currently allowed by CIFS is 57344 (fourteen 4096 byte pages)

Optimize network for CIFS: smb - TCP offloading/NetDMA & magic commands - Server Fault Use hidden Windows tweaks to speed up your Internet and network connections

Querying information

Check active parameters in active mounts with:

cat /proc/mounts

Get kernel module parameters (possible parameters)

modinfo cifs

And modify something in kernel driver:

ls /proc/fs/cifs/
cat /proc/fs/cifs/LinuxExtensionsEnabled

Just info: Bug #1113395 “mount.cifs on 13.04 fails to mount a samba share w…” : Bugs : “cifs-utils” package : Ubuntu

CIFS is a kernel part, smbclient was the part of Samba package.

For SMBclient, there is a parameter in /etc/samba/smb.conf under [Global], like strict allocate = yes. But, the whole file is not processed with CIFS.

Comparing smbclient with cifs, it’s definitely CIFS problem: Bug #810606 “Slow cifs in 11.04”

Final solution

Define that nothing is needed, from security standpoint, and specially NOT packet signing. This is done by typing:

echo 0x30030 >/proc/fs/cifs/SecurityFlags

or better yet:

sudo sh -c "echo 0x30030 >/proc/fs/cifs/SecurityFlags"
# Default is 0x81

It is nice to increase value of:

rmmod cifs
modprobe cifs CIFSMaxBufSize=130048

or make it permanent with:

echo "options cifs CIFSMaxBufSize=130048" >/etc/modprobe.d/cifs.conf

On Windows side

Set two registry settings: LargeSystemCache i onaj drugi

Show settings:

powershell -NoProfile -ExecutionPolicy unrestricted -Command "Get-SmbServerConfiguration"

but everything looks normal.

Turn off SMB1, to try: Admin Share » Overview of SMB Multichannel How to enable and disable SMBv1, SMBv2, and SMBv3 in Windows Vista, Windows Server 2008, Windows 7, Windows Server 2008 R2, Windows 8, and Windows Server 2012 Nothing.

Optimizing windows

powershell
Set-SmbServerConfiguration –RejectUnencryptedAccess $false


Find out with wich version we have clients connected:
Get-SmbSession | Select Dialect,ClientComputerName,ClientUserName

Still, linux is connection with 1.01 dialect.

Maybe: TreatHostAsStableStorage

Sources: http://blogs.technet.com/b/filecab/archive/2012/05/03/smb-3-security-enhancements-in-windows-server-2012.aspx https://wiki.archlinux.org/index.php/Samba/Troubleshooting

What is this:

Query the values:

reg query "HKLM\System\CurrentControlSet\Services\LanManServer\Parameters" /v RequireSecuritySignature
reg query "HKLM\System\CurrentControlSet\Services\LanManServer\Parameters" /v EnableSecuritySignature

Set the new values:

reg add "HKLM\System\CurrentControlSet\Services\LanManServer\Parameters" /f /v RequireSecuritySignature /t REG_DWORD /d 0
reg add "HKLM\System\CurrentControlSet\Services\LanManServer\Parameters" /f /v EnableSecuritySignature  /t REG_DWORD /d 0

You must reboot after that.

Source: The Basics of SMB Signing covering both SMB1 and SMB2

reg add "HKLM\System\CurrentControlSet\Services\LanmanServer\Parameters" /v "MaxWorkItems" /t REG_DWORD /d 512 /f
reg add "HKLM\System\CurrentControlSet\Services\LanmanServer\Parameters" /v "MaxMpxCt" /t REG_DWORD /d 2048 /f
reg add "HKLM\System\CurrentControlSet\Services\LanmanServer\Parameters" /v "MaxFreeConnections" /t REG_DWORD /d 100 /f
reg add "HKLM\System\CurrentControlSet\Services\LanmanServer\Parameters" /v "MinFreeConnections" /t REG_DWORD /d 32 /f
reg add "HKLM\SYSTEM\CurrentControlSet\Services\Lanmanserver\Parameters" /v "EnableOplocks" /t REG_DWORD /d 0 /f
reg add "HKLM\SYSTEM\CurrentControlSet\Services\lanmanserver\Parameters" /v "Size" /t REG_DWORD /d 3 /f

These two are working (LargeSystemCache & DisableLastAccess):

reg add "HKLM\SYSTEM\CurrentControlSet\Control\Session Manager\Memory Management" /f /v LargeSystemCache /t REG_DWORD /d 1
fsutil behavior query DisableLastAccess

Speed up unit tests

The one inside virtual machine.

my :root@local -e “show variables like ‘%innodb_flush_log_at_trx_commit%’;” must be 2 php - How can I speed up my PHPUnit + DBUnit test suite execution? - Stack Overflow
man mount | grep barrier centos - Should we mount with data=writeback and barrier=0 on ext3? - Server Fault maybe: data=writeback,barrier=0,commit=10
database in ramdisk mathieuk/mysql-ramdisk-helper · GitHub

Windows - 22 minuta na shared diru

windows - projekti su na CIFS shared diru, ali 1 procesor: 7.40 minuta podesio mysql Tests: 4531, Assertions: 20195, Failures: 31, Errors: 92, Skipped: 1.

Kada sam podesio O_DIRECT_NO_FSYNC, još se ubrzao na tačno oko 7 minuta. Kada sam stavio na O_DSYNC, trajalo je 10 minuta
linux - 31 minut linux na shared diru (i to kao NFS, kao najbrze) Tests: 4531, Assertions: 20563, Failures: 20, Errors: 18.
linux - Kada sam prebacio na neki direktorij unutra 28 minuta - ubrzalo se: ali malo. SUPER! nije disk. Tests: 4531, Assertions: 20566, Failures: 20, Errors: 18.
linux - na normalnoj shared lokaciji prebaceno NA 1 CPU Spustio se na 8 minuta. Tests: 4531, Assertions: 20564, Failures: 19, Errors: 18. Ostao mi onaj fstab sa optimizacijama - iako mislim da stvarno nema veze…

Za to vreme, FreeBSD: linux: 8.35 minuta Tests: 4523, Assertions: 20662, Failures: 1. windows: x

Paralel phpunit

Brian Scaturro - Parallel PHPUnit - Running Tests Concurrently With ParaTest brianium/paratest · GitHub verkkokauppacom/parallel-phpunit · GitHub A Brief History-In-Progress of Running Tests in Parallel with PHPUnit | Selenium Testing? Do Cross Browser Testing with Sauce Labs

Speed up PHPUnit

Tweak APC

echo " ; enabling apc gives ‘Segmentation fault’ ; due the Bug #63005 :: Segfault on PHP 5.4.6+ and APC 3.1.13 ; Bug is fixed and will be available in later releases. ; ; apc.enable_cli=1 ; apc.ttl=7200 apc.user_ttl=7200 " > /etc/php5/conf.d/20-phpunit_apc.ini

Optimize disk access

Main system disk

Add noatime option to /etc/fstab or just execute: mount -o remount,noatime /

Optimize MySQL specially for unit tests

From sergei’s settings

Very much: my :root@local -e “SHOW GLOBAL STATUS LIKE ‘%Created_tmp%’;” on tests…

Looks like important thing in sergei’s my.cnf is: tmp_table_size = 1G

tmp_table_size = 1G # The maximum size of internal in-memory temporary tables max_heap_table_size = 1G

which was 100MB by default

and: innodb_flush_log_at_trx_commit = 0

Moving TMP tables in shared memory

only tmpdir = /dev/shm should be ok

Moving MySQL databases to ramdisk in Ubuntu linux - Tomislav Šantek What Is /dev/shm And Its Practical Usage Experiment: MySQL tmpdir on tmpfs | Open Query blog

Probably would have to:

In unit tests, specify Unix socket: https://github.com/sebastianbergmann/dbunit/blob/master/phpunit.xml.dist

I can set: new PDO PHP: PDO::__construct - Manual

Syncdump performance

Timings for syncdump.sh

datingvip_a6 - 2000 sec (around 30 minutes)
datingvip_m5 - 1904 sec (around 30 minutes)

Optimize web server

Apache is the fastest? ChiperSoft :: PHP on Nginx vs Apache.

Perfect manual: Install PHP-FPM on Debian — Biapy Help Desk

Apache and Nginx bugs and fixed on VirtualBox (sendfile):

Vagrant and VirtualBox, modified files do not update when viewed in either NGINX or Apache | Dan Hart VirtualBox, Shared Folders and Cache | Onur Güzel shared folder problem · Issue #351 · mitchellh/vagrant

How to speed up lighttpd?

Apache mod_php

service lighttpd stop sudo apt-get install apache2 sudo a2enmod ssl mkdir -p /etc/httpd/conf

tamo iskopiraj nase konfiguracije i iskopiraj sertifikate

service apache2 restart

Ovo je bolja varijanta:

ln -s /srv/projects/dating /srv/http/dgvip sudo apt-get install apache2-mpm-worker apt-get install libapache2-mod-fastcgi a2enmod actions fastcgi alias ssl expires redirects rewrite mkdir -p /var/www cp phpinfo.php /var/www/

mkdir -p /etc/httpd/conf cp /srv/projects/dating/resources/vhosts/ssl/* /etc/httpd/conf/ cp /srv/projects/dating/resources/vhosts/* /etc/apache2/sites-enabled/

promena fpm porta echo ' AddHandler php5-fcgi .php Action php5-fcgi /php5-fcgi Alias /php5-fcgi /usr/lib/cgi-bin/php5-fcgi FastCgiExternalServer /usr/lib/cgi-bin/php5-fcgi -host 127.0.0.1:8833 -pass-header Authorization ’ >/etc/apache2/conf.d/php5-fpm.conf

apache2ctl -M apache2 -l nano /etc/apache2/sites-available/default

JAO: NE:

apt-get install libapache2-mod-php5 rm /etc/php5/apache2/conf.d/20-runkit.ini rm /etc/php5/apache2/conf.d/20-xdebug.ini rm /etc/php5/apache2/conf.d/20-phpunit_test_helpers.ini service apache2 restart

Very important!!!

Fixing High I/O usage on Amazon EBS | Lantean Quick Setup: Apache Worker, FastCGI, PHP-FPM and APC | Failover Web Development Blog

apc.mmap_file_mask=/apc.shm.XXXXXX apc.shm_size=96M

if [ -e ‘/etc/php5/conf.d/apc.ini’ -o -e ‘/etc/php5/conf.d/20-apc.ini’ ]; then echo ‘; Enable APC ressources sharing. apc.mmap_file_mask=/apc.shm.XXXXXX apc.shm_size=96’
> ‘/etc/php5/conf.d/apc-fpm.ini’ fi

OVO jE najbitnije:

echo ' ; Enable APC resource sharing apc.mmap_file_mask=/apc.shm.XXXXXX apc.shm_size=96M ’ » /etc/php5/mods-available/apc.ini

!!!! JEBEM TI MATER !!!!

lighttpd

server.network-backend = “gthread-aio” server.use-noatime = “enable”

WOW: Server network-backendDetails - Lighttpd - lighty labs raw IO performance - Lighttpd - fly light Lighttpd (lighty)Web服务器性能优化 optimizing lighttpd for file hosting | HostOnNet.com

To check possible values: lighttpd -V

very important one:

server.max-worker = 16

Docs MultiProcessor - Lighttpd - lighty labs Docs Performance - Lighttpd - lighty labs

Kao najbrze, ali potencijalni problem sa sendfile:

echo ' server.event-handler = “linux-sysepoll” server.network-backend = “linux-sendfile” server.use-noatime = “enable” server.max-worker = 16 server.stat-cache-engine = “fam” ’ >/etc/lighttpd/conf-available/20-devbox.conf

lighttpd-enable-mod devbox odnosno: ln -s /etc/lighttpd/conf-available/20-devbox.conf /etc/lighttpd/conf-enabled/20-devbox.conf

service lighttpd force-reload

to create symlink

Bezbedno: server.event-handler = “linux-sysepoll” server.network-backend = “writev” server.use-noatime = “enable” server.max-worker = 16

Sa opcijom fam, nekako mi ipak sporije server.stat-cache-engine = “fam”

Inace, da bi radila mora Gamin a ne FAM. sudo apt-get purge fam sudo apt-get install gamin

FAM was the original, and a lighter-weight workalike called Gamin.

Po meni najbolji:

server.max-worker = 8

Maximum number of file descriptors, default = 1024

ovo ne moze, osim ako se ne pokrece kao root: server.max-fds = 2048

Maximum number of request within a keep-alive session before the server terminates the connection, default = 16

server.max-keep-alive-requests = 16

Maximum number of seconds until an idling keep-alive connection is dropped, default = 5

server.max-keep-alive-idle = 4

Maximum number of seconds until a waiting, non keep-alive read times out and closes the connection, default = 60

server.max-read-idle = 60

Maximum number of seconds until a waiting write call times out and closes the connection, default = 360

server.max-write-idle = 360

Which event handler to use, default = poll

server.event-handler = “linux-sysepoll”

How to handle network writes, default = writev

server.network-backend = “linux-sendfile”

Requires FAM or Gamin to be installed, default = simple

server.stat-cache-engine = “fam”

Whether to update the atime setting on file access, default = disable

server.use-noatime = “enable”

server.max-connections = 1024

single client connection bandwidth limit in kilobytes (0=unlimited)

connection.kbytes-per-second = 0

global server bandwidth limit in kilobytes (0=unlimited)

server.kbytes-per-second = 0

Tuning Lighttpd For Linux

/etc/lighttpd/lighttpd.conf

lighttpd load testing and optimization - Stack Overflow

sysctl -w

Some settings on server:

Web server performance benchmark | Root Users

APC was the reason!

Fuck you, fuck you, fuck you:

https://code.google.com/p/hppg/wiki/APCConfiguration

Test:

Lighty ab -n 100 -c 10 http://10.0.3.10/phpinfo.php - 40 ab -n 100 -c 10 http://m1.datingvip.dev/ - 3

Apache ab -n 100 -c 10 http://10.0.3.10/phpinfo.php - 240 ab -n 100 -c 10 http://m1.datingvip.dev/ - takodje 2.95

Exactly like this: APCConfiguration - hppg - Explains how to setup APC as cache engine - HPPG - high performance photo gallery - Google Project Hosting

And APC like this:

Read from: /etc/lighttpd/conf-available/15-fastcgi-php.conf

“max-procs” => 1 (vrlo vazno - ne dele kes…) “PHP_FCGI_CHILDREN” => “4”, “PHP_FCGI_MAX_REQUESTS” => “10000”

If you are using an opcode cache such as eAccelerator, XCache or APC it’s advisable to keep max-procs at a very low number (1 is perfectly fine) and raise PHP_FCGI_CHILDREN instead. Those opcode caches will create a separate memory space for each parent process, otherwise. If you leave max-procs at 4, you’ll end up with four separate opcode memory cache segments.

date 01. Jan 0001 | modified 10. Jun 2024

filename: Devbox Performance Tuning