Problems with number of files in a folder

Problems with number of files in a folder

Problems seems to exist.

Rule of thumb is to aim for a directory size of <= 20k files, although I’ve seen relatively decent performance with up to 100k files/directory.

Handling large number of files in a directory on EXT3

One, restructure the directory so that it does not contain that many files. I did some tests, and in a default (untuned) Ext3 partition, each subsequent write degrades horribly past the 2000 file limit. So, keeping the items in a directory to within 2000 files should be fine.

linux - Best options to create and mount an ext4 volume containing > 10M files in one directory - Server Fault

I checked my filesystem with:

tune2fs -l /dev/sda3 | grep features

and discovered that I have dir_index enabled, so no need to do anything there.

I also checked if my directory is indexed with:

lsattr -d /var/www/

and noticed the big letter I in a result (----------I--e--), so everything is ok. Should be no problem.

Folders that grow beyond a single filesystem block (normally 4k) will be indexed once dir_index is enabled.

Tune your ext3 filesystem | linuxnlenux Linux - Performance

Should I set noatime?

No - not really, anymore. Today, relatime is part of the default mount options, unless overridden. It gives the same speed benefits as noatime.

Still, the the number of writes to a disk for relatime mount is close to double relative to a noatime. See more here in thsi resources:

Is it worth to tune Ext4 with noatime? Gain 30% Linux Disk Performance with noatime, nodiratime, and relatime

Change I/O scheduler?

To see what I/O scheduler is currently in use:

cat /sys/block/sda/queue/scheduler

No need, as mine is by default deadline

SSD benchmark of I/O schedulers

SSD benchmark of I/O schedulers

date 01. Jan 0001 | modified 13. May 2024
filename: Server - Speed - Disk