Making old machines run a modern Linux distribution surprisingly well (using Zswap)

In between study sessions, I've been experimenting with some old hardware that I want to get rid of on eBay. I happen to have managed to get an old IBM A31 with 384 MB of RAM up and browsing the Web while using a modern operating system, namely: Manjaro Linux 0.8.13 (LXDE), kernel version 4.2.

The challenges here are plenty: low RAM, extremely slow and old rotational hard drive (20 GB), and the ever-increasing presence of complex multimedia and Javascript on the modern Web.

The key to my success has been the use of Zswap. In a nutshell, Zswap compresses pages that would be normally swapped out of RAM (and onto disk) into a separate storage area (in RAM) that is first compressed before being queued for swapping. This effectively increases the RAM of the computer in exchange for CPU time. With a system that is heavily I/O constrained, this is a very reasonable trade-off to make.

In order to make the default installation of Manjaro XFCE functional under such constraints:

1. Edit /etc/default/grub and configure zswap to use the more space-efficient deflate compressor:
1. Append "zswap.enabled=1 zswap.compressor=deflate zswap.max_pool_percent=90" to GRUB_CMDLINE_LINUX_DEFAULT
   
2. update-grub
   
2. Add the deflate module to the initial ramdisk by editing /etc/mkinitcpio.conf:
1. Append "deflate" to MODULES
   
2. Look inside of /etc/mkinitcpio.d/ to determine what profile to specify for rebuilding the ramdisk. If you are using the Linux 4.2 kernel (recommended):
   mkinitcpio -p linux42
3. Uninstall the resource-hungry pulseaudio subsystem:
1. pacman -R pulseaudio pulseaudio-ctl pulseaudio-alsa
4. Install and use QupZilla as the default web browser
1. pacman -Sy qupzilla
   
5. For even better performance, change QupZilla's User-Agent string to an Android Webkit-based mobile device:
1. Edit -> Preferences -> Other -> Change browser identification / User Agent Manager -> Change global User Agent
   
   I suggest:
   Mozilla/5.0 (Linux; U; Android 4.0.3; en-us; HTC Sensation Build/IML74K) AppleWebKit/538.1 (KHTML, like Gecko) Version/4.0 Mobile Safari/538.1

Remarkably, these changes allow a full Web experience. Even Youtube videos seem to function.

On such a machine -- and in order to reduce slow disk I/O -- it might be advantageous to use a btrfs root filesystem mounted with gzip compression.

ZFS compression results in workload starvation, partially ameliorated by async_write_max_active

I have a particular dataset that consists of old backup files. Rather than storing them in .tar.gz archives, where the contents remain buried (and the files cannot be culled with useful tools such as fdupes), I elected to create a zfs dataset that uses gzip-9 compression.

However, on my system, I noticed that writing to the compression=gzip-9 zfs dataset resulted in starvation to other I/O processes  -- in particular, read processes -- rendering the system unusable.

In order to analyze the situation, I used this dtrace script to analyze different classes of I/O operations on my system.

See below for the sysctl tuneable that I modified.

vfs.zfs.vdev.async_write_max_active=10	vfs.zfs.vdev.async_write_max_active=3
Delete value ------------- Distribution ------------- count 256 | 0 512 |@@@ 2 1024 |@@@@ 3 2048 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ 24 4096 | 0 Flush value ------------- Distribution ------------- count 512 | 0 1024 |@@@@ 4 2048 |@ 1 4096 |@@@@ 4 8192 |@@@@ 4 16384 |@@@ 3 32768 |@@@@@@@@@@@@@@@ 14 65536 |@@@@@@@@ 7 131072 | 0 Write value ------------- Distribution ------------- count 16 | 0 32 | 96 64 |@ 531 128 |@@ 735 256 |@@ 1067 512 |@@@@@@@@@@@@@@@@@@@@@@@@@@ 11639 1024 |@@@ 1573 2048 |@@ 878 4096 |@@ 785 8192 |@ 612 16384 | 183 32768 | 25 65536 | 16 131072 | 13 262144 | 3 524288 | 0 Read value ------------- Distribution ------------- count 32 | 0 64 | 12 128 | 48 256 |@ 124 512 |@@@@@@@@@@ 2046 1024 |@@@@ 906 2048 |@@@@@@ 1244 4096 |@@@@@@@ 1523 8192 |@@@@@@ 1228 16384 |@@@@ 774 32768 |@ 173 65536 | 26 131072 | 22 262144 | 5 524288 | 0	Delete value ------------- Distribution ------------- count 256 | 0 512 |@@ 1 1024 |@@@@@@@@@@@@@@@@@@@@ 12 2048 |@@@@@@@@ 5 4096 |@@ 1 8192 |@@@@@@@@ 5 16384 | 0 Flush value ------------- Distribution ------------- count 512 | 0 1024 |@@@@@@@@@ 5 2048 |@@@@ 2 4096 |@@@@@ 3 8192 |@@@@@@@@@ 5 16384 | 0 32768 |@@@@@@@ 4 65536 |@@@@@ 3 131072 | 0 Write value ------------- Distribution ------------- count 16 | 0 32 |@ 468 64 |@@@@@@ 3741 128 |@@@@@@ 3627 256 |@@@@@ 3234 512 |@@@@@@@@@@@@@@@@@ 10520 1024 |@ 916 2048 |@ 608 4096 |@ 476 8192 |@ 476 16384 | 246 32768 | 80 65536 | 54 131072 | 11 262144 | 5 524288 | 0 Read value ------------- Distribution ------------- count 32 | 0 64 | 18 128 |@ 70 256 |@ 153 512 |@@@@@@@@ 890 1024 |@@@@ 445 2048 |@@@@@ 594 4096 |@@@@@@ 667 8192 |@@@@@@ 709 16384 |@@@@@ 609 32768 |@@@ 381 65536 |@ 130 131072 | 26 262144 | 10 524288 | 0
avg latency stddev iops throughput Write 1892us 8057us 302/s 33416k/s Delete 2303us 732us 0/s 34k/s Read 7747us 15904us 135/s 17011k/s Flush 42742us 37072us 0/s 0k/s	avg latency stddev iops throughput Write 1521us 8316us 407/s 30107k/s Delete 3557us 3001us 0/s 41k/s Read 14255us 27405us 78/s 8844k/s Flush 25081us 32016us 0/s 0k/s

In particular, it seems that the average latency of I/O Flush dropped considerably.

A more exhaustive study under more carefully controlled conditions seems like a reasonable next step.