Linux

Been enjoying Linux for ~25 years, but have never been happy with how it handled low memory situations. Swapping have always killed the system, tho it have improved a little. It's been a while since I've messed with it. I've buckled up and are using more ram now, but afair, you can play with:

(0. reduce running software, and optimize them for less memory, yada yada)

1. use a better OOM (out of memory) manager that activates sooner and more gracefully. Search in your OS' repository for it.
2. use zram as a more intelligent buffer and to remove same (zero) pages. It can lightly compress lesser used memory pages and use a partition backend for storing uncompressible pages. You spend a little cpu, to minimize swap, and when needed, only swap out what can't be compressed.
3. play with all the sysctl vm settings like swappiness and such, but be aware that there's SO much misinformation out there, so seek the official kernel docs. For instance, you can adapt the system to swap more often, but in much smaller chunks, so you avoid spending 5 minutes to hours regaining control - the system may get 'sluggish', but you have control.
4. use cgroups to divide you resources, so firefox/chrome (or compilers/memory-hogs) can only use X amount before their memory have to swap out (if they don't adapt to lower mem conditions automatically). That leaves a system for you that can still react to your input (while ff/chrome would freeze). Not perfect, tho.
5. when gaming, activate a low-system mode, where unnecessary services etc are disabled. I think there's a library/command that helps with that (and raise priority etc), but forgot its name.

EDIT: 6. when NOT gaming, add some of your vram as swap space. Its much faster than your ssd. Search github or your repository for 'vram cache' or something like that. It works via opencl, so everyone with dedicated vram can use it as super fast cache. Perhaps others can remember the name/link ?

Something like that anyway, others will know more about each point.

Also, perhaps ask an AI to create a small interface for you to fiddle with vm settings and cgroups in an automated/permanent way ? just a quick thought. Good luck.

What is your swappiness? If you have all that ram you should not be swapping. You are probably still at a swappiness for an average system, 60. Try 10. sudo sysctl vm.swappiness=10

Rather than guessing at whether it's swapping, why not check your swap usage? Running free -h in a shell will give you a brief overview of memory usage. Various GUI system monitors will graph it.

You can also find out what process is hitting your storage so hard with a tool like iotop.

https://wiki.archlinux.org/title/Zram

In the sysstat package, there's a command, iostat, that shows the amount of time that the average CPU (core) is waiting for something.

avg-cpu:  %user   %nice %system %iowait  %steal   %idle
           0.57    0.01    0.55    3.14    0.00   95.73

"User" is the CPU time used by regular processes. "Nice" is the time used by "niced" processes, stuff running at low priority. "System" is time used by the kernel. "Iowait" is the time waiting for I/O operations to complete.

If you're bounded on I/O, then the iowait number is probably going to be relatively high.

It'll also show you I/O load on each device (and the sysstat package can log this over time, let you use sar to see historical numbers).

You can artificially induce load on a given I/O device to see whether load on it induces your problem. Something like this:

$ dd if=/dev/zero of=filename-that-you-want bs=1M

That'll start your system just writing zeroes to a file. If it's I/O contention on the mSATA drive, then I'd expect you to see the problems you mentioned.

One thing you might check is whether your system is logging errors during this time. I don't know what a failing SSD looks like, but in my experience a failing rotational drive would often see errors and then attempts to re-perform I/O operations. That produces seriously-degraded disk I/O performance. These errors will show up in the kernel log. To see kernel errors since the current boot:

$ journalctl -kb

The smartmontools package contains a command, smartctl, which will let you see what your drive thinks of its own health. I'm not really familiar with what a failing SSD would look like, but with rotational drives, it can indicate that something's wrong with the drive, and it looks like my SSDs do report SMART status.

# smartctl -a /dev/sda

Moving swap

Linux can, if it needs to, use a swap file instead of a swap partition, so you don't need to repartition your NVMe if you just want to try putting your swap on the NVMe.

If you want to, first, for the current boot, disable all the active swap partitions.

# swapoff -a

If you run top, you'll see the available swap be at zero.

Then you want to create the swap file. Say we drop it in /var:

# mkswap --file /var/swapfile --size 32GiB

And then you can enable it:

# swapon /var/swapfile

Then see how things perform.

If it makes your problem go away and you want to make the change persistent:

In /etc/fstab, you'll have a line for your swap partition that the system reads at each boot. Will probably look something like this:

/dev/mapper/system--vg-swap_1 none            swap    sw              0       0

Comment that out by sticking a "#" at the beginning of your line, and put a line in for your new swapfile on the NVMe:

/var/swapfile none            swap    sw              0       0

Probably not it, but what DE/WM are you using? I had a very similar issue on KDE, but it would go away if I switched DEs.

Have any metrics on your heat? Dmesg logs? Power? Minutes of freezing certainly seems like a heat problem, but could also be a device going offline until power is restored.

You shouldn't really use swap on an SSD in the first place. It reduces the drive's lifespan.