this post was submitted on 08 Apr 2025
78 points (97.6% liked)

Selfhosted

45728 readers
924 users here now

A place to share alternatives to popular online services that can be self-hosted without giving up privacy or locking you into a service you don't control.

Rules:

  1. Be civil: we're here to support and learn from one another. Insults won't be tolerated. Flame wars are frowned upon.

  2. No spam posting.

  3. Posts have to be centered around self-hosting. There are other communities for discussing hardware or home computing. If it's not obvious why your post topic revolves around selfhosting, please include details to make it clear.

  4. Don't duplicate the full text of your blog or github here. Just post the link for folks to click.

  5. Submission headline should match the article title (don’t cherry-pick information from the title to fit your agenda).

  6. No trolling.

Resources:

Any issues on the community? Report it using the report flag.

Questions? DM the mods!

founded 2 years ago
MODERATORS
all 19 comments
sorted by: hot top controversial new old
[–] [email protected] 10 points 4 days ago (1 children)

you most likely need a backup of your /bin/,/lib/, /usr/ and /etc/ folders, because they contain files, essential for your system operation.

I guess we disagree about the point of backups then.

For me, backups are about data, as in, if I switch operating systems or something, it's what I'll need to get set up. /bin and /lib will just cause more problems than they solve since they likely won't be compatible w/ whatever system libraries are there. /usr is only relevant for the config files, and 90% of those are at whatever the distribution's defaults are (and on Arch, everything is in /usr anyway). /etc is useful for configs, but like /usr/etc, 90% of it is just defaults.

If I'm restoring from backup, I don't want to have random breakage, like config files, libraries, and binaries being incompatible. Most of the stuff outside /home is irrelevant, and the parts I do care about can be manually backed up through improving processes.

For example, I stick my self-hosted stuff into a git repo, and any changes I make get committed. I have scripts that deploy the changes wherever they need to go (e.g. I use quadlet, so this means systemd files), and those can be tweaked if something changes in a different OS. 99% of the things I host are containerized, so there is zero system-level impact (i.e. something that can't easily live in git or /home). For the rest, I largely accept that I'll probably have to fix a few things when restoring after an emergency , such as any configs for system services (e.g. /etc/sub(uid|gid), podman tweaks, users and groups, etc). I try to keep these changes extremely limited, and I keep notes in the README of my git repo, but it's never going to be 100% accurate since I often just want to get whatever it is working and promise that I'll update my docs later.

That said, the most important thing you can do, IMO, is test restoring from backup. I'm moving my systems toward containerization, which gives me a chance to fix all the stuff I totally forgot about. I keep the old system around for a while to reference (on an old drive or something), pulling files (and properly backing them up) as I go, and then I'll nuke it some months later. The important data is all in /home, which gets backed up, so it's mostly about service configuration (which I can look up later).

snapper

Snapper absolutely rocks. I'm spoiled on the openSUSE line of distributions that this is baked in. When an upgrade does sideways on my Tumbleweed desktop, I simply roll back and retry the upgrade in a couple days. It's magical.

snapborg

Looks really cool! I won't be backing up most of /, but I'll instead be a bit selective about which subvolumes I use. I already use btrfs everywhere, so this should be relatively drop-in.

[–] [email protected] 3 points 3 days ago (1 children)

I guess we disagree about the point of backups then.

We just use different threat models.)

For me, the main threat is disk failure, so I want to get new disk, restore system from backup and continue as if nothing happened.

Surely, if your hardware or OS configuration changes, you should not backup /usr, /etc and other folders.

However, the proposed workflow could be adapted to both scenarios: a single snapborg config backs up snapshots from a single subvolume, so I, actually, use two configs: one for /home excluding /home/.home_unbacked and another one for / excluding /var and some other directories. This two configs have different backup schedule and different retention policies, so in case of hardware/OS change, I'll just restore only /home backup without restoring /.

[–] [email protected] 2 points 3 days ago

Makes sense.

I'm more interested in cutting off-site backup costs, so my NAS has RAID mirror to reduce chance of total failure, and offsite backup only stores important data. I don't even backup the bulk of it (ripped movies and whatnot), just the important data.

Restore from backup looks like this for my NAS:

  1. Install OS + podman
  2. Restore data from backup
  3. Build and deploy containers from config

Personal devices are similar, but installing packages is manual (perhaps I'll backup my explicitly stored package list or something to speed it up a little). Setup takes longer than your method, but I think it's worth the reduced storage costs since I've never actually needed to do it and a few hours of downtime is totally fine for me.

[–] [email protected] 8 points 4 days ago (1 children)
[–] [email protected] 1 points 3 days ago
[–] uninvitedguest 3 points 4 days ago (1 children)

btrbk handles to automated generation of snapshots, as well as sending/receiving to another drive for backup. What this workflow accomplish that btrbk doesn't do on its own? Compression?

[–] [email protected] 1 points 3 days ago

btrbk requires, that destination disk is also formatted as btrfs.

I didn't want to have such constrain.

[–] [email protected] 5 points 4 days ago (3 children)

I don't really understand the advantage of backing up the whole btrfs volume.

[–] [email protected] 2 points 3 days ago

Snapshots are made atomically, so this workflow allows you to separate snapshot creation from actual backing up.

As subvolumes are dynamically sized, you can create as many subvols as you like and backup those, that need it.

[–] [email protected] 4 points 4 days ago (1 children)

It's most useful for something like /home so you have a full backup of your home directories without having to worry if some app has dumped its settings on a folder you don't have marked for backup. But also backing up / is useful because it gives you an easy recovery in the event of say disk failure. Just restore the entire system from a backup.

[–] [email protected] 2 points 4 days ago* (last edited 4 days ago) (1 children)

I use borg with borgmatic. I just back up / (which includes home) and exclude some folders I don't want (like /mnt or /tmp).

It does the same as you just said.

I have 20 borg snapshots of my nearly full 1tb drive which takes about 400gb of space on my NAS.

I do it at the file structure level, not at the block device level as the article suggests. Why would I want to back it up at the block device level instead?

[–] [email protected] 4 points 4 days ago* (last edited 4 days ago) (1 children)

You can have btrfs subvolumes that contain only the stuff you care about. In the article, they have a subvolume for untracked files, and this doesn't get backed up.

I have one for /home and /srv (my self-hosted stuff), and those are the ones I care about backing up. For everything else, I keep manual copies in either /home or /srv (e.g. edits to system configs and container configs are in a git repo). So I could back up those two subvolumes and get 99% of what I care about, and hope I've done a good job documenting the other 1%, which are reproducible w/ a bit of time searching how to configure stuff.

This keeps backups efficient since you can ask the filesystem to only send the changes instead of going over all the files and figuring out what has changed, so it's probably a lot faster.

[–] [email protected] 1 points 4 days ago (1 children)

This keeps backups efficient since you can ask the filesystem to only send the changes instead of going over all the files and figuring out what has changed, so it’s probably a lot faster.

Aaaah!

[–] [email protected] 3 points 4 days ago

?? Maybe you don't have enough data for it to matter, but when you have tons of files, manually looking through the filesystem can take a while, whereas the FS can keep track of what's actually changed.

[–] Lem453 3 points 4 days ago (1 children)

With snapshots it costs nothing if the OS files don't change much

[–] [email protected] 2 points 4 days ago (3 children)

You misunderstood my question, because what you said is true either way with borg.

The question is, what is the advantage of backing up the whole subvolume "block device" vs just / file structure.

[–] [email protected] 3 points 3 days ago

Backing up via snapborg allows you to see file structure, because actually it is a file-based backup. snapper here allows me to separate snapshot creation from actual backups.

[–] Lem453 3 points 4 days ago

Oh i see.

that is a good point. Being about the mount the borg repo and pull out a specific file is very useful. Not sure I would do it as a bloc device since I don't think there is an easy way to see specific files.