Recovery

RC-01 // RECOVERY

FreeBSD provides multiple independent recovery paths. Each works without the other. Together they make catastrophic system failure uncommon and recoverable.

Recovery Layers

Layer	What It Protects Against	Requires Running System
ZFS Boot Environment	Bad update, broken packages	No (bootloader menu)
/rescue subsystem	Broken shared libraries, missing /usr	No (single-user mode)
Base system separation	Corrupt /usr/local	Yes (pkg reinstall)
ZFS snapshot rollback	Accidental file deletion	Yes (zfs rollback)

/rescue Subsystem

FreeBSD ships approximately 150 statically linked binaries in /rescue. These binaries have no shared library dependencies — they work even if /usr/lib is missing or corrupted.

ls /rescue
[         chmod     df        echo      hostname  ln        mv        rm        stat
cat       chown     dmesg     expr      kill      ls        ps        sh        tar
...

When to Use It

If a package upgrade breaks a shared library (libc.so, libssl.so, etc.), dynamically linked binaries may fail to start. /rescue/sh still works:

/rescue/sh

# From rescue shell, you have: cat, ls, cp, mv, rm, tar, mount, etc.
# Fix the broken package:
/rescue/sh -c 'pkg install -f broken-package'

Accessing It

Boot to single-user mode (interrupt the bootloader, type boot -s) or just run /rescue/sh from a normal session.

ZFS Boot Environment Rollback

This is the fastest recovery path for a system that boots but is broken after an update.

# See available environments
boot-env list

# Roll back
boot-env rollback before-update-2026-03-20

# Reboot to complete
sudo reboot

If the system will not boot at all, select the previous boot environment from the bootloader menu. Press any key during the countdown, navigate to Boot Environments, and select the working one.

For full details, see Boot Environments.

Base System Separation

The FreeBSD base system lives in /bin, /sbin, /usr/bin, /usr/sbin, /usr/lib, and /usr/libexec. Packages install to /usr/local exclusively.

If the package tree is corrupted:

# Nuclear reinstall of all packages — base system untouched
sudo rm -rf /usr/local
sudo pkg bootstrap
sudo pkg install -y $(cat ~/.omfreebdy/packages.txt)

The kernel, libc, OpenSSL, and all base utilities remain intact. The system is immediately usable.

Step-by-Step Recovery Guide

Scenario 1: System boots, packages broken

# Reinstall a single broken package
sudo pkg install -f package-name

# Verify shared library links
ldd /usr/local/bin/some-binary

# Reinstall all packages (nuclear)
sudo pkg-static install -yf $(pkg query '%n-%v')

Scenario 2: System boots, display broken

# Drop to TTY
Ctrl+Alt+F2

# Or connect via SSH
ssh user@hostname

# Roll back to last working boot environment
boot-env list
boot-env rollback <name>
sudo reboot

Scenario 3: System will not boot

1. Restart and interrupt the bootloader (press any key)
2. Navigate to Boot Environments
3. Select a working environment
4. Boot into it
5. Once running, diagnose what broke

# After booting into recovery BE, check what changed
pkg version -v | grep '!'   # packages newer than repo
zfs diff zroot/ROOT/<old-be> zroot/ROOT/default

Scenario 4: /usr/lib damaged, binaries won’t run

# Boot to single-user mode
# At bootloader: type 'boot -s'

# Use rescue shell
/rescue/sh

# Mount filesystems
/rescue/mount -a

# Reinstall base system from network
# (or use freebsd-update)
freebsd-update fetch
freebsd-update install

Scenario 5: Accidental file deletion

# List ZFS snapshots for a dataset
zfs list -t snapshot zroot/usr/home/youruser

# Restore a file from snapshot
cp /zroot/usr/home/youruser/.zfs/snapshot/autosnap_2026-03-19/file.txt ~/file.txt

# Full rollback of home dataset (destructive — loses newer changes)
zfs rollback zroot/usr/home/youruser@autosnap_2026-03-19

ZFS dataset rollback is destructive — all changes after the snapshot are lost. Copy important files out before rolling back.

Preparing Before Experiments

Before making significant changes:

# Create a named boot environment
boot-env create before-experiment

# Snapshot home directory separately
sudo zfs snapshot zroot/usr/home/$(whoami)@before-experiment

# Verify
boot-env list
zfs list -t snapshot | grep before-experiment