How to enable persistent journal storage in systemd

Persistent journal storage keeps systemd log history across reboots so boot failures, short-lived service crashes, and one-off kernel events remain available after the next startup instead of disappearing with the runtime journal.

The systemd-journald service writes logs either to the memory-backed /run/log/journal hierarchy or to the on-disk /var/log/journal hierarchy. The Storage= setting controls that behavior, and the default Storage=auto mode uses persistent storage only when /var/log/journal is available soon enough for journald to treat the host as persistent.

On a running host that already booted without /var/log/journal, creating the directory alone does not always move the active journal onto disk immediately. Writing an explicit Storage=persistent drop-in, applying the shipped directory permissions, and restarting systemd-journald gives a predictable result and increases local disk use until normal journal retention limits are tuned or old logs are vacuumed.

1. Inspect the merged journald storage setting before changing anything.

   $ sudo systemd-analyze cat-config systemd/journald.conf | grep --extended-regexp '^# /(etc|run|usr/lib)/systemd/|^[[:space:]]*(#)?Storage='
   # /etc/systemd/journald.conf
   #Storage=auto
   # /usr/lib/systemd/journald.conf.d/syslog.conf

   The last uncommented Storage= value wins. If a later drop-in already forces Storage=volatile or Storage=none, remove or replace that override before continuing.

2. Write a late-sorting journald drop-in that forces persistent storage on future starts.

   $ sudo install --directory --mode=0755 /etc/systemd/journald.conf.d
   $ printf '%s\n' '[Journal]' 'Storage=persistent' | sudo tee /etc/systemd/journald.conf.d/zz-persistent-storage.conf >/dev/null

   The drop-in keeps the distro default /etc/systemd/journald.conf file unchanged and removes the dependency on /var/log/journal existing during boot.

3. Create the persistent journal directory under /var/log.

   $ sudo mkdir --parents /var/log/journal

   systemd-journald stores the system journal on disk below this path when persistent storage is enabled.

4. Apply the packaged ownership and access modes for the journal directory.

   $ sudo systemd-tmpfiles --create --prefix /var/log/journal

   This sets the access mode that systemd-journald expects before it creates the machine-id subdirectory and journal files.

5. Restart systemd-journald so it reloads the new storage setting and opens the persistent journal tree.

   $ sudo systemctl restart systemd-journald

   A running host may not begin writing under /var/log/journal until systemd-journald is restarted or the system is rebooted.

   Related: How to manage a service using systemctl

6. Check that systemd-journald created an on-disk journal file below /var/log/journal.

   $ sudo find /var/log/journal -maxdepth 2 -type f
   /var/log/journal/0123456789abcdef0123456789abcdef/system.journal

   The 32-character directory name is the local machine ID, so it differs on every host.

7. Confirm that the persistent journal contains data.

   $ sudo journalctl --directory=/var/log/journal --disk-usage
   Archived and active journals take up 8.0M in the file system.

   A non-zero result from the /var/log/journal tree confirms that logs are now being retained on disk across reboots.

   Related: How to check systemd journal size
   Related: How to clear old journal logs