No space left on device: find what is filling your Linux disk

You run a command, a deploy, a database write — and it fails with one of these:

write error: No space left on device
cp: error writing 'backup.tar.gz': No space left on device
ERROR:  could not extend file "base/16384/2619": No space left on device

The kernel is telling you it can’t write any more data to a filesystem. Usually the disk really is full, but not always — this guide covers both cases. Don’t start deleting at random: first find which filesystem is full, then what is filling it.

TL;DR

df -h                                                 # which filesystem is full?
df -i                                                 # ...or is it out of inodes?
sudo du -xh --max-depth=1 / 2>/dev/null | sort -h     # biggest directories
sudo find / -xdev -type f -size +100M -printf '%s %p\n' 2>/dev/null \
  | sort -nr | head -20 | numfmt --to=iec --field=1   # biggest files
sudo lsof +L1                                         # deleted files still holding space

1. Find the full filesystem

df -h

Filesystem      Size  Used Avail Use% Mounted on
/dev/sda1        50G   49G  1.1G  98% /
tmpfs           1.0G     0  1.0G   0% /dev/shm

Look at Use% and Mounted on. A server often has several filesystems (/, /var, /home, a data disk), and only one of them is full. Everything below targets that mount point. To check the filesystem behind a specific path:

df -h /var/lib/postgresql

2. Rule out inodes

Every file uses one inode, and a filesystem has a fixed number of them. Millions of tiny files — session files, mail queues, cache entries — can use up all inodes while df -h still shows free space. You get exactly the same “No space left on device” error.

df -i

If IUse% is at or near 100%, count files per directory to find the culprit:

sudo find /var -xdev -type f | cut -d/ -f2-3 | sort | uniq -c | sort -n | tail

3. Find the biggest directories

du adds up sizes per directory. -x stays on one filesystem (so it doesn’t count other mounts), and sort -h understands sizes like 4.5G:

sudo du -xh --max-depth=1 / 2>/dev/null | sort -h

Then drill down into the biggest one, level by level:

sudo du -xh --max-depth=1 /var 2>/dev/null | sort -h
sudo du -xh --max-depth=1 /var/log 2>/dev/null | sort -h

2>/dev/null hides “Permission denied” noise; sudo makes sure nothing is skipped.

4. Find the biggest files

Sometimes it’s one huge file — a runaway log, a forgotten backup, an ISO in someone’s home directory:

sudo find / -xdev -type f -size +100M -printf '%s %p\n' 2>/dev/null \
  | sort -nr | head -20 | numfmt --to=iec --field=1

This prints the size in bytes and the path, sorts from biggest to smallest and converts sizes to a readable form. A simpler variant, if you just want a list:

sudo find / -xdev -type f -size +500M -exec ls -lh {} + 2>/dev/null

For an interactive view, install ncdu (sudo apt install ncdu or sudo dnf install ncdu, from EPEL on RHEL) and run sudo ncdu -x /.

5. Free space safely

The usual suspects, and how to clean them without breaking anything:

6. “I deleted it, but df still says the disk is full”

On Linux, deleting a file only removes its name. If a process still has the file open, the data stays on disk until that process closes it — typically a log file deleted while the service keeps writing to it. Find such files with:

sudo lsof +L1

The output shows the process (COMMAND, PID) and the deleted file. Restart or reload that service — for example sudo systemctl restart myapp — and the space comes back. If you can’t restart it, you can empty the file through the process’s file descriptor: sudo truncate -s 0 /proc/<PID>/fd/<FD>.

7. “df shows free space, but I still can’t write”

Three things to check, in this order:

1. Inodes — df -i (step 2).
2. Reserved blocks — ext4 keeps 5% of the space for root by default, so regular users get “No space left on device” while root can still write. On a 1 TB data disk that’s 50 GB. See it with sudo tune2fs -l /dev/sdX1 | grep -i 'reserved block count'.
3. The wrong filesystem — the directory you write to may live on a different, smaller mount. df -h <path> tells you which one.

Prevent it next time

• Alert before it’s full: monitor disk usage and alert at 80–85%, not 100%.
• Rotate logs: make sure every application log is covered by logrotate, with compression and a limit on the number of files.
• Cap the journal: set SystemMaxUse=1G in the [Journal] section of /etc/systemd/journald.conf, then sudo systemctl restart systemd-journald.
• Clean up Docker regularly: unused images and build cache grow quietly.