A Linux host is slow and someone says it is out of memory. How do you check, and what do free and df actually tell you?

TL;DR: Read available in free -h, not used, because cache counts as used and is reclaimable. For disk, df reports the filesystem's accounting and du walks directories, so they disagree when a deleted file is still held open. Check inodes with df -i as well as blocks.

How to approach it

Correct the premise first, politely: high memory usage is normal and expected on Linux. Then give the commands and say which number in each is the one that matters, since the value of this question is entirely in knowing which column people misread.

A strong answer

Memory.

free -h
#               total    used    free   shared  buff/cache   available
# Mem:           31Gi    9.1Gi   1.2Gi   0.4Gi       21Gi        21Gi

free at 1.2Gi looks alarming and is not. The kernel uses otherwise idle memory for page cache, keeping recently read file data in RAM, and hands it back the moment a process needs it. available is the number that matters: memory that could be given to a new process, including reclaimable cache. 21Gi available on this box means it is fine.

The signal that memory is genuinely a problem is swap activity, not low free memory:

vmstat 1 5
# si/so columns: pages swapped in and out per second

Sustained non-zero si and so means the machine is actively paging and everything on it is slow. Combined with high wa (IO wait) and high system time, that is thrashing.

To find the consumer:

ps aux --sort=-%mem | head -10

And for a process killed without explanation, the OOM killer records it:

dmesg -T | grep -i 'killed process'
journalctl -k | grep -i oom

Worth knowing: the OOM killer chooses by a score that weights how much memory a process uses, so it frequently kills the largest process rather than the one that caused the pressure, which is how a monitoring agent gets killed instead of the leaking worker.

Disk.

df -h          # space by filesystem
df -i          # inodes by filesystem
du -sh /var/*  # what is actually in a directory

df asks the filesystem how many blocks are allocated. du walks the directory tree and adds up what it finds. They disagree in two situations, both common.

A deleted file with an open handle. Unlinking removes the directory entry, so du no longer sees it, but the inode and its blocks survive until the last file descriptor closes. A 40GB log deleted while the service still has it open is charged by df and invisible to du:

lsof +L1        # open files with no directory entry
: > /proc/2412/fd/3   # truncate through the descriptor to reclaim it

And a filesystem mounted over a directory that already contained data: du on the mount point sees the mounted filesystem, while the hidden files underneath still occupy the parent.

The other trap is inodes. Millions of tiny files can exhaust the inode table while blocks remain free, and every write then fails with "no space left on device" while df -h shows plenty. Always check df -i before concluding the disk is full.

What interviewers probe next

"Should you drop caches to free memory?" Almost never. echo 3 > /proc/sys/vm/drop_caches throws away useful cache and the kernel would have reclaimed it on demand anyway. It is a benchmarking tool, not a fix.

"What is the difference between RSS and virtual size?" Virtual is address space reserved, which can be huge and mostly untouched. RSS is physical memory actually resident. Alarm at a large VSZ is usually misplaced.

"How do you find what is writing to a full disk right now?" iotop, or du -xh / --max-depth=2 | sort -h to narrow by directory, staying on one filesystem with -x so it does not wander into mounts.

Common mistakes

Reading free instead of available and concluding a healthy machine is out of memory.

Dropping caches as a remedy, which removes a performance optimisation and fixes nothing.

Concluding a disk is full from df -h without checking df -i.

Deleting a large file while a process holds it open, then being confused that nothing was reclaimed.