Posted: December 9, 2022 | | by Giancarlo del Rossi (Red Hat)
Image
One of the things I appreciate most about Linux is the chance to explore my operating system. Nothing is off limits in Linux. That's a huge opportunity—a very comfortable benefit—for anyone using Linux daily, whether you're a sysadmin, developer, network admin, or hobbyist eager to learn more.
It has real-world ramifications, too, because sometimes you really do need to probe into the inner workings of a system to understand a problem. Every sysadmin knows the feeling of getting a vague message like "The server is slow" or "This application isn't responding." That lucky sysadmin gets to perform a "treasure hunt" to find the problem. However, Linux provides all the tools you need.
[Cheat sheet:Old Linux commands and their modern replacements]
The lsof command, in combination with other tools like top or ps, can be very helpful when diagnosing a potential bottleneck or similar issue.
List open files
The lsof command is an acronym for "list open files," but its potential isn't limited to just that role. It's commonly said that in Linux, everything is a file. In many ways, that's true, so a utility that lists open files is actually pretty useful. The lsof utility is a robust interface for the information inside the /proc virtual filesystem.
For example, suppose you need to know what process is using a particular directory:
$ lsof /runCOMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAMEsystemd 1 root 31u FIFO 0,24 0t0 19314 /run/initctlsystemd 1 root 38u FIFO 0,24 0t0 19331 /run/dmeventd-serversystemd 1 root 39u FIFO 0,24 0t0 19332 /run/dmeventd-clientsystemd-j 713 root mem REG 0,24 8388608 12871 /run/log/journal/aaff550ef54147ddb4d1776d86b04055/system.journalNetworkMa 881 root 20w FIFO 0,24 0t0 24896 /run/systemd/inhibit/1.refrhsmcertd 906 root 3wW REG 0,24 0 22479 /run/lock/subsys/rhsmcertdcrond 910 root 3uW REG 0,24 4 24002 /run/crond.pidrsyslogd 1224 root mem REG 0,24 8388608 12871 /run/log/journal/aaff550ef54147ddb4d1776d86b04055/system.journalrsyslogd 1224 root 10r REG 0,24 8388608 12871 /run/log/journal/aaff550ef54147ddb4d1776d86b04055/system.journalsshd 1651 root 11w FIFO 0,24 0t0 27390 /run/systemd/sessions/1.refsshd 1666 admin 11w FIFO 0,24 0t0 27390 /run/systemd/sessions/1.refOr maybe you need the exact opposite. Perhaps you need to know what files a particular process has open. This example looks at sshd:
$ lsof -p 890CMD PID USER FD TYPE DEVICE SIZE/OFF NODE NAMEsshd 890 root cwd DIR 253,0 224 128 /sshd 890 root rtd DIR 253,0 224 128 /sshd 890 root txt REG 253,0 886344 646480 /usr/sbin/sshdsshd 890 root mem REG 253,0 6940392 34389337 /var/lib/sss/mc/groupsshd 890 root mem REG 253,0 9253600 34389336 /var/lib/sss/mc/passwdsshd 890 root mem REG 253,0 46312 570898 ...libnss_sss.sosshd 890 root mem REG 253,0 144392 1847 ...libgpg-error.so[...]sshd 890 root 0r CHR 1,3 0t0 1027 /dev/nullsshd 890 root 1u unix 0xff... 0t0 22395 type=STREAMsshd 890 root 2u unix 0xff... 0t0 22395 type=STREAMsshd 890 root 4u unix 0xff... 0t0 24654 type=STREAMsshd 890 root 5u IPv4 24658 0t0 TCP *:ssh (LISTEN)sshd 890 root 6r REG 253,0 6940392 34389337 /var/lib/sss/mc/groupsshd 890 root 7u IPv6 24666 0t0 TCP *:ssh (LISTEN)Skip to the bottom of list
Download now
Here's an explanation of each column:
- COMMAND: The command name
- PID: Process ID (PID) of the process
- USER: Owner of the process
- FD: File descriptor definition
- TYPE: Type of file descriptor
- DEVICE: Device number or, in the case of a block device, characteror other
- SIZE/OFF: Dimension of the file or offset (the suffix
0tis the offset) - NODE: Node description of the local file; this could be the number of the local file, TCP, UDP, or STR (stream)
- NAME: The name of the mount point where the file resides
Open files
You can also discover what files a particular user has open:
$ lsof -u adminThis information is especially useful when you can't unmount an external device because a file on it is being actively accessed.
[ Get the guide to installing applications on Linux. ]
Network debugging
As I've said, everything on Linux is a file, so lsof isn't limited to the local filesystem. You can also use it for network debugging.
For example, suppose you need to know what process uses a particular TCP port (like 22, for example):
$ lsof -i TCP:22COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAMEsshd 890 root 5u IPv4 24658 0t0 TCP *:ssh (LISTEN)sshd 890 root 7u IPv6 24666 0t0 TCP *:ssh (LISTEN)You can also obtain a report on all network processes associated with a particular interface:
$ lsof -i TCP@127.0.0.1COMMAND PID USER FD TYPE DEVICE SZ/OFF NODE NAMEcupsd 4408 root 8u IPv4 32331 0t0 TCP ...ipp (LISTEN)qemu-s 13462 qemu 11u IPv4 98536 0t0 TCP ...rfb (LISTEN)[ For more tips on Linux administration, readHow to analyze a Linux process' memory map with pmap. ]
Great potential
Despite its seemingly humble purpose, the lsof tool is a powerful tool on a system that can treat almost anything as a file. Read the lsof man page to get more ideas of how this simple utility can help you perform some complex tasks.
Topics: Command line utilities Troubleshooting Linux administration
