有没有办法弄清楚正在使用Linux内核模块的原因? [英] Is there a way to figure out what is using a Linux kernel module?

问题描述

如果我加载内核模块并用lsmod列出已加载的模块，则可以获得模块的使用计数"(其他模块的数量以及对该模块的引用).但是，有没有办法弄清楚什么正在使用模块?

If I load a kernel module and list the loaded modules with lsmod, I can get the "use count" of the module (number of other modules with a reference to the module). Is there a way to figure out what is using a module, though?

问题是我正在开发的模块坚持认为其使用计数为1，因此我无法使用rmmod卸载它，但是其"by"列为空.这意味着每次我想重新编译并重新加载模块时，都必须重新启动计算机(或者，至少我无法找到其他方法来卸载它).

The issue is that a module I am developing insists its use count is 1 and thus I cannot use rmmod to unload it, but its "by" column is empty. This means that every time I want to re-compile and re-load the module, I have to reboot the machine (or, at least, I can't figure out any other way to unload it).

推荐答案

实际上，似乎有一种方法可以列出要求模块/驱动程序的进程-但是，我还没有看到它的公告(Linux内核文档之外) )，所以我在这里记下我的笔记:

Actually, there seems to be a way to list processes that claim a module/driver - however, I haven't seen it advertised (outside of Linux kernel documentation), so I'll jot down my notes here:

首先，非常感谢 @haggai_e 的回答；指向负责管理使用计数(引用计数)的函数try_module_get和try_module_put的指针是使我能够跟踪该过程的关键.

First of all, many thanks for @haggai_e's answer; the pointer to the functions try_module_get and try_module_put as those responsible for managing the use count (refcount) was the key that allowed me to track down the procedure.

我想在网上进一步寻找，我偶然发现了 Linux-内核存档:[PATCH 1/2]跟踪:减少模块跟踪点的开销；最终指向内核中存在的设施，即(我猜)跟踪"；该文档位于目录文档中/trace-Linux内核源代码树.特别是，有两个文件说明了跟踪工具， events.txt 和

Looking further for this online, I somehow stumbled upon the post Linux-Kernel Archive: [PATCH 1/2] tracing: Reduce overhead of module tracepoints; which finally pointed to a facility present in the kernel, known as (I guess) "tracing"; the documentation for this is in the directory Documentation/trace - Linux kernel source tree. In particular, two files explain the tracing facility, events.txt and ftrace.txt.

但是，在/sys/kernel/debug/tracing/README中正在运行的Linux系统上也有一个简短的跟踪mini-HOWTO"(另请参见我真的很讨厌别人说没有文档……)；请注意，在内核源代码树中，此文件实际上是由文件

But, there is also a short "tracing mini-HOWTO" on a running Linux system in /sys/kernel/debug/tracing/README (see also I'm really really tired of people saying that there's no documentation…); note that in the kernel source tree, this file is actually generated by the file kernel/trace/trace.c. I've tested this on Ubuntu natty, and note that since /sys is owned by root, you have to use sudo to read this file, as in sudo cat or

sudo less /sys/kernel/debug/tracing/README

...，这几乎适用于/sys下的所有其他操作，将在此处进行描述.

... and that goes for pretty much all other operations under /sys which will be described here.

首先，这是一个简单的最小模块/驱动程序代码(我从引用的资源中将其放在一起)，它仅创建一个/proc/testmod-sample文件节点，该节点返回字符串"This is testmod".当它被读取时；这是 testmod.c :

First of all, here is a simple minimal module/driver code (which I put together from the referred resources), which simply creates a /proc/testmod-sample file node, which returns the string "This is testmod." when it is being read; this is testmod.c:

/*
https://github.com/spotify/linux/blob/master/samples/tracepoints/tracepoint-sample.c
https://www.linux.com/learn/linux-training/37985-the-kernel-newbie-corner-kernel-debugging-using-proc-qsequenceq-files-part-1
*/

#include <linux/module.h>
#include <linux/sched.h>
#include <linux/proc_fs.h>
#include <linux/seq_file.h> // for sequence files

struct proc_dir_entry *pentry_sample;

char *defaultOutput = "This is testmod.";


static int my_show(struct seq_file *m, void *v)
{
  seq_printf(m, "%s\n", defaultOutput);
  return 0;
}

static int my_open(struct inode *inode, struct file *file)
{
  return single_open(file, my_show, NULL);
}

static const struct file_operations mark_ops = {
  .owner    = THIS_MODULE,
  .open = my_open,
  .read = seq_read,
  .llseek   = seq_lseek,
  .release  = single_release,
};


static int __init sample_init(void)
{
  printk(KERN_ALERT "sample init\n");
  pentry_sample = proc_create(
    "testmod-sample", 0444, NULL, &mark_ops);
  if (!pentry_sample)
    return -EPERM;
  return 0;
}

static void __exit sample_exit(void)
{
    printk(KERN_ALERT "sample exit\n");
    remove_proc_entry("testmod-sample", NULL);
}

module_init(sample_init);
module_exit(sample_exit);

MODULE_LICENSE("GPL");
MODULE_AUTHOR("Mathieu Desnoyers et al.");
MODULE_DESCRIPTION("based on Tracepoint sample");

可以使用以下 Makefile 构建此模块(只需将其与testmod.c放在同一目录中，然后在同一目录中运行make):

This module can be built with the following Makefile (just have it placed in the same directory as testmod.c, and then run make in that same directory):

CONFIG_MODULE_FORCE_UNLOAD=y
# for oprofile
DEBUG_INFO=y
EXTRA_CFLAGS=-g -O0

obj-m += testmod.o

# mind the tab characters needed at start here:
all:
    make -C /lib/modules/$(shell uname -r)/build M=$(PWD) modules

clean:
    make -C /lib/modules/$(shell uname -r)/build M=$(PWD) clean

构建此模块/驱动程序时，输出为内核目标文件testmod.ko.

When this module/driver is built, the output is a kernel object file, testmod.ko.

这时，我们可以准备与try_module_get和try_module_put相关的事件跟踪；这些在/sys/kernel/debug/tracing/events/module中:

At this point, we can prepare the event tracing related to try_module_get and try_module_put; those are in /sys/kernel/debug/tracing/events/module:

$ sudo ls /sys/kernel/debug/tracing/events/module
enable  filter  module_free  module_get  module_load  module_put  module_request

请注意，在我的系统上，默认情况下启用了跟踪:

Note that on my system, tracing is by default enabled:

$ sudo cat /sys/kernel/debug/tracing/tracing_enabled
1

...但是，模块跟踪(具体而言)不是:

... however, the module tracing (specifically) is not:

$ sudo cat /sys/kernel/debug/tracing/events/module/enable
0

现在，我们首先应该创建一个过滤器，该过滤器将对module_get，module_put等事件做出反应，但仅针对testmod模块.为此，我们应该首先检查事件的格式:

Now, we should first make a filter, that will react on the module_get, module_put etc events, but only for the testmod module. To do that, we should first check the format of the event:

$ sudo cat /sys/kernel/debug/tracing/events/module/module_put/format
name: module_put
ID: 312
format:
...
    field:__data_loc char[] name;   offset:20;  size:4; signed:1;

print fmt: "%s call_site=%pf refcnt=%d", __get_str(name), (void *)REC->ip, REC->refcnt

在这里，我们看到有一个名为name的字段，其中包含驱动程序名称，我们可以对其进行过滤.要创建过滤器，我们只需将过滤器字符串echo放入相应的文件中:

Here we can see that there is a field called name, which holds the driver name, which we can filter against. To create a filter, we simply echo the filter string into the corresponding file:

sudo bash -c "echo name == testmod > /sys/kernel/debug/tracing/events/module/filter"

在这里，首先请注意，由于必须调用sudo，因此必须将整个echo重定向包装为sudo -ed bash的自变量命令.其次，请注意，由于我们写入的是父" module/filter，而不是特定事件(可能是module/module_put/filter等)，因此此过滤器将应用于所有列为module目录的子代"的事件.

Here, first note that since we have to call sudo, we have to wrap the whole echo redirection as an argument command of a sudo-ed bash. Second, note that since we wrote to the "parent" module/filter, not the specific events (which would be module/module_put/filter etc), this filter will be applied to all events listed as "children" of module directory.

最后，我们启用模块跟踪:

Finally, we enable tracing for module:

sudo bash -c "echo 1 > /sys/kernel/debug/tracing/events/module/enable"

从这一点开始，我们可以读取跟踪日志文件；对我来说，阅读障碍 跟踪文件的管道"版本可以正常工作-像这样:

From this point on, we can read the trace log file; for me, reading the blocking, "piped" version of the trace file worked - like this:

sudo cat /sys/kernel/debug/tracing/trace_pipe | tee tracelog.txt

在这一点上，我们将在日志中看不到任何内容-因此是时候加载(并使用和删除)驱动程序了(在与读取trace_pipe的位置不同的终端中):

At this point, we will not see anything in the log - so it is time to load (and utilize, and remove) the driver (in a different terminal from where trace_pipe is being read):

$ sudo insmod ./testmod.ko
$ cat /proc/testmod-sample 
This is testmod.
$ sudo rmmod testmod

如果我们回到正在读取trace_pipe的终端，我们应该看到类似以下内容的东西:

If we go back to the terminal where trace_pipe is being read, we should see something like:

# tracer: nop
#
#           TASK-PID    CPU#    TIMESTAMP  FUNCTION
#              | |       |          |         |
          insmod-21137 [001] 28038.101509: module_load: testmod
          insmod-21137 [001] 28038.103904: module_put: testmod call_site=sys_init_module refcnt=2
           rmmod-21354 [000] 28080.244448: module_free: testmod

这几乎是我们将为testmod驱动程序获得的所有内容-仅在加载驱动程序(insmod)或卸载驱动程序(rmmod)时refcount才会更改，而当我们对cat进行读操作时则不会更改.因此，我们只需在该终端中使用 CTRL + C 中断对trace_pipe的读取即可；并完全停止跟踪:

That is pretty much all we will obtain for our testmod driver - the refcount changes only when the driver is loaded (insmod) or unloaded (rmmod), not when we do a read through cat. So we can simply interrupt the read from trace_pipe with CTRL+C in that terminal; and to stop the tracing altogether:

sudo bash -c "echo 0 > /sys/kernel/debug/tracing/tracing_enabled"

在这里，请注意，大多数示例都涉及读取文件/sys/kernel/debug/tracing/trace而不是此处的trace_pipe.但是，一个问题是，该文件不是要通过管道传输"的(因此，您不应在此trace文件上运行tail -f).但是，您应该在每次操作后重新阅读trace.在第一个insmod之后，我们将从cat中获得相同的输出-将trace和trace_pipe都包含在内；但是，在rmmod之后，读取trace文件将得到:

Here, note that most examples refer to reading the file /sys/kernel/debug/tracing/trace instead of trace_pipe as here. However, one problem is that this file is not meant to be "piped" (so you shouldn't run a tail -f on this trace file); but instead you should re-read the trace after each operation. After the first insmod, we would obtain the same output from cat-ing both trace and trace_pipe; however, after the rmmod, reading the trace file would give:

   <...>-21137 [001] 28038.101509: module_load: testmod
   <...>-21137 [001] 28038.103904: module_put: testmod call_site=sys_init_module refcnt=2
   rmmod-21354 [000] 28080.244448: module_free: testmod

...即:此时，insmod已经退出了很长时间，因此它不再存在于进程列表中，因此无法通过记录的进程ID(PID)来找到当时-因此我们得到一个空白的<...>作为进程名称.因此，在这种情况下，最好记录(通过tee)trace_pipe的运行输出.另外，请注意，为了清除/重置/擦除trace文件，只需向其写入0:

... that is: at this point, the insmod had already been exited for long, and so it doesn't exist anymore in the process list - and therefore cannot be found via the recorded process ID (PID) at the time - thus we get a blank <...> as process name. Therefore, it is better to log (via tee) a running output from trace_pipe in this case. Also, note that in order to clear/reset/erase the trace file, one simply writes a 0 to it:

sudo bash -c "echo 0 > /sys/kernel/debug/tracing/trace"

如果这似乎违反直觉，请注意trace是一个特殊文件，并且无论如何总会报告文件大小为零:

If this seems counterintuitive, note that trace is a special file, and will always report a file size of zero anyways:

$ sudo ls -la /sys/kernel/debug/tracing/trace
-rw-r--r-- 1 root root 0 2013-03-19 06:39 /sys/kernel/debug/tracing/trace

...即使它已满.

... even if it is "full".

最后，请注意，如果不实施过滤器，我们将在运行的系统上获得所有 all 模块调用的日志-该日志会将任何调用(也包括后台)记录到，例如使用binfmt_misc模块的人:

Finally, note that if we didn't implement a filter, we would have obtained a log of all module calls on the running system - which would log any call (also background) to grep and such, as those use the binfmt_misc module:

...
  tr-6232  [001] 25149.815373: module_put: binfmt_misc call_site=search_binary_handler refcnt=133194
..
  grep-6231  [001] 25149.816923: module_put: binfmt_misc call_site=search_binary_handler refcnt=133196
..
  cut-6233  [000] 25149.817842: module_put: binfmt_misc call_site=search_binary_handler refcnt=129669
..
  sudo-6234  [001] 25150.289519: module_put: binfmt_misc call_site=search_binary_handler refcnt=133198
..
  tail-6235  [000] 25150.316002: module_put: binfmt_misc call_site=search_binary_handler refcnt=129671

...这会增加很多开销(在日志数据量和生成该数据所需的处理时间方面).

... which adds quite a bit of overhead (in both log data ammount, and processing time required to generate it).

在查找时，我偶然发现了使用Ftrace PDF调试Linux内核，它指的是工具 trace-cmd ，与上面的功能几乎相同-但通过更简单的命令行界面.还有一个用于trace-cmd的前端阅读器" GUI，称为 KernelShark ；这两个文件也都通过sudo apt-get install trace-cmd kernelshark在Debian/Ubuntu存储库中.这些工具可以替代上述过程.

While looking this up, I stumbled upon Debugging Linux Kernel by Ftrace PDF, which refers to a tool trace-cmd, which pretty much does the similar as above - but through an easier command line interface. There is also a "front-end reader" GUI for trace-cmd called KernelShark; both of these are also in Debian/Ubuntu repositories via sudo apt-get install trace-cmd kernelshark. These tools could be an alternative to the procedure described above.

最后，我只想指出，虽然上面的testmod示例并未真正显示出在多个声明的上下文中的用法，但我使用了相同的跟踪过程来发现我正在编码的USB模块是插入USB设备后，pulseaudio反复声明-该过程似乎适用于此类用例.

Finally, I'd just note that, while the above testmod example doesn't really show use in context of multiple claims, I have used the same tracing procedure to discover that an USB module I'm coding, was repeatedly claimed by pulseaudio as soon as the USB device was plugged in - so the procedure seems to work for such use cases.

这篇关于有没有办法弄清楚正在使用Linux内核模块的原因?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！