使用 Cpuset 将内核模块隔离到特定内核 [英] Isolate Kernel Module to a Specific Core Using Cpuset
问题描述
从用户空间我们可以使用 cpuset实际上隔离我们系统中的一个特定核心，并且只对该核心执行一个特定过程.
From user-space we can use cpuset to actually isolate a specific core in our system and execute just one specific process to that core.
我正在尝试用内核模块做同样的事情.所以我希望模块在隔离的核心中执行.换句话说:如何在内核模块中使用 cpuset 的? *
I'm trying to do the same thing with a kernel module. So I want the module to get executed in an isolated core. In other words: How do I use cpuset's from inside a kernel module? *
在我的内核中使用 linux/cpuset.h模块不起作用.所以，我有一个这样的模块:
Using linux/cpuset.h in my kernel module doesn't work. So, I have a module like this:
#include <linux/module.h>
#include <linux/cpuset.h>
...
#ifdef CONFIG_CPUSETS
printk(KERN_INFO, "cpusets is enabled!");
#endif
cpuset_init(); // this function is declared in cpuset.h
...
当我尝试加载这个模块时，我得到(在 dmesg 中)以下消息 cpusets is enabled!.但我也收到消息Unknown symbol cpu_init (err 0).
When trying to load this module I get (in dmesg) the following message cpusets is enabled!. But I also receive the message Unknown symbol cpu_init (err 0).
同样，我尝试使用 linux/sched.h 中的 sched_setaffinity 将所有正在运行的过程移动到特定的核心，然后将我的模块运行到隔离的核心.我收到了同样的错误信息:Unknown symbol sched_setaffinity (err 0).我想我得到了未知符号"，因为这些函数在内核中没有 EXPORT_SYMBOL .所以我去尝试调用 sys_sched_setaffinity 系统调用(基于这个 问题)，但再次收到此消息:未知符号 sys_sched_setaffinity (err 0)！
Similarly, I tried using sched_setaffinity from linux/sched.h in order to move all running procceses to a specific core and then run my module to an isolated core. I got the same error mesage: Unknown symbol sched_setaffinity (err 0). I guess I got the "unknown symbols" because those functions have no EXPORT_SYMBOL in the kernel. So I went and tried to call the sys_sched_setaffinity system call (based on this question) but again got this mesage: Unknown symbol sys_sched_setaffinity (err 0)!
此外，我不是在寻找使用 isolcpus 的解决方案，它是在启动时设置的.我只想加载模块，然后进行隔离.
Furthermore, I am not looking for a solution that uses isolcpus, which is set while booting. I would like to just load the module and afterwards the isolationo to occur.
- (更准确地说，我希望它的内核线程在隔离内核中执行.我知道我可以使用亲和性将线程绑定到特定内核，但这并不能保证内核会运行被运行在它们上面的其他进程隔离.)
- (More precise, I want its kernel threads to execute in isolated cores. I know that I can use affinity to bind threads to specific cores, but this does not guarantee me that the cores are going to be isolated by other processes running on them.)
推荐答案
所以我希望模块在一个独立的内核中执行.
So I want the module to get executed in an isolated core.
和
实际上在我们的系统中隔离一个特定的核心并且只执行一个那个核心的具体流程
actually isolate a specific core in our system and execute just one specific process to that core
这是使用内核 3.16 在 Debian 机器上编译和测试的工作源代码.我先介绍一下如何加载和卸载以及传入的参数是什么意思.
This is a working source code compiled and tested on a Debian box using kernel 3.16. I'll describe how to load and unload first and what the parameter passed means.
所有资源都可以在 github 上找到...
All sources can be found on github here...
https://github.com/harryjackson/doc/tree/master/linux/kernel/toy/toy
构建并加载模块...
make
insmod toy param_cpu_id=2
卸载模块使用
rmmod toy
我没有使用 modprobe，因为它需要一些配置等.我们传递给 toy 内核模块的参数是我们想要隔离的 CPU.除非在该 CPU 上执行，否则所有被调用的设备操作都不会运行.
I'm not using modprobe because it expects some configuration etc. The parameter we're passing to the toy kernel module is the CPU we want to isolate. None of the device operations that get called will run unless they're executing on that CPU.
加载模块后，您可以在此处找到它
Once the module is loaded you can find it here
/dev/toy
简单的操作如
cat /dev/toy
创建内核模块捕获并产生一些输出的事件.您可以使用 dmesg 查看输出.
create events that the kernel module catches and produces some output. You can see the output using dmesg.
源代码...
#include <linux/module.h>
#include <linux/fs.h>
#include <linux/miscdevice.h>
MODULE_LICENSE("GPL");
MODULE_AUTHOR("Harry");
MODULE_DESCRIPTION("toy kernel module");
MODULE_VERSION("0.1");
#define DEVICE_NAME "toy"
#define CLASS_NAME "toy"
static int param_cpu_id;
module_param(param_cpu_id , int, (S_IRUSR | S_IRGRP | S_IROTH));
MODULE_PARM_DESC(param_cpu_id, "CPU ID that operations run on");
//static void bar(void *arg);
//static void foo(void *cpu);
static int toy_open( struct inode *inodep, struct file *fp);
static ssize_t toy_read( struct file *fp , char *buffer, size_t len, loff_t * offset);
static ssize_t toy_write( struct file *fp , const char *buffer, size_t len, loff_t *);
static int toy_release(struct inode *inodep, struct file *fp);
static struct file_operations toy_fops = {
.owner = THIS_MODULE,
.open = toy_open,
.read = toy_read,
.write = toy_write,
.release = toy_release,
};
static struct miscdevice toy_device = {
.minor = MISC_DYNAMIC_MINOR,
.name = "toy",
.fops = &toy_fops
};
//static int CPU_IDS[64] = {0};
static int toy_open(struct inode *inodep, struct file *filep) {
int this_cpu = get_cpu();
printk(KERN_INFO "open: called on CPU:%d
", this_cpu);
if(this_cpu == param_cpu_id) {
printk(KERN_INFO "open: is on requested CPU: %d
", smp_processor_id());
}
else {
printk(KERN_INFO "open: not on requested CPU:%d
", smp_processor_id());
}
put_cpu();
return 0;
}
static ssize_t toy_read(struct file *filep, char *buffer, size_t len, loff_t *offset){
int this_cpu = get_cpu();
printk(KERN_INFO "read: called on CPU:%d
", this_cpu);
if(this_cpu == param_cpu_id) {
printk(KERN_INFO "read: is on requested CPU: %d
", smp_processor_id());
}
else {
printk(KERN_INFO "read: not on requested CPU:%d
", smp_processor_id());
}
put_cpu();
return 0;
}
static ssize_t toy_write(struct file *filep, const char *buffer, size_t len, loff_t *offset){
int this_cpu = get_cpu();
printk(KERN_INFO "write called on CPU:%d
", this_cpu);
if(this_cpu == param_cpu_id) {
printk(KERN_INFO "write: is on requested CPU: %d
", smp_processor_id());
}
else {
printk(KERN_INFO "write: not on requested CPU:%d
", smp_processor_id());
}
put_cpu();
return 0;
}
static int toy_release(struct inode *inodep, struct file *filep){
int this_cpu = get_cpu();
printk(KERN_INFO "release called on CPU:%d
", this_cpu);
if(this_cpu == param_cpu_id) {
printk(KERN_INFO "release: is on requested CPU: %d
", smp_processor_id());
}
else {
printk(KERN_INFO "release: not on requested CPU:%d
", smp_processor_id());
}
put_cpu();
return 0;
}
static int __init toy_init(void) {
int cpu_id;
if(param_cpu_id < 0 || param_cpu_id > 4) {
printk(KERN_INFO "toy: unable to load module without cpu parameter
");
return -1;
}
printk(KERN_INFO "toy: loading to device driver, param_cpu_id: %d
", param_cpu_id);
//preempt_disable(); // See notes below
cpu_id = get_cpu();
printk(KERN_INFO "toy init called and running on CPU: %d
", cpu_id);
misc_register(&toy_device);
//preempt_enable(); // See notes below
put_cpu();
//smp_call_function_single(1,foo,(void *)(uintptr_t) 1,1);
return 0;
}
static void __exit toy_exit(void) {
misc_deregister(&toy_device);
printk(KERN_INFO "toy exit called
");
}
module_init(toy_init);
module_exit(toy_exit);
上面的代码包含您要求的两种方法，即 CPU 隔离和 init 在隔离核心上运行.
The code above contains the two methods you asked for ie isolation of CPU and on init run on an isolated core.
在初始化时 get_cpu 禁用抢占，即在它之后的任何内容都不会被内核抢占，而是在一个内核上运行.请注意，这是使用 3.16 完成的内核，您的里程可能因您的内核版本而异，但我认为这些 API 已经存在很长时间了
On init get_cpu disables preemption ie anything that comes after it will not be preempted by the kernel and will run on one core. Note, this was done kernel using 3.16, your mileage may vary depending on your kernel version but I think these API's have been around a long time
这是生成文件...
obj-m += toy.o
all:
make -C /lib/modules/$(shell uname -r)/build M=$(PWD) modules
clean:
make -C /lib/modules/$(shell uname -r)/build M=$(PWD) clean
注意事项.get_cpu 在 linux/smp.h 中声明为
Notes. get_cpu is declared in linux/smp.h as
#define get_cpu() ({ preempt_disable(); smp_processor_id(); })
#define put_cpu() preempt_enable()
所以你实际上不需要在调用 get_cpu 之前调用 preempt_disable.get_cpu 调用是以下调用序列的包装器...
so you don't actually need to call preempt_disable before calling get_cpu.
The get_cpu call is a wrapper around the following sequence of calls...
preempt_count_inc();
barrier();
而且 put_cpu 真的在做这个...
and put_cpu is really doing this...
barrier();
if (unlikely(preempt_count_dec_and_test())) {
__preempt_schedule();
}
您可以随意使用上述内容.几乎所有这些都来自以下来源..
You can get as fancy as you like using the above. Almost all of this was taken from the following sources..
Google for... smp_call_function_single
Google for... smp_call_function_single
Linux 内核开发，Robert Love 所著.
Linux Kernel Development, book by Robert Love.
http://derekmolloy.ie/writing-a-linux-kernel-module-part-2-a-character-device/
https://github.com/vsinitsyn/reverse/blob/master/反向.c
这篇关于使用 Cpuset 将内核模块隔离到特定内核的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
