使用Cpuset将内核模块隔离到特定的内核 [英] Isolate Kernel Module to a Specific Core Using Cpuset
问题描述
在用户空间中，我们可以使用 cpuset 来实际上在我们的系统中隔离一个特定的内核，并仅对该内核执行一个特定的过程.
From user-space we can use cpuset to actually isolate a specific core in our system and execute just one specific process to that core.
我正在尝试对内核模块执行相同的操作.因此，我希望模块在隔离的内核中执行.换句话说:如何从内核模块内部使用cpuset? *
I'm trying to do the same thing with a kernel module. So I want the module to get executed in an isolated core. In other words: How do I use cpuset's from inside a kernel module? *
在我的内核中使用 linux/cpuset.h 模块不起作用.所以，我有一个像这样的模块:
Using linux/cpuset.h in my kernel module doesn't work. So, I have a module like this:
#include <linux/module.h>
#include <linux/cpuset.h>
...
#ifdef CONFIG_CPUSETS
printk(KERN_INFO, "cpusets is enabled!");
#endif
cpuset_init(); // this function is declared in cpuset.h
...
当尝试加载此模块时，我在dmesg中收到以下消息cpusets is enabled!.但是我也收到消息Unknown symbol cpu_init (err 0).
When trying to load this module I get (in dmesg) the following message cpusets is enabled!. But I also receive the message Unknown symbol cpu_init (err 0).
类似地，我尝试使用linux/sched.h中的sched_setaffinity以便将所有正在运行的过程移至特定的内核，然后将模块运行至隔离的内核.我得到了相同的错误消息:Unknown symbol sched_setaffinity (err 0).我猜我得到了未知符号"，因为这些函数在内核中没有EXPORT_SYMBOL.因此，我去尝试调用sys_sched_setaffinity 系统调用(基于此
Similarly, I tried using sched_setaffinity from linux/sched.h in order to move all running procceses to a specific core and then run my module to an isolated core. I got the same error mesage: Unknown symbol sched_setaffinity (err 0). I guess I got the "unknown symbols" because those functions have no EXPORT_SYMBOL in the kernel. So I went and tried to call the sys_sched_setaffinity system call (based on this question) but again got this mesage: Unknown symbol sys_sched_setaffinity (err 0)!
此外，我没有在寻找使用isolcpus的解决方案，该解决方案是在引导时设置的.我只想加载模块，然后进行隔离操作.
Furthermore, I am not looking for a solution that uses isolcpus, which is set while booting. I would like to just load the module and afterwards the isolationo to occur.
- (更准确地说，我希望其内核线程在隔离的内核中执行.我知道我可以使用亲和力将线程绑定到特定的内核，但这并不能保证我的内核正在运行被运行在其上的其他进程隔离.)
- (More precise, I want its kernel threads to execute in isolated cores. I know that I can use affinity to bind threads to specific cores, but this does not guarantee me that the cores are going to be isolated by other processes running on them.)
推荐答案
所以我希望模块在隔离的内核中执行.
So I want the module to get executed in an isolated core.
和
实际上隔离了我们系统中的特定核心并仅执行一个 该核心的具体过程
actually isolate a specific core in our system and execute just one specific process to that core
这是使用内核3.16在Debian机器上编译和测试的有效源代码.我将描述如何首先加载和卸载以及传递的参数的含义.
This is a working source code compiled and tested on a Debian box using kernel 3.16. I'll describe how to load and unload first and what the parameter passed means.
所有资源都可以在github上找到...
All sources can be found on github here...
https://github.com/harryjackson/doc/树/主/Linux/内核/玩具/玩具
构建并加载模块...
Build and load the module...
make
insmod toy param_cpu_id=2
要卸载模块，请使用
rmmod toy
我不使用modprobe，因为它需要一些配置等.我们传递给toy内核模块的参数是我们要隔离的CPU.除非它们在该CPU上执行，否则任何被调用的设备操作都不会运行.
I'm not using modprobe because it expects some configuration etc. The parameter we're passing to the toy kernel module is the CPU we want to isolate. None of the device operations that get called will run unless they're executing on that CPU.
加载模块后，您可以在此处找到它
Once the module is loaded you can find it here
/dev/toy
简单的操作，例如
cat /dev/toy
创建内核模块捕获的事件并产生一些输出.您可以使用dmesg看到输出.
create events that the kernel module catches and produces some output. You can see the output using dmesg.
源代码...
#include <linux/module.h>
#include <linux/fs.h>
#include <linux/miscdevice.h>
MODULE_LICENSE("GPL");
MODULE_AUTHOR("Harry");
MODULE_DESCRIPTION("toy kernel module");
MODULE_VERSION("0.1");
#define DEVICE_NAME "toy"
#define CLASS_NAME "toy"
static int param_cpu_id;
module_param(param_cpu_id , int, (S_IRUSR | S_IRGRP | S_IROTH));
MODULE_PARM_DESC(param_cpu_id, "CPU ID that operations run on");
//static void bar(void *arg);
//static void foo(void *cpu);
static int toy_open( struct inode *inodep, struct file *fp);
static ssize_t toy_read( struct file *fp , char *buffer, size_t len, loff_t * offset);
static ssize_t toy_write( struct file *fp , const char *buffer, size_t len, loff_t *);
static int toy_release(struct inode *inodep, struct file *fp);
static struct file_operations toy_fops = {
.owner = THIS_MODULE,
.open = toy_open,
.read = toy_read,
.write = toy_write,
.release = toy_release,
};
static struct miscdevice toy_device = {
.minor = MISC_DYNAMIC_MINOR,
.name = "toy",
.fops = &toy_fops
};
//static int CPU_IDS[64] = {0};
static int toy_open(struct inode *inodep, struct file *filep) {
int this_cpu = get_cpu();
printk(KERN_INFO "open: called on CPU:%d\n", this_cpu);
if(this_cpu == param_cpu_id) {
printk(KERN_INFO "open: is on requested CPU: %d\n", smp_processor_id());
}
else {
printk(KERN_INFO "open: not on requested CPU:%d\n", smp_processor_id());
}
put_cpu();
return 0;
}
static ssize_t toy_read(struct file *filep, char *buffer, size_t len, loff_t *offset){
int this_cpu = get_cpu();
printk(KERN_INFO "read: called on CPU:%d\n", this_cpu);
if(this_cpu == param_cpu_id) {
printk(KERN_INFO "read: is on requested CPU: %d\n", smp_processor_id());
}
else {
printk(KERN_INFO "read: not on requested CPU:%d\n", smp_processor_id());
}
put_cpu();
return 0;
}
static ssize_t toy_write(struct file *filep, const char *buffer, size_t len, loff_t *offset){
int this_cpu = get_cpu();
printk(KERN_INFO "write called on CPU:%d\n", this_cpu);
if(this_cpu == param_cpu_id) {
printk(KERN_INFO "write: is on requested CPU: %d\n", smp_processor_id());
}
else {
printk(KERN_INFO "write: not on requested CPU:%d\n", smp_processor_id());
}
put_cpu();
return 0;
}
static int toy_release(struct inode *inodep, struct file *filep){
int this_cpu = get_cpu();
printk(KERN_INFO "release called on CPU:%d\n", this_cpu);
if(this_cpu == param_cpu_id) {
printk(KERN_INFO "release: is on requested CPU: %d\n", smp_processor_id());
}
else {
printk(KERN_INFO "release: not on requested CPU:%d\n", smp_processor_id());
}
put_cpu();
return 0;
}
static int __init toy_init(void) {
int cpu_id;
if(param_cpu_id < 0 || param_cpu_id > 4) {
printk(KERN_INFO "toy: unable to load module without cpu parameter\n");
return -1;
}
printk(KERN_INFO "toy: loading to device driver, param_cpu_id: %d\n", param_cpu_id);
//preempt_disable(); // See notes below
cpu_id = get_cpu();
printk(KERN_INFO "toy init called and running on CPU: %d\n", cpu_id);
misc_register(&toy_device);
//preempt_enable(); // See notes below
put_cpu();
//smp_call_function_single(1,foo,(void *)(uintptr_t) 1,1);
return 0;
}
static void __exit toy_exit(void) {
misc_deregister(&toy_device);
printk(KERN_INFO "toy exit called\n");
}
module_init(toy_init);
module_exit(toy_exit);
上面的代码包含您要求的两种方法，即隔离CPU和在隔离内核上运行的init.
The code above contains the two methods you asked for ie isolation of CPU and on init run on an isolated core.
在初始化时，get_cpu会禁用抢占，即之后发生的任何事情都不会被内核抢占，而是将在一个内核上运行.请注意，这是使用3.16内核完成的，根据您的内核版本，您的工作量可能会有所不同，但是我认为这些API已经存在了很长时间了
On init get_cpu disables preemption ie anything that comes after it will not be preempted by the kernel and will run on one core. Note, this was done kernel using 3.16, your mileage may vary depending on your kernel version but I think these API's have been around a long time
这是Makefile ...
This is the Makefile...
obj-m += toy.o
all:
make -C /lib/modules/$(shell uname -r)/build M=$(PWD) modules
clean:
make -C /lib/modules/$(shell uname -r)/build M=$(PWD) clean
注意. get_cpu在linux/smp.h中声明为
#define get_cpu() ({ preempt_disable(); smp_processor_id(); })
#define put_cpu() preempt_enable()
因此您实际上不需要在调用get_cpu之前先调用preempt_disable.
get_cpu调用是以下调用序列的包装.
so you don't actually need to call preempt_disable before calling get_cpu.
The get_cpu call is a wrapper around the following sequence of calls...
preempt_count_inc();
barrier();
而put_cpu确实在这样做...
and put_cpu is really doing this...
barrier();
if (unlikely(preempt_count_dec_and_test())) {
__preempt_schedule();
}
使用上面的方法，您可以根据自己的喜好进行选择.几乎所有这些都来自以下来源.
You can get as fancy as you like using the above. Almost all of this was taken from the following sources..
Google for ... smp_call_function_single
Google for... smp_call_function_single
Linux内核开发，罗伯特·洛夫(Robert Love)的书.
Linux Kernel Development, book by Robert Love.
http://derekmolloy.ie /writing-a-linux-kernel-module-part-2-a-character-device/
https://github.com/vsinitsyn/reverse/blob/master/reverse.c
这篇关于使用Cpuset将内核模块隔离到特定的内核的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
