用Java限制线程的CPU /内存使用率？ [英] Throttling CPU/Memory usage of a Thread in Java?

问题描述

我正在编写一个运行多个线程的应用程序，并希望限制这些线程的CPU /内存使用量。

I'm writing an application that will have multiple threads running, and want to throttle the CPU/memory usage of those threads.

有一个类似的C ++问题，但我想尽量避免使用C ++和JNI。我意识到使用更高级别的语言可能无法做到这一点，但我很想知道是否有人有任何想法。

There is a similar question for C++, but I want to try and avoid using C++ and JNI if possible. I realize this might not be possible using a higher level language, but I'm curious to see if anyone has any ideas.

编辑：增加了赏金;我想要一些非常好的，深思熟虑的想法。

Added a bounty; I'd like some really good, well thought out ideas on this.

编辑2：我需要的情况是执行其他人的我服务器上的代码。基本上它是完全任意的代码，唯一的保证是类文件上会有一个main方法。目前，在运行时加载的多个完全不同的类作为单独的线程并发执行。

EDIT 2: The situation I need this for is executing other people's code on my server. Basically it is completely arbitrary code, with the only guarantee being that there will be a main method on the class file. Currently, multiple completely disparate classes, which are loaded in at runtime, are executing concurrently as separate threads.

编写它的方式，重构创建将是一件痛苦的事情。为每个执行的类分别处理。如果这是通过VM参数限制内存使用的唯一好方法，那就这样吧。但我想知道是否有办法用线程来做。即使作为一个单独的进程，我也希望能够以某种方式限制其CPU使用率，因为正如我之前提到的，其中一些将立即执行。我不希望无限循环占用所有资源。

The way it's written, it would be a pain to refactor to create separate processes for each class that gets executed. If that's the only good way to limit memory usage via the VM arguments, then so be it. But I'd like to know if there's a way to do it with threads. Even as a separate process, I'd like to be able to somehow limit its CPU usage, since as I mentioned earlier, several of these will be executing at once. I don't want an infinite loop to hog up all the resources.

编辑3：近似对象大小的简单方法是使用java的 Instrumentation 课程;特别是getObjectSize方法。请注意，使用此工具需要一些特殊设置。

EDIT 3: An easy way to approximate object size is with java's Instrumentation classes; specifically, the getObjectSize method. Note that there is some special setup needed to use this tool.

推荐答案

如果我理解您的问题，一种方法是自适应睡眠线程，就像用Java完成视频播放一样。如果你知道你想要50％的核心利用率，你的算法应该睡眠大约0.5秒 - 可能在一秒钟内分布（例如0.25秒计算，0.25秒睡眠，等等）。这是示例来自我的视频播放器。

If I understand your problem, one way would be to adaptively sleep the threads, similarly as video playback is done in Java. If you know you want 50% core utilization, the your algorithm should sleep approximately 0.5 seconds - potentially distributed within a second (e.g. 0.25 sec computation, 0.25 sec sleep, e.t.c.). Here is an example from my video player.

long starttime = 0; // variable declared
//...
// for the first time, remember the timestamp
if (frameCount == 0) {
    starttime = System.currentTimeMillis();
}
// the next timestamp we want to wake up
starttime += (1000.0 / fps);
// Wait until the desired next time arrives using nanosecond
// accuracy timer (wait(time) isn't accurate enough on most platforms) 
LockSupport.parkNanos((long)(Math.max(0, 
    starttime - System.currentTimeMillis()) * 1000000));

此代码将根据帧数/秒值休眠。

This code will sleep based on the frames/second value.

要限制内存使用量，可以将对象创建包装到工厂方法中，并使用某种具有有限许可证的信号量作为字节来限制估计的总对象大小（需要估计对象的大小）用于定量信号量的各种对象。）

To throttle the memory usage, you could wrap your object creation into a factory method, and use some kind of semaphore with a limited permits as bytes to limit the total estimated object size (you need to estimate the size of various objects to ration the semaphore).

package concur;

import java.util.Random;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.Semaphore;
import java.util.concurrent.TimeUnit;

public class MemoryLimited {
    private static Semaphore semaphore = new Semaphore(1024 * 1024, true);
    // acquire method to get a size length array
    public static byte[] createArray(int size) throws InterruptedException {
        // ask the semaphore for the amount of memory
        semaphore.acquire(size);
        // if we get here we got the requested memory reserved
        return new byte[size];
    }
    public static void releaseArray(byte[] array) {
        // we don't need the memory of array, release
        semaphore.release(array.length);
    }
    // allocation size, if N > 1M then there will be mutual exclusion
    static final int N = 600000;
    // the test program
    public static void main(String[] args) {
        // create 2 threaded executor for the demonstration
        ExecutorService exec = Executors.newFixedThreadPool(2);
        // what we want to run for allocation testion
        Runnable run = new Runnable() {
            @Override
            public void run() {
                Random rnd = new Random();
                // do it 10 times to be sure we get the desired effect
                for (int i = 0; i < 10; i++) {
                    try {
                        // sleep randomly to achieve thread interleaving
                        TimeUnit.MILLISECONDS.sleep(rnd.nextInt(100) * 10);
                        // ask for N bytes of memory
                        byte[] array = createArray(N);
                        // print current memory occupation log
                        System.out.printf("%s %d: %s (%d)%n",
                            Thread.currentThread().getName(),
                            System.currentTimeMillis(), array,
                            semaphore.availablePermits());
                        // wait some more for the next thread interleaving
                        TimeUnit.MILLISECONDS.sleep(rnd.nextInt(100) * 10);
                        // release memory, no longer needed
                        releaseArray(array);
                    } catch (InterruptedException e) {
                        e.printStackTrace();
                    }
                }
            }
        };
        // run first task
        exec.submit(run);
        // run second task
        exec.submit(run);
        // let the executor exit when it has finished processing the runnables
        exec.shutdown();
    }
}

这篇关于用Java限制线程的CPU /内存使用率？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！