如何在JavaFX中以大频率显示图像? [英] How to show images in a large frequency in JavaFX?
问题描述
我的应用程序生成热图图像的速度与 CPU 一样快(大约每秒 30-60 张)，我想将它们显示在单个实时热图"中.在 AWT/Swing 中，我可以将它们绘制成一个 JPanel，它就像一个魅力.最近，我切换到JavaFX，想在这里实现相同的功能；起初，我尝试使用 Canvas，它很慢但还可以，但有严重的内存泄漏问题，导致应用程序崩溃.现在，我尝试了 ImageView组件 - 由于图像变得非常滞后，这显然太慢了(使用 ImageView.setImage 每次新迭代).据我了解，setImage并不能保证在函数执行完毕后实际显示图像.
My application generates heatmap images as fast as the CPU can (around 30-60 per second) and I want to display them in a single "live heatmap". In AWT/Swing, I could just paint them into a JPanel which worked like a charm. Recently, I switched to JavaFX and want to achieve the same here; at first, I tried with a Canvas, which was slow but okay-ish but had a severe memory leak problem, causing the application to crash. Now, I tried the ImageView component - which apparently is way too slow as the image gets quite laggy (using ImageView.setImage on every new iteration). As far as I understand, setImage does not guarantee that the image is actually displayed when the function finishes.
我的印象是我走错了路，以一种他们不适合的方式使用这些组件.如何每秒显示 30-60 张图片?
I am getting the impression that I am on the wrong track, using those components in a manner they are not made to. How can I display my 30-60 Images per second?
一个非常简单的测试应用程序.您将需要 JHeatChart 库.请注意，在台式机上，我获得了大约 70-80 FPS，并且可视化效果还不错且流畅，但在较小的树莓派(我的目标机器)上，我获得了大约 30 FPS，但可视化效果非常卡.
A very simple test application. You will need the JHeatChart library. Note that on a desktop machine, I get around 70-80 FPS and the visualization is okay and fluid, but on a smaller raspberry pi (my target machine), I get around 30 FPS but an extremly stucking visualization.
package sample;
import javafx.application.Application;
import javafx.embed.swing.SwingFXUtils;
import javafx.scene.Scene;
import javafx.scene.image.ImageView;
import javafx.scene.layout.VBox;
import javafx.stage.Stage;
import org.tc33.jheatchart.HeatChart;
import java.awt.*;
import java.awt.geom.AffineTransform;
import java.awt.image.AffineTransformOp;
import java.awt.image.BufferedImage;
import java.util.LinkedList;
public class Main extends Application {
ImageView imageView = new ImageView();
final int scale = 15;
@Override
public void start(Stage primaryStage) {
Thread generator = new Thread(() -> {
int col = 0;
LinkedList<Long> fps = new LinkedList<>();
while (true) {
fps.add(System.currentTimeMillis());
double[][] matrix = new double[48][128];
for (int i = 0; i < 48; i++) {
for (int j = 0; j < 128; j++) {
matrix[i][j] = col == j ? Math.random() : 0;
}
}
col = (col + 1) % 128;
HeatChart heatChart = new HeatChart(matrix, 0, 1);
heatChart.setShowXAxisValues(false);
heatChart.setShowYAxisValues(false);
heatChart.setLowValueColour(java.awt.Color.black);
heatChart.setHighValueColour(java.awt.Color.white);
heatChart.setAxisThickness(0);
heatChart.setChartMargin(0);
heatChart.setCellSize(new Dimension(1, 1));
long currentTime = System.currentTimeMillis();
fps.removeIf(elem -> currentTime - elem > 1000);
System.out.println(fps.size());
imageView.setImage(SwingFXUtils.toFXImage((BufferedImage) scale(heatChart.getChartImage(), scale), null));
}
});
VBox box = new VBox();
box.getChildren().add(imageView);
Scene scene = new Scene(box, 1920, 720);
primaryStage.setScene(scene);
primaryStage.show();
generator.start();
}
public static void main(String[] args) {
launch(args);
}
private static Image scale(Image image, int scale) {
BufferedImage res = new BufferedImage(image.getWidth(null) * scale, image.getHeight(null) * scale,
BufferedImage.TYPE_INT_ARGB);
AffineTransform at = new AffineTransform();
at.scale(scale, scale);
AffineTransformOp scaleOp =
new AffineTransformOp(at, AffineTransformOp.TYPE_NEAREST_NEIGHBOR);
return scaleOp.filter((BufferedImage) image, res);
}
}
推荐答案
您的代码从后台线程更新 UI，这是绝对不允许的.您需要确保从 FX 应用程序线程进行更新.您还想尝试限制"实际的 UI 更新，使每个 JavaFX 帧渲染不超过一次.最简单的方法是使用 AnimationTimer，它的 handle() 方法在每次渲染帧时调用.
Your code updates the UI from a background thread, which is definitely not allowed. You need to ensure you update from the FX Application Thread. You also want to try to "throttle" the actual UI updates to occur no more than once per JavaFX frame rendering. The easiest way to do this is with an AnimationTimer, whose handle() method is invoked each time a frame is rendered.
这是执行此操作的代码版本:
Here's a version of your code which does that:
import java.awt.Dimension;
import java.awt.Image;
import java.awt.geom.AffineTransform;
import java.awt.image.AffineTransformOp;
import java.awt.image.BufferedImage;
import java.util.LinkedList;
import java.util.concurrent.atomic.AtomicReference;
import org.tc33.jheatchart.HeatChart;
import javafx.animation.AnimationTimer;
import javafx.application.Application;
import javafx.embed.swing.SwingFXUtils;
import javafx.scene.Scene;
import javafx.scene.image.ImageView;
import javafx.scene.layout.VBox;
import javafx.stage.Stage;
public class Main extends Application {
ImageView imageView = new ImageView();
final int scale = 15;
@Override
public void start(Stage primaryStage) {
AtomicReference<BufferedImage> image = new AtomicReference<>();
Thread generator = new Thread(() -> {
int col = 0;
LinkedList<Long> fps = new LinkedList<>();
while (true) {
fps.add(System.currentTimeMillis());
double[][] matrix = new double[48][128];
for (int i = 0; i < 48; i++) {
for (int j = 0; j < 128; j++) {
matrix[i][j] = col == j ? Math.random() : 0;
}
}
col = (col + 1) % 128;
HeatChart heatChart = new HeatChart(matrix, 0, 1);
heatChart.setShowXAxisValues(false);
heatChart.setShowYAxisValues(false);
heatChart.setLowValueColour(java.awt.Color.black);
heatChart.setHighValueColour(java.awt.Color.white);
heatChart.setAxisThickness(0);
heatChart.setChartMargin(0);
heatChart.setCellSize(new Dimension(1, 1));
long currentTime = System.currentTimeMillis();
fps.removeIf(elem -> currentTime - elem > 1000);
System.out.println(fps.size());
image.set((BufferedImage) scale(heatChart.getChartImage(), scale));
}
});
VBox box = new VBox();
box.getChildren().add(imageView);
Scene scene = new Scene(box, 1920, 720);
primaryStage.setScene(scene);
primaryStage.show();
generator.setDaemon(true);
generator.start();
AnimationTimer animation = new AnimationTimer() {
@Override
public void handle(long now) {
BufferedImage img = image.getAndSet(null);
if (img != null) {
imageView.setImage(SwingFXUtils.toFXImage(img, null));
}
}
};
animation.start();
}
public static void main(String[] args) {
launch(args);
}
private static Image scale(Image image, int scale) {
BufferedImage res = new BufferedImage(image.getWidth(null) * scale, image.getHeight(null) * scale,
BufferedImage.TYPE_INT_ARGB);
AffineTransform at = new AffineTransform();
at.scale(scale, scale);
AffineTransformOp scaleOp = new AffineTransformOp(at, AffineTransformOp.TYPE_NEAREST_NEIGHBOR);
return scaleOp.filter((BufferedImage) image, res);
}
}
使用 AtomicReference 来包装缓冲图像确保它在两个线程之间安全共享.
Using the AtomicReference to wrap the buffered image ensures that it is safely shared between the two threads.
在我的机器上，这每秒生成大约 130 个图像；请注意，并非所有都显示，因为每次 JavaFX 图形框架显示一帧(通常限制为 60fps)时，只会显示最新的一个.
On my machine, this generates about 130 images per second; note that not all are displayed, as only the latest one is shown each time the JavaFX graphics framework displays a frame (which is typically throttled at 60fps).
如果您想确保显示所有生成的图像，即您通过 JavaFX 帧率限制图像生成，那么您可以使用 BlockingQueue 来存储图片:
If you want to ensure you show all images that are generated, i.e. you throttle the image generation by the JavaFX framerate, then you can use a BlockingQueue to store the images:
// AtomicReference<BufferedImage> image = new AtomicReference<>();
// Size of the queue is a trade-off between memory consumption
// and smoothness (essentially works as a buffer size)
BlockingQueue<BufferedImage> image = new ArrayBlockingQueue<>(5);
// ...
// image.set((BufferedImage) scale(heatChart.getChartImage(), scale));
try {
image.put((BufferedImage) scale(heatChart.getChartImage(), scale));
} catch (InterruptedException exc) {
Thread.currentThread.interrupt();
}
和
@Override
public void handle(long now) {
BufferedImage img = image.poll();
if (img != null) {
imageView.setImage(SwingFXUtils.toFXImage(img, null));
}
}
代码效率很低，因为每次迭代都会生成一个新矩阵、新的 HeatChart 等.这会导致在堆上创建许多对象并迅速丢弃，这会导致 GC 运行过于频繁，尤其是在小内存机器上.也就是说，我在最大堆大小设置为 64MB (-Xmx64m) 的情况下运行它，它仍然表现良好.您或许可以优化代码，但使用如上所示的 AnimationTimer，更快地生成图像不会对 JavaFX 框架造成任何额外压力.我建议调查使用 HeatChart(即 setZValues())的可变性以避免创建太多对象，和/或使用 PixelBuffer 直接将数据写入图像视图(这个需要在 FX 应用线程上完成).
The code is pretty inefficient, as you generate a new matrix, new HeatChart, etc, on every iteration. This causes many objects to be created on the heap and quickly discarded, which can cause the GC to be run too often, particularly on a small-memory machine. That said, I ran this with the maximum heap size set at 64MB, (-Xmx64m), and it still performed fine. You may be able to optimize the code, but using the AnimationTimer as shown above, generating images more quickly will not cause any additional stress on the JavaFX framework. I would recommend investigating using the mutability of HeatChart (i.e. setZValues()) to avoid creating too many objects, and/or using PixelBuffer to directly write data to the image view (this would need to be done on the FX Application Thread).
这是一个不同的例子，它(几乎)完全减少了对象的创建，使用一个屏幕外的 int[] 数组来计算数据，一个屏幕上的 int[]数组来显示它.有一些低级线程细节以确保屏幕上的数组只能在一致的状态下看到.屏幕上的数组用于 PixelBuffer 的底层，后者又用于 WritableImage.
Here's a different example, which (almost) completely minimizes object creation, using one off-screen int[] array to compute data, and one on-screen int[] array to display it. There's a little low-level threading details to ensure the on-screen array is only seen in a consistent state. The on-screen array is used to underly a PixelBuffer, which in turn is used for a WritableImage.
这个类生成图像数据:
import java.util.concurrent.atomic.AtomicLong;
import java.util.concurrent.locks.ReentrantLock;
import java.util.function.Consumer;
public class ImageGenerator {
private final int width;
private final int height;
// Keep two copies of the data: one which is not exposed
// that we modify on the fly during computation;
// another which we expose publicly.
// The publicly exposed one can be viewed only in a complete
// state if operations on it are synchronized on this object.
private final int[] privateData ;
private final int[] publicData ;
private final long[] frameTimes ;
private int currentFrameIndex ;
private final AtomicLong averageGenerationTime ;
private final ReentrantLock lock ;
private static final double TWO_PI = 2 * Math.PI;
private static final double PI_BY_TWELVE = Math.PI / 12; // 15 degrees
public ImageGenerator(int width, int height) {
super();
this.width = width;
this.height = height;
privateData = new int[width * height];
publicData = new int[width * height];
lock = new ReentrantLock();
this.frameTimes = new long[100];
this.averageGenerationTime = new AtomicLong();
}
public void generateImage(double angle) {
// compute in private data copy:
int minDim = Math.min(width, height);
int minR2 = minDim * minDim / 4;
for (int x = 0; x < width; x++) {
int xOff = x - width / 2;
int xOff2 = xOff * xOff;
for (int y = 0; y < height; y++) {
int index = x + y * width;
int yOff = y - height / 2;
int yOff2 = yOff * yOff;
int r2 = xOff2 + yOff2;
if (r2 > minR2) {
privateData[index] = 0xffffffff; // white
} else {
double theta = Math.atan2(yOff, xOff);
double delta = Math.abs(theta - angle);
if (delta > TWO_PI - PI_BY_TWELVE) {
delta = TWO_PI - delta;
}
if (delta < PI_BY_TWELVE) {
int green = (int) (255 * (1 - delta / PI_BY_TWELVE));
privateData[index] = (0xff << 24) | (green << 8); // green, fading away from center
} else {
privateData[index] = 0xff << 24; // black
}
}
}
}
// copy computed data to public data copy:
lock.lock();
try {
System.arraycopy(privateData, 0, publicData, 0, privateData.length);
} finally {
lock.unlock();
}
frameTimes[currentFrameIndex] = System.nanoTime() ;
int nextIndex = (currentFrameIndex + 1) % frameTimes.length ;
if (frameTimes[nextIndex] > 0) {
averageGenerationTime.set((frameTimes[currentFrameIndex] - frameTimes[nextIndex]) / frameTimes.length);
}
currentFrameIndex = nextIndex ;
}
public void consumeData(Consumer<int[]> consumer) {
lock.lock();
try {
consumer.accept(publicData);
} finally {
lock.unlock();
}
}
public long getAverageGenerationTime() {
return averageGenerationTime.get() ;
}
}
这是用户界面:
import java.nio.IntBuffer;
import javafx.animation.AnimationTimer;
import javafx.application.Application;
import javafx.scene.Scene;
import javafx.scene.control.Label;
import javafx.scene.image.ImageView;
import javafx.scene.image.PixelFormat;
import javafx.scene.image.PixelWriter;
import javafx.scene.image.WritableImage;
import javafx.scene.layout.BorderPane;
import javafx.stage.Stage;
public class AnimationApp extends Application {
private final int size = 400 ;
private IntBuffer buffer ;
@Override
public void start(Stage primaryStage) throws Exception {
// background image data generation:
ImageGenerator generator = new ImageGenerator(size, size);
// Generate new image data as fast as possible:
Thread thread = new Thread(() -> {
while( true ) {
long now = System.currentTimeMillis() ;
double angle = 2 * Math.PI * (now % 10000) / 10000 - Math.PI;
generator.generateImage(angle);
}
});
thread.setDaemon(true);
thread.start();
generator.consumeData(data -> buffer = IntBuffer.wrap(data));
PixelFormat<IntBuffer> format = PixelFormat.getIntArgbPreInstance() ;
PixelBuffer<IntBuffer> pixelBuffer = new PixelBuffer<>(size, size, buffer, format);
WritableImage image = new WritableImage(pixelBuffer);
BorderPane root = new BorderPane(new ImageView(image));
Label fps = new Label("FPS: ");
root.setTop(fps);
Scene scene = new Scene(root);
primaryStage.setScene(scene);
primaryStage.setTitle("Give me a ping, Vasili. ");
primaryStage.show();
AnimationTimer animation = new AnimationTimer() {
@Override
public void handle(long now) {
// Update image, ensuring we only see the underlying
// data in a consistent state:
generator.consumeData(data -> {
pixelBuffer.updateBuffer(pb -> null);
});
long aveGenTime = generator.getAverageGenerationTime() ;
if (aveGenTime > 0) {
double aveFPS = 1.0 / (aveGenTime / 1_000_000_000.0);
fps.setText(String.format("FPS: %.2f", aveFPS));
}
}
};
animation.start();
}
public static void main(String[] args) {
Application.launch(args);
}
}
对于不依赖于 JavaFX 13 PixelBuffer 的版本，您可以修改此类以使用 PixelWriter(AIUI 这不会像高效，但在本例中同样流畅):
For a version that doesn't rely on the JavaFX 13 PixelBuffer, you can just modify this class to use a PixelWriter (AIUI this won't be quite as efficient, but works just as smoothly in this example):
// generator.consumeData(data -> buffer = IntBuffer.wrap(data));
PixelFormat<IntBuffer> format = PixelFormat.getIntArgbPreInstance() ;
// PixelBuffer<IntBuffer> pixelBuffer = new PixelBuffer<>(size, size, buffer, format);
// WritableImage image = new WritableImage(pixelBuffer);
WritableImage image = new WritableImage(size, size);
PixelWriter pixelWriter = image.getPixelWriter() ;
和
AnimationTimer animation = new AnimationTimer() {
@Override
public void handle(long now) {
// Update image, ensuring we only see the underlying
// data in a consistent state:
generator.consumeData(data -> {
// pixelBuffer.updateBuffer(pb -> null);
pixelWriter.setPixels(0, 0, size, size, format, data, 0, size);
});
long aveGenTime = generator.getAverageGenerationTime() ;
if (aveGenTime > 0) {
double aveFPS = 1.0 / (aveGenTime / 1_000_000_000.0);
fps.setText(String.format("FPS: %.2f", aveFPS));
}
}
};
这篇关于如何在JavaFX中以大频率显示图像?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
