如何在JavaFX中大量显示图像? [英] How to show images in a large frequency in JavaFX?
问题描述
我的应用程序以最快的速度(大约30-60每秒)生成热图图像，我想在一个实时热图"中显示它们.在AWT/Swing中，我可以将它们绘制到JPanel中，就像工作一样. 最近，我切换到JavaFX，并希望在这里实现相同的目标.最初，我尝试使用画布，虽然速度较慢但还可以，但是存在严重的内存泄漏问题，导致应用程序崩溃.现在，我尝试了 ImageView 组件-显然太慢了，因为图像变得很滞后(使用
My application generates heatmap images as fast as the CPU can (around 30-60 per second) and I want to display them in a single "live heatmap". In AWT/Swing, I could just paint them into a JPanel which worked like a charm. Recently, I switched to JavaFX and want to achieve the same here; at first, I tried with a Canvas, which was slow but okay-ish but had a severe memory leak problem, causing the application to crash. Now, I tried the ImageView component - which apparently is way too slow as the image gets quite laggy (using ImageView.setImage on every new iteration). As far as I understand, setImage does not guarantee that the image is actually displayed when the function finishes.
我给人的印象是，我以错误的方式使用了这些组件，从而走上了错误的道路.如何每秒显示30-60张图片?
I am getting the impression that I am on the wrong track, using those components in a manner they are not made to. How can I display my 30-60 Images per second?
一个非常简单的测试应用程序.您将需要 JHeatChart 库. 请注意，在台式机上，我得到约70-80 FPS，并且可视化效果还不错，但在较小的树莓派(我的目标机)上，我得到了约30 FPS，但可视化效果极强.
A very simple test application. You will need the JHeatChart library. Note that on a desktop machine, I get around 70-80 FPS and the visualization is okay and fluid, but on a smaller raspberry pi (my target machine), I get around 30 FPS but an extremly stucking visualization.
package sample;
import javafx.application.Application;
import javafx.embed.swing.SwingFXUtils;
import javafx.scene.Scene;
import javafx.scene.image.ImageView;
import javafx.scene.layout.VBox;
import javafx.stage.Stage;
import org.tc33.jheatchart.HeatChart;
import java.awt.*;
import java.awt.geom.AffineTransform;
import java.awt.image.AffineTransformOp;
import java.awt.image.BufferedImage;
import java.util.LinkedList;
public class Main extends Application {
ImageView imageView = new ImageView();
final int scale = 15;
@Override
public void start(Stage primaryStage) {
Thread generator = new Thread(() -> {
int col = 0;
LinkedList<Long> fps = new LinkedList<>();
while (true) {
fps.add(System.currentTimeMillis());
double[][] matrix = new double[48][128];
for (int i = 0; i < 48; i++) {
for (int j = 0; j < 128; j++) {
matrix[i][j] = col == j ? Math.random() : 0;
}
}
col = (col + 1) % 128;
HeatChart heatChart = new HeatChart(matrix, 0, 1);
heatChart.setShowXAxisValues(false);
heatChart.setShowYAxisValues(false);
heatChart.setLowValueColour(java.awt.Color.black);
heatChart.setHighValueColour(java.awt.Color.white);
heatChart.setAxisThickness(0);
heatChart.setChartMargin(0);
heatChart.setCellSize(new Dimension(1, 1));
long currentTime = System.currentTimeMillis();
fps.removeIf(elem -> currentTime - elem > 1000);
System.out.println(fps.size());
imageView.setImage(SwingFXUtils.toFXImage((BufferedImage) scale(heatChart.getChartImage(), scale), null));
}
});
VBox box = new VBox();
box.getChildren().add(imageView);
Scene scene = new Scene(box, 1920, 720);
primaryStage.setScene(scene);
primaryStage.show();
generator.start();
}
public static void main(String[] args) {
launch(args);
}
private static Image scale(Image image, int scale) {
BufferedImage res = new BufferedImage(image.getWidth(null) * scale, image.getHeight(null) * scale,
BufferedImage.TYPE_INT_ARGB);
AffineTransform at = new AffineTransform();
at.scale(scale, scale);
AffineTransformOp scaleOp =
new AffineTransformOp(at, AffineTransformOp.TYPE_NEAREST_NEIGHBOR);
return scaleOp.filter((BufferedImage) image, res);
}
}
推荐答案
您的代码从后台线程更新UI，这绝对是不允许的.您需要确保从FX Application Thread中进行更新.您还希望尝试限制"实际的UI更新，以使每个JavaFX框架渲染不超过一次.最简单的方法是使用AnimationTimer，每次渲染帧时都会调用handle()方法.
Your code updates the UI from a background thread, which is definitely not allowed. You need to ensure you update from the FX Application Thread. You also want to try to "throttle" the actual UI updates to occur no more than once per JavaFX frame rendering. The easiest way to do this is with an AnimationTimer, whose handle() method is invoked each time a frame is rendered.
以下是您的代码的版本:
Here's a version of your code which does that:
import java.awt.Dimension;
import java.awt.Image;
import java.awt.geom.AffineTransform;
import java.awt.image.AffineTransformOp;
import java.awt.image.BufferedImage;
import java.util.LinkedList;
import java.util.concurrent.atomic.AtomicReference;
import org.tc33.jheatchart.HeatChart;
import javafx.animation.AnimationTimer;
import javafx.application.Application;
import javafx.embed.swing.SwingFXUtils;
import javafx.scene.Scene;
import javafx.scene.image.ImageView;
import javafx.scene.layout.VBox;
import javafx.stage.Stage;
public class Main extends Application {
ImageView imageView = new ImageView();
final int scale = 15;
@Override
public void start(Stage primaryStage) {
AtomicReference<BufferedImage> image = new AtomicReference<>();
Thread generator = new Thread(() -> {
int col = 0;
LinkedList<Long> fps = new LinkedList<>();
while (true) {
fps.add(System.currentTimeMillis());
double[][] matrix = new double[48][128];
for (int i = 0; i < 48; i++) {
for (int j = 0; j < 128; j++) {
matrix[i][j] = col == j ? Math.random() : 0;
}
}
col = (col + 1) % 128;
HeatChart heatChart = new HeatChart(matrix, 0, 1);
heatChart.setShowXAxisValues(false);
heatChart.setShowYAxisValues(false);
heatChart.setLowValueColour(java.awt.Color.black);
heatChart.setHighValueColour(java.awt.Color.white);
heatChart.setAxisThickness(0);
heatChart.setChartMargin(0);
heatChart.setCellSize(new Dimension(1, 1));
long currentTime = System.currentTimeMillis();
fps.removeIf(elem -> currentTime - elem > 1000);
System.out.println(fps.size());
image.set((BufferedImage) scale(heatChart.getChartImage(), scale));
}
});
VBox box = new VBox();
box.getChildren().add(imageView);
Scene scene = new Scene(box, 1920, 720);
primaryStage.setScene(scene);
primaryStage.show();
generator.setDaemon(true);
generator.start();
AnimationTimer animation = new AnimationTimer() {
@Override
public void handle(long now) {
BufferedImage img = image.getAndSet(null);
if (img != null) {
imageView.setImage(SwingFXUtils.toFXImage(img, null));
}
}
};
animation.start();
}
public static void main(String[] args) {
launch(args);
}
private static Image scale(Image image, int scale) {
BufferedImage res = new BufferedImage(image.getWidth(null) * scale, image.getHeight(null) * scale,
BufferedImage.TYPE_INT_ARGB);
AffineTransform at = new AffineTransform();
at.scale(scale, scale);
AffineTransformOp scaleOp = new AffineTransformOp(at, AffineTransformOp.TYPE_NEAREST_NEIGHBOR);
return scaleOp.filter((BufferedImage) image, res);
}
}
使用AtomicReference包装缓冲的图像可确保在两个线程之间安全地共享该图像.
Using the AtomicReference to wrap the buffered image ensures that it is safely shared between the two threads.
在我的机器上，每秒生成约130张图像；请注意，不会全部显示，因为每次JavaFX图形框架显示一个帧(通常以60fps的速度)时，仅显示最新的帧.
On my machine, this generates about 130 images per second; note that not all are displayed, as only the latest one is shown each time the JavaFX graphics framework displays a frame (which is typically throttled at 60fps).
如果要确保显示所有生成的图像，即以JavaFX帧速率限制图像生成，则可以使用BlockingQueue存储图像:
If you want to ensure you show all images that are generated, i.e. you throttle the image generation by the JavaFX framerate, then you can use a BlockingQueue to store the images:
// AtomicReference<BufferedImage> image = new AtomicReference<>();
// Size of the queue is a trade-off between memory consumption
// and smoothness (essentially works as a buffer size)
BlockingQueue<BufferedImage> image = new ArrayBlockingQueue<>(5);
// ...
// image.set((BufferedImage) scale(heatChart.getChartImage(), scale));
try {
image.put((BufferedImage) scale(heatChart.getChartImage(), scale));
} catch (InterruptedException exc) {
Thread.currentThread.interrupt();
}
和
@Override
public void handle(long now) {
BufferedImage img = image.poll();
if (img != null) {
imageView.setImage(SwingFXUtils.toFXImage(img, null));
}
}
由于每次迭代都会生成一个新的矩阵，新的HeatChart等，因此代码效率很低.这导致在堆上创建许多对象并迅速将其丢弃，这可能导致GC过于频繁地运行，尤其是在小型内存计算机上.就是说，我以最大堆大小设置为64MB(-Xmx64m)来运行它，但它仍然运行良好.您可能可以优化代码，但是使用如上所示的AnimationTimer，更快地生成图像不会对JavaFX框架造成任何额外的压力.我建议使用HeatChart(即setZValues())的可变性进行调查，以避免创建过多的对象，和/或使用
The code is pretty inefficient, as you generate a new matrix, new HeatChart, etc, on every iteration. This causes many objects to be created on the heap and quickly discarded, which can cause the GC to be run too often, particularly on a small-memory machine. That said, I ran this with the maximum heap size set at 64MB, (-Xmx64m), and it still performed fine. You may be able to optimize the code, but using the AnimationTimer as shown above, generating images more quickly will not cause any additional stress on the JavaFX framework. I would recommend investigating using the mutability of HeatChart (i.e. setZValues()) to avoid creating too many objects, and/or using PixelBuffer to directly write data to the image view (this would need to be done on the FX Application Thread).
这是另一个示例，它使用一个屏幕外的int[]数组计算数据，然后使用一个屏幕上的int[]数组显示数据，(几乎)完全最小化了对象的创建.有一些底层线程细节，以确保仅以一致的状态看到屏幕上的阵列.屏幕上的数组用于PixelBuffer的下方，而PixelBuffer则用于WritableImage.
Here's a different example, which (almost) completely minimizes object creation, using one off-screen int[] array to compute data, and one on-screen int[] array to display it. There's a little low-level threading details to ensure the on-screen array is only seen in a consistent state. The on-screen array is used to underly a PixelBuffer, which in turn is used for a WritableImage.
此类生成图像数据:
import java.util.concurrent.atomic.AtomicLong;
import java.util.concurrent.locks.ReentrantLock;
import java.util.function.Consumer;
public class ImageGenerator {
private final int width;
private final int height;
// Keep two copies of the data: one which is not exposed
// that we modify on the fly during computation;
// another which we expose publicly.
// The publicly exposed one can be viewed only in a complete
// state if operations on it are synchronized on this object.
private final int[] privateData ;
private final int[] publicData ;
private final long[] frameTimes ;
private int currentFrameIndex ;
private final AtomicLong averageGenerationTime ;
private final ReentrantLock lock ;
private static final double TWO_PI = 2 * Math.PI;
private static final double PI_BY_TWELVE = Math.PI / 12; // 15 degrees
public ImageGenerator(int width, int height) {
super();
this.width = width;
this.height = height;
privateData = new int[width * height];
publicData = new int[width * height];
lock = new ReentrantLock();
this.frameTimes = new long[100];
this.averageGenerationTime = new AtomicLong();
}
public void generateImage(double angle) {
// compute in private data copy:
int minDim = Math.min(width, height);
int minR2 = minDim * minDim / 4;
for (int x = 0; x < width; x++) {
int xOff = x - width / 2;
int xOff2 = xOff * xOff;
for (int y = 0; y < height; y++) {
int index = x + y * width;
int yOff = y - height / 2;
int yOff2 = yOff * yOff;
int r2 = xOff2 + yOff2;
if (r2 > minR2) {
privateData[index] = 0xffffffff; // white
} else {
double theta = Math.atan2(yOff, xOff);
double delta = Math.abs(theta - angle);
if (delta > TWO_PI - PI_BY_TWELVE) {
delta = TWO_PI - delta;
}
if (delta < PI_BY_TWELVE) {
int green = (int) (255 * (1 - delta / PI_BY_TWELVE));
privateData[index] = (0xff << 24) | (green << 8); // green, fading away from center
} else {
privateData[index] = 0xff << 24; // black
}
}
}
}
// copy computed data to public data copy:
lock.lock();
try {
System.arraycopy(privateData, 0, publicData, 0, privateData.length);
} finally {
lock.unlock();
}
frameTimes[currentFrameIndex] = System.nanoTime() ;
int nextIndex = (currentFrameIndex + 1) % frameTimes.length ;
if (frameTimes[nextIndex] > 0) {
averageGenerationTime.set((frameTimes[currentFrameIndex] - frameTimes[nextIndex]) / frameTimes.length);
}
currentFrameIndex = nextIndex ;
}
public void consumeData(Consumer<int[]> consumer) {
lock.lock();
try {
consumer.accept(publicData);
} finally {
lock.unlock();
}
}
public long getAverageGenerationTime() {
return averageGenerationTime.get() ;
}
}
这是用户界面:
import java.nio.IntBuffer;
import javafx.animation.AnimationTimer;
import javafx.application.Application;
import javafx.scene.Scene;
import javafx.scene.control.Label;
import javafx.scene.image.ImageView;
import javafx.scene.image.PixelFormat;
import javafx.scene.image.PixelWriter;
import javafx.scene.image.WritableImage;
import javafx.scene.layout.BorderPane;
import javafx.stage.Stage;
public class AnimationApp extends Application {
private final int size = 400 ;
private IntBuffer buffer ;
@Override
public void start(Stage primaryStage) throws Exception {
// background image data generation:
ImageGenerator generator = new ImageGenerator(size, size);
// Generate new image data as fast as possible:
Thread thread = new Thread(() -> {
while( true ) {
long now = System.currentTimeMillis() ;
double angle = 2 * Math.PI * (now % 10000) / 10000 - Math.PI;
generator.generateImage(angle);
}
});
thread.setDaemon(true);
thread.start();
generator.consumeData(data -> buffer = IntBuffer.wrap(data));
PixelFormat<IntBuffer> format = PixelFormat.getIntArgbPreInstance() ;
PixelBuffer<IntBuffer> pixelBuffer = new PixelBuffer<>(size, size, buffer, format);
WritableImage image = new WritableImage(pixelBuffer);
BorderPane root = new BorderPane(new ImageView(image));
Label fps = new Label("FPS: ");
root.setTop(fps);
Scene scene = new Scene(root);
primaryStage.setScene(scene);
primaryStage.setTitle("Give me a ping, Vasili. ");
primaryStage.show();
AnimationTimer animation = new AnimationTimer() {
@Override
public void handle(long now) {
// Update image, ensuring we only see the underlying
// data in a consistent state:
generator.consumeData(data -> {
pixelBuffer.updateBuffer(pb -> null);
});
long aveGenTime = generator.getAverageGenerationTime() ;
if (aveGenTime > 0) {
double aveFPS = 1.0 / (aveGenTime / 1_000_000_000.0);
fps.setText(String.format("FPS: %.2f", aveFPS));
}
}
};
animation.start();
}
public static void main(String[] args) {
Application.launch(args);
}
}
对于不依赖JavaFX 13 PixelBuffer的版本，您可以修改该类以使用PixelWriter(AIUI效率不高，但在此示例中运行同样平稳):
For a version that doesn't rely on the JavaFX 13 PixelBuffer, you can just modify this class to use a PixelWriter (AIUI this won't be quite as efficient, but works just as smoothly in this example):
// generator.consumeData(data -> buffer = IntBuffer.wrap(data));
PixelFormat<IntBuffer> format = PixelFormat.getIntArgbPreInstance() ;
// PixelBuffer<IntBuffer> pixelBuffer = new PixelBuffer<>(size, size, buffer, format);
// WritableImage image = new WritableImage(pixelBuffer);
WritableImage image = new WritableImage(size, size);
PixelWriter pixelWriter = image.getPixelWriter() ;
和
AnimationTimer animation = new AnimationTimer() {
@Override
public void handle(long now) {
// Update image, ensuring we only see the underlying
// data in a consistent state:
generator.consumeData(data -> {
// pixelBuffer.updateBuffer(pb -> null);
pixelWriter.setPixels(0, 0, size, size, format, data, 0, size);
});
long aveGenTime = generator.getAverageGenerationTime() ;
if (aveGenTime > 0) {
double aveFPS = 1.0 / (aveGenTime / 1_000_000_000.0);
fps.setText(String.format("FPS: %.2f", aveFPS));
}
}
};
这篇关于如何在JavaFX中大量显示图像?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
