使用qtpaint（qt）或rdyncall（SDL / OpenGL）软件包快速显示光栅图像的R中的高性能2D OpenGL图形？ [英] Performant 2D OpenGL graphics in R for fast display of raster image using qtpaint (qt) or rdyncall (SDL/OpenGL) packages?

问题描述

对于我在R& C中进行的实时交互式Mandelbrot查看器， Rcpp + OpenMP&闪亮我正在寻找一种高性能的方式来显示1920x1080矩阵作为光栅图像，以期能够实现ca. 5-10 fps（计算Mandelbrot图像现在可以达到约20-30 fps适度放大，当然应该快速滚动）。使用 image（）选项 useRaster = TRUE ， plot.raster 甚至> grid.raster（）仍然没有完全切断它，所以我正在寻找更高性能的选项，理想情况下使用OpenGL加速。

我注意到有 qt 包装包 qtutils 和 qtpaint
http://finzi.psych.upenn.edu/R/library/qtutils/html/sceneDevice.html
您可以在其中设置参数 opengl = TRUE 和
http://finzi.psych.upenn.edu/R/library/qtpaint/html/qplotView.html
再次参数 opengl = TRUE 和 http://finzi.psych.upenn.edu/R/library/qtpaint/html/painting.html 。

我也注意到应该可以使用 rdyncall 包调用SDL和GL / OpenGL函数（从 https://cran.r-project.org/src/contrib/Archive/rdyncall/ 以及 https://www.libsdl.org/download-1.2.php ），可在 http://hg.dyncall.org/pub/dyncall/bindings/file/87fd9f34eaa0/R/rdyncall/demo/ 00Index ，例如 http://hg.dyncall.org /pub/dyncall/bindings/file/87fd9f34eaa0/R/rdyncall/demo/randomfield.R ）。

我正确地说，使用这些软件包时，应该能够使用 opengl 加速显示2D图像栅格？如果是这样，有任何想法如何做到这一点（我问，因为我不是专家在 qt 或 SDL / OpenGL ）？

一些非OpenGL选项的时序对于我的应用程序来说太慢了：

 ＃一些示例数据& [0-1]范围数据矩阵的预期颜色映射
 library（RColorBrewer）
 ncol = 1080 
 cols = colorRampPalette（RColorBrewer :: brewer.pal（11，RdYlBu））（ ncol）
 colfun = colorRamp（RColorBrewer :: brewer.pal（11，RdYlBu））
 col = rgb（colfun（seq（0,1，length.out = ncol）），max = 255）
 mat = matrix（seq（1：1080）/ 1080，nrow = 1920，ncol = 1080，byrow = TRUE）
 mat2rast = function（mat，col）{
 idx = findInterval（mat，seq（0，1，length.out = length（col）））
 colours = col [idx] 
 rastmat = t（matrix（colors，ncol = ncol（mat），nrow = brow（mat），byrow = TRUE））
 class（rastmat）=raster
 return（rastmat）
} 
 system.time（mat2rast（mat，col） ）＃0.24s 
 
＃plot.raster方法 - 最好的方法之一？ 
 par（mar = c（0,0,0,0））
 system.time（plot（mat2rast（mat，col），asp = NA））＃0.26s 
 
＃网格图形 - 配合plot.raster？ 
 library（grid）
 system.time（grid.raster（mat2rast（mat，col），interpolate = FALSE））＃0.28s 
 
＃base R image（）
 par（mar = c（0,0,0,0））
 system.time（image（mat，axes = FALSE，useRaster = TRUE，col = cols））＃0.74s＃note Y is与上面的2个选项相比，翻转 - 但不是那么重要，因为我可以按照我希望
 
＃magick  - 浏览器查看器的方式填充矩阵，所以没有什么好处...... 
＃library（magick ）
＃image_read（mat2rast（mat，col））
 
＃imager  - 不会在基本R图形设备上绘图，所以这个图标不会与Shiny 
一起使用＃如果你不必按ESC退出控制到R这
＃可能有一些潜力，虽然... 
 library（imager）
 display（as.cimg（mat2rast（mat ，col）））
 
＃ggplot2  - 仅用于记录... 
 df = expand.grid（y = 1：1080，x = 1：1920）
 df $ z = seq（1,1080）/ 1080 
 library（ggplot2）
 system.time（{q <-qplot（data = df，x = x，y = y，fill = z， geom =raster）+ 
 scale_x_continuous（expand = c（0,0））+ 
 scale_y_continuous（expand = c（0,0））+ 
 scale_fill_gradientn（colors = cols）+ 
 theme_void（）+ theme（legend.position =none）; print（q）}）＃11s 
  

解决方案

RGL包介绍，它是：

< blockquote>

用于R的可视化设备系统，使用OpenGL作为渲染后端。其核心的rgl设备是用C ++编写的实时3D引擎。它提供了一个交互式的视点导航工具（鼠标+车轮支持）和一个R编程接口。

由于RGL是一个实时3D引擎，我预计使用RGL for 2D会给你一个快速的显示。

请注意，这是一个旧的项目，所以我不确定它适合你的

你可以看看本文并查看一些结果图像在这个画廊

For a real-time interactive Mandelbrot viewer I was making in R & Rcpp+OpenMP & Shiny I am on the lookout for a performant way to display 1920x1080 matrices as raster images in the hope of being able to achieve ca. 5-10 fps (calculating the Mandelbrot images themselves now achieves ca. 20-30 fps at moderate zooms, and certainly scrolling around should go fast). Using image() with option useRaster=TRUE, plot.raster or even grid.raster() still doesn't quite cut it, so I am on the lookout for a more performant option, ideally using OpenGL acceleration.

I noticed that there are qt wrapper packages qtutils and qtpaint http://finzi.psych.upenn.edu/R/library/qtutils/html/sceneDevice.html where you can set argument opengl=TRUE and http://finzi.psych.upenn.edu/R/library/qtpaint/html/qplotView.html again with argument opengl=TRUE and http://finzi.psych.upenn.edu/R/library/qtpaint/html/painting.html.

And I also noticed that one should be able to call SDL and GL/OpenGL functions using the rdyncall package (install from https://cran.r-project.org/src/contrib/Archive/rdyncall/ and SDL from https://www.libsdl.org/download-1.2.php)`, demos available at http://hg.dyncall.org/pub/dyncall/bindings/file/87fd9f34eaa0/R/rdyncall/demo/00Index, e.g. http://hg.dyncall.org/pub/dyncall/bindings/file/87fd9f34eaa0/R/rdyncall/demo/randomfield.R).

Am I correct that with these packages one should be able to display a 2D image raster using opengl acceleration? If so, has anyone any thoughts how to do this (I'm asking because I'm not an expert in either qt or SDL/OpenGL)?

Some timings of non-OpenGL options which are too slow for my application:

# some example data & desired colour mapping of [0-1] ranged data matrix
library(RColorBrewer)
ncol=1080
cols=colorRampPalette(RColorBrewer::brewer.pal(11, "RdYlBu"))(ncol)
colfun=colorRamp(RColorBrewer::brewer.pal(11, "RdYlBu"))
col = rgb(colfun(seq(0,1, length.out = ncol)), max = 255)
mat=matrix(seq(1:1080)/1080,nrow=1920,ncol=1080,byrow=TRUE)
mat2rast = function(mat, col) {
  idx = findInterval(mat, seq(0, 1, length.out = length(col)))
  colors = col[idx]
  rastmat = t(matrix(colors, ncol = ncol(mat), nrow = nrow(mat), byrow = TRUE))
  class(rastmat) = "raster"
  return(rastmat)
}
system.time(mat2rast(mat, col)) # 0.24s

# plot.raster method - one of the best?
par(mar=c(0, 0, 0, 0))
system.time(plot(mat2rast(mat, col), asp=NA)) # 0.26s

# grid graphics - tie with plot.raster?
library(grid)
system.time(grid.raster(mat2rast(mat, col),interpolate=FALSE)) # 0.28s

# base R image()
par(mar=c(0, 0, 0, 0))
system.time(image(mat,axes=FALSE,useRaster=TRUE,col=cols)) # 0.74s # note Y is flipped to compared to 2 options above - but not so important as I can fill matrix the way I want

# magick - browser viewer, so no good....
# library(magick)
# image_read(mat2rast(mat, col))

# imager - doesn't plot in base R graphics device, so this one won't work together with Shiny
# If you wouldn't have to press ESC to return control to R this
# might have some potential though...
library(imager)
display(as.cimg(mat2rast(mat, col)))

# ggplot2 - just for the record...
df=expand.grid(y=1:1080,x=1:1920)
df$z=seq(1,1080)/1080
library(ggplot2)
system.time({q <- qplot(data=df,x=x,y=y,fill=z,geom="raster") + 
                scale_x_continuous(expand = c(0,0)) + 
                scale_y_continuous(expand = c(0,0)) +
                scale_fill_gradientn(colours = cols) + 
                theme_void() + theme(legend.position="none"); print(q)}) # 11s 

解决方案

According to the RGL package introduction, it is :

a visualization device system for R, using OpenGL as the rendering backend. An rgl device at its core is a real-time 3D engine written in C++. It provides an interactive viewpoint navigation facility (mouse + wheel support) and an R programming interface.

As RGL is a real time 3D engine, I expect that using RGL for 2D will give you a fast display.

Please note that this is an old project so I am not sure that it fit your requirement.

You can take a look on this paper and see some result images in this gallery

这篇关于使用qtpaint（qt）或rdyncall（SDL / OpenGL）软件包快速显示光栅图像的R中的高性能2D OpenGL图形？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！