如何在Flutter中渲染单个像素? [英] How do I Render Individual Pixels in Flutter?

问题描述

我正在使用自定义的 RenderBox 进行绘制.
以下代码中的canvas对象来自paint方法.

I am using a custom RenderBox to draw.
The canvas object in the code below comes from the PaintingContext in the paint method.

我正在尝试使用

I am trying to render pixels individually by using Canvas.drawRect.
I should point out that these are sometimes larger and sometimes smaller than the pixels on screen they actually occupy.

for (int i = 0; i < width * height; i++) {
  // in this case the rect size is 1
  canvas.drawRect(
      Rect.fromLTWH(index % (width * height), 
          (index / (width * height)).floor(), 1, 1), Paint()..color = colors[i]);
}

存储

我将像素存储为 List<List<Color>> (在上面的代码中为colors).我以前尝试过使用不同的嵌套列表，但是它们在性能方面没有引起任何明显的差异. 当使用999x999图片填充列表时，我的Android模拟器测试设备上的内存增加了282.7MB.请注意，它只是暂时增加282.7MB.大约半分钟后，增加量下降到153.6MB并停留在该位置(无需任何用户交互).

Storage

I am storing the pixels as a List<List<Color>> (colors in the code above). I tried differently nested lists previously, but they did not cause any noticable discrepancies in terms of performance. The memory on my Android Emulator test device increases by 282.7MB when populating the list with a 999x999 image. Note that it only temporarily increases by 282.7MB. After about half a minute, the increase drops to 153.6MB and stays there (without any user interaction).

分辨率为999x999，上面的代码导致 GPU max 为250.1 ms/frame，并且 UI max 为1835.9 ms/frame，这显然是不可接受的.尝试绘制999x999图像时，UI冻结两秒钟，考虑到4k视频在同一设备上的流畅运行，这应该是小菜一碟(我想).

With a resolution of 999x999, the code above causes a GPU max of 250.1 ms/frame and a UI max of 1835.9 ms/frame, which is obviously unacceptable. The UI freezes for two seconds when trying to draw a 999x999 image, which should be a piece of cake (I would guess) considering that 4k video runs smoothly on the same device.

我不确定如何使用Android探查器正确跟踪此问题，但是在 填充或更改列表时，即绘制像素(对于这种情况以上指标)，CPU使用率也从0%上升到60%.这是AVD性能设置:

I am not exactly sure how to track this properly using the Android profiler, but while populating or changing the list, i.e. drawing the pixels (which is the case for the above metrics as well), CPU usage goes from 0% to up to 60%. Here are the AVD performance settings:

我不知道从哪里开始，因为我什至不确定代码的哪一部分会导致冻结.它是内存使用量吗?还是图纸本身?
我一般会如何处理?我究竟做错了什么?我应该如何存储这些像素呢?

I have no idea where to start since I am not even sure what part of my code causes the freezing. Is it the memory usage? Or the drawing itself?
How would I go about this in general? What am I doing wrong? How should I store these pixels instead.

我已经尝试了很多，但都无济于事，我只想指出最引人注目的那些:

I have tried so much that did not help at all that I will try to only point out the most notable ones:

• 我尝试将List<List<Color>>转换为 dart:ui库中的c19> ，希望使用 c21> .为此，我尝试对自己的PNG进行编码，但是我无法渲染多于一行的文字 .但是，它看起来并不能提高性能.尝试转换9999x9999图像时，遇到内存不足异常.现在，我想知道视频的渲染效果如何，因为任何4k视频如果在内存中存储几秒钟，它会比9999x9999图像轻松占用更多的内存.

• I tried converting the List<List<Color>> to an Image from the dart:ui library hoping to use Canvas.drawImage. In order to do that, I tried encoding my own PNG, but I have not been able to render more than a single row. However, it did not look like that would boost performance. When trying to convert a 9999x9999 image, I ran into an out of memory exception. Now, I am wondering how video is rendered as all as any 4k video will easily take up more memory than a 9999x9999 image if a few seconds of it are in memory.

我尝试实现了 image 包.但是，我在完成它之前停了下来，因为我注意到它不是在Flutter中使用，而是在HTML中使用.我不会用它得到任何东西.

I tried implementing the image package. However, I stopped before completing it as I noticed that it is not meant to be used in Flutter but rather in HTML. I would not have gained anything using that.

对于以下得出的结论而言，这一点非常重要:我试图 仅绘制而不存储像素，即使用Random.nextInt产生随机的颜色.尝试随机生成999x999图像时，导致 GPU max 为1824.7 ms/frames，而 UI max 为2362.7 ms/frame，这甚至更糟，尤其是在GPU部门.

This one is pretty important for the following conclusion I will draw: I tried to just draw without storing the pixels, i.e. is using Random.nextInt to generate random colors. When trying to randomly generate a 999x999 image, this resulted in a GPU max of 1824.7 ms/frames and a UI max of 2362.7 ms/frame, which is even worse, especially in the GPU department.

这是我在尝试使用Canvas.drawImage进行失败的尝试之前得出的结论:Canvas.drawRect不能完成此任务，因为它甚至不能绘制简单的图像.

This is the conclusion I reached before trying my failed attempt at rendering using Canvas.drawImage: Canvas.drawRect is not made for this task as it cannot even draw simple images.

您如何在Flutter中做到这一点?

How do you do this in Flutter?

• 这基本上是两个月前我尝试询问的内容(是的，我一直在努力解决这个问题已有很长时间)，但是我认为当时我并没有正确表达自己的意思，而且我什至不知道实际的问题是什么.

• This is basically what I tried to ask over two months ago (yes, I have been trying to resolve this issue for that long), but I think that I did not express myself properly back then and that I knew even less what the actual problem was.

我可以正确渲染的最高分辨率约为10k像素.我至少需要1m.

The highest resolution I can properly render is around 10k pixels. I need at least 1m.

我认为放弃Flutter并转向本机可能是我唯一的选择.但是，我想相信我所面对的这个问题是完全错误的.我花了大约三个月的时间来弄清这个问题，但没有发现任何东西可以引导我到任何地方.

I am thinking that abandoning Flutter and going for native might be my only option. However, I would like to believe that I am just approaching this problem completely wrong. I have spent about three months trying to figure this out and I did not find anything that lead me anywhere.

推荐答案

解决方案

dart:ui具有将像素轻松转换为Image的功能: decodeImageFromPixels

Solution

dart:ui has a function that converts pixels to an Image easily: decodeImageFromPixels

• 示例实现

性能问题

在当前的master频道中无效

Does not work in the current master channel

当我创建此答案时，我根本不知道这个原因，这就是为什么我写了" Alternative "(" Alternative ")部分.

I was simply not aware of this back when I created this answer, which is why I wrote the "Alternative" section.

感谢 @pslink ，让我想起BMP，因为我写了我自己的自己的编码PNG.
我以前曾研究过它，但是我认为如果没有足够的文档，它看起来会很复杂.现在，我发现这篇不错的文章解释了必要的BMP标头并通过复制.我浏览了所有资料，但找不到此示例的原始资料.也许Wikipedia文章的作者是自己写的.

Thanks to @pslink for reminding me of BMP after I wrote that I had failed to encode my own PNG.
I had looked into it previously, but I thought that it looked to complicated without sufficient documentation. Now, I found this nice article explaining the necessary BMP headers and implemented 32-bit BGRA (ARGB but BGRA is the order of the default mask) by copying Example 2 from the "BMP file format" Wikipedia article. I went through all sources but could not find an original source for this example. Maybe the authors of the Wikipedia article wrote it themselves.

使用Canvas.drawImage和我的999x999像素从BMP字节列表转换为图像，我得到的 GPU max 为9.9 ms/frame和 UI max 7.1 ms/frame的strong>，很棒！

Using Canvas.drawImage and my 999x999 pixels converted to an image from a BMP byte list, I get a GPU max of 9.9 ms/frame and a UI max of 7.1 ms/frame, which is awesome!

|  ms/frame |  Before (Canvas.drawRect) | After (Canvas.drawImage) |
|-----------|---------------------------|--------------------------|
|  GPU max  | 1824.7                    | 9.9                      |
|  UI max   | 2362.7                    | 7.1                      |

结论

像Canvas.drawRect这样的画布操作并不意味着那样使用.

Conclusion

Canvas operations like Canvas.drawRect are not meant to be used like that.

首先，这很简单，但是，您需要正确填充字节列表，否则，将得到一个错误，指出您的数据格式不正确并且看不到结果，这可能会令人沮丧

First of, this is quite straight-forward, however, you need to correctly populate the byte list, otherwise, you are going to get an error that your data is not correctly formatted and see no results, which can be quite frustrating.

您将需要在绘制前准备图像 ，因为您无法在绘画调用中使用async操作.

在代码中，您需要使用 Codec 将您的字节列表转换为图像.

In code, you need to use a Codec to transform your list of bytes into an image.

final list = [
            0x42, 0x4d, // 'B', 'M'
            ...];
// make sure that you either know the file size, data size and data offset beforehand
//        or that you edit these bytes afterwards

final Uint8List bytes = Uint8List.fromList(list);

final Codec codec = await instantiateImageCodec(bytes));

final Image image = (await codec.getNextFrame()).image;

您需要将此图像传递到绘图小部件，例如使用 FutureBuilder .
现在，您可以在绘图中使用 Canvas.drawImage 打电话.

You need to pass this image to your drawing widget, e.g. using a FutureBuilder.
Now, you can just use Canvas.drawImage in your draw call.

这篇关于如何在Flutter中渲染单个像素?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！