如何在 Flutter 中渲染单个像素? [英] How do I Render Individual Pixels in Flutter?

问题描述

设置

我正在使用自定义

原因

我不知道从哪里开始，因为我什至不确定代码的哪一部分导致了冻结.是内存占用吗?还是图纸本身?
我一般会怎么做?我究竟做错了什么?我应该如何存储这些像素.

努力

我尝试了很多都没有帮助，所以我将尝试只指出最显着的:

• 我尝试将 List> 转换为 Image 来自 dart:ui 库 希望使用 Canvas.drawImage.为了做到这一点，我尝试对自己的 PNG 进行编码，但是 我无法渲染多于一行.但是，看起来这不会提高性能.在尝试转换 9999x9999 图像时，我遇到了内存不足异常.现在，我想知道视频是如何渲染的，因为任何 4k 视频都比 9999x9999 图像更容易占用更多内存，如果它的几秒钟在内存中.

• 我尝试实施 image 包.然而，我在完成它之前停下来，因为我注意到它并不打算在 Flutter 中使用，而是在 HTML 中使用.我不会用它获得任何东西.

• 这对于我将得出的以下结论非常重要:我尝试只绘制而不存储像素，即使用 Random.nextInt 生成随机颜色.当尝试随机生成 999x999 图像时，这导致 GPU 最大值 1824.7 毫秒/帧 和 UI 最大值2362.7 ms/frame，更差，尤其是在GPU部门.

结论

这是我在尝试使用 Canvas.drawImage 进行渲染失败之前得出的结论:Canvas.drawRect 不是为此任务制作的，因为它甚至无法绘制简单的图片.

你如何在 Flutter 中做到这一点?

注意事项

• 这基本上是两个月前我尝试问的问题(是的，我一直在努力解决这个问题)，但我认为当时我没有正确表达自己，我更不知道实际问题是什么.

• 我可以正确渲染的最高分辨率约为 10k 像素.我至少需要1m.

• 我认为放弃 Flutter 转而使用 Native 可能是我唯一的选择.但是，我想相信我只是完全错误地解决了这个问题.我花了大约三个月的时间试图弄清楚这一点，但我没有找到任何可以引导我到任何地方的东西.

解决方案

解决方案

dart:ui 有一个函数可以轻松地将像素转换为 Image:decodeImageFromPixels

• 示例实现

• 性能问题

• 在当前的master 频道

我在创建这个答案时根本没有意识到这一点，这就是我写下替代方案"的原因部分.

替代方案

感谢 @pslink 在我写完失败后提醒我BMP编码我自己的PNG.
我以前研究过它，但我认为它在没有足够文档的情况下看起来很复杂.现在，我发现这篇不错的文章解释了必要的BMP 标头并通过复制 来自BMP 文件格式"的示例 2维基百科文章.我浏览了所有来源，但找不到此示例的原始来源.也许维基百科文章的作者自己写的.

结果

使用 Canvas.drawImage 和我的 999x999 像素从 BMP 字节列表转换为图像，我得到了 GPU 最大值 9.9 ms/frame 和 UI 最大值 7.1 ms/frame，太棒了！

|毫秒/帧 |之前 (Canvas.drawRect) |之后 (Canvas.drawImage) ||-----------|---------------------------|--------------------------||最大 GPU |1824.7 |9.9 ||用户界面最大值 |2362.7 |7.1 |

结论

像 Canvas.drawRect 这样的画布操作不应该这样使用.

说明

首先，这是非常简单的，但是，您需要正确填充字节列表，否则，您将收到数据格式不正确且看不到结果的错误，这可能会令人非常沮丧.

您需要在绘制之前准备图像，因为您不能在绘制调用中使用 async 操作.

在代码中，需要使用Codec 将您的字节列表转换为图像.

final list = [0x42, 0x4d,//'B', 'M'...];//确保您事先知道文件大小、数据大小和数据偏移量//或者你之后编辑这些字节最终 Uint8List 字节 = Uint8List.fromList(list);最终编解码器编解码器 = await instantiateImageCodec(bytes));最终图像图像 = (await codec.getNextFrame()).image;

您需要将此图像传递给您的绘图小部件，例如使用 FutureBuilder.
现在，您只需使用 Canvas.drawImage 在您的绘图调用中.

Setup

I am using a custom RenderBox to draw.
The canvas object in the code below comes from the PaintingContext in the paint method.

Drawing

I am trying to render pixels individually by using Canvas.drawRect.
I should point out that these are sometimes larger and sometimes smaller than the pixels on screen they actually occupy.

for (int i = 0; i < width * height; i++) {
  // in this case the rect size is 1
  canvas.drawRect(
      Rect.fromLTWH(index % (width * height), 
          (index / (width * height)).floor(), 1, 1), Paint()..color = colors[i]);
}

Storage

I am storing the pixels as a List<List<Color>> (colors in the code above). I tried differently nested lists previously, but they did not cause any noticable discrepancies in terms of performance. The memory on my Android Emulator test device increases by 282.7MB when populating the list with a 999x999 image. Note that it only temporarily increases by 282.7MB. After about half a minute, the increase drops to 153.6MB and stays there (without any user interaction).

Rendering

With a resolution of 999x999, the code above causes a GPU max of 250.1 ms/frame and a UI max of 1835.9 ms/frame, which is obviously unacceptable. The UI freezes for two seconds when trying to draw a 999x999 image, which should be a piece of cake (I would guess) considering that 4k video runs smoothly on the same device.

CPU

I am not exactly sure how to track this properly using the Android profiler, but while populating or changing the list, i.e. drawing the pixels (which is the case for the above metrics as well), CPU usage goes from 0% to up to 60%. Here are the AVD performance settings:

Cause

I have no idea where to start since I am not even sure what part of my code causes the freezing. Is it the memory usage? Or the drawing itself?
How would I go about this in general? What am I doing wrong? How should I store these pixels instead.

Efforts

I have tried so much that did not help at all that I will try to only point out the most notable ones:

• I tried converting the List<List<Color>> to an Image from the dart:ui library hoping to use Canvas.drawImage. In order to do that, I tried encoding my own PNG, but I have not been able to render more than a single row. However, it did not look like that would boost performance. When trying to convert a 9999x9999 image, I ran into an out of memory exception. Now, I am wondering how video is rendered as all as any 4k video will easily take up more memory than a 9999x9999 image if a few seconds of it are in memory.

• I tried implementing the image package. However, I stopped before completing it as I noticed that it is not meant to be used in Flutter but rather in HTML. I would not have gained anything using that.

• This one is pretty important for the following conclusion I will draw: I tried to just draw without storing the pixels, i.e. is using Random.nextInt to generate random colors. When trying to randomly generate a 999x999 image, this resulted in a GPU max of 1824.7 ms/frames and a UI max of 2362.7 ms/frame, which is even worse, especially in the GPU department.

Conclusion

This is the conclusion I reached before trying my failed attempt at rendering using Canvas.drawImage: Canvas.drawRect is not made for this task as it cannot even draw simple images.

How do you do this in Flutter?

Notes

• This is basically what I tried to ask over two months ago (yes, I have been trying to resolve this issue for that long), but I think that I did not express myself properly back then and that I knew even less what the actual problem was.

• The highest resolution I can properly render is around 10k pixels. I need at least 1m.

• I am thinking that abandoning Flutter and going for native might be my only option. However, I would like to believe that I am just approaching this problem completely wrong. I have spent about three months trying to figure this out and I did not find anything that lead me anywhere.

解决方案

Solution

dart:ui has a function that converts pixels to an Image easily: decodeImageFromPixels

• Example implementation

• Issue on performance

• Does not work in the current master channel

I was simply not aware of this back when I created this answer, which is why I wrote the "Alternative" section.

Alternative

Thanks to @pslink for reminding me of BMP after I wrote that I had failed to encode my own PNG.
I had looked into it previously, but I thought that it looked to complicated without sufficient documentation. Now, I found this nice article explaining the necessary BMP headers and implemented 32-bit BGRA (ARGB but BGRA is the order of the default mask) by copying Example 2 from the "BMP file format" Wikipedia article. I went through all sources but could not find an original source for this example. Maybe the authors of the Wikipedia article wrote it themselves.

Results

Using Canvas.drawImage and my 999x999 pixels converted to an image from a BMP byte list, I get a GPU max of 9.9 ms/frame and a UI max of 7.1 ms/frame, which is awesome!

|  ms/frame |  Before (Canvas.drawRect) | After (Canvas.drawImage) |
|-----------|---------------------------|--------------------------|
|  GPU max  | 1824.7                    | 9.9                      |
|  UI max   | 2362.7                    | 7.1                      |

Conclusion

Canvas operations like Canvas.drawRect are not meant to be used like that.

Instructions

First of, this is quite straight-forward, however, you need to correctly populate the byte list, otherwise, you are going to get an error that your data is not correctly formatted and see no results, which can be quite frustrating.

You will need to prepare your image before drawing as you cannot use async operations in the paint call.

In code, you need to use a Codec to transform your list of bytes into an image.

final list = [
            0x42, 0x4d, // 'B', 'M'
            ...];
// make sure that you either know the file size, data size and data offset beforehand
//        or that you edit these bytes afterwards

final Uint8List bytes = Uint8List.fromList(list);

final Codec codec = await instantiateImageCodec(bytes));

final Image image = (await codec.getNextFrame()).image;

You need to pass this image to your drawing widget, e.g. using a FutureBuilder.
Now, you can just use Canvas.drawImage in your draw call.

这篇关于如何在 Flutter 中渲染单个像素?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！