将Flutter动画直接渲染到视频 [英] Render Flutter animation directly to video

问题描述

考虑到Flutter使用自己的图形引擎，是否可以将Flutter动画直接呈现到视频中，或者以逐帧的方式创建屏幕截图?

Considering that Flutter uses its own graphics engine, is there a way to render Flutter animations directly to video, or create screenshots in a frame by frame fashion?

一个用例是，这样可以使观众更容易地进行演示.

One use case would be that this allows easier demonstration for the audience.

例如，某位作者想要创建Flutter动画教程，在该教程中，他们将使用直接由Flutter渲染的GIF动画/视频来构建演示应用程序并撰写同伴博客文章.

For example, an author wants to create a Flutter animation tutorial, where they builds a demo app and writes a companion blog post, using the animation GIF/videos rendered directly with Flutter.

另一个例子是，UI团队之外的一名开发人员发现复杂的动画中几乎没有错误.他们无需实际学习动画代码，就可以将动画渲染为视频并使用批注编辑该短片，然后将其发送给UI团队进行诊断.

Another example would be one developer outside the UI team discovers a complex animation has tiny mistake in it. Without actually learning the animation code, they can render the animation into a video and edit that short clip with annotations, then send it to the UI team for diagnoses.

推荐答案

虽然不漂亮，但是我设法使原型工作了.首先，所有动画都需要由单个主动画控制器驱动，以便我们可以逐步进入所需动画的任何部分.其次，我们要记录的小部件树必须用全局键包装在 RepaintBoundary 中.RepaintBoundary及其键可以生成小部件树的快照，如下所示:

It's not pretty, but I have managed to get a prototype working. Firstly, all animations need to be powered by a single main animation controller so that we can step through to any part of the animation we want. Secondly, the widget tree that we want to record has to be wrapped inside a RepaintBoundary with a global key. The RepaintBoundary and it's key can produce snapshots of the widget tree as such:

Future<Uint8List> _capturePngToUint8List() async {
    // renderBoxKey is the global key of my RepaintBoundary
    RenderRepaintBoundary boundary = renderBoxKey.currentContext.findRenderObject(); 
    
    // pixelratio allows you to render it at a higher resolution than the actual widget in the application.
    ui.Image image = await boundary.toImage(pixelRatio: 2.0);
    ByteData byteData = await image.toByteData(format: ui.ImageByteFormat.png);
    Uint8List pngBytes = byteData.buffer.asUint8List();

    return pngBytes;
  }

然后可以在循环中使用上述方法，该循环将小部件树捕获为pngBytes，并按照所需的帧速率指定的deltaT向前移动animationController:

The above method can then be used inside a loop which captures the widget tree into pngBytes, and steps the animationController forward by a deltaT specified by the framerate you want as such:

double t = 0;
int i = 1;

setState(() {
  animationController.value = 0.0;
});

Map<int, Uint8List> frames = {};
double dt = (1 / 60) / animationController.duration.inSeconds.toDouble();

while (t <= 1.0) {
  print("Rendering... ${t * 100}%");
  var bytes = await _capturePngToUint8List();
  frames[i] = bytes;

  t += dt;
  setState(() {
    animationController.value = t;
  });
  i++;
}

最后，所有这些png帧都可以通过管道传输到ffmpeg子进程中，然后再写入视频中.我还没有设法使这部分工作正常，所以我要做的是将所有png帧写到实际的png文件中，然后在编写它们的文件夹内手动运行ffmpeg.(注意:我已经使用flutter桌面访问了ffmpeg的安装，但是在发布商上有一个一个软件包.dev也可以在移动设备上获取ffmpeg )

Finally, all these png frames can be piped into an ffmpeg subprocess to be written into a video. I haven't managed to get this part working nicely yet, so what I've done instead is write out all the png-frames into actual png files and I then manually run ffmpeg inside the folder where they are written. (Note: I've used flutter desktop to be able to access my installation of ffmpeg, but there is a package on pub.dev to get ffmpeg on mobile too)

List<Future<File>> fileWriterFutures = [];

frames.forEach((key, value) {
  fileWriterFutures.add(_writeFile(bytes: value, location: r"D:\path\to\my\images\folder\" + "frame_$key.png"));
});

await Future.wait(fileWriterFutures);

_runFFmpeg();

这是我的文件编写器帮助功能:

Here is my file-writer help-function:

Future<File> _writeFile({@required String location, @required Uint8List bytes}) async {
  File file = File(location);
  return file.writeAsBytes(bytes);
}

这是我的FFmpeg运行器功能:

And here is my FFmpeg runner function:

void _runFFmpeg() async {
  // ffmpeg -y -r 60 -start_number 1 -i frame_%d.png -c:v libx264 -preset medium -tune animation -pix_fmt yuv420p test.mp4
  var process = await Process.start(
      "ffmpeg",
      [
        "-y", // replace output file if it already exists
        "-r", "60", // framrate
        "-start_number", "1",
        "-i", r"./test/frame_%d.png", // <- Change to location of images
        "-an", // don't expect audio
        "-c:v", "libx264rgb", // H.264 encoding
        "-preset", "medium",
        "-crf",
        "10", // Ranges 0-51 indicates lossless compression to worst compression. Sane options are 0-30
        "-tune", "animation",
        "-preset", "medium",
        "-pix_fmt", "yuv420p",
        r"./test/test.mp4" // <- Change to location of output
      ],
      mode: ProcessStartMode.inheritStdio // This mode causes some issues at times, so just remove it if it doesn't work. I use it mostly to debug the ffmpeg process' output
   );

  print("Done Rendering");
}

这篇关于将Flutter动画直接渲染到视频的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！