SKScene 内的 GPUImageView 作为 SKNode 材质 - 在 ARKit 上播放透明视频 [英] GPUImageView inside SKScene as SKNode material - Playing transparent video on ARKit

问题描述

在项目 -A- 中，我使用 GPUImageView 以透明方式显示视频(在绿屏上录制).使用 GPUImageChromaKeyBlendFilter 等.并且工作出色.

In Project -A- I used GPUImageView to display Video (recorded on greenscreen) with transparency. Using the GPUImageChromaKeyBlendFilter, and so on. and works Superb.

另一个基于 ARKIT 的项目 -B- 向我展示了一个带有 VIDEO 的平原，它也可以使用 SKVideoNode 和 AVPlayer 正常工作.

Another project -B- based on ARKIT shows me in the space a plain with VIDEO and it also works fine using SKVideoNode and AVPlayer.

现在的问题是将它们合二为一:) 所以在空间中我想显示视频但具有透明度......不幸的是，我无法在任何 SpriteKit 元素上渲染 GPUImageView，然后添加到 SKScene，这是 SCNPlane 的动画纹理，这可能吗?或者也许还有其他方法可以使用 ARKit 渲染带有透明度的视频.?

Now the question is to combine it all together in one :) So in space I want to display Video but with transparency ... Unfortunately, I can not render a GPUImageView on any SpriteKit element, and then add to SKScene, which is an animated texture for SCNPlane, is it possible at all? Or maybe there is other way to render Video with transparencies with ARKit.?

感谢您的任何建议

推荐答案

我有同样的任务！在此处查看我的解决方案.

I had the same task! See my solution here.

我基本上实现了 ChromaKeyMaterial 来自 href="https://stackoverflow.com/users/6192057/l%C3%ABsha-turkowski">Lësha Turkowski 并编写了此代码来放置和播放视频.

I basically implemented ChromaKeyMaterial from Lësha Turkowski and wrote this code to place and play the video.

import UIKit
import ARKit

class ARTransVC: UIViewController{

@IBOutlet weak var arSceneView: ARSCNView!
let configuration = ARWorldTrackingConfiguration()

private var player: AVPlayer = {
    guard let url = Bundle.main.url(forResource: "FY3A4278", withExtension: "mp4") else { fatalError() }
    return AVPlayer(url: url)
}()

override func viewDidLoad() {
    super.viewDidLoad()
    self.arSceneView.debugOptions = [ARSCNDebugOptions.showWorldOrigin, ARSCNDebugOptions.showFeaturePoints]
    self.arSceneView.session.run(configuration)

    //a delay for ARKit to capture the surroundings
    DispatchQueue.main.asyncAfter(deadline: .now() + 3) {

        // A SpriteKit scene to contain the SpriteKit video node
        let spriteKitScene = SKScene(size: CGSize(width: self.arSceneView.frame.width, height: self.arSceneView.frame.height))
        spriteKitScene.scaleMode = .aspectFit
        spriteKitScene.backgroundColor = .clear
        spriteKitScene.scaleMode = .aspectFit

        //Create the SpriteKit video node, containing the video player
        let videoSpriteKitNode = SKVideoNode(avPlayer: self.player)
        videoSpriteKitNode.position = CGPoint(x: spriteKitScene.size.width / 2.0, y: spriteKitScene.size.height / 2.0)
        videoSpriteKitNode.size = spriteKitScene.size
        videoSpriteKitNode.yScale = -1.0
        videoSpriteKitNode.play()
        spriteKitScene.addChild(videoSpriteKitNode)

        // To make the video loop
        self.player.actionAtItemEnd = .none
        NotificationCenter.default.addObserver(
            self,
            selector: #selector(ARTransVC.playerItemDidReachEnd),
            name: NSNotification.Name.AVPlayerItemDidPlayToEndTime,
            object:  self.player.currentItem)

        // Create the SceneKit scene
        let scene = SCNScene()
        self.arSceneView.scene = scene

        //Create a SceneKit plane and add the SpriteKit scene as its material
        let background = SCNPlane(width: CGFloat(1), height: CGFloat(1))
        background.firstMaterial?.diffuse.contents = spriteKitScene
        let chromaKeyMaterial = ChromaKeyMaterial()
        chromaKeyMaterial.diffuse.contents = self.player

        let backgroundNode = SCNNode(geometry: background)
        backgroundNode.geometry?.firstMaterial?.isDoubleSided = true
        backgroundNode.geometry!.materials = [chromaKeyMaterial]

        backgroundNode.position = SCNVector3(0,0,-2.0)
        scene.rootNode.addChildNode(backgroundNode)

        //video does not start without delaying the player
        //playing the video before just results in [SceneKit] Error: Cannot get pixel buffer (CVPixelBufferRef)
        DispatchQueue.main.asyncAfter(deadline: .now() + 1) {
            self.player.seek(to:CMTimeMakeWithSeconds(1, 1000))
            self.player.play()
        }
    }
}

@objc func playerItemDidReachEnd(notification: NSNotification) {
    if let playerItem: AVPlayerItem = notification.object as? AVPlayerItem {
        playerItem.seek(to: kCMTimeZero, completionHandler: nil)
    }
}

这篇关于SKScene 内的 GPUImageView 作为 SKNode 材质 - 在 ARKit 上播放透明视频的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！