SKScene中的GPUImageView作为SKNode素材-在ARKit上播放透明视频 [英] GPUImageView inside SKScene as SKNode material - Playing transparent video on ARKit

问题描述

在项目-A-中，我使用了GPUImageView来透明显示视频(记录在绿屏上).使用GPUImageChromaKeyBlendFilter，依此类推. 并表现出色.

In Project -A- I used GPUImageView to display Video (recorded on greenscreen) with transparency. Using the GPUImageChromaKeyBlendFilter, and so on. and works Superb.

另一个基于ARKIT的项目-B-向我展示了使用VIDEO的平原，使用SKVideoNode和AVPlayer也可以正常工作.

Another project -B- based on ARKIT shows me in the space a plain with VIDEO and it also works fine using SKVideoNode and AVPlayer.

现在的问题是将所有内容组合成一个:)所以在太空中，我想显示视频但要透明... 不幸的是，我无法在任何SpriteKit元素上渲染GPUImageView，然后将其添加到SKScene(这是SCNPlane的动画纹理)，有可能吗?也许还有其他方法可以通过ARKit呈现具有透明胶片的视频.

Now the question is to combine it all together in one :) So in space I want to display Video but with transparency ... Unfortunately, I can not render a GPUImageView on any SpriteKit element, and then add to SKScene, which is an animated texture for SCNPlane, is it possible at all? Or maybe there is other way to render Video with transparencies with ARKit.?

感谢任何建议

推荐答案

我有相同的任务！在此处查看我的解决方案.

I had the same task! See my solution here.

我基本上实现了 ChromaKeyMaterial href ="https://stackoverflow.com/users/6192057/l%C3%ABsha-turkowski">LëshaTurkowski 并编写了这段代码来放置和播放视频.

I basically implemented ChromaKeyMaterial from Lësha Turkowski and wrote this code to place and play the video.

import UIKit
import ARKit

class ARTransVC: UIViewController{

@IBOutlet weak var arSceneView: ARSCNView!
let configuration = ARWorldTrackingConfiguration()

private var player: AVPlayer = {
    guard let url = Bundle.main.url(forResource: "FY3A4278", withExtension: "mp4") else { fatalError() }
    return AVPlayer(url: url)
}()

override func viewDidLoad() {
    super.viewDidLoad()
    self.arSceneView.debugOptions = [ARSCNDebugOptions.showWorldOrigin, ARSCNDebugOptions.showFeaturePoints]
    self.arSceneView.session.run(configuration)

    //a delay for ARKit to capture the surroundings
    DispatchQueue.main.asyncAfter(deadline: .now() + 3) {

        // A SpriteKit scene to contain the SpriteKit video node
        let spriteKitScene = SKScene(size: CGSize(width: self.arSceneView.frame.width, height: self.arSceneView.frame.height))
        spriteKitScene.scaleMode = .aspectFit
        spriteKitScene.backgroundColor = .clear
        spriteKitScene.scaleMode = .aspectFit

        //Create the SpriteKit video node, containing the video player
        let videoSpriteKitNode = SKVideoNode(avPlayer: self.player)
        videoSpriteKitNode.position = CGPoint(x: spriteKitScene.size.width / 2.0, y: spriteKitScene.size.height / 2.0)
        videoSpriteKitNode.size = spriteKitScene.size
        videoSpriteKitNode.yScale = -1.0
        videoSpriteKitNode.play()
        spriteKitScene.addChild(videoSpriteKitNode)

        // To make the video loop
        self.player.actionAtItemEnd = .none
        NotificationCenter.default.addObserver(
            self,
            selector: #selector(ARTransVC.playerItemDidReachEnd),
            name: NSNotification.Name.AVPlayerItemDidPlayToEndTime,
            object:  self.player.currentItem)

        // Create the SceneKit scene
        let scene = SCNScene()
        self.arSceneView.scene = scene

        //Create a SceneKit plane and add the SpriteKit scene as its material
        let background = SCNPlane(width: CGFloat(1), height: CGFloat(1))
        background.firstMaterial?.diffuse.contents = spriteKitScene
        let chromaKeyMaterial = ChromaKeyMaterial()
        chromaKeyMaterial.diffuse.contents = self.player

        let backgroundNode = SCNNode(geometry: background)
        backgroundNode.geometry?.firstMaterial?.isDoubleSided = true
        backgroundNode.geometry!.materials = [chromaKeyMaterial]

        backgroundNode.position = SCNVector3(0,0,-2.0)
        scene.rootNode.addChildNode(backgroundNode)

        //video does not start without delaying the player
        //playing the video before just results in [SceneKit] Error: Cannot get pixel buffer (CVPixelBufferRef)
        DispatchQueue.main.asyncAfter(deadline: .now() + 1) {
            self.player.seek(to:CMTimeMakeWithSeconds(1, 1000))
            self.player.play()
        }
    }
}

@objc func playerItemDidReachEnd(notification: NSNotification) {
    if let playerItem: AVPlayerItem = notification.object as? AVPlayerItem {
        playerItem.seek(to: kCMTimeZero, completionHandler: nil)
    }
}

这篇关于SKScene中的GPUImageView作为SKNode素材-在ARKit上播放透明视频的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！