使用WebRTC通过ReplayKit发送iOS设备的屏幕截图 [英] Using WebRTC to send an iOS devices’ screen capture using ReplayKit

问题描述

我们想使用WebRTC通过ReplayKit发送iOS设备的屏幕截图. ReplayKit具有一个processSampleBuffer回调，该回调提供了CMSampleBuffer.

We would like to use WebRTC to send an iOS devices’ screen capture using ReplayKit. The ReplayKit has a processSampleBuffer callback which gives CMSampleBuffer.

但是，这就是我们所困的地方，我们似乎无法将CMSampleBuffer发送到连接的对等方. 我们尝试从sampleBuffer创建pixelBuffer，然后创建RTCVideoFrame.

But here is where we are stuck, we can’t seem to get the CMSampleBuffer to be sent to the connected peer. We have tried to create pixelBuffer from the sampleBuffer, and then create RTCVideoFrame.

我们还从RTCPeerConnectionFactory中提取了RTCVideoSource，然后使用RTCVideoCapturer并将其流式传输到localVideoSource.

we also extracted the RTCVideoSource from RTCPeerConnectionFactory and then used an RTCVideoCapturer and stream it to the localVideoSource.

知道我们在做什么错吗?

Any idea what we are doing wrong?

var peerConnectionFactory: RTCPeerConnectionFactory?

override func processSampleBuffer(_ sampleBuffer: CMSampleBuffer, with sampleBufferType: RPSampleBufferType) {
 switch sampleBufferType {
           case RPSampleBufferType.video:

        // create the CVPixelBuffer
        let pixelBuffer:CVPixelBuffer = CMSampleBufferGetImageBuffer(sampleBuffer)!;

        // create the RTCVideoFrame
        var videoFrame:RTCVideoFrame?;
        let timestamp = NSDate().timeIntervalSince1970 * 1000
        videoFrame = RTCVideoFrame(pixelBuffer: pixelBuffer, rotation: RTCVideoRotation._0, timeStampNs: Int64(timestamp))

        // connect the video frames to the WebRTC
        let localVideoSource = self.peerConnectionFactory!.videoSource()
        let videoCapturer = RTCVideoCapturer()
        localVideoSource.capturer(videoCapturer, didCapture: videoFrame!)

        let videoTrack : RTCVideoTrack =   self.peerConnectionFactory!.videoTrack(with: localVideoSource, trackId: "100")

        let mediaStream: RTCMediaStream = (self.peerConnectionFactory?.mediaStream(withStreamId: "1"))!
        mediaStream.addVideoTrack(videoTrack)
        self.newPeerConnection!.add(mediaStream)

        break
    }
}

推荐答案

这是实现您的好主意，只需使用代码段中使用的方法呈现 RTCVideoFrame ，然后所有其他对象都将初始化超出方法的最佳方法.为了更好地理解我，我给你一个片段.

This is a great idea to implement you just have to render the RTCVideoFrame in the method that you have used in the snippet, and all the other object will initialize outsize the method, best way. for better understanding, I am giving you a snippet.

    var peerConnectionFactory: RTCPeerConnectionFactory?
    var localVideoSource: RTCVideoSource?
    var videoCapturer: RTCVideoCapturer?
    func setupVideoCapturer(){
          // localVideoSource and videoCapturer will use 
            localVideoSource = self.peerConnectionFactory!.videoSource() 
            videoCapturer = RTCVideoCapturer()
    //      localVideoSource.capturer(videoCapturer, didCapture: videoFrame!)

            let videoTrack : RTCVideoTrack =   self.peerConnectionFactory!.videoTrack(with: localVideoSource, trackId: "100")

            let mediaStream: RTCMediaStream = (self.peerConnectionFactory?.mediaStream(withStreamId: "1"))!
            mediaStream.addVideoTrack(videoTrack)
            self.newPeerConnection!.add(mediaStream)
        }


 override func processSampleBuffer(_ sampleBuffer: CMSampleBuffer, with sampleBufferType: RPSampleBufferType) {
     switch sampleBufferType {
               case RPSampleBufferType.video:

            // create the CVPixelBuffer
            let pixelBuffer:CVPixelBuffer = CMSampleBufferGetImageBuffer(sampleBuffer)!;

            // create the RTCVideoFrame
            var videoFrame:RTCVideoFrame?;
            let timestamp = NSDate().timeIntervalSince1970 * 1000
            videoFrame = RTCVideoFrame(pixelBuffer: pixelBuffer, rotation: RTCVideoRotation._0, timeStampNs: Int64(timestamp))
            // connect the video frames to the WebRTC
            localVideoSource.capturer(videoCapturer, didCapture: videoFrame!)

            break
        }
    }

希望这会对您有所帮助.

Hope this will help you.

这篇关于使用WebRTC通过ReplayKit发送iOS设备的屏幕截图的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！