在播放iOS时从HLS流（视频）中提取/录制音频 [英] Extract/Record Audio from HLS stream (video) while playing iOS
问题描述
我正在使用AVPlayer播放HLS流。我还需要在用户按下记录按钮时记录这些流。
我使用的方法是分别录制音频和视频，然后在最后合并这些文件以制作最终视频。并且它对远程mp4文件很成功。
I am playing HLS streams using AVPlayer. And I also need to record these streams as user presses record button. The approach I am using is to record audio and video separately then at the end merge these file to make the final video. And It is successful with remote mp4 files.
但是现在对于HLS（.m3u8）文件，我能够使用AVAssetWriter录制视频但是录音有问题。
But now for the HLS (.m3u8) files I am able to record the video using AVAssetWriter but having problems with audio recording.
我正在使用MTAudioProccessingTap来处理原始音频数据并将其写入文件。我跟着这篇文章。我能够录制远程mp4音频但不能使用HLS流。
最初我无法使用AVAssetTrack * audioTrack = [asset tracksWithMediaType：AVMediaTypeAudio] [0]从流中提取音轨。
I am using MTAudioProccessingTap to process the raw audio data and write it to a file. I followed this article. I am able to record remote mp4 audio but its not working with HLS streams. Initially I wasn't able to extract the audio tracks from the stream using AVAssetTrack *audioTrack = [asset tracksWithMediaType:AVMediaTypeAudio][0];
但是我能够使用KVO提取audioTracks来初始化MTAudioProcessingTap。
But I was able to extract the audioTracks using KVO to initialize the MTAudioProcessingTap.
-(void)observeValueForKeyPath:(NSString *)keyPath ofObject:(id)object change:(NSDictionary *)change context:(void *)context{
AVPlayer *player = (AVPlayer*) object;
if (player.status == AVPlayerStatusReadyToPlay)
{
NSLog(@"Ready to play");
self.previousAudioTrackID = 0;
__weak typeof (self) weakself = self;
timeObserverForTrack = [player addPeriodicTimeObserverForInterval:CMTimeMakeWithSeconds(1, 100) queue:nil usingBlock:^(CMTime time)
{
@try {
for(AVPlayerItemTrack* track in [weakself.avPlayer.currentItem tracks]) {
if([track.assetTrack.mediaType isEqualToString:AVMediaTypeAudio])
weakself.currentAudioPlayerItemTrack = track;
}
AVAssetTrack* audioAssetTrack = weakself.currentAudioPlayerItemTrack.assetTrack;
weakself.currentAudioTrackID = audioAssetTrack.trackID;
if(weakself.previousAudioTrackID != weakself.currentAudioTrackID) {
NSLog(@":::::::::::::::::::::::::: Audio track changed : %d",weakself.currentAudioTrackID);
weakself.previousAudioTrackID = weakself.currentAudioTrackID;
weakself.audioTrack = audioAssetTrack;
/// Use this audio track to initialize MTAudioProcessingTap
}
}
@catch (NSException *exception) {
NSLog(@"Exception Trap ::::: Audio tracks not found!");
}
}];
}
}
我也是跟踪trackID以检查曲目是否已更改。
I am also keeping track of trackID to check if track is changed.
这是我初始化MTAudioProcessingTap的方法。
This is how I initialize the MTAudioProcessingTap.
-(void)beginRecordingAudioFromTrack:(AVAssetTrack *)audioTrack{
// Configure an MTAudioProcessingTap to handle things.
MTAudioProcessingTapRef tap;
MTAudioProcessingTapCallbacks callbacks;
callbacks.version = kMTAudioProcessingTapCallbacksVersion_0;
callbacks.clientInfo = (__bridge void *)(self);
callbacks.init = init;
callbacks.prepare = prepare;
callbacks.process = process;
callbacks.unprepare = unprepare;
callbacks.finalize = finalize;
OSStatus err = MTAudioProcessingTapCreate(
kCFAllocatorDefault,
&callbacks,
kMTAudioProcessingTapCreationFlag_PostEffects,
&tap
);
if(err) {
NSLog(@"Unable to create the Audio Processing Tap %d", (int)err);
NSError *error = [NSError errorWithDomain:NSOSStatusErrorDomain
code:err
userInfo:nil];
NSLog(@"Error: %@", [error description]);;
return;
}
// Create an AudioMix and assign it to our currently playing "item", which
// is just the stream itself.
AVMutableAudioMix *audioMix = [AVMutableAudioMix audioMix];
AVMutableAudioMixInputParameters *inputParams = [AVMutableAudioMixInputParameters
audioMixInputParametersWithTrack:audioTrack];
inputParams.audioTapProcessor = tap;
audioMix.inputParameters = @[inputParams];
_audioPlayer.currentItem.audioMix = audioMix;
}
但现在有了这个音频轨道MTAudioProcessingTap回调准备和处理是从未打电话过
But Now with this audio track MTAudioProcessingTap callbacks "Prepare" and "Process" are never called.
我是通过KVO获得audioTrack的问题吗？
Is the problem with the audioTrack I am getting through KVO?
现在我真的很感激，如果有的话可以帮助我。或者可以告诉我是否使用写入方法来记录HLS流？
Now I would really appreciate if some one can help me with this. Or can tell am I using the write approach to record HLS Streams?
推荐答案
我找到了解决方案并在我的应用中使用它。想早点发布，但没有时间。
I Found solution for this and using it in my app. Wanted to post it earlier but didn't get the time.
因此，要使用HLS，您应该知道它们究竟是什么。为此，请在Apple网站上查看。
HLS Apple
So to play with HLS you should have some knowledge what they are exactly. For that please see it here on Apple Website. HLS Apple
以下是我的步骤以下。
1.首先获取m3u8并解析它。
您可以使用这个有用的工具包 M3U8Kit 来解析它。
使用此套件，您可以获得M3U8MediaPlaylist或M3U8MasterPlaylist（如果它是主播放列表）
如果您获得主播放列表，您也可以解析它以获得M3U8MediaPlaylist
Here are the steps I am following. 1. First get the m3u8 and parse it. You can parse it using this helpful kit M3U8Kit. Using this kit you can get the M3U8MediaPlaylist or M3U8MasterPlaylist(if it is a master playlist) if you get the master playlist you can also parse it to get M3U8MediaPlaylist
(void) parseM3u8
{
NSString *plainString = [self.url m3u8PlanString];
BOOL isMasterPlaylist = [plainString isMasterPlaylist];
NSError *error;
NSURL *baseURL;
if(isMasterPlaylist)
{
M3U8MasterPlaylist *masterList = [[M3U8MasterPlaylist alloc] initWithContentOfURL:self.url error:&error];
self.masterPlaylist = masterList;
M3U8ExtXStreamInfList *xStreamInfList = masterList.xStreamList;
M3U8ExtXStreamInf *StreamInfo = [xStreamInfList extXStreamInfAtIndex:0];
NSString *URI = StreamInfo.URI;
NSRange range = [URI rangeOfString:@"dailymotion.com"];
NSString *baseURLString = [URI substringToIndex:(range.location+range.length)];
baseURL = [NSURL URLWithString:baseURLString];
plainString = [[NSURL URLWithString:URI] m3u8PlanString];
}
M3U8MediaPlaylist *mediaPlaylist = [[M3U8MediaPlaylist alloc] initWithContent:plainString baseURL:baseURL];
self.mediaPlaylist = mediaPlaylist;
M3U8SegmentInfoList *segmentInfoList = mediaPlaylist.segmentList;
NSMutableArray *segmentUrls = [[NSMutableArray alloc] init];
for (int i = 0; i < segmentInfoList.count; i++)
{
M3U8SegmentInfo *segmentInfo = [segmentInfoList segmentInfoAtIndex:i];
NSString *segmentURI = segmentInfo.URI;
NSURL *mediaURL = [baseURL URLByAppendingPathComponent:segmentURI];
[segmentUrls addObject:mediaURL];
if(!self.segmentDuration)
self.segmentDuration = segmentInfo.duration;
}
self.segmentFilesURLs = segmentUrls;
}
你可以看到你将从m3u8获得.ts文件的链接解析它。
You can see that you will get the links to the .ts files from the m3u8 parse it.
- 现在下载所有的.ts文件进入本地文件夹。
- 将这些.ts文件合并到一个mp4文件和导出中。
您可以使用这个精彩的C库来实现这一目标
TS2MP4
- Now download all the .ts file into a local folder.
- Merge these .ts files in to one mp4 file and Export. You can do that using this wonderful C library TS2MP4
然后您可以删除.ts文件或在需要时保留它们。
and then you can delete the .ts files or keep them if you need them.
这篇关于在播放iOS时从HLS流（视频）中提取/录制音频的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
