如何从一台iOS设备到另一台iOS设备无线传输AVAsset音频？ [英] How do I stream AVAsset audio wirelessly form one iOS device to another?
问题描述
我正在制作类似于从iPod库传输音频，通过网络或蓝牙发送数据，以及使用音频队列播放的内容。
I'm making something like streaming the audio from iPod library, send the data via network or bluetooth, and playback using audio queue.
感谢您的支持问题和代码。帮助我很多。
Thanks for this question and code. Help me a lot.
我有两个问题。
-
我应该从一台设备发送到另一台设备？ CMSampleBufferRef？ AudioBuffer？ MDATA？ AudioQueueBuffer？包？我不知道。
what should I send from one device to another? CMSampleBufferRef? AudioBuffer? mData? AudioQueueBuffer? packet? I have no idea.
当应用程序完成播放时，它崩溃了，我收到了错误（-12733）。我只是想知道如何处理错误而不是让它崩溃。 （检查OSState？当错误发生时，停止它？）
When the app finished the playing, it crashed, and I got error (-12733). I just want to know how to handle the errors instead of letting it crash. (check the OSState? When the error happened, stop it?)
错误：无法读取样本数据（-12733）
Error: could not read sample data (-12733)
推荐答案
我会先回答你的第二个问题 - 不要等待应用程序崩溃，你可以停止拉动音频通过检查您正在阅读的CMSampleBufferRef中可用的样本数量从轨道中获取;例如（此代码也将包含在我的答案的下半部分）：
I will answer your second question first - don't wait for the app to crash, you can stop pulling audio from the track by checking the number of samples that are available in the CMSampleBufferRef you are reading; for example (this code will also be included in the 2nd half of my answer):
CMSampleBufferRef sample;
sample = [readerOutput copyNextSampleBuffer];
CMItemCount numSamples = CMSampleBufferGetNumSamples(sample);
if (!sample || (numSamples == 0)) {
// handle end of audio track here
return;
}
关于你的第一个问题，这取决于你所抓的音频类型 - 它可以是PCM（非压缩）或VBR（压缩）格式。我甚至不打算解决PCM部分问题，因为通过网络将未压缩的音频数据从一部手机发送到另一部手机根本不聪明 - 这会不必要地昂贵并且会阻塞你的网络带宽。所以我们留下了VBR数据。为此你必须发送你从样本中提取的 AudioBuffer 和 AudioStreamPacketDescription 的内容。但话说回来，最好用代码解释我在说什么：
Regarding your first question, it depends on the type of audio you are grabbing - it could be wither PCM (non-compressed) or VBR (compressed) format. I'm not even going to bother addressing the PCM part because it's simply not smart to send uncompressed audio data from one phone to another over the network - it's unnecessarily expensive and will clog your networking bandwidth. So we're left with VBR data. For that you've got to send the contents of AudioBuffer and AudioStreamPacketDescription you pulled from the sample. But then again, it's probably best to explain what I'm saying by code:
-(void)broadcastSample
{
[broadcastLock lock];
CMSampleBufferRef sample;
sample = [readerOutput copyNextSampleBuffer];
CMItemCount numSamples = CMSampleBufferGetNumSamples(sample);
if (!sample || (numSamples == 0)) {
Packet *packet = [Packet packetWithType:PacketTypeEndOfSong];
packet.sendReliably = NO;
[self sendPacketToAllClients:packet];
[sampleBroadcastTimer invalidate];
return;
}
NSLog(@"SERVER: going through sample loop");
Boolean isBufferDataReady = CMSampleBufferDataIsReady(sample);
CMBlockBufferRef CMBuffer = CMSampleBufferGetDataBuffer( sample );
AudioBufferList audioBufferList;
CheckError(CMSampleBufferGetAudioBufferListWithRetainedBlockBuffer(
sample,
NULL,
&audioBufferList,
sizeof(audioBufferList),
NULL,
NULL,
kCMSampleBufferFlag_AudioBufferList_Assure16ByteAlignment,
&CMBuffer
),
"could not read sample data");
const AudioStreamPacketDescription * inPacketDescriptions;
size_t packetDescriptionsSizeOut;
size_t inNumberPackets;
CheckError(CMSampleBufferGetAudioStreamPacketDescriptionsPtr(sample,
&inPacketDescriptions,
&packetDescriptionsSizeOut),
"could not read sample packet descriptions");
inNumberPackets = packetDescriptionsSizeOut/sizeof(AudioStreamPacketDescription);
AudioBuffer audioBuffer = audioBufferList.mBuffers[0];
for (int i = 0; i < inNumberPackets; ++i)
{
NSLog(@"going through packets loop");
SInt64 dataOffset = inPacketDescriptions[i].mStartOffset;
UInt32 dataSize = inPacketDescriptions[i].mDataByteSize;
size_t packetSpaceRemaining = MAX_PACKET_SIZE - packetBytesFilled - packetDescriptionsBytesFilled;
size_t packetDescrSpaceRemaining = MAX_PACKET_DESCRIPTIONS_SIZE - packetDescriptionsBytesFilled;
if ((packetSpaceRemaining < (dataSize + AUDIO_STREAM_PACK_DESC_SIZE)) ||
(packetDescrSpaceRemaining < AUDIO_STREAM_PACK_DESC_SIZE))
{
if (![self encapsulateAndShipPacket:packet packetDescriptions:packetDescriptions packetID:assetOnAirID])
break;
}
memcpy((char*)packet + packetBytesFilled,
(const char*)(audioBuffer.mData + dataOffset), dataSize);
memcpy((char*)packetDescriptions + packetDescriptionsBytesFilled,
[self encapsulatePacketDescription:inPacketDescriptions[i]
mStartOffset:packetBytesFilled
],
AUDIO_STREAM_PACK_DESC_SIZE);
packetBytesFilled += dataSize;
packetDescriptionsBytesFilled += AUDIO_STREAM_PACK_DESC_SIZE;
// if this is the last packet, then ship it
if (i == (inNumberPackets - 1)) {
NSLog(@"woooah! this is the last packet (%d).. so we will ship it!", i);
if (![self encapsulateAndShipPacket:packet packetDescriptions:packetDescriptions packetID:assetOnAirID])
break;
}
}
[broadcastLock unlock];
}
我在上面的代码中使用的一些方法是你不喜欢的方法需要担心，比如为每个数据包添加标题（我创建了自己的协议，你可以创建自己的协议）。有关详细信息，请参阅这个教程。
Some methods that I've used in the above code are methods you don't need to worry about, such as adding headers to each packet (I was creating my own protocol, you can create your own). For more info see this tutorial.
- (BOOL)encapsulateAndShipPacket:(void *)source
packetDescriptions:(void *)packetDescriptions
packetID:(NSString *)packetID
{
// package Packet
char * headerPacket = (char *)malloc(MAX_PACKET_SIZE + AUDIO_BUFFER_PACKET_HEADER_SIZE + packetDescriptionsBytesFilled);
appendInt32(headerPacket, 'SNAP', 0);
appendInt32(headerPacket,packetNumber, 4);
appendInt16(headerPacket,PacketTypeAudioBuffer, 8);
// we use this so that we can add int32s later
UInt16 filler = 0x00;
appendInt16(headerPacket,filler, 10);
appendInt32(headerPacket, packetBytesFilled, 12);
appendInt32(headerPacket, packetDescriptionsBytesFilled, 16);
appendUTF8String(headerPacket, [packetID UTF8String], 20);
int offset = AUDIO_BUFFER_PACKET_HEADER_SIZE;
memcpy((char *)(headerPacket + offset), (char *)source, packetBytesFilled);
offset += packetBytesFilled;
memcpy((char *)(headerPacket + offset), (char *)packetDescriptions, packetDescriptionsBytesFilled);
NSData *completePacket = [NSData dataWithBytes:headerPacket length: AUDIO_BUFFER_PACKET_HEADER_SIZE + packetBytesFilled + packetDescriptionsBytesFilled];
NSLog(@"sending packet number %lu to all peers", packetNumber);
NSError *error;
if (![_session sendDataToAllPeers:completePacket withDataMode:GKSendDataReliable error:&error]) {
NSLog(@"Error sending data to clients: %@", error);
}
Packet *packet = [Packet packetWithData:completePacket];
// reset packet
packetBytesFilled = 0;
packetDescriptionsBytesFilled = 0;
packetNumber++;
free(headerPacket);
// free(packet); free(packetDescriptions);
return YES;
}
- (char *)encapsulatePacketDescription:(AudioStreamPacketDescription)inPacketDescription
mStartOffset:(SInt64)mStartOffset
{
// take out 32bytes b/c for mStartOffset we are using a 32 bit integer, not 64
char * packetDescription = (char *)malloc(AUDIO_STREAM_PACK_DESC_SIZE);
appendInt32(packetDescription, (UInt32)mStartOffset, 0);
appendInt32(packetDescription, inPacketDescription.mVariableFramesInPacket, 4);
appendInt32(packetDescription, inPacketDescription.mDataByteSize,8);
return packetDescription;
}
接收数据：
- (void)receiveData:(NSData *)data fromPeer:(NSString *)peerID inSession:(GKSession *)session context:(void *)context
{
Packet *packet = [Packet packetWithData:data];
if (packet == nil)
{
NSLog(@"Invalid packet: %@", data);
return;
}
Player *player = [self playerWithPeerID:peerID];
if (player != nil)
{
player.receivedResponse = YES; // this is the new bit
} else {
Player *player = [[Player alloc] init];
player.peerID = peerID;
[_players setObject:player forKey:player.peerID];
}
if (self.isServer)
{
[Logger Log:@"SERVER: we just received packet"];
[self serverReceivedPacket:packet fromPlayer:player];
}
else
[self clientReceivedPacket:packet];
}
备注：
-
我在这里没有涉及很多网络细节（例如，在接收数据部分。我使用了很多自定义对象而没有扩展他们的定义）。我没有，因为解释所有这些都超出了SO的一个答案的范围。但是，您可以按照优秀教程。他花时间解释网络原则，我上面使用的架构几乎是他的逐字逐句。但是有一个陷阱（见下一点）
There are a lot of networking details that I didn't cover here (ie, in the receiving data part. I used a lot of custom made objects without expanding on their definition). I didn't because explaining all of that is beyond the scope of just one answer on SO. However, you can follow the excellent tutorial of Ray Wenderlich. He takes his time in explaining networking principles, and the architecture I use above is almost taken verbatim from him. HOWEVER THERE IS A CATCH (see next point)
根据您的项目，GKSession可能不合适（特别是如果您的项目是实时的，或者如果您需要超过2-3个设备同时连接）它有很多限制。您将不得不深入挖掘并直接使用Bonjour。 iPhone炫酷项目有一个很好的快速章节，给了一个很好的使用Bonjour服务的示例。它并不像听起来那么可怕（苹果文档对这个问题有点霸道）。
Depending on your project, GKSession may not be suitable (especially if your project is realtime, or if you need more than 2-3 devices to connect simultaneously) it has a lot of limitations. You will have to dig down deeper and use Bonjour directly instead. iPhone cool projects has a nice quick chapter that gives a nice example of using Bonjour services. It's not as scary as it sounds (and the apple documentation is kinda overbearing on that subject).
我注意到你使用GCD进行多线程处理。同样，如果您正在处理实时，那么您不希望使用高级框架来为您做繁重的工作（GCD就是其中之一）。有关此主题的更多信息，请阅读此优秀的文章。另请阅读 justin 之间的长时间讨论。 a / 12454761/766570>这个答案。
I noticed you use GCD for your multithreading. Again, if you are dealing with real time then you don't want to use advanced frameworks that do the heavy lifting for you (GCD is one of them). For more on this subject read this excellent article. Also read the prolonged discussion between me and justin in the comments of this answer.
你可能想看看 MTAudioProcessingTap 。它可以在处理AVAssets时为您节省一些麻烦。我没有测试过这个东西。在我完成所有工作后，它就出来了。
You may want to check out MTAudioProcessingTap introduced in iOS 6. It can potentially save you some hassle while dealing with AVAssets. I didn't test this stuff though. It came out after I did all my work.
最后但并非最不重要的，你可能想看看学习核心音频书。这是关于这一主题的广泛认可的参考文献。我记得当你提出这个问题时，你会像你一样陷入困境。核心音频很重，需要时间才能接收。所以只会给你指点。您将不得不花时间自己吸收材料，然后您将弄清楚事情是如何运作的。祝你好运！
Last but not least, you may want to check out the learning core audio book. It's a widely acknowledged reference on this subject. I remember being as stuck as you were at the point you asked the question. Core audio is heavy duty and it takes time to sink in. SO will only give you pointers. You will have to take your time to absorb the material yourself then you will figure out how things work out. Good luck!
这篇关于如何从一台iOS设备到另一台iOS设备无线传输AVAsset音频？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
