DialogFlow StreamingDetectIntentResponse不返回ResponseId，QueryText，Transcript或除LanguageCode之外的任何内容 [英] DialogFlow StreamingDetectIntentResponse doesn't return ResponseId, QueryText, Transcript or anything except LanguageCode

问题描述

我已经在Unity中成功使用了Grpc，并将请求发送到Dialog流并收到了响应.您可以在此处

I have successfully used Grpc in Unity and sent request to Dialog flow and received response. You can check the details here

然而，整个返回结果仅是以下内容

However the whole returned result is the following only

{"queryResult":{"languageCode":"ja"}}

{ "queryResult": { "languageCode": "ja" } }

未返回预期的响应ID，查询文本等. 在console.dialogflow.com中进行测试时，我得到以下结果

The expected response id, query text, etc are not returned. When testing in console.dialogflow.com I get the following result

{ "responseId":"cdf8003e-6599-4a28-9314-f4462c36e21b"， "queryResult":{ "queryText":おはようございます"， "speechRecognitionConfidence":0.92638445， "languageCode":"ja" } }

{ "responseId": "cdf8003e-6599-4a28-9314-f4462c36e21b", "queryResult": { "queryText": "おはようございます", "speechRecognitionConfidence": 0.92638445, "languageCode": "ja" } }

但是，当我在console.dialogflow.com中尝试并什么也没说时

However when I tried in console.dialogflow.com and didn't say anything I got

{"queryResult":{"languageCode":"ja"}}

{ "queryResult": { "languageCode": "ja" } }

因此，也许InputAudio编码在某种程度上是错误的.

So perhaps the InputAudio encoding is wrong somehow.

这是我的方法

var serializedByteArray = convertToBytes(samples);
request.InputAudio = Google.Protobuf.ByteString.CopyFrom(serializedByteArray);

并转换为字节如下

public static byte[] convertToBytes(float[] audio)
{
    List<byte> bytes = new List<byte>();

    foreach (float audioI in audio) {
        bytes.AddRange(BitConverter.GetBytes(audioI));
    }

    return bytes.ToArray();
}

音频源定义如下，其中sampleRate为16000

The audio source is define as follows where sampleRate is 16000

audioSource.clip = Microphone.Start(null, true, 30, sampleRate);

我确保正确设置采样率hz.

I made sure to set sample rate hz properly.

queryInput.AudioConfig.SampleRateHertz = sampleRate;

我已将记录的字节从一个字节记录到一个文件(所有字节流附加在一起)，并编写了一个控制台应用程序来测试生成的二进制文件，但使用的是DetectIntent而不是流检测意图.

I have logged the recorded bytes from unity to a file (have all the bytes streamed appended together) and have written a console application to test the binary generated but using DetectIntent rather than streaming detect intent.

GoogleCredential credential = GoogleCredential.FromJson(privateKey);

var url = "dialogflow.googleapis.com";

Grpc.Core.Channel channel = new Grpc.Core.Channel(url, credential.ToChannelCredentials());


var client = SessionsClient.Create(channel);


CallOptions options = new CallOptions();

DetectIntentRequest detectIntentRequest = new DetectIntentRequest();
detectIntentRequest.Session = "projects/projectid/agent/sessions/" + "detectIntent";
QueryInput queryInput = new QueryInput();
queryInput.AudioConfig = new InputAudioConfig();
queryInput.AudioConfig.LanguageCode = "ja";
queryInput.AudioConfig.SampleRateHertz = sampleRate;//must be between 8khz and 48khz
queryInput.AudioConfig.AudioEncoding = AudioEncoding.Linear16;

detectIntentRequest.QueryInput = queryInput;

detectIntentRequest.InputAudio = Google.Protobuf.ByteString.CopyFrom(File.ReadAllBytes("D:\\temp\\audio.bytes"));
 var response = client.DetectIntent(detectIntentRequest);
        Console.WriteLine(response.ToString());
        Console.WriteLine(response.ResponseId);
Console.Read();

我仍然收到这个消息(还有空的响应.ResponseId)

I still get this (and empty response.ResponseId)

{"queryResult":{"languageCode":"ja"}}

{ "queryResult": { "languageCode": "ja" } }

感谢前进.

推荐答案

最终找到了答案.我将数据源浮点数转换为linear16字节数组的方式显然是错误的.这是起作用的代码 归功于统一论坛上的帖子.

Finally found the answer. The way I converted the datasource float to linear16 byte array was obviously wrong. Here's the code that worked Credits to that post on unity forum.

https://forum.unity.com/threads/writing-audiolistener-getoutputdata-to-wav-problem.119295/#post-899142

public static byte[] convertToBytes(float[] dataSource)
{
    var intData = new Int16[dataSource.Length];
    //converting in 2 steps : float[] to Int16[], //then Int16[] to Byte[]

    var bytesData = new Byte[dataSource.Length * 2];
    //bytesData array is twice the size of
    //dataSource array because a float converted in Int16 is 2 bytes.

    var rescaleFactor = 32767; //to convert float to Int16

    for (var i = 0; i < dataSource.Length; i++)
    {
        intData[i] = (short)(dataSource[i] * rescaleFactor);
        var byteArr = new byte[2];
        byteArr = BitConverter.GetBytes(intData[i]);
        byteArr.CopyTo(bytesData, i * 2);
    }

    return bytesData;
}

这篇关于DialogFlow StreamingDetectIntentResponse不返回ResponseId，QueryText，Transcript或除LanguageCode之外的任何内容的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！