DialogFlow StreamingDetectIntentResponse 不返回 ResponseId、QueryText、Transcript 或除 LanguageCode 之外的任何内容 [英] DialogFlow StreamingDetectIntentResponse doesn't return ResponseId, QueryText, Transcript or anything except LanguageCode

问题描述

我已成功在 Unity 中使用 Grpc 并将请求发送到对话流并收到响应.您可以在此处

I have successfully used Grpc in Unity and sent request to Dialog flow and received response. You can check the details here

但是整个返回的结果只有以下

However the whole returned result is the following only

{ "queryResult": { "languageCode": "ja" } }

{ "queryResult": { "languageCode": "ja" } }

未返回预期的响应 ID、查询文本等.在 console.dialogflow.com 中测试时，我得到以下结果

The expected response id, query text, etc are not returned. When testing in console.dialogflow.com I get the following result

{"responseId": "cdf8003e-6599-4a28-9314-f4462c36e21b",查询结果":{"queryText": "おはようございます",语音识别信心":0.92638445，语言代码":ja"}}

{ "responseId": "cdf8003e-6599-4a28-9314-f4462c36e21b", "queryResult": { "queryText": "おはようございます", "speechRecognitionConfidence": 0.92638445, "languageCode": "ja" } }

然而，当我在 console.dialogflow.com 中尝试时，我什么也没说

However when I tried in console.dialogflow.com and didn't say anything I got

{ "queryResult": { "languageCode": "ja" } }

{ "queryResult": { "languageCode": "ja" } }

所以也许 InputAudio 编码有问题.

So perhaps the InputAudio encoding is wrong somehow.

这是我的做法

var serializedByteArray = convertToBytes(samples);
request.InputAudio = Google.Protobuf.ByteString.CopyFrom(serializedByteArray);

并转换为字节如下

public static byte[] convertToBytes(float[] audio)
{
    List<byte> bytes = new List<byte>();

    foreach (float audioI in audio) {
        bytes.AddRange(BitConverter.GetBytes(audioI));
    }

    return bytes.ToArray();
}

音频源定义如下，采样率为 16000

The audio source is define as follows where sampleRate is 16000

audioSource.clip = Microphone.Start(null, true, 30, sampleRate);

我确保正确设置采样率 hz.

I made sure to set sample rate hz properly.

queryInput.AudioConfig.SampleRateHertz = sampleRate;

我已将记录的字节从 unity 记录到文件(将所有字节流附加在一起)并编写了一个控制台应用程序来测试生成的二进制文件，但使用 DetectIntent 而不是流式检测意图.

I have logged the recorded bytes from unity to a file (have all the bytes streamed appended together) and have written a console application to test the binary generated but using DetectIntent rather than streaming detect intent.

GoogleCredential credential = GoogleCredential.FromJson(privateKey);

var url = "dialogflow.googleapis.com";

Grpc.Core.Channel channel = new Grpc.Core.Channel(url, credential.ToChannelCredentials());


var client = SessionsClient.Create(channel);


CallOptions options = new CallOptions();

DetectIntentRequest detectIntentRequest = new DetectIntentRequest();
detectIntentRequest.Session = "projects/projectid/agent/sessions/" + "detectIntent";
QueryInput queryInput = new QueryInput();
queryInput.AudioConfig = new InputAudioConfig();
queryInput.AudioConfig.LanguageCode = "ja";
queryInput.AudioConfig.SampleRateHertz = sampleRate;//must be between 8khz and 48khz
queryInput.AudioConfig.AudioEncoding = AudioEncoding.Linear16;

detectIntentRequest.QueryInput = queryInput;

detectIntentRequest.InputAudio = Google.Protobuf.ByteString.CopyFrom(File.ReadAllBytes("D:\\temp\\audio.bytes"));
 var response = client.DetectIntent(detectIntentRequest);
        Console.WriteLine(response.ToString());
        Console.WriteLine(response.ResponseId);
Console.Read();

我仍然得到这个(和空的 response.ResponseId)

I still get this (and empty response.ResponseId)

{ "queryResult": { "languageCode": "ja" } }

{ "queryResult": { "languageCode": "ja" } }

感谢提前.

推荐答案

终于找到答案了.我将数据源浮点数转换为线性 16 字节数组的方式显然是错误的.这是有效的代码感谢 Unity 论坛上的那篇帖子.

Finally found the answer. The way I converted the datasource float to linear16 byte array was obviously wrong. Here's the code that worked Credits to that post on unity forum.

https://forum.unity.com/threads/writing-audiolistener-getoutputdata-to-wav-problem.119295/#post-899142

public static byte[] convertToBytes(float[] dataSource)
{
    var intData = new Int16[dataSource.Length];
    //converting in 2 steps : float[] to Int16[], //then Int16[] to Byte[]

    var bytesData = new Byte[dataSource.Length * 2];
    //bytesData array is twice the size of
    //dataSource array because a float converted in Int16 is 2 bytes.

    var rescaleFactor = 32767; //to convert float to Int16

    for (var i = 0; i < dataSource.Length; i++)
    {
        intData[i] = (short)(dataSource[i] * rescaleFactor);
        var byteArr = new byte[2];
        byteArr = BitConverter.GetBytes(intData[i]);
        byteArr.CopyTo(bytesData, i * 2);
    }

    return bytesData;
}

这篇关于DialogFlow StreamingDetectIntentResponse 不返回 ResponseId、QueryText、Transcript 或除 LanguageCode 之外的任何内容的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！