在 UWP desktop-bridge 包中发送语音识别 args.Result 作为参数 [英] Send speech recognition args.Result as parameter in UWP desktop-bridge package

问题描述

我想弄清楚，是否有可能发送 using Windows.Media.SpeechRecognition; args.Result.Text 作为 UWP 的参数 到控制台应用程序.

I'm trying to figure out, if it is possible send using Windows.Media.SpeechRecognition; args.Result.Text as parameter from UWP to Console application.

例如，通过以下场景，我发送 TextToSpeech(args.Result.Text); with args.Result.Text; 值，其中 using Windows.Media.SpeechSynthesis; 每次出现args.Result.Text;时，text-to-speech都会朗读识别结果.textBlock2.Text = args.Result.Text; 也显示结果:

For example by following scenario I'm sending TextToSpeech(args.Result.Text); with args.Result.Text; value, where using Windows.Media.SpeechSynthesis; text-to-speech pronounces the recognition result each time when args.Result.Text; appears. textBlock2.Text = args.Result.Text; also displays result:

 private async void ContinuousRecognitionSession_ResultGenerated(
            SpeechContinuousRecognitionSession sender, SpeechContinuousRecognitionResultGeneratedEventArgs args)
        {
            await dispatcher.RunAsync(CoreDispatcherPriority.Normal, () =>
            {               
                textBlock1.Text = args.Result.Text;    

                TextToSpeech(args.Result.Text);               
            });
        }

但是如果我尝试将 args.Result.Text; 作为参数发送到控制台应用程序，包含在 Desktop-Bridge 中的 UWP> 包:

but if I'm trying to send args.Result.Text; as parameter to console application, included with UWP in Desktop-Bridge package:

    private async void ContinuousRecognitionSession_ResultGenerated(
        SpeechContinuousRecognitionSession sender, SpeechContinuousRecognitionResultGeneratedEventArgs args)
    {
        await dispatcher.RunAsync(CoreDispatcherPriority.Normal, () =>
        {                            
          textBlock1.Text = args.Result.Text; 

          SendTTSResult(args.Result.Text);
        });
    }

到请求的函数:

 private async void SendTTSResult(string res)
    {
        await Dispatcher.RunAsync(CoreDispatcherPriority.Normal, () =>
        {
            if (ApiInformation.IsApiContractPresent("Windows.ApplicationModel.FullTrustAppContract", 1, 0))
            {
                ApplicationData.Current.LocalSettings.Values["parameters"] = res;
                await FullTrustProcessLauncher.LaunchFullTrustProcessForCurrentAppAsync("Parameters");
            }
        });
    }

失败的行为在我看来并不是很清楚:

Behavior of failure looks to me not really clear:

根据第一个识别结果，将参数发送到控制台应用程序，控制台应用程序成功加载、获取并显示该参数.但是对于第二个请求，问题从这个处理级别退回，即使参数发送函数明确是失败的原因Result.Text 但这在函数开始执行之前就已经发生了，因为前面的输出显示 textBlock1.Text = args.Result.Text; 也不再接收事件.

With first recognition result, it sends parameter to console application, which successfully loads, gets and displays this parameter. But with second request the problem backs away from this processing level, even if parameters sending function is unambiguously the cause of a failure SendTTSResult(args.Result.Text); function does not receives args.Result.Text but this happens already before function will comes in action, because preceding output display textBlock1.Text = args.Result.Text; also does not receives event anymore.

与 async() => 行为有点不同，它成功接收事件并将值作为参数发送到控制台，在这种情况下从执行和语音开始发生 2-3 次请求，然后事件消失，当它甚至没有通过 SendTTSResult(string res) 时，想象一下 SendTTSResult(string res) 中的东西不允许从识别，但只是停止，即使我把它放在 TextToSpeech(string text) 函数的末尾，文本到语音也会停止接收事件:

With async() => behavior is a bit different, it successfully receives event and sends value as parameter to console, in this case it happens 2-3 times from beginning of execution and voice requesting, then event disappears, when it is not even passed through SendTTSResult(string res), to imagine that the something in SendTTSResult(string res) does not allows pass string from recognition, but just stops, even if I put it in the end of TextToSpeech(string text) function, text to speech also stops receive event:

private async void SendTTSResult(string res)
        {
            await Dispatcher.RunAsync(CoreDispatcherPriority.Normal, async() =>
            {
                if (ApiInformation.IsApiContractPresent("Windows.ApplicationModel.FullTrustAppContract", 1, 0))
                {
                    ApplicationData.Current.LocalSettings.Values["parameters"] = res;
                    await FullTrustProcessLauncher.LaunchFullTrustProcessForCurrentAppAsync("Parameters");
                }
            });
        }

看起来像使用 SendTTSResult(string res) 函数发送 args.Result.Text 值作为参数工作正常，成功发送字符串，但同时存在ContinuousRecognitionSession_ResultGenerated 中的这个函数会以某种方式影响它内部事件的接收.同时 ContSpeechRecognizer_HypothesisGenerated 的行为看起来完全不同，args.Hypothesis.Text 事件每次都出现并且结果作为参数成功传递，具有相同的 SendTTSResult(string res).

It looks like sending of args.Result.Text value as parameter with SendTTSResult(string res) function works fine, sends string successfully, but at the same time presence of this function in ContinuousRecognitionSession_ResultGenerated somehow affects receiving of event inside of it. At the same time behavior of ContSpeechRecognizer_HypothesisGenerated looks completely different, args.Hypothesis.Text event appears each time and result successfully passes as parameter with same SendTTSResult(string res).

当一个事件的过程中涉及到发送参数的功能时，什么可以阻止它被执行，如果可能的话如何解决?

What can prevent an event from being executed when the function of sending a parameter is involved in its process, and how to fix it if possible?

连续语音识别的完整代码添加到我在 Windows 开发中心的问题 发送语音识别args.Result作为UWP桌面桥包中的参数

编辑 1: **********************************************************************************************************

EDIT 1: **************************************************************************************************

在参数函数后面，Console Connector.exe 只显示参数，没有运行任何应用程序或其他任何东西:

Behind parameter function, Console Connector.exe only shows parameter without running of any app or anything else:

static void Main(string[] args)
        {
            string result = Assembly.GetExecutingAssembly().Location;
            int index = result.LastIndexOf("\");
            string rootPath = $"{result.Substring(0, index)}\..\";
            if (args.Length > 2)
            {
                switch (args[2])
                {
                    case "/parameters":
                        string parameters = ApplicationData.Current.LocalSettings.Values["parameters"] as string;
                        Console.WriteLine("Parameter: " + parameters);
                        Console.ReadLine();
                        break;
                }
            }
        }

Packeage.appxmanifest:

Packeage.appxmanifest:

<uap:Extension Category="windows.appService">
  <uap:AppService Name="SampleInteropService" />
</uap:Extension>

<desktop:Extension Category="windows.fullTrustProcess" Executable="ConnectorConnector.exe">
  <desktop:FullTrustProcess>
    <desktop:ParameterGroup GroupId="Parameters" Parameters="/parameters" />
  </desktop:FullTrustProcess>
</desktop:Extension>

编辑 2: **********************************************************************************************************

EDIT 2: **************************************************************************************************

我已经尝试过 SendTTSResult(SpeechRecogVal); 改变变量值:

I've tried SendTTSResult(SpeechRecogVal); with change of variable value:

 private async void ContinuousRecognitionSession_ResultGenerated(
        SpeechContinuousRecognitionSession sender, SpeechContinuousRecognitionResultGeneratedEventArgs args)
    {
        await dispatcher.RunAsync(CoreDispatcherPriority.Normal, () =>
        {               
            SpeechRecogVal = args.Result.Text;
        });
    }

但它是相同的行为 tbRec.Text = SpeechRecogVal; 显示输出成功，直到我添加 SendTTSResult(SpeechRecogVal);,

but it is same behavior tbRec.Text = SpeechRecogVal; shows output successfully until I add SendTTSResult(SpeechRecogVal);,

   private string _srVal;

    public string SpeechRecogVal
    {
        get
        {
            return _srVal;
        }

        set
        {
            _srVal = value;
            ValueChanged();
        }
    }

    void ValueChanged()
    {
        tbRec.Text = SpeechRecogVal;
        // SendTTSResult(SpeechRecogVal);
    }

所以，似乎问题出在 await Dispatcher.RunAsync(CoreDispatcherPriority.Normal, async () => 和 if (ApiInformation.IsApiContractPresent("Windows.ApplicationModel.FullTrustAppContract", 1, 0)) 和 await dispatcher.RunAsync(CoreDispatcherPriority.Normal, () => of private async voidContinuousRecognitionSession_ResultGenerated(SpeechContinuousRecognitionSession sender, SpeechContinuousRecognitionResultGeneratedEventArgs> args)

So, seems like problem is something between await Dispatcher.RunAsync(CoreDispatcherPriority.Normal, async () => and if (ApiInformation.IsApiContractPresent("Windows.ApplicationModel.FullTrustAppContract", 1, 0)) and await dispatcher.RunAsync(CoreDispatcherPriority.Normal, () => of private async voidContinuousRecognitionSession_ResultGenerated(SpeechContinuousRecognitionSession sender, SpeechContinuousRecognitionResultGeneratedEventArgs args)

另外，我试过了:

private async void ContinuousRecognitionSession_ResultGenerated(
    SpeechContinuousRecognitionSession sender, SpeechContinuousRecognitionResultGeneratedEventArgs args)
{
       await  SendTTSResult(args.Result.Text);        
}

作为任务:

async Task SendTTSResult(string res)
    {
        await Dispatcher.RunAsync(CoreDispatcherPriority.Normal, async () =>
        {
            if (ApiInformation.IsApiContractPresent("Windows.ApplicationModel.FullTrustAppContract", 1, 0))
            {
                ApplicationData.Current.LocalSettings.Values["parameters"] = res;
                await FullTrustProcessLauncher.LaunchFullTrustProcessForCurrentAppAsync("Parameters");
            }
        });
    }

而且它也只有在第一个请求事件实例响应时才成功，然后就相当了.所以看起来 ContinuousRecognitionSession_ResultGenerated 与 Windows.Media.SpeechRecognition 命名空间 并且与 async Task SendTTSResult(string res) 中的某些内容或与此代码行内容不兼容:

And it is also successful only with first request event instance response, then goes quite. So seems like ContinuousRecognitionSession_ResultGenerated is someway different from other options in Windows.Media.SpeechRecognition Namespace and not compatible with something in async Task SendTTSResult(string res) or rather with this code lines content:

 ApplicationData.Current.LocalSettings.Values["parameters"] = args.Result.Text;
 await FullTrustProcessLauncher.LaunchFullTrustProcessForCurrentAppAsync("Parameters");

推荐答案

System.NullReferenceException 发生 appservice 断开连接的情况，你能在发送消息之前检查 apservice 的连接吗?

System.NullReferenceException occurs appservice disconnect scenario, Could you check the apservice's connection before send message?

为了解释这一点，我创建了示例项目，引用 Stefanwick 博客.当我没有在 WPF 客户端调用 InitializeAppServiceConnection 方法时，我也会重现您的问题.如果你想发送文本到 wpf，你可以调用 Connection.SendMessageAsync 方法，就像下面的 SendMesssage 点击 envent .

For explain this, I create sample project that refer Stefanwick blog. And I also reproduce your issue when I does not invoke InitializeAppServiceConnection method in WPF client. If you want to send text to wpf, you could invoke Connection.SendMessageAsync method just like below SendMesssage click envent .

能力

      <Extensions>
        <uap:Extension Category="windows.appService">
          <uap:AppService Name="SampleInteropService" />
        </uap:Extension>
        <desktop:Extension Category="windows.fullTrustProcess" Executable="AlertWindowAlertWindow.exe" />
      </Extensions>
    </Application>
  </Applications>
  <Capabilities>
    <Capability Name="internetClient" />
    <rescap:Capability Name="runFullTrust" />
  </Capabilities>

WPF

private AppServiceConnection connection = null;
public MainWindow()
{
    InitializeComponent();
    InitializeAppServiceConnection();
}
private async void InitializeAppServiceConnection()
{
    connection = new AppServiceConnection();
    connection.AppServiceName = "SampleInteropService";
    connection.PackageFamilyName = Package.Current.Id.FamilyName;
    connection.RequestReceived += Connection_RequestReceived;
    connection.ServiceClosed += Connection_ServiceClosed;

    AppServiceConnectionStatus status = await connection.OpenAsync();
    if (status != AppServiceConnectionStatus.Success)
    {
        MessageBox.Show(status.ToString());
        this.IsEnabled = false;
    }
}

private void Connection_ServiceClosed(AppServiceConnection sender, AppServiceClosedEventArgs args)
{
    Dispatcher.BeginInvoke(DispatcherPriority.Normal, new Action(() =>
    {
        Application.Current.Shutdown();
    }));
}

private async void Connection_RequestReceived(AppServiceConnection sender, AppServiceRequestReceivedEventArgs args)
{
    // retrive the reg key name from the ValueSet in the request
    string key = args.Request.Message["KEY"] as string;

    if (key.Length > 0)
    {
        Dispatcher.BeginInvoke(DispatcherPriority.Normal, new Action(() =>
        {
            InfoBlock.Text = key;

        }));
        ValueSet response = new ValueSet();
        response.Add("OK", "SEND SUCCESS");
        await args.Request.SendResponseAsync(response);
    }
    else
    {
        ValueSet response = new ValueSet();
        response.Add("ERROR", "INVALID REQUEST");
        await args.Request.SendResponseAsync(response);
    }
}
private async void Button_Click(object sender, RoutedEventArgs e)
{
    ValueSet response = new ValueSet();
    response.Add("OK", "AlerWindow Message");
    await connection.SendMessageAsync(response);
}

UWP

 protected async override void OnNavigatedTo(NavigationEventArgs e)
 {
     base.OnNavigatedTo(e);

     if (ApiInformation.IsApiContractPresent("Windows.ApplicationModel.FullTrustAppContract", 1, 0))
     {
         App.AppServiceConnected += MainPage_AppServiceConnected;
         App.AppServiceDisconnected += MainPage_AppServiceDisconnected;
         await FullTrustProcessLauncher.LaunchFullTrustProcessForCurrentAppAsync();
     }
 }

 private async void MainPage_AppServiceDisconnected(object sender, EventArgs e)
 {
     await Dispatcher.RunAsync(CoreDispatcherPriority.Normal, () =>
     {
         Reconnect();
     });
 }



private void MainPage_AppServiceConnected(object sender, AppServiceTriggerDetails e)
 {
     App.Connection.RequestReceived += AppServiceConnection_RequestReceived;

 }
 private async void AppServiceConnection_RequestReceived(AppServiceConnection sender, AppServiceRequestReceivedEventArgs args)
 {
     string value = args.Request.Message["OK"] as string;
     await Dispatcher.RunAsync(CoreDispatcherPriority.Normal, () =>
      {
          InfoBlock.Text = value;
      });


 }     
 private async void Reconnect()
 {
     if (App.IsForeground)
     {
         MessageDialog dlg = new MessageDialog("Connection to desktop process lost. Reconnect?");
         UICommand yesCommand = new UICommand("Yes", async (r) =>
         {
             await FullTrustProcessLauncher.LaunchFullTrustProcessForCurrentAppAsync();
         });
         dlg.Commands.Add(yesCommand);
         UICommand noCommand = new UICommand("No", (r) => { });
         dlg.Commands.Add(noCommand);
         await dlg.ShowAsync();
     }
 }
 private int count = 0;
 private async void SendMesssage(object sender, RoutedEventArgs e)
 {
     count++;
     ValueSet request = new ValueSet();
     request.Add("KEY", $"Test{count}");
     AppServiceResponse response = await App.Connection.SendMessageAsync(request);

     // display the response key/value pairs

     foreach (string value in response.Message.Values)
     {
         await Dispatcher.RunAsync(CoreDispatcherPriority.Normal, () =>
         {
             StatusBlock.Text = value;
         });

     }
 }

这是完整的代码示例.

这篇关于在 UWP desktop-bridge 包中发送语音识别 args.Result 作为参数的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！