从另一个应用程序的文本框中获取文本和插入符号 [英] Get text and caret from a textbox in another application

问题描述

背景信息

我正在开发 Windows Speech-To-Text 应用程序.

I am working on a Windows Speech-To-Text application.

识别语音后，应将文本插入当前具有键盘焦点的文本框(Think Word/Firefox/其他应用程序).要插入我目前正在使用的文本 InputSimulatorPlus.

When speech has been recognized, the text should be inserted into the textbox that currently has keyboard focus (Think Word/Firefox/other applications). To insert the text I am currently using InputSimulatorPlus.

插入文本时，重要的是要对识别的文本进行格式设置以适应周围的文本，例如:

When inserting the text, it is important the recognized text is formatted to fit the surrounding text, like:

• 标点后的大写字母
• 不在标点符号之后的首字母小写
• 大写首字母换行
• 标点后的空格

等等

问题

为了能够格式化文本，我需要文本和插入符号位置.

To be able to format the text I need the text, and the caret position.

目前我一直在使用 UI 自动化 NuGet 包与 文本模式.这适用于所有支持文本模式的文本框，但很多程序不支持文本模式.

Currently I have been using UI Automation NuGet Package with Text Pattern. This works well for all textboxes that support Text Pattern, but alot of programs do not support Text Pattern.

策略问题:我应该使用 UI 自动化以外的其他方法吗?

Strategy question: Should I use another approach than UI Automation?

我注意到很多我想要支持的应用程序不支持文本模式，但支持值模式或传统 IAccessible 模式(Microsoft Active Accessibility).

I have noticed that a lot of the applications I want to support do not support Text Pattern, but do support Value Pattern or Legacy IAccessible Pattern(Microsoft Active Accessibility).

我一直在研究使用 Value Pattern 并且可以获取文本框的文本，但不能插入符号位置.

I have been looking into using the Value Pattern and can get the text of the textbox, but not the caret position.

using System.Windows.Automation;
using System.Windows.Automation.Text;
...

AutomationElement automationElement = AutomationElement.FocusedElement;
var elements = automationElement.FindAll(TreeScope.Element,
    new AndCondition(
        new PropertyCondition(AutomationElement.HasKeyboardFocusProperty, true),
        new PropertyCondition(AutomationElement.IsValuePatternAvailableProperty, true)));
foreach (AutomationElement element in elements)
{
    if (element.GetCurrentPattern(ValuePattern.Pattern) is ValuePattern valuePattern)
    {
        var text = valuePattern.Current.Value;
        var caret = ? //How to get caret position?
        Console.WriteLine($"Caret: {caret}, Text: {text}");
        return (text,caret);
    }
}

我也一直在研究使用 传统 IAccessible 模式 并且可以获取文本框的文本，但不能插入符号位置.

I have also been looking into using the Legacy IAccessible Pattern and can get the text of the textbox, but not the caret position.

using System.Windows.Automation;
using System.Windows.Automation.Text;
...

AutomationElement automationElement = AutomationElement.FocusedElement;
var elements = automationElement.FindAll(TreeScope.Element,
    new AndCondition(
        new PropertyCondition(AutomationElement.HasKeyboardFocusProperty, true),
        new PropertyCondition(AutomationElement.IsLegacyIAccessiblePatternAvailableProperty, true)));
foreach (AutomationElement element in elements)
{
    if (element.GetCurrentPattern(LegacyIAccessiblePattern.Pattern) is LegacyIAccessiblePattern legazyAccessiblePattern)
    {
        var text = legazyAccessiblePattern.Current.Value;
        var caret = ? //How to get caret position?
        Console.WriteLine($"Caret: {caret}, Text: {text}");
        return (text,caret);
    }
}

主要问题:如何使用 UI 自动化获取给定文本框的插入符号位置?

Main question: How do I get the caret position of a given textbox using UI Automation?

附言我知道这永远不会适用于所有应用程序，但如果它适用于大多数普通应用程序，那就太棒了.

P.S. I know this will never work for all applications, but if it can work for most normal applications, that would be awesome.

推荐答案

你不想要 ValuePattern，你想要 IUIAutomationTextPattern2，支持GetCaretRange.

You don't want ValuePattern, you want IUIAutomationTextPattern2, which supports GetCaretRange.

如果您需要回退到 MSAA，它会更加棘手(而且您根本无法获取插入符号周围的文本).MSAA 没有像 UIAutomation 那样的文本支持.如果你真的真的不能使用 UIAutomation，那么你唯一真正的选择是使用 文本服务框架，文档不全，因此没有得到广泛实施.如果您需要文本服务框架，您可能需要查看 我的博客，它没有好久没更新了.

If you need to fall back to MSAA, it's a lot hackier (and you can't get the text around the caret at all). MSAA just doesn't have anything like the text support that UIAutomation has. If you REALLY REALLY can't use UIAutomation, then your only real option is to use Text Services Framework, which is poorly documented and therefore not widely implemented. If you need Text Services Framework, you may want to look at my blog, which hasn't been updated in quite a while.

对于 MSAA 插入符，首先调用 GetGUIThreadInfo.它返回关于 hwndCaret 的数据，它是当前包含插入符号的窗口.(或者，如果 hwndCaret 为 NULL，您可以尝试使用 hwndFocus.)使用这些窗口之一，您可以执行以下操作:

For the MSAA caret, start by calling GetGUIThreadInfo. That returns data about a hwndCaret, which is the window which currently contains the caret. (Or you could try using hwndFocus if hwndCaret is NULL.) Using one of those windows, you could do something like:

IAccessible *pAccCaret = NULL;

VARIANT varCaret;
varCaret.vt = VT_I4;
varCaret.lVal = CHILDID_SELF;

if (SUCCEEDED(AccessibleObjectFromWindow(hwnd, OBJID_CARET, IID_IAccessible, (void **)&pAccCaret)))
{
    hr = pAccCaret->accLocation( &rcCaret.left, &rcCaret.top, &rcCaret.right, &rcCaret.bottom, varCaret);

    pAccCaret->Release();
}

这篇关于从另一个应用程序的文本框中获取文本和插入符号的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！