一个同时监测申请后停止接收UI自动化事件，然后过一段时间后重新启动 [英] UI Automation events stop being received after a while monitoring an application and then restart after some time
问题描述
我们使用的是微软的UIAutomation框架开发一个客户端，它监视一个特定的应用程序的事件，并响应它们以不同的方式。我们已经开始与框架的托管版本，但由于延误的问题，转移到包裹在 UIACOMWrapper <本地版/ A>。经过与我们的（块状）WPF应用程序中的性能更多的问题，我们决定将它移动到这似乎解决所有性能问题的独立终端应用程序（通过UDP传输的事件给我们的WPF应用程序）。唯一的问题是，它似乎每几分钟，为TabSelection，StructureChanged，的windowOpened和的windowClosed事件停止被捕获为几分钟。令人惊讶的PropertyChanged事件仍收到并处理，而出现这种情况。我会后我们的活动监视器的相关code，但是这是因为我们使用微软自己的AccEvent实用程序时，也有类似的行为可能无关。我不能发布受监控应用程序的code，因为它是专有和机密，以及，我可以说，这是一个WinForms应用程序承载WPF窗口和也是相当巨大的。
有没有人见过这种行为与UI自动化框架，工作时？
感谢您的时间。
We are using Microsoft's UIAutomation framework to develop a client that monitors events of a specific application and responds to them in different ways. We've started with the managed version of the framework, but due to delay issues, moved to the native version wrapped in UIACOMWrapper. After more issues with performance inside our (massive) WPF application, we decided to move it to a separate terminal application (transfer the events to our WPF app through UDP) which seemed to fix all the performance issues. The only problem is that it seems that every several minutes, the events for TabSelection, StructureChanged, WindowOpened and WindowClosed stop being captured for a few minutes. Surprisingly PropertyChanged events are still received and handled while this happens. I will post the relevant code of our event monitor, but this is probably irrelevant as we have seen similar behavior when using Microsoft's own AccEvent utility. I can't post the code of the monitored application as it is proprietary and confidential as well, I can say that it is a WinForms application that hosts WPF windows and also quite massive. Has anyone seen this sort of behavior while working with the UI Automation framework? Thank you for your time.
下面是显示器code（我所知道的事件处理的UI自动化的线程在这里，但其移动到一个专用的线程没有改变任何东西）：
Here's the monitor code (I know the event handling is on the UI Automation threads here but moving it to a dedicated thread did not change anything):
public void registerHandlers()
{
//Register on structure changed and window opened events
System.Windows.Automation.Automation.AddStructureChangedEventHandler(
this.getMsAutomationElement(), System.Windows.Automation.TreeScope.Subtree, this.handleStructureChanged);
System.Windows.Automation.Automation.AddAutomationEventHandler(
System.Windows.Automation.WindowPattern.WindowOpenedEvent,
this.getMsAutomationElement(),
System.Windows.Automation.TreeScope.Subtree,
this.handleWindowOpened);
System.Windows.Automation.Automation.AddAutomationEventHandler(
System.Windows.Automation.WindowPattern.WindowClosedEvent,
System.Windows.Automation.AutomationElement.RootElement,
System.Windows.Automation.TreeScope.Subtree,
this.handleWindowClosed);
this.registerValueChanged();
this.registerTextNameChange();
this.registerTabSelected();
this.registerRangeValueChanged();
}
private void registerRangeValueChanged()
{
if (this.getMsAutomationElement() != null)
{
System.Windows.Automation.Automation.AddAutomationPropertyChangedEventHandler(
this.getMsAutomationElement(),
System.Windows.Automation.TreeScope.Subtree, this.handlePropertyChange,
System.Windows.Automation.RangeValuePattern.ValueProperty);
}
}
private void unregisterRangeValueChanged()
{
System.Windows.Automation.Automation.RemoveAutomationPropertyChangedEventHandler(
this.getMsAutomationElement(),
this.handlePropertyChange);
}
private void registerValueChanged()
{
if (this.getMsAutomationElement() != null)
{
System.Windows.Automation.Automation.AddAutomationPropertyChangedEventHandler(
this.getMsAutomationElement(),
System.Windows.Automation.TreeScope.Subtree, this.handlePropertyChange,
System.Windows.Automation.ValuePattern.ValueProperty);
}
}
private void unregisterValueChanged()
{
System.Windows.Automation.Automation.RemoveAutomationPropertyChangedEventHandler(
this.getMsAutomationElement(),
this.handlePropertyChange);
}
private void registerTextNameChange()
{
if (this.getMsAutomationElement() != null)
{
System.Windows.Automation.Automation.AddAutomationPropertyChangedEventHandler(
this.getMsAutomationElement(),
System.Windows.Automation.TreeScope.Subtree, this.handlePropertyChange,
System.Windows.Automation.AutomationElement.NameProperty);
}
}
private void unregisterTextNameChange()
{
System.Windows.Automation.Automation.RemoveAutomationPropertyChangedEventHandler(
this.getMsAutomationElement(),
this.handlePropertyChange);
}
private void handleWindowOpened(object src, System.Windows.Automation.AutomationEventArgs e)
{
Console.ForegroundColor = ConsoleColor.Magenta;
Console.WriteLine(DateTime.Now.ToShortTimeString() + " " + "Window opened:" + " " +
(src as System.Windows.Automation.AutomationElement).Current.Name);
System.Windows.Automation.AutomationElement element = src as System.Windows.Automation.AutomationElement;
//this.sendEventToPluginQueue(src, e, element.GetRuntimeId(), this.getAutomationParent(element).GetRuntimeId());
//Fill out the fields of the control added message
int[] parentId = this.getAutomationParent(element).GetRuntimeId();
this.copyToIcdArray(parentId,
this.protocol.getMessageSet().outgoing.ControlAddedMessage.Data.controlAdded.parentRuntimeId);
this.copyToIcdArray(element.GetRuntimeId(),
this.protocol.getMessageSet().outgoing.ControlAddedMessage.Data.controlAdded.runtimeId);
//Send the message using the protocol
this.protocol.send(this.protocol.getMessageSet().outgoing.ControlAddedMessage);
}
private void copyToIcdArray(int[] runtimeId, ICD.UI_AUTOMATION.RuntimeId icdRuntimeId)
{
icdRuntimeId.runtimeIdNumberOfItems.setVal((byte)runtimeId.Count());
for (int i = 0; i < runtimeId.Count(); i++)
{
icdRuntimeId.runtimeIdArray.getElement(i).setVal(runtimeId[i]);
}
}
private void handleWindowClosed(object src, System.Windows.Automation.AutomationEventArgs e)
{
if (src != null)
{
Console.ForegroundColor = ConsoleColor.Cyan;
Console.WriteLine(DateTime.Now.ToShortTimeString() + " " + "Window closed:" + " " +
(src as System.Windows.Automation.AutomationElement).GetRuntimeId().ToString());
System.Windows.Automation.AutomationElement element = src as System.Windows.Automation.AutomationElement;
this.copyToIcdArray(element.GetRuntimeId(),
this.protocol.getMessageSet().outgoing.ControlRemovedMessage.Data.controlRemoved.runtimeId);
//Send the message using the protocol
this.protocol.send(this.protocol.getMessageSet().outgoing.ControlRemovedMessage);
//this.sendEventToPluginQueue(src, e, element.GetRuntimeId());
}
}
编辑：
我忘了提，我强烈怀疑的问题是，用户界面，自动化事件处理程序的一个线程被莫名其妙地卡住。我相信这样做的原因是，当问题在我的显示器发生后，我开始AccEvent的实例，并收到所有我的显示器没有得到丢失事件。这意味着该事件被触发，但不传递到我的显示器
I forgot to mention that I strongly suspect that the issue is that one of the UI-Automation event handler threads gets stuck somehow. The reason I believe this, is that when the problem occurred in my monitor, I started an instance of AccEvent and it received all the missing events that my monitor was not getting. This means that the events are being fired but not passed to my monitor.
EDIT2：
我忘了提这种情况的发生运行在Windows 8的具体目标应用程序，我还没有看到我自己的Windows 7机器上的这种现象与其他应用程序。另一个有趣的事情是，它似乎发生周期性有多有少，但无论何时我订阅的事件，即其几乎可以立即订阅以后的事情，但随后需要几分钟再次发生。
I forgot to mention that this happens running in Windows 8 with the specific target application, I have not seen this phenomenon on my own Windows 7 machine with other applications. Another interesting thing is that it seems to happen periodically more or less, but regardless of when I subscribe to events, i.e. it can happen almost immediately after subscribing but then it takes several minutes to reoccur.
推荐答案
我害怕，我不知道你看到延迟的原因，但这里有一些这方面的想法......
I'm afraid I don't know the cause of the delays that you're seeing, but here are some thoughts on this...
一切我下面说的涉及到Windows原生API UIA，而不是管理的.NET API UIA。近年来UIA所有方面做了改进到Windows UIA的API。所以每当我写UIA客户端C＃code，我叫UIA通过托管包装，我与tlbimp.exe是SDK工具生成。
Everything I say below relates to the native UIA API in Windows, not the managed .NET UIA API. All improvements to UIA in recent years have been made to the Windows UIA API. So whenever I write UIA client C# code, I call UIA through a managed wrapper that I generate with the tlbimp.exe SDK tool.
这是我第一次用命令生成包装像...
That is, I first generate the wrapper with a command like...
C：\\ Program Files文件（x86）的\\微软的SDK \\ WINDOWS \\ v8.1A \\ BIN \\ NETFX 4.5.1工具\\ 64 \\ tlbimp.exe是C：\\ WINDOWS \\ SYSTEM32 \\ uiautomationcore.dll /输出：互操作.UIAutomationCore.dll
"C:\Program Files (x86)\Microsoft SDKs\Windows\v8.1A\bin\NETFX 4.5.1 Tools\x64\tlbimp.exe" c:\windows\system32\uiautomationcore.dll /out:Interop.UIAutomationCore.dll
然后我包括我的C＃项目，添加到Interop.UIAutomationCore.dll参考使用Interop.UIAutomationCore;我的C＃文件，然后我可以做这样的事情...
Then I include a reference to the Interop.UIAutomationCore.dll in my C# project, add "using Interop.UIAutomationCore;" to my C# file, and then I can do things like...
IUIAutomation uiAutomation = new CUIAutomation8();
IUIAutomationElement rootElement = uiAutomation.GetRootElement();
uiAutomation.AddAutomationEventHandler(
20016, // UIA_Window_WindowOpenedEventId
rootElement,
TreeScope.TreeScope_Descendants,
null,
this);
...
public void HandleAutomationEvent(IUIAutomationElement sender, int eventId)
{
// Got a window opened event...
}
在Windows 7中，有大约UIA事件处理一些重要的制约因素。这是很容易编写事件处理程序而没有考虑这些限制，并与UIA进行交互时，可能会导致长时间的延迟。例如，它不添加或事件处理中删除UIA事件处理程序是很重要的。所以在那个时候，我故意再没UIA从我的事件处理程序内调用的。相反，我会发布自己的消息，或添加一些动作到队列中，让我的事件处理程序返回，并采取任何行动，我想响应事件不久之后在另一个线程。这需要对我而言更多的工作，但我不想冒险打延误。我创建的任何线程将在MTA中运行。
In Windows 7, there were some important constraints around UIA event handlers. It was easy to write event handlers which didn't account for those constraints, and that could lead to long delays when interacting with UIA. For example, it was important to not add or remove a UIA event handler from inside an event handler. So at the time, I intentionally made no UIA calls at all from inside my event handlers. Instead, I'd post myself a message or add some action to a queue, allow my event handler to return, and take whatever action I wanted to in response to the event shortly afterwards on another thread. This required some more work on my part, but I didn't want to risk hitting delays. And any threads I created would be running in an MTA.
上述动作的一个例子是我的旧的焦点追踪样本达在<一个href=\"https://$c$c.msdn.microsoft.com/windowsapps/Windows-7-UI-Automation-6390614a/source$c$c?fileId=21469&pathId=715901329\" rel=\"nofollow\">https://$c$c.msdn.microsoft.com/windowsapps/Windows-7-UI-Automation-6390614a/source$c$c?fileId=21469&pathId=715901329.该文件FocusEventHandler.cs创建MTA线程和队列的消息，以避免UIA事件处理程序调用里面
An example of the action described above is in my old focus tracking sample up at https://code.msdn.microsoft.com/windowsapps/Windows-7-UI-Automation-6390614a/sourcecode?fileId=21469&pathId=715901329. The file FocusEventHandler.cs creates the MTA thread and queues messages to avoid making UIA calls inside the event hander.
由于微软Windows 7，我知道在UIA与线程的限制和延迟已放宽，以及遇到的延迟的可能性已经减少。最近，有这方面的的Windows 8.1和Windows 10之间的一些改进，所以如果它会是实际在Windows 10上运行您的code，就看是否延迟仍然瑞普有很有趣。
Since Window 7, I know the constraints in UIA relating to threading and delays have been relaxed, and the likelihood of encountering delays has been reduced. More recently, there were some improvements between Windows 8.1 and Windows 10 in this area, so if it'd be practical to run your code on Windows 10, it would be interesting to see if the delays still repro there.
我知道这是耗时的，但你可能会感兴趣的删除与UIA的互动事件处理程序内，如果延迟消失有看头。如果他们这样做，这将会是决定哪些行动似乎引发的问题，并查看是否有实现自己的目标而无需在事件处理程序进行交互UIA的另一种方式的情况。
I know this is time consuming, but you might be interested in removing the interaction with UIA inside your event handlers and seeing if the delays go away. If they do, it'd be a case of determining which action seems to trigger the problem, and seeing if there's an alternative way of achieving your goals without performing the UIA interaction in the event handlers.
例如，在事件处理程序，你呼唤我，
For example, in your event handler, you call...
this.getAutomationParent（元素）.GetRuntimeId（）;
this.getAutomationParent(element).GetRuntimeId();
我预计这将导致两个呼叫回生成事件的提供应用程序。第一次调用是获取源元素的父元素，第二个电话是得到了父母的RuntimeId。因此，尽管UIA等待事件处理程序返回，你叫了两声回UIA。虽然我不知道，这是一个问题吗，我会避免。
I expect this will lead to two calls back into the provider app which generated the event. The first call is to get the parent of the source element, and the second call is to get the RuntimeId of that parent. So while UIA is waiting for your event handler to return, you've called twice back into UIA. While I don't know that that's a problem, I'd avoid it.
有时你可以通过与事件本身的利益缓存的一些数据，避免交叉调用proc回到供应商的过程。例如，假设我知道我会想引发一个事件的windowOpened元素的RuntimeId。我可以问UIA缓存数据与我收到的时候我注册事件的事件。
Sometimes you can avoid a cross-proc call back to the provider process by having some data of interest cached with the event itself. For example, say I know I'm going to want the RuntimeId of an element that raised a WindowOpened event. I can ask UIA to cache that data with the events I receive, when I register for the events.
int propertyRuntimeId = 30000; // UIA_RuntimeIdPropertyId
...
IUIAutomationCacheRequest cacheRequestRuntimeId = uiAutomation.CreateCacheRequest();
cacheRequestRuntimeId.AddProperty(propertyRuntimeId);
uiAutomation.AddAutomationEventHandler(
20016, // UIA_Window_WindowOpenedEventId
rootElement,
TreeScope.TreeScope_Descendants,
cacheRequestRuntimeId,
this);
...
public void HandleAutomationEvent(IUIAutomationElement sender, int eventId)
{
// Got a window opened event...
// Get the RuntimeId from the source element. Because that data is cached with the
// event, we don't have to call back through UIA into the provider process here.
int[] runtimeId = sender.GetCachedPropertyValue(propertyRuntimeId);
}
在一个侧面说明，当与事件（）通过调用诸如FindFirstBuildCache（，）解决实际的，我总是缓存中的数据时，或通过UIA访问元素，因为我想避免尽可能多的跨进程内通话成为可能。
On a side note, when practical, I always cache data when dealing with events or accessing elements through UIA, (by using calls such as FindFirstBuildCache(),) as I want to avoid as many cross-proc calls as possible.
所以，我的建议是：
- 使用本地的Windows API UIA与Tlbimp.exe将产生的托管包装。
- 缓存尽可能多的数据尽可能的活动，以避免不必要的后期回调到供应商的过程。
- 从一个UIA事件处理中避免回调到UIA。
谢谢，
盖伊
这篇关于一个同时监测申请后停止接收UI自动化事件，然后过一段时间后重新启动的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
