如何从NSView或NSWindow创建AXUIElementRef？ [英] How to create an AXUIElementRef from an NSView or NSWindow?

问题描述

关于macOS Accessibility API，是否有必要创建对应于NSView或NSWindow的AXUIElementRef？

似乎有一种方法在Carbon使用AXUIElementCreateWithHIObjectAndIdentifier的年代，但是该功能不再可用。

我知道的唯一方法是使用Accessibility API进行递归搜索应用程序的UI元素的整个层次结构，寻找与NSView或NSWindow相匹配的元素。但是除了是一个繁重的解决方案之外，它甚至无法保证成功，因为可能没有办法仅通过使用AXUIElementRef的可用属性来肯定地对应AXUIElementRef和Cocoa对象。

我愿意考虑可以帮助实现此目标的未记录的API。

解决方案

我找到了解决方法

我知道这不是您问题的直接答案，但我将尽力解释我在iOS中找到它的方法，希望您能够在macOS中执行相同的操作。此外，这可能对其他读者也有用...

我首先猜测过程本身正在创建 AXUIElementRef ，因此当我请求具有 AXUIElementRef 值的可访问性属性（例如 kAXUIElementAttributeChildChild ）时，它必须创建它们。 / p>

然后我创建了一个应用，并对 _AXUIElementCreateAppElementWithPid（int pid）进行了调用，并使用 [[[NSProcessInfo processInfo] processIdentifier] 。
我收到了根 AXUIElementRef ，然后将其传递给 AXError AXUIElementCopyMultipleAttributeValues（AXUIElementRef元素，CFArrayRef属性，AXCopyMultipleAttributeOptions选项，CFArrayRef _Nullable *值），请求 kAXUIElementAttributeChildren ，它就起作用了（应该在主线程上运行！）

我开始将 AXUIElementCopyMultipleAttributeValues 调用仔细调试到汇编代码中，该过程非常像这样（这是很伪的代码，当然……）：

  //序列化MIG调用
的参数，如果（axUIElementRef-> pid == self pid）{
 //调用我们自己的进程很好，我们可以轻松地进行调试！ 
 _MIGXAAXUIElementCopyMultipleAttributeValues（序列化参数）{
 //从AXUIElementRef获取原始元素：
 UIView * view = _AXElementForAXUIElementUniqueId（id）; 
 [view accessibilityAttributeValue：kAXUIElementAttributeChildren] {
 [view _accessibilityUserTestingChildren] {
 //由于这是UIApplication元素，因此只获取窗口：
 NSArray *< UIWindow *> ; winArr = [（UIApplication *）view _accessibilityWindows]; 
 //对于结果中的每个值，转换为AX值：
 ... 
 AXConvertOutgoingValue（winArr）{
 //对于每个UIView，转换为AXUIElementRef：
 AXUIElementRef e = _AXCreateAXUIElementWithElement（winArr [i]）; 
} 
} 
} 
} 
} else {
 //仅在获得授权并在过程之外
进行相同的操作} 
  

因此，在iOS中，您只需调用 AXUIElementRef _AXCreateAXUIElementWithElement（UIView *） ; 从UI转换为可访问性元素，然后 UIView * _AXElementForAXUIElementUniqueId（AXUIElementRefGetUniqueID（AXUIElementRef））; 相反。

所有符号均来自 AXRuntime.framework 。

在Mac中，您'需要链接到 ApplicationServices.framework 并尝试类似的方法。

希望这会有所帮助...

Concerning the macOS Accessibility API, is there anyway to create an AXUIElementRef that corresponds to either an NSView or an NSWindow?

There appears to have been a way of doing this back in the days of Carbon using AXUIElementCreateWithHIObjectAndIdentifier but that function isn't available anymore.

The only method I'm aware of is to use the Accessibility API to recursively search the entire hierarchy of UI elements of your application looking for one that matches the NSView or NSWindow. But in addition to being an onerous solution, it's not even guaranteed to succeed as there might not be a way to positively correspond an AXUIElementRef and a Cocoa object by just using the available attributes of the AXUIElementRef.

I am willing to consider undocumented APIs that help accomplish this.

解决方案

I found the way to do the same thing in iOS.

I know this isn't a direct answer to your question, but I'll try to explain what I have done to find it in iOS, and hopefully you'll be able to do the same in macOS. Also, this might be useful for other readers...

I started by guessing that the process itself is creating the AXUIElementRef, so it has to create them when I request accessibility attributes that have AXUIElementRef values, such as kAXUIElementAttributeChildren.

I then created an app, and dlsym'ed _AXUIElementCreateAppElementWithPid(int pid), calling it with [[NSProcessInfo processInfo] processIdentifier]. I received the root AXUIElementRef, which I then passed into AXError AXUIElementCopyMultipleAttributeValues(AXUIElementRef element, CFArrayRef attributes, AXCopyMultipleAttributeOptions options, CFArrayRef _Nullable *values), requesting kAXUIElementAttributeChildren, and it worked (should be run on main thread)!

I started debugging the AXUIElementCopyMultipleAttributeValues call carefully into the assembly code, which went pretty much like that (this is very pseudo-code, off course...):

// serialize the arguments for MIG call
if (axUIElementRef->pid == self pid) {
    // that's good that we are calling our own process, we can easily keep debugging!
    _MIGXAAXUIElementCopyMultipleAttributeValues(serialized arguments) {
         // Get the original element from the AXUIElementRef:
         UIView* view = _AXElementForAXUIElementUniqueId(id);
         [view accessibilityAttributeValue:kAXUIElementAttributeChildren] {
              [view _accessibilityUserTestingChildren] {
                   // since this is the UIApplication element, it just gets the windows:
                   NSArray*<UIWindow*> winArr = [(UIApplication*)view _accessibilityWindows];
                   // for each value in result, convert to AX value:
                   ...
                   AXConvertOutgoingValue(winArr) {
                       // For each UIView, convert to AXUIElementRef:
                       AXUIElementRef e = _AXCreateAXUIElementWithElement(winArr[i]);
                   }
              }
         }
    }
} else {
    // Do the same only if we are entitled, and outside our process
}

So, in iOS, you simply call AXUIElementRef _AXCreateAXUIElementWithElement(UIView*); to convert from UI to accessibility element, and UIView* _AXElementForAXUIElementUniqueId(AXUIElementRefGetUniqueID(AXUIElementRef)); in the opposite direction.

All symbols are from AXRuntime.framework.

In mac, you'll need to link against ApplicationServices.framework and try something similar.

Hope this helps...

这篇关于如何从NSView或NSWindow创建AXUIElementRef？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！