如何读取窗口内容(使用accessibilityService)并在Android中使用draw over其他应用程序权限来调用UI? [英] How to read window content (using accessibilityService) and evoking UI using draw over other app permission in Android?
问题描述
我关于同一主题的
现在我想从社区了解什么:
- 如何在辅助功能"屏幕和产品详细信息页面上显示我的帮助 UI.
- 如何检测产品页面何时出现在屏幕上并唤起我的浮动按钮.
- 如何获取屏幕上显示的产品名称.
- 如何仅针对某些应用限制此屏幕阅读功能?(因为我不想以版权问题告终)
- 有什么教程可以帮助我吗?虽然我已经在 Google 上尝试过任何直接教程，但没有取得任何成功.
现在为什么我需要这些信息:
我正在为学生计划一个应用程序，如果它在我的服务器上或某个可用的网络位置上可用，它将帮助他们免费获得所需的(或类似的)书籍(电子书).由于某些书籍的版权问题，我想将其功能限制在某些应用程序中.
现在我从我对这个主题的研究中了解到的(但我不确定我是否在正确的轨道上)
我曾尝试在博客文章中总结我是如何解决问题的.任何需要帮助的人都可以查看.
我之前的回答被版主删除了，可能是因为我刚刚发布了一个博客链接.所以我也在这里再次发布我的答案并提供详细信息.
最简单的方法是android自己提供的.如果您正在自己的应用程序上构建某些东西，这是最好和最简单的方法.你必须使用findAccessibilityNodeInfosByViewId()"如下所述
@Override公共无效 onAccessibilityEvent(AccessibilityEvent 事件){AccessibilityNodeInfo source = event.getSource();如果(源==空){返回;}列表findAccessibilityNodeInfosByViewId = source.findAccessibilityNodeInfosByViewId("您的包裹名称:id/RESOURCE ID FROM WHERE YOU WANT DATA");如果(findAccessibilityNodeInfosByViewId.size()> 0){AccessibilityNodeInfo parent = (AccessibilityNodeInfo) findAccessibilityNodeInfosByViewId.get(0);//如果所需数据在视图层次结构中很深，您也可以遍历列表.String requiredText = parent.getText().toString();Log.i("必填文本", requiredText);}} 如果您正在其他应用程序上构建某些内容并且不知道 res id.您必须使用父级、子级计数、数据所在级别、事件类型的组合来构建逻辑.您必须在事件或来源或上述值的组合中寻找模式，才能获得所需的宝贵数据.此外，您必须在代码中设置一些调整以根据您的任务获得准确的数据.就像在我们的任务中，我们有一些正则表达式来获取宝贵的数据.
您必须注意，如果视图或 UI 更改或 res id 更改，您当前的逻辑可能会失败.因此，您必须密切关注正在构建服务的应用程序.以下是一些示例代码，我们在其中获取没有 res id 的数据，可能有助于您了解其工作原理.
打印源和事件
@Override公共无效 onAccessibilityEvent(AccessibilityEvent 事件){AccessibilityNodeInfo source = event.getSource();如果(源==空){返回;}Log.i("事件", event.toString() + "");Log.i("Source", source.toString() + "");}获取childcount
source.getChildCount();如果孩子数 >0，您可能需要调查一下.
示例代码 1
if (level == 0 && source.getClassName().equals("android.widget.TextView") && source.getText()!=null && !source.getText().toString().isEmpty()){//这里的级别是for循环的迭代String recivedText = source.getText().toString();if(source.getClassName().equals("android.widget.TextView") && source.getParent()!=null && source.getParent().getClassName().equals("android.widget.FrameLayout") && source.getParent().getParent()==null){返回接收文本；}}示例代码 2
if (source.getPackageName().equals("PACKAGE NAME OF APP FOR WHICH YOUR EXPECTING EVENT")) {如果(event.getEventType()== AccessibilityEvent.TYPE_WINDOW_STATE_CHANGED和放大器;&安培;活动类包名全名(如果你想为一些特定的屏幕)"的equals(event.getClassName())){if(source.getText()!=null && source.getClassName().equals("android.widget.TextView") && source.getParent()!=null && source.getParent().getClassName().equals("android.widget.RelativeLayout")&&source.getParent().getParent()!=null &&source.getParent().getParent().getClassName().equals("android.widget.ScrollView") && source.getParent().getParent().getParent()!=null && source.getParent().getParent().getParent().getClassName().equals("android.support.v4.view.ViewPager")&& source.getParent().getParent().getParent().getParent()!=null && source.getParent().getParent().getParent().getParent().getClassName().equals("android.widget.FrameLayout")&&source.getParent().getParent().getParent().getParent().getParent()==空){返回 source.getText().toString();}}这有点笨拙，但要使其正常工作，您必须深入查看日志以找到可将您带到所需数据的模式.
2017 年 11 月 20 日更新谷歌将暂停所有使用无障碍服务的应用程序用于无障碍功能以外的其他事情.我收到了一封关于我的一个应用程序的电子邮件，其他开发人员也收到了来自谷歌的相同电子邮件(Reddit).所以，我认为我们应该寻找一些其他的做事方式，我请求你也在这里更新它，如果有人提出其他实现来实现相同的行为，它也会帮助其他人.
谢谢乌梅什·乔汉
My last question about the same topic was not clear enough and was put on hold by community and later it was automatically deleted. So, I am explaining that Question in detailed manner so that community can understand and help in a better manner.
I want functionality similar to Voodoo App and MySmartPrice offers.
Now what they do
Step 1: When we open the Voodoo App for the first time they show a small tutorial. At the end of tutorial there is a "Activate Now" button which when pressed take us to Accessibility settings screen.
Step 2: In Accessibility Screen it further guides how to find and unable the Voodoo Service.
Step 3: When we enable it, it further ask to grant "Observe Your Action" and "Retrieve Window Content" permissions.
Step 4: Once we are done granting permissions on accessibility screen, and move to some shopping app or access shopping site via browser and reach product page a voodoo floating button pops up automatically.
Step 5: Once we click on it, it shows the price of same/related/similar products available on other apps or website so that user can check for best deal available.
Reference Screenshot of Voodoo
Now what I want to know form community:
- How I can show my help UI over Accessibility screen and product detail page.
- How to detect when a product page is there on screen and to evoke my floating button.
- How to fetch name of product shown on screen.
- How to restrict this screen reading functionality for some apps only? (as I don't want to end up in some copyright issue)
- Is there any tutorial which can help me out? Although I had already tried on Google for any direct tutorial and didn't got any success.
Now why I need this information:
I am planning a app for students which will help them to get there desired (or similar) book (ebook) for free if it is available on my server or on some web location which is available web. I want to restrict it the functionality to some apps only due to copyright issues on some books.
Now what I know from my study regarding this topic (but I am not sure whether I am on right track or not)
- I can take help form Accessibility Service
- Draw Over Other Apps. Example 1 , Example 2
I had tried to summarize how i was able to get thru the problem in a blog post. Anyone who needs some help can take a look into it.
Accessibility Service to the next level
My Previous answer was deleted by moderators may due to i just posted a link to blog. So i am posting my answer again with details here too.
The easiest way is provided by android itself. If you are building something on your own app this is the best and easiest possible way of doing it. You have to use "findAccessibilityNodeInfosByViewId ()" as explained below
@Override
public void onAccessibilityEvent(AccessibilityEvent event) {
AccessibilityNodeInfo source = event.getSource();
if (source == null) {
return;
}
List<AccessibilityNodeInfo> findAccessibilityNodeInfosByViewId = source.findAccessibilityNodeInfosByViewId("YOUR PACKAGE NAME:id/RESOURCE ID FROM WHERE YOU WANT DATA");
if (findAccessibilityNodeInfosByViewId.size() > 0) {
AccessibilityNodeInfo parent = (AccessibilityNodeInfo) findAccessibilityNodeInfosByViewId.get(0);
// You can also traverse the list if required data is deep in view hierarchy.
String requiredText = parent.getText().toString();
Log.i("Required Text", requiredText);
}
}
If you are building something on other apps and don’t know the res id. You have to build a logic with the combination of parent, child count, level on which your data is found, type of event. You have to look for a pattern in event or source or the combination of above said values to get the required data preciously. Further, you have to setup some tweak in your code to get the accurate data according to your task. Like in our task we have some regex to get precious data.
You have to take care that if the view or UI is changed or res ids are changed your current logic might fail. So, you have to keep a close look at the app on which you are building your service. Following are some example code in which we are getting data without res id that might be useful for you to understand how it will work.
To print source and event
@Override
public void onAccessibilityEvent(AccessibilityEvent event) {
AccessibilityNodeInfo source = event.getSource();
if (source == null) {
return;
}
Log.i("Event", event.toString() + "");
Log.i("Source", source.toString() + "");
}
To get the childcount
source.getChildCount();
If child count is >0 you might have to look into it.
Example Code 1
if (level == 0 && source.getClassName().equals("android.widget.TextView") && source.getText()!=null && !source.getText().toString().isEmpty()){
// here level is iteration of for loop
String recivedText = source.getText().toString();
if(source.getClassName().equals("android.widget.TextView") && source.getParent()!=null && source.getParent().getClassName().equals("android.widget.FrameLayout") && source.getParent().getParent()==null){
return recivedText;
}
}
Example Code 2
if (source.getPackageName().equals("PACKAGE NAME OF APP FOR WHICH YOUR EXPECTING EVENT")) {
if(event.getEventType()==AccessibilityEvent.TYPE_WINDOW_STATE_CHANGED && "COMPLETE NAME OF ACTIVITY CLASS WITH PACKAGE NAME (if you want it for some specific screen)".equals(event.getClassName())){
if(source.getText()!=null && source.getClassName().equals("android.widget.TextView") && source.getParent()!=null && source.getParent().getClassName().equals("android.widget.RelativeLayout")&& source.getParent().getParent()!=null && source.getParent().getParent().getClassName().equals("android.widget.ScrollView") && source.getParent().getParent().getParent()!=null && source.getParent().getParent().getParent().getClassName().equals("android.support.v4.view.ViewPager")&& source.getParent().getParent().getParent().getParent()!=null && source.getParent().getParent().getParent().getParent().getClassName().equals("android.widget.FrameLayout")&& source.getParent().getParent().getParent().getParent().getParent()==null){
return source.getText().toString();
}
}
It is bit clumsy but to get it working you have to look deep into logs to find a pattern which will take you to your required data.
Update 20/11/2017 Google is suspending all apps using accessibility services for things other than accessibility. I had received a email regarding the same for one of my app and other developers are also receiving same email from google (Reddit). So, I think we should look for some other way of doing things and I request you to please update it here also, if some one comes up with other implementation to achieve same behavior it would help others also.
Thanks Umesh Chauhan
这篇关于如何读取窗口内容(使用accessibilityService)并在Android中使用draw over其他应用程序权限来调用UI?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
