获取图像中特定区域的主导颜色：网页元素的颜色查询 [英] Get dominating color of a specific area in an image: Color Query for Web Page Elements

问题描述

问题描述

我编写一个Java应用程序，让程序员通过指定可见属性来查询网页上的页面元素。最重要和最困难的之一是颜色。

具体来说，我需要一种方法来获得用户可见的颜色的网页元素使用Selenium 2和webdriver。我想要能够查询颜色值（＃ff0000 ）或名称（红色）。

一个参数应该控制需要主导的类似颜色的百分比。如果设置为 100％，则不允许该元素具有任何其他颜色。如果设置为 50％，则元素需要填充一半的颜色。

控制这些颜色的容差。具有更高的容忍度， red 也可以匹配Stackoverflow上的橙色提问按钮。

/ h2>

给出了众所周知的Stackoverflow网页，我突出显示了要检查的页面元素：

具有较高的颜色容差和不太高的支配百分比，以下查询应返回指定的结果：

  color（'＃FFEFC6'）//完全匹配：true 
 color（'yellow'）//匹配公差范围：true 
 color（'orange'）// true 
 color（'blue'）// false 
 color（'green'）// false 
  

我的第一种方法

最好使用 color 和 background-color 。但是这些不考虑图像，这是良好的颜色查询所需要的。此外，它们可能产生困难，因为css选择器继承和透明度的处理。此外，在当前元素之上具有高于 z-index 的绝对定位的元素可能产生意外的结果。

是要检查的网页元素。它在JavaScript驱动程序的Java绑定中表示为JavaScript DOM元素（或JQuery对象）或 RemoteWebElement 。

可以采取自动屏幕截图的网页（我使用Firefox）的当前状态，参见这里：使用Selenium WebDriver截图

要检查的页面元素的坐标是已知的。因此，屏幕截图图像可以裁剪为该大小和区域，并以某种方式分析以检查查询是否返回真或假。

实现

在这种情况下，我不限于Java。 JavaScript 会非常好，因为我在JQuery的帮助下做其他查询。性能重要。我指望你，我恐怕这是一个非常困难的任务。因此我需要你的输入。

UPDATE

我通过截取屏幕截图并分析像素数据相关部分。这样我可以处理各种背景图像和透明度。它是Abmash框架的一部分，它是开源的，可供任何人免费使用： GmHub上的Abmash

解决方案

最简单的方法：

• 屏幕截图（保存在内存中）


• 剪切元素的截图top = el.offsetTop，left = el.offsetLeft，width = el.offsetWidth，height = el.offsetHeight


• 获取裁剪图像的像素数据


• 循环遍历像素，获得R，G，B元素的总和，然后将总和除以像素数得到平均值。

如果您真的想使用JavaScript

• 如果您打算在JavaScript中进行最后检查，您可以将像素数据发送到JavaScript进行处理。


• IMAGE剪裁图像的URI。然后将该IMG绘制到CANVAS，然后通过ctx2d.getImageData（...）循环通过像素

元素是IMG或具有背景图像CSS。只是使用颜色和背景颜色CSS检查否则。

Problem Description

I am writing a Java application that lets programmers query for page elements on a web page by specifying visible attributes. One of the most important and difficult is Color.

To be specific, i need a way to get the user-visible color of web page elements using Selenium 2 and Webdriver. I want to be able to query for color values (#ff0000) or names (red).

One parameter should control the percentage of similar colors needed to be "dominating" enough. If set to 100% the element is not allowed to have any other color. If set to 50% the element needs to be halfway filled with the color.

There should be another parameter to control the "tolerance" of these colors. With a higher tolerance, red could also match the orange "Ask Question" button here on Stackoverflow.

Example

Given the well-known Stackoverflow web page, i highlighted the page element to check: With a higher color tolerance and a not too high domination percentage, the following queries should return the specified result:

color('#FFEFC6') // exact match: true
color('yellow') // match in tolerance range: true
color('orange') // true
color('blue') // false
color('green') // false

My first approach

Best bet would be using CSS attributes like color and background-color. But these do not take images into account, which are needed for good color queries. Also, they could produce difficulties because of css selector inheritance and the handling of transparency. In addition, absolutely positioned elements with a higher z-index above the current element could produce unexpected results.

Given is the web page element to check. It is represented either as JavaScript DOM element (or JQuery object) or as RemoteWebElement in the Java bindings of Webdriver.

It is possible to take automated Screenshots of the current state of the web page (i am using Firefox), see here: Take a screenshot with Selenium WebDriver

The coordinates of the page element to check are known. Therefore, the screenshot image could cropped to that size and area and be analyzed somehow to check if the query returns true or false.

Implementation

I am not limited to Java in this case. JavaScript would be very nice because i am doing the other queries with the help of JQuery too. Performance matters. I am counting on you, i fear this is a very difficult task. Therefore i need your input.

UPDATE

I solved this issue by taking screenshots and analyzing the pixel data of the relevant part. That way i can deal with all kinds of background images and transparency. It's part of the Abmash framework, which is open source and free to anybody to use: Abmash on Github

解决方案

Easiest way:

• Get screenshot (save in memory)
• Crop screenshot to the element top = el.offsetTop, left = el.offsetLeft, width = el.offsetWidth, height = el.offsetHeight
• Get the pixel data for the cropped image
• Loop through the pixels getting the total sums of the R, G, B elements then divide the total sums by the pixel count to get the average. Test the average color against your constraints.

If you really want to use JavaScript

• You could send the pixel data to JavaScript for processing if you're intent on doing the final check in JavaScript.
• Or you could send JavaScript the IMAGE URI for the cropped image. Then draw that IMG to a CANVAS then loop through the pixel with ctx2d.getImageData(...)

Only do the above if the element is an IMG or a has a background-image CSS. Just use color and background-color CSS checks otherwise.

这篇关于获取图像中特定区域的主导颜色：网页元素的颜色查询的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！