如何在Java和Selenium中找到画布上按钮的坐标，然后单击它们? [英] How to find the coordinates of the buttons on a canvas, and click on them after using Java and Selenium?

问题描述

我有一个在线计算器，我想自动执行一些操作，例如减法，除法等，但事实是，因为它是canvas应用程序计算器，所以没有任何元素(下面的链接).我想知道如何单击按钮以使某些操作自动化?

I have an online calculator that i want to automate some operations for, like subtraction, division, etc. but the thing is that there are no elements since it is a canvas app calculator (link below). I'm wondering how can i click the buttons so im able to automate some operations?

在线计算器正在尝试实现自动化:

https://www.online-calculator.com/full-screen-计算器/

画布HTML代码

<canvas id="canvas" width="335" height="434" style="position: absolute; display: block; background-color: rgb(255, 255, 255); width: 167.939px; height: 217px;"></canvas>

我的Selenium-Java代码

driver.get("https://www.online-calculator.com/full-screen-calculator/");
driver.manage().window().maximize();        
WebElement li = driver.findElements(By.tagName("iframe")).get(0);
driver.switchTo().frame(li);        
WebElement canv = driver.findElements(By.tagName("canvas")).get(0);
System.out.println(canv.getSize());
System.out.println(canv.getLocation());

try {
 Actions builder = new Actions(driver);
 builder.moveToElement(canv, x, y);
 builder.click();
 builder.perform();
    } catch (Exception e) 
    {
      // do nothing
    }

因此，如您在上面的代码中看到的那样，我仍然必须找到我要运行的操作的x和y.例如，如果我想计算"10-4"，然后验证它等于6，那么我如何找到按钮10、4和减法运算符-"的坐标.我们将不胜感激.谢谢:)

So as you see in above code, I still have to find the x and y of the operations i want to run. For ex, how can i find the coordinates of the buttons 10, 4 and the subtraction operation '-', if I want to calculate '10 - 4' and then validate that it is equal to 6. Any help would be highly appreciated. Thanks :)

注意:如果窗口大小发生变化((稍后我想锁定窗口大小，以便在不同的屏幕上进行测试时不会感到困惑))，画布的宽度，高度和位置都会发生变化.

Note: The canvas width, height and location will change if the window size changes ((Im thinking later of locking the window size so my tests are not flaky on different screens)).

推荐答案

<canvas>元素在<iframe>中.因此，要对<canvas>中的元素调用click()，您必须:

The <canvas> element is within an <iframe>. So to invoke click() on the elements within the <canvas> you have to:

• 诱导 WebDriverWait 以使所需的框架可用并切换到.
• 诱导 WebDriverWait 以使所需的元素可点击.

• 您可以使用以下解决方案:

• Induce WebDriverWait for the desired frame to be available and switch to it.
• Induce WebDriverWait for the desired element to be clickable.

• You can use the following solution:

• 代码块:

• Code Block:

driver.get("https://www.online-calculator.com/full-screen-calculator/")
new WebDriverWait(driver, 10).until(ExpectedConditions.frameToBeAvailableAndSwitchToIt(By.id("fullframe")));
WebElement canvas = new WebDriverWait(driver, 10).until(ExpectedConditions.elementToBeClickable(By.id("canvas")));

该元素仅是用于图形的容器，并且是HTML页面上的矩形区域.默认情况下，画布没有边框且没有内容.但是，指定了id属性(将在脚本中引用)，width和height属性来定义画布的大小.要添加边框，请使用style属性.一个例子:

The element is only a container for graphics and is a rectangular area on an HTML page. By default, a canvas has no border and no content. However, an id attribute (to be referred to in a script), a width and height attribute are specified to define the size of the canvas. To add a border, the style attribute is used. An example:

<canvas id="myCanvas" width="200" height="100" style="border:1px solid #000000;"></canvas>

HTML画布是一个二维网格.画布的左上角的坐标为(0,0).

The HTML canvas is a two-dimensional grid. The upper-left corner of the canvas has the coordinates (0,0).

在文章使用Selenium WebDriver自动测试HTML5 Canvas应用程序 @Aaron Mulder提到，可以使用

In the article Automated Testing of HTML5 Canvas Applications with Selenium WebDriver @Aaron Mulder mentions, interacting with the elements within <canvas> is possible using event support of the Actions Class API:

• moveToElement(WebElement target, int xOffset, int yOffset): Moves the mouse to an offset from the top-left corner of the element.

new Actions(driver).moveToElement(canvas, xWithinCanvas, yWithinCanvas).click().perform();

但是并不是100％可靠的，因为每个mouse down，mouse up或mouse click都发生在元素的中心处.因此，上面的代码会在提供的坐标( x ， y )中产生一个mouse move事件，然后在<canvas>，然后是<canvas>中心的mouse down，mouse up和click.对于<button>来说应该没问题，但对于<canvas>来说却不值得，因为您想要在其中单击特定位置.

But is not 100% reliable as, every mouse down, mouse up, or mouse click happens at the center of the element. So the code above produces a mouse move event to the provided coordinates (x,y), then a mouse move event to the center of the <canvas>, then a mouse down, mouse up, and click at the center of the <canvas>. That should have been fine for a <button> but is not worth for a <canvas>, where you want to be able to click at a specific location.

解决方法是使用JavaScript调度合成的鼠标事件，如下所示:

The workaround, is to dispatch synthesized mouse events using JavaScript as follows:

// pageX and pageY are offsets which you need to know through mouse coordinates.
((JavascriptExecutor)driver).executeScript("var evt = $.Event('click', { pageX: " + x +
   ", pageY: " + (y + 55) + " } );" +
   "$('#myCanvas').trigger(evt);");

但是，要单击<canvas>中的元素，您可以使用 firefox ，因为mouse move事件在Firefox中运行良好，并且可以避免使用鼠标坐标作为事件处理，如下所示:

However, to click on the elements within the <canvas> you can be at ease using firefox as the mouse move event works well in Firefox and you can avoid using the mouse coordinates as the event processing as follows:

new Actions(driver).moveToElement(
   canvas, xWithinCanvas, yWithinCanvas).perform();
((JavascriptExecutor)driver).executeScript("$('#canvas').click();");

---

此用例

要自动执行 subscription 操作，例如 3 - 1 = 使用

---

This usecase

To automate a substruction operation e.g. 3-1= using Selenium you can use the following solution:

• 代码块:

• Code Block:

driver.get("https://www.online-calculator.com/full-screen-calculator/");
new WebDriverWait(driver, 10).until(ExpectedConditions.frameToBeAvailableAndSwitchToIt(By.id("fullframe")));
WebElement canvas = new WebDriverWait(driver, 10).until(ExpectedConditions.elementToBeClickable(By.id("canvas")));
//clicking on 3
new Actions(driver).moveToElement(canvas, 0, 0).moveByOffset(0,(255/6)*3).click().build().perform();
//clicking on the substract sign (-)
new Actions(driver).moveToElement(canvas, 0, 0).moveByOffset((174/5)*2,(255/6)*3).click().build().perform();
//clicking on 1
new Actions(driver).moveToElement(canvas, 0, 0).moveByOffset(-(174/5)*4,(255/6)*3).click().build().perform();
//clicking on equals to sign (=)
new Actions(driver).moveToElement(canvas, 0, 0).moveByOffset((174/5)*4,(255/6)*4).click().build().perform();

执行视频:

Execution Video:

您可以在以下位置找到几个相关的详细讨论:

You can find a couple of relevant detailed discussion in:

• 如何执行鼠标滚轮在Selenium中的HTML5 Canvas上滚动?

这篇关于如何在Java和Selenium中找到画布上按钮的坐标，然后单击它们?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！