双轨并行任务时死亡情况的webdriver [英] Track dead WebDriver instances during parallel task

问题描述

我看到一些死者实例怪事运行使用硒的webdriver，简单的例子是，比如并行嵌套循环网络压力测试，打每100次展示300独特的网页。

我成功获得4 - 8 webdriver的情况下使用将一个的ThreadLocal< FirefoxWebDriver> 来隔离它们每个任务线程，并MaxDegreeOfParallelism在ParallelOptions实例限制螺纹。我在的ThreadLocal<分区，只有并行外环（页面的集合），并检查 .IsValueCreated ;> 每个分区的长期运行的任务方法的开头容器内。为了便于以后清理，我每一个新的实例添加到由线程ID键控ConcurrentDictionary。

不管我用什么并行化或分区策略，webdriver的情况下，偶尔会做一以下内容：

• 启动，但从来没有显示一个URL或运行一个印象


• 启动，运行任何展示次数罚款，然后只是闲置在某个时刻

在其中任一发生，并行循环的最后的似乎注意到一个线程没有做任何事情，它产生一个新的分区。如果的 N 的是允许的线程数量，这导致其的 N 的生产线只是时间约为50-60％。

清理仍然工作在最后罚款;有可能2n个打开的浏览器或以上，但生产性和非生产性的人都得到清理。

有没有一种方法来监视这些无用的webdriver实例和）清除它们向右走，加上B）得到并行循环立即更换任务段，而不是落后了好几分钟，因为它往往现在呢？

解决方案

由于没有 OnReady 事件，也不是的isReady 属性，我的工作围绕它通过创建每个实例后，睡在线程几秒钟。 。这样做，似乎给我100％耐用，运转情况的webdriver

感谢您的建议，我实现了的isReady 功能-source项目 Webinator 。使用，如果你想，或者使用下面列出的代码。

我试过实例25个实例，所有这些都是功能性的，所以我在算法相当有信心在这一点（我的杠杆作用HtmlAgilityPack，看是否存在的元素，但在这里我要跳过它为简单起见）：

 公共无效WaitForReady（IWebDriver司机）
 {
 VAR JS = @{VAR临时=使用document.createElement（'DIV'）; temp.id ='browserReady'; + 
 @B = document.getElementsByTagName（体）[0]; b.appendChild（临时）;}; 
（（IJavaScriptExecutor）驱动程序）.ExecuteScript（JS）; 
 
 WaitForSuccess（（）=> 
 {
 IWebElement元= NULL; 
试
 {
元素= driver.FindElement（通过.ID（browserReady））;！
} 
抓
 {
 //未找到要素
} 
 
返回元素=空; 
}，
 timeoutInMilliseconds：10000）; 
 
 JS = @{VAR临时=的document.getElementById（'browserReady'）; + 
 @temp.parentNode.removeChild（TEMP）;}; 
（（IJavaScriptExecutor）驱动程序）.ExecuteScript（JS）; 
} 
 
私人布尔WaitForSuccess（Func键<布尔>的行动，诠释timeoutInMilliseconds）
 {
如果（动作== NULL）返回false; 
 
布尔成功; 
 const int的PollRate = 250; 
 VAR maxTries = timeoutInMilliseconds / PollRate; 
 INT试图= 0; 
做
 {
成功=行动（）; 
试图++; 
如果（成功&安培;！&安培;尝试< = maxTries）
 {
 Thread.sleep代码（PollRate）; 
} 
} 
，而（成功&安培;！&安培;尝试< maxTries）; 
返回成功; 
} 
  

的假设是，如果浏览器响应的JavaScript功能，并寻找元素，那么它可能是一个可靠的实例，并准备好被使用。

I am seeing some dead-instance weirdness running parallelized nested-loop web stress tests using Selenium WebDriver, simple example being, say, hit 300 unique pages with 100 impressions each.

I'm "successfully" getting 4 - 8 WebDriver instances going using a ThreadLocal<FirefoxWebDriver> to isolate them per task thread, and MaxDegreeOfParallelism on a ParallelOptions instance to limit the threads. I'm partitioning and parallelizing the outer loop only (the collection of pages), and checking .IsValueCreated on the ThreadLocal<> container inside the beginning of each partition's "long running task" method. To facilitate cleanup later, I add each new instance to a ConcurrentDictionary keyed by thread id.

No matter what parallelizing or partitioning strategy I use, the WebDriver instances will occasionally do one of the following:

• Launch but never show a URL or run an impression
• Launch, run any number of impressions fine, then just sit idle at some point

When either of these happen, the parallel loop eventually seems to notice that a thread isn't doing anything, and it spawns a new partition. If n is the number of threads allowed, this results in having n productive threads only about 50-60% of the time.

Cleanup still works fine at the end; there may be 2n open browsers or more, but the productive and unproductive ones alike get cleaned up.

Is there a way to monitor for these useless WebDriver instances and a) scavenge them right away, plus b) get the parallel loop to replace the task segment immediately, instead of lagging behind for several minutes as it often does now?

解决方案

Since there is no OnReady event nor an IsReady property, I worked around it by sleeping the thread for several seconds after creating each instance. Doing that seems to give me 100% durable, functioning WebDriver instances.

Thanks to your suggestion, I've implemented IsReady functionality in my open-source project Webinator. Use that if you want, or use the code outlined below.

I tried instantiating 25 instances, and all of them were functional, so I'm pretty confident in the algorithm at this point (I leverage HtmlAgilityPack to see if elements exist, but I'll skip it for the sake of simplicity here):

public void WaitForReady(IWebDriver driver)
{
    var js = @"{ var temp=document.createElement('div'); temp.id='browserReady';" +
             @"b=document.getElementsByTagName('body')[0]; b.appendChild(temp); }";
    ((IJavaScriptExecutor)driver).ExecuteScript(js);

    WaitForSuccess(() =>
    {
        IWebElement element = null;
        try
        {
            element = driver.FindElement(By.Id("browserReady"));
        }
        catch
        {
            // element not found
        }

        return element != null;
    },
    timeoutInMilliseconds: 10000);

    js = @"{var temp=document.getElementById('browserReady');" +
         @" temp.parentNode.removeChild(temp);}";
    ((IJavaScriptExecutor)driver).ExecuteScript(js);
}

private bool WaitForSuccess(Func<bool> action, int timeoutInMilliseconds)
{
    if (action == null) return false;

    bool success;
    const int PollRate = 250;
    var maxTries = timeoutInMilliseconds / PollRate;
    int tries = 0;
    do
    {
        success = action();
        tries++;
        if (!success && tries <= maxTries)
        {
            Thread.Sleep(PollRate);
        }
    }
    while (!success && tries < maxTries);
    return success;
}

The assumption is if the browser is responding to javascript functions and is finding elements, then it's probably a reliable instance and ready to be used.

这篇关于双轨并行任务时死亡情况的webdriver的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！