Python，Selenium和Chromedriver-使用find_element_by_id的无限循环会导致CPU问题 [英] Python, Selenium and Chromedriver - endless loop using find_element_by_id causes CPU problem

问题描述

大家好！我已经遇到这个问题一个星期了，但我认为我无法解决，而且我也看不到任何基于在线文章的解决方案。希望有人可以在这里为我提供帮助...

Good day to all! I've been experiencing this problem for a week now but I don't think I can solve it and I also do not see any solution based on articles online. Hopefully someone can help me here...

我的情况：
我需要在一页中监控6个不同表的价格几乎每秒都会改变。到一天结束时，我将关闭浏览器（按X按钮）并终止脚本（按Control + C），然后在早晨再次运行并让它运行一整天。该脚本是用python编写的，并且正在使用硒读取价格。我使用的浏览器是Chrome。我的操作系统是Windows 2008 R2； Selenium版本为3.14.1

My scenario: I need to monitor prices from 6 different tables in one page that changes almost every second. By end of day, I would close the browser (by pressing the X button) and terminate the script (by pressing Control+C) then run again in the morning and let it run through out the day. The script is written in python and is using selenium to read the prices. The browser I use is Chrome. My OS is Windows 2008 R2; Selenium version is 3.14.1

这是部分代码。它只是在 find_elements_by_id 内以1秒的间隔无限循环地读取表中的价格。

here is partial part of the code. It is just plainly reading the prices within the tables using find_elements_by_id inside an infinite loop with 1 second interval.

While True:
    close1 = float(browser.find_element_by_id('bnaBox1').find_elements_by_id('lastprc1')[0].text.encode('ascii','ignore'))
    close2 = float(browser.find_element_by_id('bnaBox2').find_elements_by_id('lastprc2')[0].text.encode('ascii','ignore'))
    close3 = float(browser.find_element_by_id('bnaBox3').find_elements_by_id('lastprc3')[0].text.encode('ascii','ignore'))
    close4 = float(browser.find_element_by_id('bnaBox4').find_elements_by_id('lastprc4')[0].text.encode('ascii','ignore'))
    close5 = float(browser.find_element_by_id('bnaBox5').find_elements_by_id('lastprc5')[0].text.encode('ascii','ignore'))
    close6 = float(browser.find_element_by_id('bnaBox6').find_elements_by_id('lastprc6')[0].text.encode('ascii','ignore'))
    time.sleep(1)
...

在运行的前几分钟内，脚本消耗的CPU量最少（大约20％到30％），但在几分钟后，消耗量缓慢上升到100％！除脚本外，计算机中没有其他进程在运行。

During the first few minutes of the run, the scripts consumes minimal amount of CPU (approx 20~30 percent) but after few more minutes, consumption slowly shoots up to 100%! There is no other processes running in the machine than the script.

到目前为止，我已完成故障排除（它们都不能解决我的问题）

Troubleshooting I've done so far (they all did not solve my issue)

• 将我的Chrome升级到最新版本-v71和chromerdriver 2.44


• 将Chrome浏览器还原到了以前的版本版本（v62，v68，v69，v70）


• 将Chromedriver版本降到2.42和2.43


• 清除了我的％TEMP％文件-


• 重新启动计算机（多次）

• upgraded my chrome to latest version - v71 and chromerdriver 2.44
• rolled back Chrome to previous versions (v62, v68, v69, v70)
• rolled back Chromedriver version to 2.42 and 2.43
• cleared my %TEMP% files -
• rebooted machine (multiple times)

该程序仅在表中获取值，但我怀疑在在后台运行脚本时，会堆积不必要的数据，从而导致CPU崩溃。

The program only gets values within tables but I suspect that somewhere in the background, as the the script runs, unnecessary data is piling-up which causes the CPU to hit the ceiling.

希望有人可以帮助我找出导致CPU出现此问题的原因并解决问题。

Hoping that someone can help me figure out what causes this problem in the CPU and resolve the issue.

推荐答案

要准确猜测 100％CPU使用率的确切原因，而又不能对代码块（特别是 WebDriver 配置）进行任何可视化，将很难。因此，答案将很大程度上基于通用准则，如下所示：

It would be tough to guess the exact reason of 100% CPU Usage without any visibility to your code blocks specifically the WebDriver configuration. So the answer will be pretty much based on generic guidelines as follows:

• 从不关闭浏览器（通过按X按钮）。始终在 tearDown（）{} 方法内调用 driver.quit（）来关闭&优雅地销毁 WebDriver 和 Web Client 实例。
  
  
  
  
  • 您可以在 PhantomJS Web驱动程序保留在内存中
  
  
  • Never close the browser (by pressing the X button). Always invoke driver.quit() within tearDown(){} method to close & destroy the WebDriver and Web Client instances gracefully.
    • You can find a detailed discussion in PhantomJS web driver stays in memory
    
    
    • 您可以在 Selenium：如何在不调用driver.quit（）的情况下停止geckodriver进程对PC内存的影响？
    
    
    • You can find a detailed discussion in Selenium : How to stop geckodriver process impacting PC memory, without calling driver.quit()?

    几个有用的 ChromeOptions（）及其用法如下：

    A couple of useful ChromeOptions() and their usage are as follows:

    options.addArguments("start-maximized"); // open Browser in maximized mode
    options.addArguments("disable-infobars"); // disabling infobars
    options.addArguments("--disable-extensions"); // disabling extensions
    options.addArguments("--disable-gpu"); // applicable to windows os only
    options.addArguments("--disable-dev-shm-usage"); // overcome limited resource problems
    options.addArguments("--no-sandbox"); // Bypass OS security model
    

  
  

  • 使用 time.sleep（1）是很大的否。

    
    
    • 您可以在如何在python中休眠webdriver来找到详细的讨论毫秒
    
    
    • You can find a detailed discussion in How to sleep webdriver in python for milliseconds
    
    
    • 您可以在限制Chrome无头CPU和内存的使用量
    
    
    • You can find a detailed discussion in Limit chrome headless CPU and memory usage
    
    
    • 将 ChromeDriver 升级到当前的 ChromeDriver v2.44 级。
    
    
    • 将 Chrome 版本保持在 Chrome v69-71 级别之间。 （根据ChromeDriver v2.44发行说明）
    $ b通过您的 IDE 和 Rebuild 仅通过必需的依赖项$ b
    • 清理您的项目工作区。 / li>
      
    • 如果您的基本 Web客户端版本太旧，请通过 Revo卸载程序 ，并安装最新的GA和 Web Client 的发行版。
    
    
    • 进行系统重启。
    
    
    • 执行 @Test 。
    
    
    • Upgrade ChromeDriver to current ChromeDriver v2.44 level.
    • Keep Chrome version between Chrome v69-71 levels. (as per ChromeDriver v2.44 release notes)
    • Clean your Project Workspace through your IDE and Rebuild your project with required dependencies only.
    • If your base Web Client version is too old, then uninstall it through Revo Uninstaller and install a recent GA and released version of Web Client.
    • Take a System Reboot.
    • Execute your @Test.
    
    
    • （仅限Windows操作系统）使用 CCleaner 工具清除所有在执行 Test Su之前和之后的操作系统杂务ite 。
    
    
    • （仅仅LinuxOS ）在执行以下命令之前和之后释放并释放Ubuntu / Linux Mint中未使用/缓存的内存您的 Test Suite 。
    
    
    • (WindowsOS only) Use CCleaner tool to wipe off all the OS chores before and after the execution of your Test Suite.
    • (LinuxOS only) Free Up and Release the Unused/Cached Memory in Ubuntu/Linux Mint before and after the execution of your Test Suite.

    这篇关于Python，Selenium和Chromedriver-使用find_element_by_id的无限循环会导致CPU问题的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！