Selenium 在 Firefox 中使用过多内存 [英] Selenium using too much RAM with Firefox

问题描述

我在 Firefox 中使用 selenium 来自动化 Instagram 上的一些任务.它基本上会在用户个人资料和通知页面之间来回切换，并根据找到的内容执行任务.

它有一个无限循环，可确保任务继续进行.我每隔几步就有 sleep() 函数，但内存使用量不断增加.我在 Python 中有这样的东西:

while(True):预期条件()...doTask()driver.back()预期条件()...doAnotherTask()driver.forward()预期条件()

我从不关闭驱动程序，因为这会大大减慢程序的速度，因为它有很多查询要处理.有什么办法可以不关闭或不退出驱动程序，不让内存使用增加超时?

添加了明确的条件，但这也无济于事.我正在使用 Firefox 的无头模式.

解决方案

从 Selenium 开始，对 Firefox 使用的 RAM 的控制很少.正如您提到的，浏览器客户端，即 Mozilla 在 Instagram 上的用户个人资料和通知页面之间来回切换，并根据它发现的内容执行任务作为单个用例过于广泛.因此，首要任务是将与您的用例相关的无限循环分解为更小的测试.

<小时>

time.sleep()

引入 time.sleep() 实际上是掩盖了根本问题.但是在使用 Selenium 和

然而，根据本次讨论，一些用户认为 使用的内存越多越好，因为这意味着您不会浪费 RAM.Firefox 使用 RAM 来加快其进程，因为应用程序数据在 RAM 中的传输速度要快得多.

<小时>

解决方案

您可以执行以下任一/所有通用/特定步骤:

• 将 Selenium 升级到当前级别版本 3.141.59.
• 将GeckoDriver 升级到GeckoDriver v0.24.0 级别.
• 将 Firefox 版本升级到 Firefox v65.0.2 级别.
• 清理您的项目工作区，通过您的 IDE 并重建您的项目，仅使用所需的依赖项.
• 如果您的基本 Web Client 版本太旧，请将其卸载并安装 Web Client 的最新 GA 和发布版本.

• 某些扩展允许您阻止此类不必要的内容，例如:

  • uBlock Origin 允许您隐藏广告网站.
  • NoScript 允许您有选择地启用和禁用所有脚本在网站上运行.

  • 要打开带有扩展程序的 Firefox 客户端，您可以从 https://addons.mozilla.org 并使用 add_extension(extension='webdriver.xpi') 方法将扩展添加到一个 FirefoxProfile 如下:

    from selenium import webdriverprofile = webdriver.FirefoxProfile()profile.add_extension(extension='extension_name.xpi')driver = webdriver.Firefox(firefox_profile=profile, executable_path=r'C:path	ogeckodriver.exe')

• 如果您的测试不需要CSS，您可以在这个讨论.

I am using selenium with Firefox to automate some tasks on Instagram. It basically goes back and forth between user profiles and notifications page and does tasks based on what it finds.

It has one infinite loop that makes sure that the task keeps on going. I have sleep() function every few steps but the memory usage keeps increasing. I have something like this in Python:

while(True):
    expected_conditions()
    ...doTask()
    driver.back()
    expected_conditions()
    ...doAnotherTask()
    driver.forward()
    expected_conditions()

I never close the driver because that will slow down the program by a lot as it has a lot of queries to process. Is there any way to keep the memory usage from increasing overtime without closing or quitting the driver?

EDIT: Added explicit conditions but that did not help either. I am using headless mode of Firefox.

解决方案

To start with Selenium have very little control over the amount of RAM used by Firefox. As you mentioned the Browser Client i.e. Mozilla goes back and forth between user profiles and notifications page on Instagram and does tasks based on what it finds is too broad as a single usecase. So, the first and foremost task would be to break up the infinite loop pertaining to your usecase into smaller Tests.

---

time.sleep()

Inducing time.sleep() virtually puts a blanket over the underlying issue. However while using Selenium and WebDriver to execute tests through your Automation Framework, using time.sleep() without any specific condition defeats the purpose of automation and should be avoided at any cost. As per the documentation:

time.sleep(secs) suspends the execution of the current thread for the given number of seconds. The argument may be a floating point number to indicate a more precise sleep time. The actual suspension time may be less than that requested because any caught signal will terminate the sleep() following execution of that signal’s catching routine. Also, the suspension time may be longer than requested by an arbitrary amount because of the scheduling of other activity in the system.

You can find a detailed discussion in How to sleep webdriver in python for milliseconds

---

Analysis

There were previous instances when Firefox consumed about 80% of the RAM.

However as per this discussion some of the users feels that the more memory is used the better because it means you don't have RAM wasted. Firefox uses RAM to make its processes faster since application data is transferred much faster in RAM.

---

Solution

You can implement either/all of the generic/specific steps as follows:

• Upgrade Selenium to current levels Version 3.141.59.
• Upgrade GeckoDriver to GeckoDriver v0.24.0 level.
• Upgrade Firefox version to Firefox v65.0.2 levels.
• Clean your Project Workspace through your IDE and Rebuild your project with required dependencies only.
• If your base Web Client version is too old, then uninstall it and install a recent GA and released version of Web Client.

• Some extensions allow you to block such unnecessary content, as an example:

  • uBlock Origin allows you to hide ads on websites.
  • NoScript allows you to selectively enable and disable all scripts running on websites.

  • To open the Firefox client with an extension you can download the extension i.e. the XPI file from https://addons.mozilla.org and use the add_extension(extension='webdriver.xpi') method to add the extension in a FirefoxProfile as follows:

    from selenium import webdriver
    
    profile = webdriver.FirefoxProfile() 
    profile.add_extension(extension='extension_name.xpi')
    driver = webdriver.Firefox(firefox_profile=profile, executable_path=r'C:path	ogeckodriver.exe') 
    

• If your Tests doesn't requires the CSS you can disable the CSS following the this discussion.

这篇关于Selenium 在 Firefox 中使用过多内存的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！