是否可以在python中并行化Selenium Webdriver get_attribute调用? [英] Is it possible to parallelize selenium webdriver get_attribute calls in python?

问题描述

我正在运行这段代码

from multiprocessing.Pool import ThreadPool
from selenium import webdriver
driver = webdriver.Firefox()
driver.get(url)
elements = driver.find_elements_by_class_name("class-name")
pool = ThreadPool(4)
async = [pool.apply_async(fn_which_calls_get_attribute,(element,)) for element in elements]
results = [result.get() for result in async]

对于某些结果工作正常，但对其他结果抛出 ResponseNotReady 错误.如果我使用"pool.apply"而不是异步版本，它将按预期运行.

which works fine for some of the results, but throws an error of ResponseNotReady for other results. It runs as expected if I use "pool.apply" instead of the async version.

我一次对selenium驱动程序进行多次调用是一个问题，错误是因为它无法处理吗?还是我的并行化有问题?

Is it a problem that I am making multiple calls to the selenium driver at once, and the error is because it cannot handle it? Or is something wrong with my parallelization?

推荐答案

仅暗示Selenium在单个线程和单个核心系统中运行.因此，不可能在Selenium Webdriver上执行多线程.是的，您可以创建一个单独的实例并附加到多核系统的另一个核.

Just Hint that Selenium run in a single thread and in a single core system. So its not possible to exercise multi-threading over selenium webdriver . Yes you can create a separate instance and attach to another core of a multi core system.

我可能不会回答您的问题，但是如果您要尝试做类似的好事，那就不要做.

I may not answer your question but if you are trying to do something similar good not to do.

这篇关于是否可以在python中并行化Selenium Webdriver get_attribute调用?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！