如何在Python上使用Selenium Webdriver和browsermob代理捕获网络流量? [英] How to capture network traffic using selenium webdriver and browsermob proxy on Python?

问题描述

我想通过在Python上使用Selenium Webdriver捕获网络流量.因此，我必须使用代理(例如BrowserMobProxy)

I would like to capture network traffic by using Selenium Webdriver on Python. Therefore, I must use a proxy (like BrowserMobProxy)

当我使用webdriver.Chrome时:

When I use webdriver.Chrome:

from browsermobproxy import Server

server = Server("~/browsermob-proxy")

server.start()
proxy = server.create_proxy()

from selenium import webdriver
co = webdriver.ChromeOptions()
co.add_argument('--proxy-server={host}:{port}'.format(host='localhost', port=proxy.port))

driver = webdriver.Chrome(executable_path = "~/chromedriver", chrome_options=co)

proxy.new_har
driver.get(url)
proxy.har # returns a HAR 

for ent in proxy.har['log']['entries']:
    print ent['request']['url']

该网页已正确加载，并且所有请求都可以在HAR文件中访问. 但是当我使用webdriver.Firefox时:

the webpage is loaded properly and all requests are available and accessible in the HAR file. But when I use webdriver.Firefox:

# The same as above
# ...
from selenium import webdriver
profile  = webdriver.FirefoxProfile()
driver = webdriver.Firefox(firefox_profile=profile, proxy = proxy.selenium_proxy())

proxy.new_har
driver.get(url)
proxy.har # returns a HAR

for ent in proxy.har['log']['entries']:
    print ent['request']['url']

无法正确加载网页，并且HAR文件中的请求数量少于应有的请求数量.

The webpage cannot be loaded properly and the number of requests in the HAR file is smaller than the number of requests that should be.

您是否知道第二个代码中的代理设置有什么问题?我应该如何修复它以正确使用webdriver.Firefox?

Do you have any idea what the problem of proxy settings in the second code? How should I fix it to use webdriver.Firefox properly for my purpose?

推荐答案

只是偶然发现了这个项目 https://github.com/derekargueta/selenium-profiler .吐出所有网络数据以获取URL.应该很容易被黑客入侵并集成到您正在运行的任何测试中.

Just stumbled across this project https://github.com/derekargueta/selenium-profiler. Spits out all network data for a URL. Shouldn't be hard to hack and integrate into whatever tests you're running.

原始来源: https://www.openhub.net/p/selenium-profiler

这篇关于如何在Python上使用Selenium Webdriver和browsermob代理捕获网络流量?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！