带有Firebug + NetExport + FireStarter的Selenium Webdriver无法在Python中创建har文件 [英] Selenium Webdriver with Firebug + NetExport + FireStarter not creating a har file in Python
问题描述
我目前正在Python中使用Firebug，NetExport和FireStarter(正在试用)运行Selenium，试图获取URL的网络流量.我希望HAR文件会出现在列出的目录中，但是什么也没出现.当我在Firefox中进行测试并通过UI时，会导出并保存一个HAR文件，因此我知道代码本身可以正常工作.查看多个示例后，我看不到我所缺少的内容.
I am currently running Selenium with Firebug, NetExport, and (trying out) FireStarter in Python trying to get the network traffic of a URL. I expect a HAR file to appear in the directory listed, however nothing appears. When I test it in Firefox and go through the UI, a HAR file is exported and saved so I know the code itself functions as expected. After viewing multiple examples I do not see what I am missing.
我正在使用Firefox 29.0.1 萤火虫1.12.8 FireStarter 0.1a6 NetExport 0.9b6
I am using Firefox 29.0.1 Firebug 1.12.8 FireStarter 0.1a6 NetExport 0.9b6
还有其他人遇到过此问题吗?我收到正确填写的"webFile.txt"文件.
Has anyone else encountered this issue? I am receiving a "webFile.txt" file being filled out correctly.
在查找附加组件的每个版本之后，它们应该与我使用的Firefox版本兼容.我尝试使用Firefox 20版，但是没有帮助.我目前正在提取源代码.
After looking up each version of the add-ons they are supposed to be compatible with the version of Firefox I am using. I tried using Firefox version 20, however that did not help. I am currently pulling source code.
此外，我在有和没有FireStarter的情况下都进行了尝试，并且在两种情况下都尝试手动刷新页面以尝试生成HAR.
In addition I have tried it with and without FireStarter, and I have tried refreshing the page manually in both cases to try to generate a HAR.
我的代码如下:
import urllib2
import sys
import re
import os
import subprocess
import hashlib
import time
import datetime
from browsermobproxy import Server
from selenium import webdriver
import selenium
a=[];
theURL='';
fireBugPath = '/Users/tai/Documents/workspace/testSelenium/testS/firebug.xpi';
netExportPath = '/Users/tai/Documents/workspace/testSelenium/testS/netExport.xpi';
fireStarterPath = '/Users/tai/Documents/workspace/testSelenium/testS/fireStarter.xpi';
profile = webdriver.firefox.firefox_profile.FirefoxProfile();
profile.add_extension( fireBugPath);
profile.add_extension(netExportPath);
profile.add_extension(fireStarterPath);
#firefox preferences
profile.set_preference("app.update.enabled", False)
profile.native_events_enabled = True
profile.set_preference("webdriver.log.file", "/Users/tai/Documents/workspace/testSelenium/testS/webFile.txt")
profile.set_preference("extensions.firebug.DBG_STARTER", True);
profile.set_preference("extensions.firebug.currentVersion", "1.12.8");
profile.set_preference("extensions.firebug.addonBarOpened", True);
profile.set_preference("extensions.firebug.addonBarOpened", True);
profile.set_preference('extensions.firebug.consoles.enableSite', True)
profile.set_preference("extensions.firebug.console.enableSites", True);
profile.set_preference("extensions.firebug.script.enableSites", True);
profile.set_preference("extensions.firebug.net.enableSites", True);
profile.set_preference("extensions.firebug.previousPlacement", 1);
profile.set_preference("extensions.firebug.allPagesActivation", "on");
profile.set_preference("extensions.firebug.onByDefault", True);
profile.set_preference("extensions.firebug.defaultPanelName", "net");
#set net export preferences
profile.set_preference("extensions.firebug.netexport.alwaysEnableAutoExport", True);
profile.set_preference("extensions.firebug.netexport.autoExportToFile", True);
profile.set_preference("extensions.firebug.netexport.saveFiles", True);
profile.set_preference("extensions.firebug.netexport.autoExportToServer", False);
profile.set_preference("extensions.firebug.netexport.Automation", True);
profile.set_preference("extensions.firebug.netexport.showPreview", False);
profile.set_preference("extensions.firebug.netexport.pageLoadedTimeout", 15000);
profile.set_preference("extensions.firebug.netexport.timeout", 10000);
profile.set_preference("extensions.firebug.netexport.defaultLogDir", "/Users/tai/Documents/workspace/testSelenium/testS/har");
profile.update_preferences();
browser = webdriver.Firefox(firefox_profile=profile);
def openURL(url,s):
theURL = url;
time.sleep(6);
#browser = webdriver.Chrome();
browser.get(url); #load the url in firefox
time.sleep(3); #wait for the page to load
browser.execute_script("window.scrollTo(0, document.body.scrollHeight/5);")
time.sleep(1); #wait for the page to load
browser.execute_script("window.scrollTo(0, document.body.scrollHeight/4);")
time.sleep(1); #wait for the page to load
browser.execute_script("window.scrollTo(0, document.body.scrollHeight/3);")
time.sleep(1); #wait for the page to load
browser.execute_script("window.scrollTo(0, document.body.scrollHeight/2);")
time.sleep(1); #wait for the page to load
browser.execute_script("window.scrollTo(0, document.body.scrollHeight);")
searchText='';
time.sleep(20); #wait for the page to load
if(s.__len__() >0):
for x in range(0, s.__len__()):
searchText+= ("" + browser.find_element_by_id(x));
else:
searchText+= browser.page_source;
a=getMatches(searchText)
#print ("\n".join(swfLinks));
print('\n'.join(removeNonURL(a)));
# print(browser.page_source);
browser.quit();
return a;
def found_window(name):
try: browser.switch_to_window(name)
except NoSuchWindowException:
return False
else:
return True # found window
def removeFirstQuote(tex):
for x in tex:
b = x[1:];
if not b in a:
a.append(b);
return a;
def getMatches(t):
return removeFirstQuote(re.findall('([\"|\'][^\"|\']*\.swf)', t));
def removeNonURL(t):
a=[];
for b in t:
if(b.lower()[:4] !="http" ):
if(b[0] == "//"):
a.append(theURL+b[2:b.__len__()]);
else:
while(b.lower()[:4] !="http" and b.__len__() >5):
b=b[1:b.__len__()];
a.append(b);
else:
a.append(b);
return a;
openURL("http://www.chron.com",a);
推荐答案
我为自己的工作解决了此问题，方法是在关闭浏览器之前设置更长的等待时间.我认为您当前正在将netexport设置为在程序退出后导出，因此不会写入任何文件.导致此的行是:
I fixed this issue for my own work by setting a longer wait before closing the browser. I think you are currently setting netexport to export after the program has quit, so no file is written. The line causing this is:
profile.set_preference("extensions.firebug.netexport.pageLoadedTimeout", 15000);
在 netexport源代码中，我们认为pageLoadedTimeout是在最后一个页面请求声明已加载页面之后要等待的毫秒数".因此，我怀疑您的所有次要页面加载都阻止netexport拥有足够的时间来写入文件.需要注意的是，您将系统设置为在10秒后自动导出，因此我不确定为什么您不获取一半加载的json文件.
From the netexport source code we have that pageLoadedTimeout is the `Number of milliseconds to wait after the last page request to declare the page loaded'. So I suspect all your minor page loads are preventing netexport from having enough time to write the file. One caveat is you set the system to automatically export after 10s so I'm not sure why you are not acquiring half loaded json files.
这篇关于带有Firebug + NetExport + FireStarter的Selenium Webdriver无法在Python中创建har文件的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
