Selenium Webdriver:修改navigator.webdriver标志以防止硒检测 [英] Selenium webdriver: Modifying navigator.webdriver flag to prevent selenium detection
问题描述
我正在尝试使用硒和铬在网站中自动化一个非常基本的任务,但是以某种方式,该网站会检测到铬是由硒驱动的,并阻止了每个请求.我怀疑网站依赖于这样的公开DOM变量 https://stackoverflow.com/a/41904453/648236检测硒驱动的浏览器.
I'm trying to automate a very basic task in a website using selenium and chrome but somehow the website detects when chrome is driven by selenium and blocks every request. I suspect that the website is relying on an exposed DOM variable like this one https://stackoverflow.com/a/41904453/648236 to detect selenium driven browser.
我的问题是,有没有办法使navigator.webdriver标志为假?我愿意尝试修改后重新尝试编译硒源,但似乎无法在资源库中的任何地方找到NavigatorAutomationInformation源 https://github.com/SeleniumHQ/selenium
My question is, is there a way I can make the navigator.webdriver flag false? I am willing to go so far as to try and recompile the selenium source after making modifications, but I cannot seem to find the NavigatorAutomationInformation source anywhere in the repository https://github.com/SeleniumHQ/selenium
非常感谢您的帮助
PS:我还从 https://w3c.github.io/webdriver/#尝试了以下操作界面
Object.defineProperty(navigator, 'webdriver', {
get: () => false,
});
但是它只会在初始页面加载后更新属性.我认为网站会在执行脚本之前检测到该变量.
But it only updates the property after the initial page load. I think the site detects the variable before my script is executed.
推荐答案
首先更新 1
execute_cdp_cmd()
:借助execute_cdp_cmd(cmd, cmd_args)
命令的可用性,您现在可以轻松执行 google-chrome-devtools Selenium 的"nofollow noreferrer>命令一个>.使用此功能,您可以轻松地修改navigator.webdriver
以防止检测到硒.
First the update 1
execute_cdp_cmd()
: With the availability of execute_cdp_cmd(cmd, cmd_args)
command now you can easily execute google-chrome-devtools commands using Selenium. Using this feature you can modify the navigator.webdriver
easily to prevent Selenium from getting detected.
要防止检测到硒驱动的 WebDriver ,利基方法将包括以下所有步骤之一或全部:
To prevent Selenium driven WebDriver getting detected a niche approach would include either/all of the below mentioned steps:
-
旋转用户代理通过
execute_cdp_cmd()
命令如下:
#Setting up Chrome/83.0.4103.53 as useragent
driver.execute_cdp_cmd('Network.setUserAgentOverride', {"userAgent": 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.53 Safari/537.36'})
将 webdriver 的 navigator
的属性值更改为未定义
Change the property value of the navigator
for webdriver to undefined
driver.execute_cdp_cmd("Page.addScriptToEvaluateOnNewDocument", {
"source": """
Object.defineProperty(navigator, 'webdriver', {
get: () => undefined
})
"""
})
排除enable-automation
开关的集合
Exclude the collection of enable-automation
switches
options.add_experimental_option("excludeSwitches", ["enable-automation"])
关闭useAutomationExtension
options.add_experimental_option('useAutomationExtension', False)
总结上面提到的所有步骤,有效的代码块将是:
Clubbing up all the steps mentioned above and effective code block will be:
from selenium import webdriver
options = webdriver.ChromeOptions()
options.add_argument("start-maximized")
options.add_experimental_option("excludeSwitches", ["enable-automation"])
options.add_experimental_option('useAutomationExtension', False)
driver = webdriver.Chrome(options=options, executable_path=r'C:\WebDrivers\chromedriver.exe')
driver.execute_cdp_cmd("Page.addScriptToEvaluateOnNewDocument", {
"source": """
Object.defineProperty(navigator, 'webdriver', {
get: () => undefined
})
"""
})
driver.execute_cdp_cmd('Network.setUserAgentOverride', {"userAgent": 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.53 Safari/537.36'})
print(driver.execute_script("return navigator.userAgent;"))
driver.get('https://www.httpbin.org/headers')
历史
根据 W3C编辑器草案,当前的实现严格提到:
History
As per the W3C Editor's Draft the current implementation strictly mentions:
当用户代理位于
webdriver-active
标志设置为true
. >远程控制,其初始设置为false
.
The
webdriver-active
flag is set totrue
when the user agent is under remote control which is initially set tofalse
.
进一步
Navigator includes NavigatorAutomationInformation;
请注意:
NavigatorAutomationInformation
界面不应在 WorkerNavigator 上公开.
NavigatorAutomationInformation
接口定义为:
interface mixin NavigatorAutomationInformation {
readonly attribute boolean webdriver;
};
如果设置了webdriver-active
flag ,则返回 true
,否则返回false.
which returns true
if webdriver-active
flag is set, false otherwise.
最后,navigator.webdriver
定义了一种标准方法,用于使用户代理协作来通知文档它由 WebDriver 控制,以便可以在自动化过程中触发备用代码路径.
Finally, the navigator.webdriver
defines a standard way for co-operating user agents to inform the document that it is controlled by WebDriver, so that alternate code paths can be triggered during automation.
警告:更改/调整上述参数可能会阻止导航并检测到 WebDriver 实例.
Caution: Altering/tweaking the above mentioned parameters may block the navigation and get the WebDriver instance detected.
更新(2019年11月6日)
从当前的实现开始,访问网页而不被检测到的理想方法是使用ChromeOptions()
类向以下参数添加几个参数:
Update (6-Nov-2019)
As of the current implementation an ideal way to access a web page without getting detected would be to use the ChromeOptions()
class to add a couple of arguments to:
- 排除
enable-automation
开关的集合 - 关闭
useAutomationExtension
- Exclude the collection of
enable-automation
switches - Turn-off
useAutomationExtension
通过ChromeOptions
的实例如下:
-
Java示例:
Java Example:
System.setProperty("webdriver.chrome.driver", "C:\\Utility\\BrowserDrivers\\chromedriver.exe");
ChromeOptions options = new ChromeOptions();
options.setExperimentalOption("excludeSwitches", Collections.singletonList("enable-automation"));
options.setExperimentalOption("useAutomationExtension", false);
WebDriver driver = new ChromeDriver(options);
driver.get("https://www.google.com/");
Python示例
Python Example
from selenium import webdriver
options = webdriver.ChromeOptions()
options.add_experimental_option("excludeSwitches", ["enable-automation"])
options.add_experimental_option('useAutomationExtension', False)
driver = webdriver.Chrome(options=options, executable_path=r'C:\path\to\chromedriver.exe')
driver.get("https://www.google.com/")
1 :仅适用于Selenium的Python客户端.
1: Applies to Selenium's Python clients only.
2 :仅适用于Selenium的Python客户端.
2: Applies to Selenium's Python clients only.
3 :仅适用于Selenium的Python客户端.
3: Applies to Selenium's Python clients only.
这篇关于Selenium Webdriver:修改navigator.webdriver标志以防止硒检测的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!