网站拒绝使用Selenium获取请求 [英] Website denies get request using Selenium
本文介绍了网站拒绝使用Selenium获取请求的处理方法，对大家解决问题具有一定的参考价值，需要的朋友们下面随着小编来一起学习吧！
问题描述
我正在尝试使用硒来自动登录网站 https://afghanbids.com/，但是当我运行python程序时，浏览器会加载一个页面，显示访问被拒绝.您可能会看到下面的屏幕截图.
I am trying to use selenium in order to automate for login into a website https://afghanbids.com/ but when I run my python program, the browser loads a page which says access denied. You may please see the screen shot below.
我的代码:
driver = webdriver.Chrome(executable_path="chromedriver.exe")
driver.get("https://afghanbids.com/")
它表明:
如何提出请求，以使目标网站不拒绝我的请求?
How can I make the request so that the target website does not deny my request?
推荐答案
我接受了您的代码，进行了一些调整，并遇到了 access_denied 页面.这里是观察结果:
I took your code, added a few tweaks and encountered the access_denied page. Here are the observation results:
代码块:
from selenium import webdriver
options = webdriver.ChromeOptions()
options.add_argument("start-maximized")
options.add_experimental_option("excludeSwitches", ["enable-automation"])
options.add_experimental_option('useAutomationExtension', False)
driver = webdriver.Chrome(options=options, executable_path=r'C:\WebDrivers\chromedriver.exe')
driver.get("https://afghanbids.com/")
print(driver.page_source)
浏览器快照:
控制台输出:
<html lang="en"><head>
<meta charset="utf-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1">
<title>AfghanBids</title>
<!--favicon-->
<!-- <link rel="apple-touch-icon" href="/theme/images/apple-touch-icon.png"> -->
<link rel="shortcut icon" href="https://afghanbids.com/assets/backend/images/favicon_1.ico" type="image/x-icon">
<script type="text/javascript">
var base_url = "https://afghanbids.com/";
var number = 1;
var total_size = 0;
var allow = false;
</script>
<!-- bootstrap -->
<link href="https://afghanbids.com/assets/plugins/bootstrap-3.3.2/css/bootstrap.min.css" rel="stylesheet">
<!-- Icons -->
<link href="https://afghanbids.com/assets/plugins/font-awesome-4.2.0/css/font-awesome.min.css" rel="stylesheet">
<!-- Themes styles-->
<link href="https://afghanbids.com/assets/theme/css/theme.css" rel="stylesheet">
<!-- Your custom css -->
<link href="https://afghanbids.com/assets/theme/css/theme-custom.css" rel="stylesheet">
<link href="https://afghanbids.com/assets/custom/custom.css" rel="stylesheet">
<script type="text/javascript" src="https://afghanbids.com/assets/backend/js/jquery.min.js"></script>
<!-- TO SET TIMEZONE OF VISITOR -->
<script type="text/javascript" src="//cdnjs.cloudflare.com/ajax/libs/jstimezonedetect/1.0.4/jstz.min.js"></script>
<script type="text/javascript">
$(document).ready(function(){
var tz = jstz.determine(); // Determines the time zone of the browser client
var timezone = tz.name(); //'Asia/Kolhata' for Indian Time.
url = base_url+"admin/user/set_timezone";
$.post(url, {timezone: timezone}, function(data){
// alert(data);
});
});
</script>
<!-- FOR GOOGLE RECAPTCHA -->
<!-- HTML5 Shim and Respond.js IE8 support of HTML5 elements and media queries -->
<!-- WARNING: Respond.js doesn't work if you view the page via file:// -->
<!--[if lt IE 9]>
<script src="https://oss.maxcdn.com/html5shiv/3.7.2/html5shiv.min.js"></script>
<script src="https://oss.maxcdn.com/respond/1.4.2/respond.min.js"></script>
<![endif]-->
</head>
<body>
<div class="loading-page overlay" id="myNav">
<!-- <a href="javascript:void(0)" class="closebtn" onclick="closeNav()">×</a> -->
<div class="counter">
<p>File Uploading</p>
<p style="font-size:12px">Please wait ...</p>
<h1>0%
<!--
h1.abs loading
h1.abs.color loading
-->
</h1>
<hr>
</div>
</div>
<!-- wrapper page -->
<div class="wrapper">
<!-- main-header -->
<header class="main-header">
<!-- main navbar -->
<nav class="navbar navbar-default main-navbar hidden-sm hidden-xs">
<div class="container">
<div class="collapse navbar-collapse" id="bs-example-navbar-collapse-1">
<ul class="nav navbar-nav">
<li class=""><a class="my-nav-a" href="https://afghanbids.com/home"><strong>HOME</strong></a></li>
<li class=""><a class="my-nav-a" href="https://afghanbids.com/buyer_front"><strong>BUYERS</strong></a></li>
<li class=""><a class="my-nav-a" href="https://afghanbids.com/seller_front"><strong>SELLERS</strong></a></li>
<li class=""><a class="my-nav-a" href="https://afghanbids.com/home/contact"><strong>CONTACT US</strong></a></li>
<li class=""><a class="my-nav-a" href="https://afghanbids.com/home/about"><strong>ABOUT US</strong></a></li>
<li class=""><a href="https://afghanbids.com/admin/user/register"><strong>REGISTER</strong></a></li> </ul>
<form action="https://afghanbids.com/admin/user/login" id="quick_login" hidden="" method="post"></form>
<ul class="nav navbar-nav navbar-right">
<li class="link-btn"><input type="text" maxlength="64" class="form-control" required="" form="quick_login" name="email_username" placeholder="Email / Username"></li>
<li class="link-btn"><input type="password" maxlength="32" class="form-control" required="" form="quick_login" name="password" pattern=".{6,}" placeholder="Password"></li>
<li class="link-btn"><button class="btn btn-theme btn-pill btn-xs btn-line" form="quick_login" type="submit">Login</button></li>
</ul>
</div>
</div>
</nav><!-- end main navbar -->
<!-- mobile navbar -->
<div class="container">
<nav class="mobile-nav hidden-md hidden-lg">
<a href="#" class="btn-nav-toogle first">
<span class="bars"></span>
Menu
</a>
<div class="mobile-nav-block">
<h4>Navigation</h4>
<a href="#" class="btn-nav-toogle">
<span class="barsclose"></span>
Close
</a>
<ul class="nav navbar-nav">
<li class=""><a href="https://afghanbids.com/home"><strong>Home</strong></a></li>
<li class=""><a href="https://afghanbids.com/buyer_front"><strong>BUYERS</strong></a></li>
<li class=""><a href="https://afghanbids.com/seller_front"><strong>SELLERS</strong></a></li>
<li class=""><a href="https://afghanbids.com/home/contact"><strong>CONTACT US</strong></a></li>
<li class=""><a href="https://afghanbids.com/home/about"><strong>ABOUT US</strong></a></li>
<li class=""><a href="https://afghanbids.com/admin/user/register"><strong>Registration</strong></a></li>
<li><a href="https://afghanbids.com/admin/user/login_page/1"><strong>Login</strong></a></li>
</ul>
</div>
</nav>
</div><!-- mobile navbar -->
<!-- <div class="container"> -->
<!-- <img class="banner-image" src="header-banner.jpg" width="100%"> -->
<!-- </div> -->
<!-- form search area-->
<div class="container">
<div class="row" style="margin-top: -30px">
<div class="col-md-5">
<div class="col-md-8">
<div class="logo text-center inline">
<!-- <img src="assets/theme/images/logo.png" alt=""> -->
<h3 class="banner-msg"><strong></strong></h3>
<!-- <div class="label-add">Resume</div> -->
<!-- </a> -->
</div>
</div>
<div class="col-md-4">
<div class="logo text-center inline">
</div>
</div>
</div>
<div class="col-md-7">
<!-- form search -->
<form method="get" action="https://afghanbids.com/home/search" class="form-search-list">
<div class="row">
<div class="col-sm-10 col-xs-12">
<div class="form-group" style="margin-top: 3px;">
<label class="color-white">Search For Opportunities</label>
<input class="form-control" maxlength="128" required="" name="keywords" placeholder="Type your keywords here...">
</div>
</div>
<div class="col-sm-2 col-xs-12">
<div class="form-group">
<label class="hidden-xs"> </label>
<button class="btn btn-block btn-theme btn-success">Search</button>
</div>
</div>
</div>
<!--<p class="text-right"><a href="#modal-advanced" data-toggle="modal" class="link-white">Advanced Search</a></p>-->
</form><!-- end form search -->
</div>
</div>
</div><!-- end form search area-->
</header><!-- end main-header --> <!-- body-content -->
<div class="body-content clearfix">
<div class="block-section bg-color2">
<div class="container">
<!-- text centered -->
<div class="text-center">
<h1>Sorry, page not found!</h1>
<!-- <ul class="list-inline">
<li> <a href="#">Home</a></li>
<li><a href="#">Find a Job</a></li>
<li> <a href="#">Blog</a></li>
<li> <a href="#">Contact Us</a></li>
</ul> -->
</div><!-- end text centered -->
<!-- big text error -->
<div class="big-error">404</div><!-- end big text error -->
</div>
</div>
</div><!--end body-content -->
<!-- main-footer -->
<footer class="main-footer">
<div class="container">
<div class="row">
<div class="col-sm-6">
<ul class="list-inline link-footer text-center-xs">
<li><a href="https://afghanbids.com/">Home</a></li>
<li><a href="https://afghanbids.com/home/about">About Us</a></li>
<li><a href="https://afghanbids.com/home/contact">Contact Us</a></li>
</ul>
</div>
<div class="col-sm-6 ">
<span class="pull-right">Powered By: <a target="_blank" href="http://www.codezone.af">CodeZone</a></span>
</div>
</div>
</div>
</footer><!-- end main-footer -->
</div><!-- end wrapper page -->
<!-- jQuery (necessary for Bootstrap's JavaScript plugins) -->
<!-- <script src="/plugins/jquery.js"></script> -->
<!-- jQuery Bootstrap -->
<script src="https://afghanbids.com/assets/plugins/bootstrap-3.3.2/js/bootstrap.min.js"></script>
<script src="https://afghanbids.com/assets/plugins/jquery.easing-1.3.pack.js"></script>
<!-- Lightbox -->
<!-- <script src="https://afghanbids.com/assets/plugins/magnific-popup/jquery.magnific-popup.min.js"></script> -->
<!-- Theme JS -->
<script src="https://afghanbids.com/assets/theme/js/theme.js"></script>
</body></html>
分析
最初，我怀疑硒驱动了 ChromeDriver 启动了 google-chrome 浏览上下文是被检测到.
但是令人惊讶的是，即使 windows 来访问 google-chrome-在手动发起的会话上，可以通过以下两种方式之一禁用devtools :
But surprisingly, even the windows based shortcuts to access the google-chrome-devtools remains disabled through either of the following ways on the manually inititiated session:
- F12
- ctrl + shift + I
- ctrl + shift + J
- ctrl + shift + C
- F12
- ctrl+shift+I
- ctrl+shift+J
- ctrl+shift+C
此外，如果您强制打开google-chrome-devtools 通过更多工具->开发人员工具，手动启动的会话开始显示抱歉，找不到页面！.因此，即使分析HTML也会有些困难.
Additionally, If you forcefully open the google-chrome-devtools through More tools -> Developer tools, the manually inititiated session starts showing Sorry, page not found!. Hence it would be a bit difficult even to analyze the HTML.
最后，我必须承认开发人员在限制自动化机器人方面所做的出色工作.
Finally, I must admit the developers did a commendable job in restricting the automated bots.
这篇关于网站拒绝使用Selenium获取请求的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
查看全文
