Scrapy飞溅登录 [英] Scrapy splash log-in
本文介绍了Scrapy飞溅登录的处理方法,对大家解决问题具有一定的参考价值,需要的朋友们下面随着小编来一起学习吧!
问题描述
我以前使用splash form请求登录其中一个站点。但是,开发人员改变了它,增加了更多的javascript,我无法弄清楚我做错了什么。我添加了javascript,也用于该网站。
I used to use splash form request to login in one of the site. However, developers changed it, added more javascript, and I can't figure out what I'm doing wrong. I added javascript, which beeing used in that site as well.
class MySpider(scrapy.Spider):
name = "lost"
start_urls = ["mysite",] ###########changed main loggin form
def start_requests(self):
for url in self.start_urls:
yield SplashRequest(
url,
self.parse,
args={'wait': 1},
)
def parse(self, response):
return SplashFormRequest.from_response(
response,
formdata={'mail': 'mymail', 'pass': 'mypasswd'},
callback=self.after_login
)
def after_login(self,response):
print('This is body '+response.body+' The end of body')
### Going to film list ######
if "Username" in response.body:
self.logger.error("##Success##")
Javascript:
Javascript:
$(document).ready(function(){
$('input[name="mail"],input[name="pass"]').keydown(function (e)
{
if(e.keyCode == 13)
{
login();
}
});
});
function login()
{
mail = $('input[name="mail"]').val();
pass = $('input[name="pass"]').val();
if($('input[name="rem"]:checked').length)
rem = 1;
else
rem = 0;
if(mail.length && pass.length > 5)
{
metrikaEvents('LOGIN');
console.log('OK!');
$.ajax({
type: "POST",
url: "/ajaxik.php",
dataType : "json",
data:
{
act:'users',
type:'login',
mail:encodeURIComponent(mail),
pass:encodeURIComponent(pass),
rem:encodeURIComponent(rem)
},
success: function(msg)
{
if(msg.result == 'ok')
{
if(msg.error)
{
switch(msg.error)
{
default:
text = lf_config.errors.user.login_error;
break;
}
ntfctn(text,'error');
}
else if(msg.success)
{
ntfctn(msg.name+lf_config.notifications['user_login'],'information');
setTimeout('goTo("/",false)',1000);
// goTo('/',false);
}
}
},
});
}
}
function loginTogglePass(t)
{
if($('input[name="'+t+'"]').attr('type') == 'password')
{
$('input[name="'+t+'"]').attr('type','text');
$('input[name="'+t+'"]').prev('div.eye- icon').removeClass('closed').addClass('opened');
}
else
{
$('input[name="'+t+'"]').attr('type','password');
$('input[name="'+t+'"]').prev('div.eye-icon').removeClass('opened').addClass('closed');
}
}
我看到javascript正在寻找输入键。但点击按钮也应该有效。谁能让我朝着正确的方向前进?谢谢
I see that javascript looking for "enter" key down. But clicking button should work as well. Can anyone put me in the right direction? Thanks
推荐答案
从这个主题借来的想法
使用formdata登录,使用splash并逐个检测页面元素
Instead using formdata to login, use splash and detect page's elements one by one
class MySpider(scrapy.Spider):
name = "lost"
allowed_domains = ["mydomain"]
start_urls = ['myurl']
req = 10
series = {}
def start_requests(self):
script = """
function main(splash)
local url = splash.args.url
assert(splash:go(url))
assert(splash:wait(10))
splash:set_viewport_full()
local search_input = splash:select('input[name=mail]')
search_input:send_text("email")
local search_input = splash:select('input[name=pass]')
search_input:send_text("password")
assert(splash:wait(5))
local submit_button = splash:select('input[class^=primary-btn]')
submit_button:click()
assert(splash:wait(10))
return {
html = splash:html(),
png = splash:png(),
}
end
"""
yield SplashRequest(
'myurl',
callback = self.after_login, ###inserting callabck
endpoint='execute',
args={
'lua_source': script,
'ua': "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.106 Safari/537.36"
}
)
def parse(self, response):
script = response.body
这篇关于Scrapy飞溅登录的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!
查看全文