使用安全登录来刮除网站内容 [英] Scrape a site content With a Secure Login

问题描述

我试图用登录保护
擦除一个网站的内容但无法做到
网站的登录有三个选项用户名，密码，密码
这里是代码I使用

I am trying to scrape the contents of the a site with login secured but unable to do it The site's login has three options username,password,passcode here is the code I am using

<?php

// HTTP authentication

$url = "http://aftabcurrency.com/login_script.php";

$ch = curl_init();    

curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1); 

curl_setopt($ch, CURLOPT_URL, $url); 
$cookie = 'cookies.txt';
$timeout = 30;
curl_setopt($curl, CURLOPT_TIMEOUT,         10); 
curl_setopt($curl, CURLOPT_CONNECTTIMEOUT,  $timeout );
curl_setopt($curl, CURLOPT_COOKIEJAR,       $cookie);
curl_setopt($curl, CURLOPT_COOKIEFILE,      $cookie);

curl_setopt ($ch, CURLOPT_POST, 1); 
curl_setopt ($ch,CURLOPT_POSTFIELDS,"user_name=user&user_password=pass&passcode=code");             

$result = curl_exec($ch); 

curl_close($ch); 

echo $result;

?>

推荐答案

POST 到 http://aftabcurrency.com/login_script.php
您的curl还需要接受Cookie。

身份验证后，脚本将重定向您，因此您还需要添加 CURLOPT_FOLLOWACTION 。

这里是您脚本的编辑版本，我无法在 http://aftabcurrency.com/ 希望它能正常工作：

here is a edited version of your script, I can't test it on http://aftabcurrency.com/ hope it works:

$url = "http://aftabcurrency.com/login_script.php";

$ch = curl_init();    
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1); 

curl_setopt($ch, CURLOPT_URL, $url); 
$cookie = 'cookies.txt';
$timeout = 30;

curl_setopt ($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($ch, CURLOPT_TIMEOUT,         10); 
curl_setopt($ch, CURLOPT_CONNECTTIMEOUT,  $timeout );
curl_setopt($ch, CURLOPT_COOKIEJAR,       $cookie);
curl_setopt($ch, CURLOPT_COOKIEFILE,      $cookie);

curl_setopt ($ch, CURLOPT_POST, 1); 
curl_setopt ($ch,CURLOPT_POSTFIELDS,"user_name=user&user_password=pass&passcode=code");     

$result = curl_exec($ch);

/* //OPTIONAL - Redirect to another page after login
$url = "http://aftabcurrency.com/some_other_page";
curl_setopt ($ch, CURLOPT_POST, 0); 
curl_setopt($ch, CURLOPT_URL, $url);
$result = curl_exec($ch);
 */ //end OPTIONAL 

curl_close($ch); 
echo $result;

这篇关于使用安全登录来刮除网站内容的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！