CURLOPT_FOLLOWLOCATION无效 [英] CURLOPT_FOLLOWLOCATION not working

问题描述

我正试图通过此链接搜索数据：
http： //www.treasurydirect.gov/NP/BPDLogin?application=np

I'm trying to scrape the data at this link: http://www.treasurydirect.gov/NP/BPDLogin?application=np

包含元刷新。

我正在使用curl_exec并将CURLOPT_FOLLOWLOCATION设置为true。

I'm using curl_exec with CURLOPT_FOLLOWLOCATION set to true.

在phpinfo（）中，我确认未设置安全模式和basedir。但是CURLOPT_FOLLOWLOCATION仍然无效。这是我的代码：

In phpinfo() I confirmed that safe mode and basedir are not set. However CURLOPT_FOLLOWLOCATION is still not working. Here's my code:

<?php 
error_reporting(E_ALL);

$url = 'http://www.treasurydirect.gov/NP/BPDLogin?application=np';

// READ THE WEB PAGE
$ch = curl_init();
curl_setopt($ch,CURLOPT_URL,$url);
curl_setopt($ch,CURLOPT_RETURNTRANSFER,1);
curl_setopt($ch,CURLOPT_CONNECTTIMEOUT,20);
curl_setopt($ch,CURLOPT_FOLLOWLOCATION,true);
curl_setopt($ch,CURLOPT_COOKIEFILE,"cookies.txt");
curl_setopt($ch,CURLOPT_COOKIEJAR,"cookies.txt");
curl_error($ch);
$htm = curl_exec($ch);
curl_close($ch);

// MAKE OUR OUTPUT EASY TO READ
echo "<pre>";

// GET THE TITLE
$txt = strip_tags($htm, '<title>');
$rgx
= '#'            // REGEX DELIMITER - START
. '\<title\>'    // TITLE TAG WITH ANGLE BRACKETS ESCAPED - START
. '(.*?)'        // GROUP OF ANYTHING
. '\</title\>'   // TITLE TAG WITH ANGLE BRACKETS ESCAPED - END
. '#'            // REGEX DELIMITER - END
. 'is'           // CASE-INSENSITIVE, SINGLE LINE
;
preg_match($rgx, $txt, $arr);

// DISPLAY THE TITLE
echo
'<strong>'
. $arr[1]
. '</strong>'
. PHP_EOL
. PHP_EOL
;

// SHOW THE PAGE SOURCE
$src = htmlentities($htm);
echo $src;

?>

推荐答案

问题不在于元刷新标签（由方式永远不会被CURLOPT_FOLLOWLOCATION选项所遵循）但HTTP用户代理标头。网站根据接受的用户代理列表检查HTTP用户代理标头字段。您可以通过在为$ ch设置选项时添加以下行来解决此问题：

The problem is not the meta refresh tag (which by the way never will be followed by CURLOPT_FOLLOWLOCATION option) but the HTTP user agent header. The web site checks the HTTP user agent header field against a list of accepted user agents. You could solve this by adding the following line when setting options for $ch:

curl_setopt($ch, CURLOPT_USERAGENT, "Mozilla/4.0 (compatible; MSIE 5.01; Windows NT 5.0)");

这篇关于CURLOPT_FOLLOWLOCATION无效的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！