获取cURL以跟随重定向 [英] Get cURL to follow redirects

问题描述

我在Laravel中有一个应用程序，它可以抓取站点并检索特定信息.最近，我看到越来越多的网站被禁止使用403，因此我决定第一次尝试使用cURL.

I have an application in Laravel that scrapes sites and retrieves specific information. Lately I've been seeing more and more sites getting 403 forbiidden so i decided to try cURL out for the first time.

我现在唯一的问题是，具有301或302重定向的网站，cURL没有跟随它们.

The only problem i have now, is that sites that have 301 or 302 redirects, cURL doesn't follow them.

这就是我所拥有的:

$curl = curl_init(); 
        curl_setopt($curl, CURLOPT_URL, $results['url_search']);  
        curl_setopt($curl, CURLOPT_RETURNTRANSFER, 1);  
        curl_setopt($curl, CURLOPT_CONNECTTIMEOUT, 10);  
        curl_setopt($curl,CURLOPT_USERAGENT,'Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.13) Gecko/20080311 Firefox/2.0.0.13');

        $str = curl_exec($curl);  

        $header_size = curl_getinfo($curl, CURLINFO_HEADER_SIZE);
        $header = curl_getinfo($curl, CURLINFO_HTTP_CODE);
        $body = substr($str, $header_size);

        curl_close($curl); 

因此，正如我所说，在存在重定向的页面上，我被卡住了.有什么建议么?我看过一些东西，但是都没用.

So as I said, on pages where there are redirects, I get stuck. Any suggestions? I've seen some things but none of them worked.

推荐答案

curl 可以选择完全实现您所寻找的目标，以下重定向:

curl has an option to achieve exactly what you are looking for, following redirects:

curl_setopt($curl, CURLOPT_FOLLOWLOCATION, true);

只需在执行该行之前将其添加到curl-options中即可.

Just add this line to your curl-options before your execution of it.

如您所料，这将跟随任何301/302重定向并最终到达一个站点，该站点不再进一步重定向您的请求.

As you might expect, this will follow any 301 / 302 redirects and ends up on a site, which doesn't redirect your request any further.

此外，请记住(没有解决方法)这可能会导致无限循环.(将站点重定向到b，将b重定向到a).

Also, remember that (without having a workaround) this might lead to an infinite loop. (site a redirects to b and b redirects to a).

也就是说，您也应该使用此选项:

That said, you should use this option as well:

curl_setopt($curl, CURLOPT_MAXREDIRS, 10);

这样，您的请求将在10次重定向后结束，并且您不必费心在无限循环中运行脚本.

This way, your requests will end after 10 redirects and you don't have to bother with your script running in an endless-loop.

有关不同选项的进一步工作的一个好资源是 php上的相关站点.net

A good source for your further work with diffrent options is the regarding site on php.net

这篇关于获取cURL以跟随重定向的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！