如何解决URL重定向？ [英] How to resolve URL redirects?

问题描述

我有许多短urls.Each URL的txt文件是由单一line.I分隔希望解决​​的URL，以获得最终link.Also某些URL重定向twice.How自动执行此得到最终的网址每行一个网址的输出格式？
更新：
输入文本文件：

  http://www.example.com/go/post-page-1
http://www.example.com/go/post-page-2
http://www.example.com/go/post-page-3
 

在txt文件所需的输出格式：

  http://www.example.org/post-page-name
http://www.example.org/post-page-name
http://www.example.org/post-page-name
 

下面是链接重定向如何：

 初始的URL：http：//www.example.com/go/post-page
    ==＆GT; 301永久重定向中间的URL：http：//click.affiliate.com/tracking URL = HTTP：//www.example.org/post-page-name
==＆GT; 302临时重定向最终网址：http://www.example.org/post-page-name
 

下面是code我试过，但它不能解决的URL到最后一个环节，而是中间环节。

 ＃！/斌/庆典
RM resolved_urls.txt
在$ URL（猫url.txt）;做
        wget的-S$网址2  - ;＆放大器; 1 | grep的^地点＆gt;＆GT; resolved_urls.txt
DONE
 

解决方案

尝试是这样的：

 ＃！/斌/庆典功能getFinalRedirect {
    本地URL = $ 1
    而真实的;做
        nextloc = $（卷曲-s -I $网址| grep的^地点：）
        如果[-n$ nextloc];然后
            URL = $ {nextloc ##地点：}
        其他
            打破
        科幻
    DONE    回声$网址
}URL =htt​​p://stackoverflow.com/q/25485374/1563512
getFinalRedirect $网址
 

谨防无限的重定向。这就产生了：

  $ ./test.bash
http://stackoverflow.com/questions/25485374/how-to-resolve-url-redirects
 

然后，打电话给你的文件中的函数：

 同时读取URL;做
    getFinalRedirect $网址
完成＆LT; urls.txt＆GT; finalurls.txt
 

I have a txt document with many short urls.Each url is seperated by a single line.I want to resolve the URLs to get the final link.Also some URLs are redirected twice.How to automate this to get the final urls with output format of one url per line? Update: Input text file:

http://www.example.com/go/post-page-1 
http://www.example.com/go/post-page-2 
http://www.example.com/go/post-page-3 

Output format needed in txt file:

http://www.example.org/post-page-name
http://www.example.org/post-page-name
http://www.example.org/post-page-name

Here is how the links are redirected:

Initial URL:http://www.example.com/go/post-page 
    ==>301 Permanent Redirect

Intermediate url:http://click.affiliate.com/tracking?url=http://www.example.org/post-page-name
==>302 Temporary Redirect

Final URL: http://www.example.org/post-page-name

Here is the code i tried but it doesn't resolve URLs to the final link but rather to the intermediate link.

#!/bin/bash
rm resolved_urls.txt
for url in $(cat url.txt); do
        wget -S "$url" 2>&1 | grep ^Location >> resolved_urls.txt
done

解决方案

Try something like this:

#!/bin/bash

function getFinalRedirect {
    local url=$1
    while true; do
        nextloc=$( curl -s -I $url | grep ^Location: )
        if [ -n "$nextloc" ]; then
            url=${nextloc##Location: }
        else
            break
        fi
    done

    echo $url
}

url="http://stackoverflow.com/q/25485374/1563512"
getFinalRedirect $url

Beware of infinite redirects. This produces:

$ ./test.bash 
http://stackoverflow.com/questions/25485374/how-to-resolve-url-redirects

Then, to call the function on your file:

while read url; do
    getFinalRedirect $url
done < urls.txt > finalurls.txt

这篇关于如何解决URL重定向？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！