如何在Python中遵循元刷新 [英] how to follow meta refreshes in Python

问题描述

Python的urllib2遵循3xx重定向以获取最终内容.是否有办法使urllib2(或其他一些库，例如 httplib2 )也遵循元数据刷新?还是我需要手动解析HTML以获取刷新元标记?

Python's urllib2 follows 3xx redirects to get the final content. Is there a way to make urllib2 (or some other library such as httplib2) also follow meta refreshes? Or do I need to parse the HTML manually for the refresh meta tags?

推荐答案

好的，似乎没有库支持它，所以我一直在使用以下代码:

OK, seems no library supports it so I have been using this code:

import urllib2
import urlparse
import re

def get_hops(url):
    redirect_re = re.compile('<meta[^>]*?url=(.*?)["\']', re.IGNORECASE)
    hops = []
    while url:
        if url in hops:
            url = None
        else:
            hops.insert(0, url)
            response = urllib2.urlopen(url)
            if response.geturl() != url:
                hops.insert(0, response.geturl())
            # check for redirect meta tag
            match = redirect_re.search(response.read())
            if match:
                url = urlparse.urljoin(url, match.groups()[0].strip())
            else:
                url = None
    return hops

这篇关于如何在Python中遵循元刷新的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！