AttributeError:'list'对象没有属性'timeout'-尝试使用BeautifulSoup处理多个URL [英] AttributeError: 'list' object has no attribute 'timeout' - Trying to process multiple URLs with BeautifulSoup

问题描述

我正在尝试使用urllib和BeautifulSoup捕获并解析多个URL，但是出现以下错误:

I am trying to grab and parse multiple URLs using urllib and BeautifulSoup, but I get the following error:

AttributeError:列表"对象没有属性超时"

据我了解，解析器告诉我我提交了一个列表，它正在寻找一个URL.如何处理多个URL?

From what I understand, the parser is telling me that I submitted a list and it is looking for a single URL. How can I process multiple URLs?

这是我的代码:

from bs4 import BeautifulSoup
from bs4.element import Comment
import urllib.request


def tag_visible(element):
    if element.parent.name in ['style', 'script', 'head', 'title', 'meta', '[document]']:
        return False
    if isinstance(element, Comment):
        return False
    return True

addresses = ["https://en.wikipedia.org", "https://stackoverflow.com", "https://techcrunch.com"]

def text_from_html(body):
    soup = BeautifulSoup(body, 'html.parser')
    texts = soup.findAll(text=True)
    visible_texts = filter(tag_visible, texts)  
    return u" ".join(t.strip() for t in visible_texts)

html = urllib.request.urlopen(addresses).read()
print(text_from_html(html))

推荐答案

您的错误明确指出'list'对象没有属性'timeout'

这是因为urlopen不在列表中.您应该将其嵌套在这样的循环中:

It's because urlopen doesn't take in a list. you should nest it in a loop like this:

my_texts = []
for each in addresses
    html = urllib.request.urlopen(addresses).read()
    print(text_from_html(html)) # or assign to variable like:
    my_texts.append(text_from_html(html))

我建议您为HTTP使用比 urllib 更好的模块，改用 requests ((导入请求)

I would suggest you to use a better module for http than urllib, use requests instead (import requests)

这篇关于AttributeError:'list'对象没有属性'timeout'-尝试使用BeautifulSoup处理多个URL的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！