我应该如何凑这些图片没有错误？ [英] How should I scrape these images without errors?

问题描述

我想刮的图像（或图像链接），本次论坛的（ http://www.xossip.com/showthread.php?t=1384077 ）。我试着美丽汤4和这里是code我尝试：

I'm trying to scrape the images (or the images link) of this forum (http://www.xossip.com/showthread.php?t=1384077) . I've tried beautiful soup 4 and here is the code I tried:

import requests
from bs4 import BeautifulSoup

def spider(max_pages):
    page = 1
    while page <= max_pages:
        url = 'http://www.xossip.com/showthread.php?t=1384077&page=' + str(page)
        sourcecode= requests.get(url)
        plaintext = sourcecode.text
        soup = BeautifulSoup(plaintext)
        for link in soup.findAll('a',{'class': 'alt1'}):
            src = link.get('src')
            print(src)


        page += 1
spider(1)

我应该如何纠正它，使我得到这样 pzy.be/example ？

推荐答案

最简单的方法是只要求每个页面和筛选的img标签：

The simplest way is to just request each page and filter the img tags:

from bs4 import BeautifulSoup
from requests import get
import re

def get_wp():
    start_url = "http://www.xossip.com/showthread.php?t=1384077&page={}"
    for i in range(73):
        r = get(start_url.format(i))
        soup = BeautifulSoup(r.content)
        for img in (i["src"] for i in  soup.find_all("img", src=re.compile("http://pzy.be.*.jpg"))):
           yield img

这篇关于我应该如何凑这些图片没有错误？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！