Discord.py:无法弄清楚如何使用 aiohttp 代替此用例的请求 [英] Discord.py: Can't figure out how to use aiohttp in place of requests for this use case

问题描述

我的原始代码(使用了 requests 库):

My original code (which made use of the requests library):

sess = requests.session()
req = sess.get(i) #i is a url
soup = BeautifulSoup(req.content, 'html.parser')
link = soup.find('a')['href']
with sess.get(link, stream=True) as req2:
    with open(os.path.join('C:\\save\\location', "download.txt"), "wb") as x_file:
        shutil.copyfileobj(req2.raw, x_file)

这段代码的作用:

1. 为我获取位于 url i
的页面
2. 使用 bs4 在所述页面(其文本文件)上查找特定下载链接 link.
3. 该网站使用会话 cookie，因此我使用 requests 中的 sessions 来保存我的 cookie 从一个请求到下一个请求，以便它可以下载文件.
4. 使用 shutil 将文本文件写入我设备中提到的目录中.

1. Gets me the page at url i
2. Finds a particular download link link on said page (its a text file), using bs4.
3. The website uses session cookies, so I used sessions from requests to keep my cookies from one request to the next so it can download the file.
4. Writes the text file to my device at the mentioned directory using shutil.

就代码的工作而言，它可以.文件被下载，我在所需的文件夹中有名为 download.txt 的所需文件.

As far as the working of the code is concerned, its OK. The file gets downloaded, and I have the required file named download.txt in the desired folder.

但是，当我尝试在 async 函数中将其用作 discord.py bot 的一部分时，问题就出现了.代码仍然做它需要做的事情但是:当它得到多个命令时，它一个接一个地执行它们，这显然是不可取的.我从 Stack Overflow 本身发现这是因为在 async 函数中使用 requests 会导致整个代码阻塞，从而导致我观察到的情况.因此，我尝试使用 aiohttp 而不是 requests，它显然适用于 async 函数.我已经设法用 aiohttp 替换了我对 requests 的大部分使用，它似乎运行良好，但是我似乎无法弄清楚正确的语法来做什么我在上面做(在使用会话 cookie 的同时下载文本文件，bs4 和 shutil.

However the problem arises when I try to use it as part of a discord.py bot, in an async function. The code still does what it needs to do but: When it gets multiple commands, it executes them one after the other, which is obviously not desirable. I found from Stack Overflow itself that this is because using requests in an async function causes the whole code to block, resulting in what I observed. So instead of requests, I tried to use aiohttp, which apparently works well with async functions. I have managed to replace most of my usage of requests with aiohttp, and it does seem to be working well, however I cannot seem to figure out the correct syntax to do what I am doing above (downloading the text file while making use of session cookies, bs4 and shutil.

在使用 aiohttp 时，我将如何保留会话 cookie、使用 bs4 并将文件写入我的设备?

How would I go about keeping my session cookies, using bs4 and writing the file to my device when using aiohttp?

编辑原始链接和文件属于个人性质，但如果您想要一个样本进行测试，这应该以类似的方式工作:

Edit The original link and file are of a personal nature, but in case you want a sample to test against, this should work in a similar way:

import requests
from bs4 import BeautifulSoup
import os
import shutil

sess = requests.session()
req = sess.get('https://www.gutenberg.org/ebooks/2600')
soup = BeautifulSoup(req.content, 'html.parser')
link = soup.find('a', text="EPUB (no images)")['href']
link="https://www.gutenberg.org"+link
with sess.get(link, stream=True) as req2:
    with open(os.path.join('C:\\save\\location', "war.epub"), "wb") as x_file:
        shutil.copyfileobj(req2.raw, x_file)

这会在您想要的目录中创建一个名为 war.epub 的文件，其中包含从古腾堡项目下载的战争与和平的 epub 版本.我将如何使用 aiohttp 而不是请求来实现这一点?

This makes a file called war.epub in your desired directory, containing the epub version of War and Peace downloaded from the Gutenberg Project. How would I go about implementing this using aiohttp instead of requests?

推荐答案

注意我已经包含了 aiofile 来异步写入文件，因为大文件会导致代码阻塞.

Note that I have included aiofile to asynchronously write the file, as big files will cause the code to block.

import aiohttp
import asyncio
from bs4 import BeautifulSoup
import os
from aiofile import AIOFile

async def write_file():
    sess = aiohttp.ClientSession()
    req = await sess.get('https://www.gutenberg.org/ebooks/2600')

    soup = BeautifulSoup(await req.read(), 'html.parser')
    link = soup.find('a', text='EPUB (no images)')['href']
    link = 'https://www.gutenberg.org' + link

    req2 = await sess.get(link)

    async with AIOFile(os.path.join('C:\\save\\location', 'war.epub'), 'wb') as x_file:
        await x_file.write(await req2.read())

    await sess.close()


loop = asyncio.get_event_loop()
loop.run_until_complete(write_file())

这篇关于Discord.py:无法弄清楚如何使用 aiohttp 代替此用例的请求的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！