“HTTPError: 429 Client Error: Too Many Requests for url"在 Python [w/Azure 开发人员帐户] [英] "HTTPError: 429 Client Error: Too Many Requests for url" in Python [w/ Azure developer account]

问题描述

尝试在包含 5-20 个图像的小示例目录上运行 Python 代码时出现持续异常.

Getting continual exception when trying to run Python code on a small sample directory of 5-20 images.

我在 Microsoft 有一个开发者帐户(截至今天早上)，并已咨询 Azure 支持，但仅通过聊天无法解决此问题.他们已经指示我把这个贴在这里，所以如果让其他人看到它，我很抱歉！

I have a developer account with Microsoft (as of this morning) and have inquired with Azure Support, but this issue wasn't solvable through chat alone. They've instructed me to post this here, so I'm sorry if it comes off as an eye-roll to everyone else!

此网络范围内的文档很少.也许这会对其他人有所帮助，因为对于像我一样拥有大量杂乱图像的图形和媒体人员来说，这可能会改变游戏规则.

There's very little documentation on this web-wide. Maybe this'll help someone else, because this is potentially a game-changer for people in graphics and media who have a huge amount of disorganized images like I do.

请注意，此代码昨晚确实有效！但只有一次.不知道出了什么问题！

Note that this code DID work once last night! But only once. No idea what went wrong!

# API reference :

# https://westus.dev.cognitive.microsoft.com/docs/services/5adf991815e1060e6355ad44/operations/56f91f2e778daf14a499e1fa

# 参考 : https://ledge.ai/microsoft-computer-vision-api/

# 機能概要 : img フォルダ中の画像をAI解析し、ファイルのリネームを行います。

# 使い方 : python3 cv_demo.py

# 注意 : サブスクリプションキーは変更してください


import requests

import glob

import os

import time

import urllib



subscription_key = "i do have a real subscription key don't worry"

assert subscription_key



vision_base_url = "https://westcentralus.api.cognitive.microsoft.com/vision/v2.0/"

analyze_url = vision_base_url + "analyze"



# ファイル名を変更

def file_rename(list_1, list_2):

    for i in range(len(list_1)):

        os.rename(list_1[i], './img/' + list_2[i] + '.jpg')



def ms_computer_vision_api(filepath):

    headers = {'Ocp-Apim-Subscription-Key': subscription_key,'Content-Type': 'application/octet-stream'}

    params = urllib.parse.urlencode({
    # Request parameters
   'visualFeatures': 'Categories,Tags,Description,Faces'
   })





    img = open(filepath, 'rb')

    img_byte = img.read()



    response = requests.post(analyze_url, data=img_byte, headers=headers, params=params)

    response.raise_for_status()



    return response.json()



if __name__ == "__main__":

    # 画像ファイルを配列に格納

    image_file = glob.glob('./img/*')



    vision_file_name = []



    start = time.time()



    # Computer Vision APIにリクエストして結果を取得

    for i in range(len(image_file)):

        json_data = ms_computer_vision_api(image_file[i])



        # 生成された文章を取得

        file_name = json_data['description']['captions'][0]['text']

        vision_file_name.append(file_name)



    # 文章の空白をファイル名用にアンダーバーに修正

    for i in range(len(vision_file_name)):

        vision_file_name[i] = vision_file_name[i].replace(' ', '_')



    file_rename(image_file,vision_file_name)



    # 経過時間を出力

    print("elapsed_time:{0}".format(time.time() - start) + "[sec]")

 File "C:\Users\MyName\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.7_qbz5n2kfra8p0\LocalCache\local-packages\Python37\site-packages\requests\models.py", line 940, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 429 Client Error: Too Many Requests for url: https://renameimages.cognitiveservices.azure.com/vision/v2.0/analyze?visualFeatures=Description

推荐答案

根据您的代码，此错误来自请求认知 API 的循环.

Based on your code this error comes from a loop which requesting a cognitive API.

其实每个认知服务都有TPS(Transaction Per Second)的限制，超过TPS就会报429错误.即使特定的认知服务具有更高的 TPS，例如 50 TPS，也许您仍然会出现 429 错误.您应该始终使用以下策略以避免将来出现 429.

Actually each Cognitive Service has the limit of TPS (Transaction Per Second), and will report 429 error when exceed the TPS. Even if specific Cognitive Service has a higer TPS such as 50 TPS, maybe you still have 429 error. You should always use the following policy to avoid 429 in the future.

以下是429的解释以及429的处理方法.

The following is the explanation 429 and how to handle 429.

1. HTTP 429 表示 RateLimitExceeded，这意味着您每秒或每分钟进行过多的 API 调用.

1. HTTP 429 would indicate RateLimitExceeded, meaning you are making too many API calls per second or minute.

HTTP 429 发生时，您必须等待一段时间再次调用 API，否则将拒绝下一次 API 调用.通常我们使用指数回退重试策略来重试操作来处理 429 错误:

When HTTP 429 happens, you must wait for some time to call API again, or else the next call of API will be refused. Usually we retry the operation using something like an exponential back off retry policy to handle the 429 error:

2.1) 您需要检查代码中的 HTTP 响应代码.

2.1) You need check the HTTP response code in your code.

2.2) 当 HTTP 响应码为 429 时，请在 N 秒后重试此操作，您可以自行定义例如 10 秒……

2.2) When HTTP response code is 429, then retry this operation after N seconds which you can define by yourself such as 10 seconds…

例如，下面是 429 的响应.您可以将等待时间设置为 (26 + n) 秒.(PS:这里可以自己定义n，比如n = 5…)

For example, the following is a response of 429. You can set your wait time as (26 + n) seconds. (PS: you can define n by yourself here, such as n = 5…)

{
    "error":{
       "statusCode": 429,
        "message": "Rate limit is exceeded. Try again in 26 seconds." 
    }
}

2.3) 如果第 2 步成功，则继续下一步操作.

2.3) If step 2 succeed, continue the next operation.

2.4) 如果第 2 步也失败并显示 429，则在 N*N 秒后重试此操作(您也可以自己定义)，这是指数退避重试策略..

2.4) If step 2 fail with 429 too, retry this operation after N*N seconds (you can define by yourself too) which is an exponential back off retry policy..

2.5) 如果第 4 步也失败并返回 429，请在 NNN 秒后重试此操作……

2.5) If step 4 fail with 429 too, retry this operation after NNN seconds…

2.6) 你应该一直等待当前操作成功，等待时间会呈指数增长.

2.6) You should always wait for current operation to succeed, and the Waiting time will be exponential growth.

这篇关于“HTTPError: 429 Client Error: Too Many Requests for url"在 Python [w/Azure 开发人员帐户]的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！