如何在不首先写入标准输出的情况下将日志从内存直接写入 AWS S3?(Python，boto3) [英] How Can I Write Logs Directly to AWS S3 from Memory Without First Writing to stdout? (Python, boto3)

问题描述

我正在尝试将 Python 日志文件直接写入 S3，而没有先将它们保存到标准输出.我希望在程序运行完成后将日志文件自动写入 S3.我想使用 boto3 put_object 方法:

I'm trying to write Python log files directly to S3 without first saving them to stdout. I want the log files to be written to S3 automatically when the program is done running. I'd like to use the boto3 put_object method:

import atexit
import logging

import boto3  


def write_logs(body, bucket, key):
    s3 = boto3.client("s3")
    s3.put_object(Body=body, Bucket=bucket, Key=key)

log = logging.getLogger("some_log_name")
log.info("Hello S3")

atexit.register(write_logs, body=log, bucket="bucket_name", key="key_name")

quit()

这在上传到 S3 时会引发错误.如果我没记错的话，它要求上传到 S3 的对象必须是字节类型的.一旦我有时间重新创建问题，我会用确切的错误更新问题.

This throws an error when uploading to S3. If i remember correctly, it requires that the object uploaded to S3 must be bytes-like. I'll update the question with the exact error once I have time to recreate the issue.

推荐答案

您需要在此处添加一些内容.首先，创建一个 StringIO 对象.然后，使用日志记录 StreamHandler 将日志写入 StringIO 对象.将处理程序添加到您的记录器.最后，调用 StringIO 对象上的 getvalue() 方法.您可以将其写入 S3.

You need to add a couple things here. First, create a StringIO object. Then, write the logs to the StringIO object using a logging StreamHandler. Add the handler to your logger. Finally, call the getvalue() method on the StringIO object. You can write that to S3.

import atexit
import io
import logging

import boto3  


def write_logs(body, bucket, key):
    s3 = boto3.client("s3")
    s3.put_object(Body=body, Bucket=bucket, Key=key)  


log = logging.getLogger("some_log_name")
log_stringio = io.StringIO()
handler = logging.StreamHandler(log_stringio)
log.addHandler(handler)

atexit.register(write_logs, body=log_stringio.getvalue(), bucket="bucket_name", key="key_name")

log.info("Hello S3")

quit()

这篇关于如何在不首先写入标准输出的情况下将日志从内存直接写入 AWS S3?(Python，boto3)的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！