如何在不先写入stdout的情况下将日志直接从内存直接写入AWS S3? (Python，boto3) [英] How Can I Write Logs Directly to AWS S3 from Memory Without First Writing to stdout? (Python, boto3)

问题描述

我正在尝试将Python日志文件直接写入S3，而不先将它们保存到stdout.我希望在程序运行完毕后将日志文件自动写入S3.我想使用boto3 put_object方法:

I'm trying to write Python log files directly to S3 without first saving them to stdout. I want the log files to be written to S3 automatically when the program is done running. I'd like to use the boto3 put_object method:

import atexit
import logging

import boto3  


def write_logs(body, bucket, key):
    s3 = boto3.client("s3")
    s3.put_object(Body=body, Bucket=bucket, Key=key)

log = logging.getLogger("some_log_name")
log.info("Hello S3")

atexit.register(write_logs, body=log, bucket="bucket_name", key="key_name")

quit()

当上传到S3时，这将引发错误.如果我没记错的话，它要求上传到S3的对象必须是字节状的.我有时间重新创建问题后，将使用确切的错误来更新问题.

This throws an error when uploading to S3. If i remember correctly, it requires that the object uploaded to S3 must be bytes-like. I'll update the question with the exact error once I have time to recreate the issue.

推荐答案

您需要在此处添加一些内容.首先，创建一个StringIO对象.然后，使用日志记录StreamHandler将日志写入StringIO对象.将处理程序添加到记录器中.最后，在StringIO对象上调用getvalue()方法.您可以将其写入S3.

You need to add a couple things here. First, create a StringIO object. Then, write the logs to the StringIO object using a logging StreamHandler. Add the handler to your logger. Finally, call the getvalue() method on the StringIO object. You can write that to S3.

import atexit
import io
import logging

import boto3  


def write_logs(body, bucket, key):
    s3 = boto3.client("s3")
    s3.put_object(Body=body, Bucket=bucket, Key=key)  


log = logging.getLogger("some_log_name")
log_stringio = io.StringIO()
handler = logging.StreamHandler(log_stringio)
log.addHandler(handler)

atexit.register(write_logs, body=log_stringio.getvalue(), bucket="bucket_name", key="key_name")

log.info("Hello S3")

quit()

这篇关于如何在不先写入stdout的情况下将日志直接从内存直接写入AWS S3? (Python，boto3)的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！