AWS Lambda函数的独立python子流程 [英] Independent python subprocess from AWS Lambda function

问题描述

我已经成功创建了可读写RDS的Lambda函数(app1).

I have successfully created a Lambda function (app1) that reads and writes to RDS.

我的Lambda函数是用python2.7编写的，并以压缩包的形式上传.

My Lambda function is written in python2.7 and uploaded as a zipped package.

我在与RDS和Lambda函数相同的VPC上的EC2实例上创建并测试了压缩包.

I created and tested the zipped package on an EC2 instance in the same VPC as my RDS and Lambda function.

接下来，我在Lambda函数中添加了该功能，以使用subprocess.popen弹出一个独立的子进程(app2)，并让app1返回，而app2子进程自行继续.我测试了app1可以成功返回其处理程序的输出，而app2则通过在app2中放置60秒的睡眠并跟踪app2的输出文件来继续返回.

Next, I added the functionality to my Lambda function to popen an independent subprocess (app2) using subprocess.popen and had app1 return while the app2 subprocess continued on its own. I tested that app1 would successfully return its handler's output while app2 continued by putting a 60 second sleep in app2 and tailed the output file of app2.

我在EC2实例中成功测试了app1和app2功能.

I successfully tested app1 and app2 functionality in the EC2 instance.

上载新程序包后，我的app1看起来完全符合预期，并立即返回其处理程序的输出，但是app2功能并未出现"到实例化中，但是没有日志，错误或要捕获的输出来自app2.

After uploading the new package, my app1 appears to behave exactly as expected, and returns its handler's output immediately, but the app2 functionality doesn't "appear" to instantiated, but there is no logs, errors, or output to capture from app2.

在app1中，我通过在独立subproccess.popen之前和之后执行subprocess.check_output(['ls'，'-la'])来测试该子流程是否正常工作，并且本地文件夹与我的文件一起显示.除了没有按预期创建的app2output文件.

In app1, I tested that subprocess worked by performing a subprocess.check_output(['ls','-la']) prior to and after the independent subproccess.popen, and the local folder is displayed with my files. Except there isn't a app2output file created as expected.

两个问题

1. AWS-Lambda概念中是否缺少某些特别之处 导致app2失败"?所谓失败"，是指既不编写与创建新文件并对其进行写入，也不以与app1成功一样的方式在Cloudwatch中创建任何日志，也不像app1一样打印到Lambda控制台.
2. 如何在AWS-Lambda环境中捕获app2的任何输出(日志信息和错误)?

1. Is there something special that I am missing in AWS-Lambda concepts that is causing app2 to "fail"? By "fail" I mean not writing creating the new file and writing to it, nor creating any logs in Cloudwatch the same way app1 successfully does, nor printing out to the Lambda console like app1 does.
2. How do I catch any output (logging info and errors) for app2 in an AWS-Lambda environment?

app1.py

import subprocess
import sys
import logging
import rds_config
import pymysql
#rds settings
rds_host  = "rdshost"
name = rds_config.db_username
password = rds_config.db_password
db_name = rds_config.db_name
port = 3306

logger = logging.getLogger()
logger.setLevel(logging.INFO)

server_address = (rds_host, port)
try:
    conn = pymysql.connect(rds_host, user=name, passwd=password, db=db_name, connect_timeout=5)
except:
    logger.error("ERROR: Unexpected error: Could not connect to MySql instance.")
    sys.exit()

def handler(event, context):

    cur = conn.cursor()
    isql = "INSERT ..."
    cur.execute(isql)
    conn.commit()
    newid = cur.lastrowid
    cur.close()

    args = [str(newid),str(event['name'])]

    logger.info('ARGS: '+str(args))
    print 'pwd: '
    output = subprocess.check_output(['pwd'])
    print output
    print 'ls -la'
    output = subprocess.check_output(['ls','-l'])
    print output

    pid = subprocess.Popen([sys.executable, "app2.py"]+args, stdout=subprocess.PIPE, stderr=subprocess.PIPE, stdin=subprocess.PIPE)

    logger.info('PID: '+str(pid))
    output = subprocess.check_output(['ls','-l'])
    print output

    return "{'status':'success','newid':'"+str(newid)+"'}";

app1.py中"logger.info('PID:'+ str(pid))"的输出

The output from "logger.info('PID: '+str(pid))" in app1.py

就像:"PID:< subprocess.Popen对象位于0x7f51aba2a550>"

is like: "PID: <subprocess.Popen object at 0x7f51aba2a550>"

app2

import sys
import logging
from datetime import datetime
import time

fo = open('app2output','a+')
fo.write("starting with: "+str(sys.argv)+"\n")

logger = logging.getLogger()
logger.setLevel(logging.INFO)
logger.info("Starting with: "+str(sys.argv)+"\n")

#log accumulated processing time
t1 = datetime.now();
sleep(60)
t2 = datetime.now();
tstring = "{'t1':'"+str(t1)+"','t2':'"+str(t2)+"','args':'"+str(sys.argv[1])+"'}"
logger.info(tstring+"\n")
fo.write(tstring+"\n")
fo.close()
sys.exit()

推荐答案

处理程序函数返回后，AWS Lambda环境将终止.处理程序功能完成后，您将无法在AWS Lambda环境中在后台运行子流程.您需要对Lambda函数进行编码，以等待子进程完成.

The AWS Lambda environment will be terminated as soon as the handler function returns. You can't run subprocesses in the background in an AWS Lambda environment after your handler function is complete. You would need to code your Lambda function to wait for the subprocess to complete.

这篇关于AWS Lambda函数的独立python子流程的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！