在AWS Lambda上运行python脚本时出错 [英] Error while running a python script on AWS Lambda

问题描述

我正在尝试在AWS Lambda上运行以下python脚本，该脚本已手动运行，并且可以在输出S3存储桶上获得结果，而没有任何问题.但是现在当我从AWS Lambda调用脚本时出现以下错误时，不确定我是否在脚本上丢失了任何内容?

I'm trying to run the below python script on AWS Lambda, which I have run manually and I could get the outcome on my output S3 bucket without any issue. But now when I invoke the script from AWS Lambda getting the below error, not sure if I am missing anything on the script?

#!/usr/bin/env python3
import boto3

#Function for executing athena queries
def run_query(Event, context):
    ...
    run_query(query, database, s3_output)
    client = boto3.client('athena')
    response = client.start_query_execution(
        QueryString=query,
        QueryExecutionContext={
            'Database': 's3_accesslog'
            },
        ResultConfiguration={
            'OutputLocation': s3_output,
            }
        )
    
#import datetime 
import datetime
year = datetime.date.today()
year = year.strftime("%Y")
month = datetime.date.today()
month = month.strftime("%m")
day = datetime.date.today()
day = day.strftime("%d")

#select bucket 
s3_input = "s3://smathena/cf-ant-prod/year=%s/month=%s/day=%s" % (year, month, day)
   
#Athena configuration
s3_ouput = 's3://smathena/athenatest/'
database = 's3_accesslog'
table = 'test_output1'

#Athena database and table definition
create_database = "CREATE DATABASE IF NOT EXISTS %s;" % (database)
delete_table = "drop table %s.%s;" % (database, table)
create_table = \
  """CREATE EXTERNAL TABLE IF NOT EXISTS %s.%s (
  `Date` DATE,
   Time STRING,
   Location STRING,
   SCBytes BIGINT,
   RequestIP STRING,
   Method STRING,
   Host STRING,
   Uri STRING,
   Status INT,
   Referrer STRING,
   UserAgent STRING,
   UriQS STRING,
   Cookie STRING,
   ResultType STRING,
   RequestId STRING,
   HostHeader STRING,
   Protocol STRING,
   CSBytes BIGINT,
   TimeTaken FLOAT,
   XForwardFor STRING,
   SSLProtocol STRING,
   SSLCipher STRING,
   ResponseResultType STRING,
   CSProtocolVersion STRING,
   FleStatus STRING,
   FleEncryptedFields INT,
   CPort INT,
   TimeToFirstByte FLOAT,
   XEdgeDetailedResult STRING,
   ScContent STRING,
   ScContentLen BIGINT,
   ScRangeStart BIGINT,
   ScRangeEnd BIGINT
   )
   ROW FORMAT DELIMITED FIELDS TERMINATED BY '\t'
   LOCATION '%s'
   TBLPROPERTIES ('skip.header.line.count' = '2');""" % (database, table, s3_input)

#Query definitions
query_1 = "SELECT * FROM %s.%s where CAST(status AS VARCHAR) = '404';" % (database, table)

#Execute all queries
queries = [ create_database, delete_table, create_table, query_1 ]
for q in queries:
   print("Executing query: %s" % (q))
   res = run_query(q, database, s3_ouput)

但是现在当我从AWS Lambda调用脚本时出现以下错误时，不确定我是否在脚本上丢失了任何内容?

But now when I invoke the script from AWS Lambda getting the below error, not sure if I am missing anything on the script?

{
  "errorMessage": "run_query() takes 2 positional arguments but 3 were given",
  "errorType": "TypeError",
  "stackTrace": [
    "  File \"/var/lang/lib/python3.7/imp.py\", line 234, in load_module\n    return load_source(name, filename, file)\n",
    "  File \"/var/lang/lib/python3.7/imp.py\", line 171, in load_source\n    module = _load(spec)\n",
    "  File \"<frozen importlib._bootstrap>\", line 696, in _load\n",
    "  File \"<frozen importlib._bootstrap>\", line 677, in _load_unlocked\n",
    "  File \"<frozen importlib._bootstrap_external>\", line 728, in exec_module\n",
    "  File \"<frozen importlib._bootstrap>\", line 219, in _call_with_frames_removed\n",
    "  File \"/var/task/lambda_function.py\", line 86, in <module>\n    res = run_query(q, database, s3_ouput)\n"
  ]
}``

推荐答案

您的函数 lambda_handle 不符合函数的

输入应该在 event 中.该链接的另一个示例:

inputs to your function should be in event. Another example from that link:

def my_handler(event, context):
    message = 'Hello {} {}!'.format(event['first_name'], 
                                    event['last_name'])  
    return { 
        'message' : message
    }  

在您的情况下，我希望 query ， database 和 s3_output 成为 event 的一部分.您可能应该返回有关正在执行的雅典娜查询的信息.

I would expect query, database, and s3_output to be part of the event in your case. You should probably return information about the executing athena query.

这篇关于在AWS Lambda上运行python脚本时出错的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！