如何在 aws lambda 中添加 mecab 库 [英] how to add mecab library in aws lambda

问题描述

我正在尝试将 mecab 库添加到 aws lambda 层，但没有奏效.

I'm trying to add mecab library to aws lambda layer but it didn't work.

我想要的是标记日语和韩语.标记化就足够了.

What I want is to tokenize Japanese and Korean languages. Tokenizing is enough.

这是我所做的.(我参考了这个网站:https://towardsdatascience.com/how-to-install-python-packages-for-aws-lambda-layer-74e193c76a91 用于为 aws lambda 层安装 python 包)

Here's what I have done. (I referred to this site: https://towardsdatascience.com/how-to-install-python-packages-for-aws-lambda-layer-74e193c76a91 for installing python packages for aws lambda layers)

1. AWS EC2 docker 安装.

1. AWS EC2 docker installation.

构建 docker 文件

Build docker file

sudo vi Dockerfile

-----------------vi editor------------------
FROM amazonlinux:2.0.20191016.0
RUN yum install -y python37 && \
    yum install -y python3-pip && \
    yum install -y zip && \
    yum clean all
RUN python3.7 -m pip install --upgrade pip && \
    python3.7 -m pip install virtualenv
-----------------vi editor------------------


docker build -t lambdalayer .

1. 运行

docker run -it --name lambdalayer lambdalayer:latest bash

1. 安装python包

python3.7 -m venv mypackages

source mypackages/bin/activate

pip install mecab-python3 -t ./python
pip install unidic-lite -t ./python
pip install --no-binary :all: mecab-python3 -t ./python
pip install -v python-mecab-ko -t ./python

deactivate

1. 压缩文件

zip -r python.zip ./python/

docker cp lambdalayer:python.zip /home/ubuntu/

1. AWS s3 上传

cd /home/ubuntu

aws s3 cp python.zip s3://bukketyounghee

1. 制作一个lmabda层

aws lambda publish-layer-version --layer-name layer-search --compatible-runtimes "python3.7" --content S3Bucket=bukketyounghee,S3Key=python.zip

我不知道接下来该怎么办.它不必是 mecab 库，但我想使用 aws lambda，因为我想要一个无服务器应用程序.请帮帮我.

I don't know what I should do next. It doesn't have to be mecab library but I want to use aws lambda because I want a serverless application. Please help me.

提前致谢！

推荐答案

您可以创建一个 lambda 层 使用 docker，如 AWS 博客.

You can create a lambda layer using docker as described in the AWS blog.

因此您可以将 mecab 添加到您的函数中，如下所示:

Thus you can add mecab to your function as follows:

1. 创建空文件夹，例如mylayer.

转到文件夹并使用

mecab-python3
unidic-lite

1. 运行以下 docker 命令:

该命令将为python3.8创建层:

The command will create layer for python3.8:

docker run -v "$PWD":/var/task "lambci/lambda:build-python3.8" /bin/sh -c "pip install -r requirements.txt -t python/lib/python3.8/site-packages/; exit"

1. 将图层存档为 zip:

zip -9 -r mylayer.zip python 

1. 在 AWS 控制台中基于 mylayer.zip 创建 lambda 层.不要忘记将 Compatible runtime 指定为 python3.8.

1. Create lambda layer based on mylayer.zip in the AWS Console. Don't forget to specify Compatible runtime to python3.8.

将第 5 步中创建的图层添加到您的函数中.

Add the the layer created in step 5 to your function.

我使用您的代码测试了图层:

I tested the layer using your code:

import json

import MeCab

def lambda_handler(event, context):
        
    wakati = MeCab.Tagger("-Owakati")

    a = wakati.parse("pythonが大好きです").split()
    return  {
        'statusCode': 200,  
        'body': json.dumps(a)
        }

它正常工作:

{
  "statusCode": 200,
  "body": "[\"python\", \"\\u304c\", \"\\u5927\\u597d\\u304d\", \"\\u3067\\u3059\"]"
}

这篇关于如何在 aws lambda 中添加 mecab 库的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！