AWS Lambda-MySQL缓存 [英] AWS Lambda - MySQL caching

问题描述

我有使用RDS的Lambda.我想改进它并使用Lambda连接缓存.据我所知，我已经找到了几篇文章，并在我这方面实施了它.但是现在，我不确定这是否是正确的方法.

I have Lambda that uses RDS. I wanted to improve it and use the Lambda connection caching. I have found several articles, and implemented it on my side, best to my knowledge. But now, I am not sure it is this the rigth way to go.

我有Lambda(运行节点8)，其中有几个文件与require一起使用.我将从主函数开始，直到到达MySQL初始化程序，这是确切的路径.一切都将非常简单，仅显示运行MySQL的代码流:

I have Lambda (running Node 8), which has several files used with require. I will start from the main function, until I reach the MySQL initializer, which is exact path. All will be super simple, showing only to flow of the code that runs MySQL:

主要Lambda:

const jobLoader = require('./Helpers/JobLoader');

exports.handler = async (event, context) => {
    const emarsysPayload = event.Records[0];
    let validationSchema;

    const body = jobLoader.loadJob('JobName');
     ...
    return;
...//

职位代码:

const MySQLQueryBuilder = require('../Helpers/MySqlQueryBuilder');

exports.runJob = async (params) => {
      const data = await MySQLQueryBuilder.getBasicUserData(userId);

MySQLBuilder:

const mySqlConnector = require('../Storage/MySqlConnector');

    class MySqlQueryBuilder {
        async getBasicUserData (id) {
            let query = `
    SELECT * from sometable WHERE id= ${id} 
    `;

            return mySqlConnector.runQuery(query);
        }
    }

最后是连接器本身:

const mySqlConnector = require('promise-mysql');
const pool = mySqlConnector.createPool({
        host: process.env.MY_SQL_HOST,
        user: process.env.MY_SQL_USER,
        password: process.env.MY_SQL_PASSWORD,
        database: process.env.MY_SQL_DATABASE,
        port: 3306
    });

    exports.runQuery = async query => {
        const con = await pool.getConnection();
        const result = con.query(query);
        con.release();
        return result;
    };

我知道评估效果会显示实际结果，但是今天是星期五，直到下周下半年，我才能在Lambda上运行它.而且，确实如此，周末的开始真是太棒了，知道我朝着正确的方向……还是不正确.

I know that measuring performance will show the actual results, but today is Friday, and I will not be able to run this on Lambda until the late next week... And really, it would be awesome start of the weekend knowing I am in right direction... or not.

感谢您的投入.

推荐答案

第一件事是了解require在NodeJS中的工作方式.我确实建议您仔细阅读文章，如果您有兴趣进一步了解它.

First thing would be to understand how require works in NodeJS. I do recommend you go through this article if you're interested in knowing more about it.

现在，一旦您需要连接，便永久拥有它，并且不再需要它.这与您要查找的内容相匹配，因为您不想每次都通过创建新的连接来使数据库不堪重负.

Now, once you have required your connection, you have it for good and it won't be required again. This matches what you're looking for as you don't want to overwhelm your database by creating a new connection every time.

但是，有一个问题...

But, there is a problem...

Lambda冷启动

每当您首次调用Lambda函数时，它都会旋转一个包含您的函数的容器，并使该容器保持活动状态约5分钟.只要您一次发出1个请求，很可能(尽管不能保证)每次都将击中相同的容器.但是，如果您同时有2个请求，会发生什么情况?然后，将另一个容器与先前已预热的容器并行旋转.您刚刚在数据库上创建了另一个连接，现在您有2个容器.现在，如果您有3个并发请求，那会发生什么?是的！再增加一个容器，等于再增加一个数据库连接.

Whenever you invoke a Lambda function for the first time, it will spin up a container with your function inside it and keep it alive for approximately 5 mins. It's very likely (although not guaranteed) that you will hit the same container every time as long as you are making 1 request at a time. But what happens if you have 2 requests at the same time? Then another container will be spun up in parallel with the previous, already warmed up container. You have just created another connection on your database and now you have 2 containers. Now, guess what happens if you have 3 concurrent requests? Yes! One more container, which equals one more DB connection.

只要对Lambda函数有新请求，默认情况下，它们就会扩展以满足需求(您可以在控制台中对其进行配置，以将执行限制为任意数量的并发执行-遵守帐户限制)

As long as there are new requests to your Lambda functions, by default, they will scale out to meet demand (you can configure it in the console to limit the execution to as many concurrent executions as you want - respecting your Account limits)

仅通过在函数调用时简单地要求您的代码，就不能安全地确保与数据库的连接数量固定.好在，这不是你的错.这就是Lambda函数的行为.

You cannot safely make sure you have a fixed amount of connections to your Database by simply requiring your code upon a Function's invocation. The good thing is that this is not your fault. This is just how Lambda functions behave.

...另一种方法是

例如，在真正的缓存系统中缓存所需的数据，例如 ElasticCache .然后，您可以通过 CloudWatch事件以一定的时间频率运行.然后，该函数将查询您的数据库并将结果存储在您的外部缓存中.这样，您可以确保一次只能由一个Lambda打开数据库连接，因为它将尊重CloudWatch Event，该事件每次触发仅运行一次.

to cache the data you want in a real caching system, like ElasticCache, for example. You could then have one Lambda function be triggered by a CloudWatch Event that runs in a certain frequency of time. This function would then query your DB and store the results in your external cache. This way you make sure your DB connection is only opened by one Lambda at a time, because it will respect the CloudWatch Event, which turns out to run only once per trigger.

编辑:OP在评论部分中发送了链接之后，我决定添加更多信息，以澄清上述文章要说的内容

EDIT: after the OP sent a link in the comment sections, I have decided to add a few more info to clarify what the mentioned article wants to say

摘自文章:

简单.您能够存储超出我们范围的变量 处理函数.这意味着您可以创建数据库 处理程序函数外部的连接池，然后可以是 与该功能的每次将来调用共享.这允许 发生合并."

"Simple. You ARE able to store variables outside the scope of our handler function. This means that you are able to create your DB connection pool outside of the handler function, which can then be shared with each future invocation of that function. This allows for pooling to occur."

这正是您在做什么.这行得通！但是问题是，如果您同时具有N个连接(Lambda请求).如果未设置任何限制，则默认情况下最多可以同时启动1000个Lambda函数.现在，如果您在接下来的5分钟内同时发出另外1000个请求，则很可能不会打开任何新连接，因为在先前的调用中已经打开了这些新连接，并且这些容器仍然处于活动状态.

And this is exactly what you're doing. And this works! But the problem is if you have N connections (Lambda Requests) at the same time. If you don't set any limits, by default, up to 1000 Lambda functions can be spun up concurrently. Now, if you then make another 1000 requests simultaneously in the next 5 minutes, it's very likely you won't be opening any new connections, because they have already been opened on previous invocations and the containers are still alive.

这篇关于AWS Lambda-MySQL缓存的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！