AWS 系统管理器“进行中"命令限制为 5? [英] AWS Systems Manager "In Progress" commands limit to 5?
问题描述
因此，首先，我四处寻找有关我面临的问题的现有线程，但没有找到任何内容.我也在 AWS 论坛 上发布了此消息，但没有得到答复.如果这里已经有一个线程已经存在，我深表歉意.此外，我将为即将发布的相对较长的帖子道歉.
So firstly, I looked around for an existing thread on the issue I'm facing, but I haven't found anything. I've also posted this on AWS forums and got no answer. If there is an existing thread here for this already, I apologize. Furthermore, I will apologize for the upcoming relative long post.
现在，我想要做的是使用 AWS-RunShellScript 文档 运行同一应用程序的多个(阻塞)进程.问题是，我不能让超过 5 个进程开始使用这种方法.如果我通过 SSH 甚至手动启动它们，我可以毫无问题地启动数十个.
Now, what I am trying to do is to run multiple (blocking) processes of the same app, using the AWS-RunShellScript document. Problem is, I can't have more than 5 processes started using this method. If I start them via SSH or even manually, I can start dozens without any issues.
我使用的实例是 Ubuntu.我正在使用 Python 3.7.4 进行 AWS 资源操作，但在使用 AWS 控制台 时也会发生同样的情况.
The instance I am using is Ubuntu. I am doing AWS resource manipulation using Python 3.7.4, but the same occurs when using the AWS Console as well.
每个命令通常会阻塞终端(即阻止您在终端的该实例中发出更多命令，如果您要手动执行)-反过来，它会设置其状态，正如 AWS SSM 所见 - 进行中.从本质上讲，从 AWS SSM 的角度来看，该命令并不完整，直到进程被终止或停止(更多内容见下文).
Each command would normally block the terminal (i.e. prevent you for issuing further commands in that instance of the terminal, if you were to do it manually) - which, in turn, sets its status, as seen by the AWS SSM - In Progress. Essentially, the command is not complete from AWS SSM point of view, until the process is killed or stopped (more on that below).
问题是我可以通过 SSM 运行多达 4 个进程，并且仍然能够使用 SSM(杀死、检查等)操纵它们 - 这意味着最多有 4 命令进行中.然而，当我启动第 5 个时，虽然它们都继续工作，但我不能再使用 SSM，没有其他命令被执行(无论是新进程还是任何其他命令)
The problem is that I can run up to 4 processes through SSM and be able to still manipulate them using SSM (killing, inspecting, etc) - meaning having a maximum of 4 commands In Progress. However, when I launch a 5th one, while they all continue to work, I can't use SSM anymore, no other command gets executed (either being a new process or any other command)
重现此问题的最简单方法是通过 AWS-RunShellScript 文档发送 5 个简单的 sleep 60 命令，然后尝试任何新命令- 您会在 SSM 中注意到它们将作为 In Progress 弹出，但是如果您拖尾 amazon-ssm-agent.log 文件，则实际上不会执行任何新命令.更奇怪的是，您会注意到日志在此块之后停止:
Easiest way to reproduce this is to send 5 simple sleep 60 commands, via AWS-RunShellScript document, and then attempt any new command - you'll notice in the SSM they will pop as In Progress, but if you tail the amazon-ssm-agent.log file, no new commands will actually be executed. What's more odd, you'll notice that the log stops after this block:
2019-08-13 08:25:12 INFO [MessagingDeliveryService] SendReply Response{
Description: "Reply e82b5dcb-0e81-4698-8f6e-fe1411f18300 was successfully sent.",
MessageId: "aws.ssm.1af47ba7-0d28-41ac-83dd-3bffbaa7db2d.i-08d3f4176a025a07b",
ReplyId: "e82b5dcb-0e81-4698-8f6e-fe1411f18300",
ReplyStatus: "QUEUED"
在此之后将不会处理更多命令，也不会记录更多信息.但是，使用我们的示例，当 sleep 结束时，QUEUED 命令将在另一个插槽打开时立即执行(假设您只能排队 5strong> 一次执行命令，我相信是这样，但没有提及).
No further commands will be processed past this point, no further information being logged. However, using our example, when the sleep ends, the QUEUED commands will get executed as soon as another slot is opened (assuming you can only queue 5 commands at a time, as I believe it's the case, but it's nowhere mentioned).
注意:正如我提到的 AWS-RunShellScript 文档，同样的问题也出现在 AWS-RunRemoteScript 文档中.
Note: As I've mentioned AWS-RunShellScript document, the same issue occurs with the AWS-RunRemoteScript document as well.
由于我必须提供一些代码，请使用 Python 从上述示例中找到以下片段:
Since I have to provide some code, please find below snippets from the example mentioned, using Python:
run_cmd_shell = lambda: ssm.send_command(
Targets=[{
'Key': 'tag:Name',
'Values': ['test_ssm']
},
{
'Key': 'tag:Role',
'Values': ['slave']
}
],
DocumentName='AWS-RunShellScript',
Parameters={'commands': [f'sleep {sleep_time}'],
'workingDirectory': [workingDirectory],
'executionTimeout': [executionTimeout]
},
OutputS3BucketName=bucket_name,
OutputS3KeyPrefix=bucket_prefix,
MaxConcurrency='150'
)
remote_cmd_script = lambda: ssm.send_command(
Targets=[{
'Key': 'tag:Name',
'Values': ['test_ssm']
},
{
'Key': 'tag:Role',
'Values': ['slave']
}
],
DocumentName='AWS-RunRemoteScript',
Parameters={'sourceType': ['S3'],
'sourceInfo': [f'{{"path":"https://s3.amazonaws.com/{bucket_name}/agents/{project_name}"}}'],
'commandLine': [f'sleep {sleep_time}'],
'workingDirectory': [workingDirectory],
'executionTimeout': [executionTimeout]
},
OutputS3BucketName=bucket_name,
OutputS3KeyPrefix=bucket_prefix,
MaxConcurrency='150'
)
我希望能够通过 SSH 或手动(比 5 多得多)运行尽可能多的阻塞命令，但是要么我在 SSM 方面做错了什么，或 AWS SSM 是有限的.
I would expect to be able to run as many blocking commands as I can via SSH or manually (which is a lot more than 5), but either I am doing something wrong SSM-wise, or AWS SSM is limited.
推荐答案
简答.增加 amazon-ssm-agent.json 文件中的 CommandWorkersLimit 设置
Short answer. Increase the CommandWorkersLimit setting in the amazon-ssm-agent.json file
对我如何追踪它的回应略长.
Slightly longer response of how I tracked it down.
来自源代码中的ReleaseNotes
去掉了最大并行执行次数的上限代理上的文档(以前最多 10 个)您可以配置这个数字通过设置CommandWorkerLimit"属性amazon-ssm-agent.json 文件
Removed the upper limit for the maximum number of parallel executing documents on the agent (previously the max was 10) You can configure this number by setting the "CommandWorkerLimit" attribute in amazon-ssm-agent.json file
如果我们达到峰值 amazon-ssm-agent.json.template 文件在 Mds 部分，您可以看到它设置为 5.
And if we take a peak amazon-ssm-agent.json.template file in the Mds section you can see it set to 5.
{
"Profile":{
"ShareCreds" : true,
"ShareProfile" : ""
},
"Mds": {
"CommandWorkersLimit" : 5,
"StopTimeoutMillis" : 20000,
"Endpoint": "",
"CommandRetryLimit": 15
},
... <LOTS DELETED>
}
这篇关于AWS 系统管理器“进行中"命令限制为 5?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
