如何在ECS实例角色中使用python ElasticSearch软件包而不是通过凭据? [英] How to use python ElasticSearch package with ECS Instance Role instead of passing credentials?

问题描述

我构建了一个可在不同时间查询ElasticSearch的应用.现在，我正在将应用程序托管在ECS上，现在，我想将查询切换为基于实例角色权限，而不是基于作为环境变量传递的凭据.

I've built an app that queries ElasticSearch at various times. Now that I'm moving the app to be hosted on a ECS, I want to switch my queries to be based on the Instance Role permissions rather than credentials that I've been passing as environment variables.

基于 docs ，如果我没有传递任何凭据给Boto3，我应该使用我的实例角色.但是为了使用python中的 ElasticSearc h包发送查询，我必须传递凭据.有没有使用实例角色来做到这一点的方法?

Based on the docs, if I pass no credentials to Boto3, I ought to get use my Instance Role. But in order to send my query using the ElasticSearch package in python, I have to pass credentials. Is there a way to do this using Instance Roles?

示例代码

The example code here gave me hope that I would just pass in the `boto3.Session().get_credentials() function to retrieve the Instance Role credentials. But it turns out, as explained by my aws admin, that Instance Roles do not actually have credentials (aws_key, aws_secret_key). As a result my code seems to generate an AWS key which is not valid on the cloud and returns only null values for my queries.

有人告诉我，唯一的选择是使用

I've been told that the only alternative is to use the ES Boto3 client and not pass credentials...but using ES with this client is not nearly as friendly as using the ElasticSearch python package interface.

有没有一种方法可以使ElasticSearch包在不传递凭据且仅依赖实例角色权限的情况下查询ES?

Is there a way to make it work where the ElasticSearch package could query the ES without passing credentials and only relying on the Instance Role permissions?

到目前为止，这是我的代码:

Here is my code so far:

if os.environ.get('LOCAL_DEV')==1:
  AWS_KEY = os.environ['AWS_ACCESS_KEY']
  AWS_SECRET_KEY = os.environ['AWS_SECRET_KEY']
else:
  credentials = boto3.Session().get_credentials() #fails to use instance role
  AWS_KEY=credentials.access_key
  AWS_SECRET_KEY=credentials.secret_key

# It pulls my environment variables when running locally, and when running on the cluster, I need it to use the Instance Role
  
#### below is how I initialize and query with the ElasticSearch package


import elasticsearch as es
from requests_aws4auth import AWS4Auth
from constants import AWS_KEY, AWS_SECRET_KEY

awsauth = AWS4Auth(AWS_KEY, AWS_SECRET_KEY, 'us-west-2', 'es') 
#requires a credential input
host ='search-xxx.xxx.xxxxxxx.xxxx.amazonaws.com'
e_s = es.Elasticsearch(
            hosts=[{'host': host, 'port': 443}],
            http_auth=awsauth,
            use_ssl=True,
            verify_certs=True,
            connection_class=es.RequestsHttpConnection
            )

   res = e_s.search(index='foo', body='bar')
    res = res['hits']['hits']

在此方面的任何帮助将不胜感激

Any help on this would be very appreciated

推荐答案

我最终要做的是使用基于InstanceRole权限的boto3提取临时凭据.由于这些是临时凭证，因此需要对其进行轮换.但是，它起作用了！在发现需要在AWS4Auth命令中使用临时令牌之前，我很沮丧.

What I ended up doing was pulling temporary credentials using boto3 that were based on the the InstanceRole permissions. Since these are temporary credentials, they need to be rotated. However, it worked! I was stumped until I discovered the need to use a temporary token in the AWS4Auth command

temp_access=boto3.Session(region_name='foo-region').get_credentials().access_key
temp_secret=boto3.Session(region_name='foo-region').get_credentials().secret_key
temp_token=boto3.Session(region_name='foo-region').get_credentials().token

########
from requests_aws4auth import AWS4Auth
awsauth=AWS4Auth(temp_access, temp_secret, 'us-west-2', 'es', session_token=temp_token)

es.Elasticsearch(hosts=[{'host':host, 'port':443}],
                 http_auth=awsauth, 
                 use_ssl=True, 
                 verify_certs=True, 
                 connection_class=es.RequestsHttpConnection)

查看全文