R + httr和EC2 API身份验证问题 [英] R + httr and EC2 api authentication issues
问题描述
我想使用R包 httr 通过其API访问EC2服务。但是我有点不确定如何开始，因为它不属于通常的 Oauth2.0身份验证格式，在该格式中您具有通常的：密钥，机密，令牌和签名系统。我认为EC2使用签名版本2方法，但是我不清楚它是如何工作的。
查看EC2提供的有关发出查询请求的文档在 http://docs.amazonwebservices.com/AWSEC2/latest/ UserGuide / using-query-api.html
我认为我需要签名的值...。但是不知道如何获取
我尝试通过 httr 使用某些给定命令，如下所示。我可以调整URL字符串中的大多数参数来表示我和我想做的事情，例如 AWSAccessKeyId ， ImageId ，端点和 Action 等...，但是只是不知道从何处获取签名值。
在给出的某些示例中，它们似乎也未提供秘密访问密钥...
因此，经过尝试的命令如下所示，已更改了一些代表我的值，但得到了以下内容：
require（httr）
GET（ https://ec2.amazonaws.com/
？Action = RunInstances
& ImageId = ami-60a54009
& MaxCount = 3
& ; MinCount = 1
& Placement.AvailabilityZone = us-east-1b
& Monitoring.Enabled = true
& AWSAccessKeyId = 0GS7553JW74RRM612K02EXAMPLE
& Version = 2012-10 -01
& Expires = 2010-10-10T12：00：00Z
& Signature = lBP67vCvGlDMBQ1dofZxg8E8SUEXAMPLE
& SignatureVersion = 2
& SignatureMet hod = HmacSHA256）
我收到响应：
响应[http://aws.amazon.com/ec2/]
状态：200
内容类型：text / html; charset = UTF-8
<！DOCTYPE HTML PUBLIC-// W3C // DTD HTML 4.01 // EN http://www.w3.org/TR/ html4 / strict.dtd>
< html>
< head>
<元http-equiv = Content-Type content = text / html; charset = utf-8>
< link rel = icon type = image / ico href = // d36cz9buwru1tt.cloudfront.net/favicon.ico\">
< link rel =快捷方式图标 type = image / ico href = // d36cz9buwru1tt.cloudfront.net/favicon.ico\">
< meta name = description content = Amazon Elastic Compute Cloud在云中提供可扩展的按需付费计算能力。 />< meta name = keywords content = /> ...
有人对EC2 api及其身份验证过程有任何经验吗，这会很容易吗足以使用R来设置和运行由我选择的AMI（已加载R和其他相关软件包）的linux实例，然后在这些实例中运行一些R命令并将输出返回？ p>
不要认为它与我的sessionInfo确实相关，以防万一，它是：
sessionInfo（）
R版本2.15.1（2012-06-22）
平台：x86_64-apple-darwin9.8.0 / x86_64（64位）
语言环境：
[1] en_GB.UTF-8 / en_GB.UTF-8 / en_GB.UTF-8 / C / en_GB.UTF-8 / en_GB.UTF-8
附加基本软件包：
[1]统计图形grDevices utils数据集方法基础
其他附加软件包：
[1] httr_0.2
通过命名空间（未附加）：
[1]摘要_0.5.2 plyr_1.7.1 RCurl_1.95-1.1 stringr_0.6.1 tools_2.15.1
编辑：
因此，在进一步尝试遵循@hadley建议的文档的过程中，这是我尝试并得到的...关于我在哪里出错的更多有用的提示将不胜感激...：
require（httr）
aws.key<- xxxxxxx
aws.secret<- xxxxxxxxxxxx
动词<- GET
区域<- ec2.amazonaws.com
func<- DescribeImages
ami.number<- ami-xxxxxxxxx
params<-list（paste0（ ImageId.1 =，ami.number），
Version = 2012-10-01，
Expires = 2012-11-20T12％3A00 ％3A00Z）
＃添加用于创建字符串以签名
的方法和关键参数orig.len.params<-length（params）
params .w.method.key<-参数
params.w.method.key [[orig.len.params + 1]]<- SignatureVersion = 2
params.w.method。 key [[orig.len.params + 3]]<- SignatureMethod = HmacSHA1
params.w.method.key [[orig.len.params + 3]] <-paste0（ AWSAccessKeyId =，aws.key）
＃要签名的字符串（s2s）
s2s<-paste（c（paste0（verb， \n ，zone， \n， / \n， AWSAccessKeyId =，aws.key），paste0（ Action =，func），paste（sort（unlist（params.w.method.key） ），collapse =&）），collapse =&）
＃签名（sig）
sig<-hmac_sha1（aws.secret，s2s）
＃为签名的请求URL生成添加签名，方法和密钥参数
params.w.sig.method.key<-参数
params.w.sig.method.key [ [orig.len.params + 1]]<-paste0（ Signature =，sig）
params.w.sig.method.key [[orig.len.params + 2]]<- SignatureVersion = 2
params.w.sig.method.key [[orig.len.params + 3]]<-- SignatureMethod = HmacSHA1
params.w.sig.method.key [ [orig.len.params + 4]]<-paste0（ AWSAccessKeyId =，aws.key）
＃签名的请求（sr）
sr<-paste（c（ paste0（ https：//，zone，paste0（？Action =，func）），paste（unlist（params.w.sig.method.key），collapse =&））），collapse = &）
＃获取签名的r equest
GET（sr）
我得到的答复：
响应[https://ec2.amazonaws.com?Action=DescribeImages&ImageId.1=[ami.number.from.before]&版本= 2012-10-01& Expires = 2012-11-20T12％3A00％3A00Z& Signature = [sig.value.from.before]& SignatureVersion = 2& SignatureMethod = HmacSHA1& AWSAccessKeyId = [aws.key.from。之前] /]
状态：401
内容类型：
<？xml version = 1.0 encoding = UTF-8？>
< Response><错误><错误><代码> AuthFailure< /代码><消息> AWS无法验证提供的访问凭据< /消息< // Error>< / Error>< RequestID> 5e10fb0b-f304-4677-9c64-98b4537c659a< / RequestID< / Response>
这是我尝试逐步转换R代码的算法。根据我的经验，您真的想单独执行每个步骤，以便可以在每个阶段检查结果是否正确。
（ httr）
require（ RCurl）
require（ stringr）
＃0：从envvars获取密钥和机密，并设置请求参数
aws.key<-Sys.getenv（ AWS_KEY）
aws.secret<-Sys.getenv（ AWS_SECRET_KEY）
动词<- GET
区域<- ec2.amazonaws.com
ami.number<- ami-xxxxxxxxx
参数<-列表（
Action = DescribeImages，
ImageId.1 = ami.number，
Version = 2012-10-01，
Expires = 2012-11-20T12： 00：00Z，
SignatureVersion = 2，
SignatureMethod = HmacSHA1，
AWSAccessKeyId = aws.key）
＃1a：对UTF-8查询进行排序由参数名称
参数组成的字符串组成部分<-params [order（names（params））]
＃1b：URL编码参数名称和值
params_e<- （params，curlEscape）
nam es（params_e）<-curlEscape（names（params_e））
params_str<-str_c（names（params_e）， =，unlist（params_e），折叠=&）
params_str <-gsub（％2E，。，gsub（％2D，-，params_str））
＃2：创建用于签名的字符串
string_to_sign< ;-str_c（
toupper（verb）， \n，
tolower（zone）， \n，
/， \n，
params_str
）
＃3：计算符合RFC 2104的HMAC
＃4：将结果值转换为base64。
hmac<-hmac_sha1（aws.secret，string_to_sign）
params $ Signature<-hmac
GET（paste0（ https：//，区域），query = params）
I would like to use the R package httr to access the EC2 services via their API. But I am a little bit unsure how to get started as it does not fall into the usual authentication format of "Oauth2.0" in which you have the usual: key, secret, token and signature system. I think EC2 uses the "signature version 2" method, but I am unclear as to how that works.
Looking at the documentation that EC2 provides with regard to making query requests at http://docs.amazonwebservices.com/AWSEC2/latest/UserGuide/using-query-api.html
I think I need the value for signature....but don't know how to get it
I have tried using some of the given commands using httr as shown below. I can adapt most of the parameters in the URL string to represent me and the things i want to do e.g AWSAccessKeyId, ImageId, endpoint and the Action etc....but just dont know where to go about getting the signature value.
Also in some of the examples given, they dont seem to provide the secret access key either...
So the tried commands are as follows having altered some of the values to represent me but got the following:
require(httr)
GET("https://ec2.amazonaws.com/
?Action=RunInstances
&ImageId=ami-60a54009
&MaxCount=3
&MinCount=1
&Placement.AvailabilityZone=us-east-1b
&Monitoring.Enabled=true
&AWSAccessKeyId=0GS7553JW74RRM612K02EXAMPLE
&Version=2012-10-01
&Expires=2010-10-10T12:00:00Z
&Signature=lBP67vCvGlDMBQ1dofZxg8E8SUEXAMPLE
&SignatureVersion=2
&SignatureMethod=HmacSHA256")
to which i get the response:
Response [http://aws.amazon.com/ec2/]
Status: 200
Content-type: text/html; charset=UTF-8
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN" "http://www.w3.org/TR/html4/strict.dtd">
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<link rel="icon" type="image/ico" href="//d36cz9buwru1tt.cloudfront.net/favicon.ico">
<link rel="shortcut icon" type="image/ico" href="//d36cz9buwru1tt.cloudfront.net/favicon.ico">
<meta name="description" content="Amazon Elastic Compute Cloud delivers scalable, pay-as-you-go compute capacity in the cloud. " /><meta name="keywords" content="" /> ...
Has anybody had any experience with the EC2 api and its authentication procedure and would it be easy enough to use R to be able to setup and run linux instances with AMIs chosen by me (that have R and other relevant packages loaded onto it), then to run a few R commands in those instances, and bring the output back?
Don't think its really related to my sessionInfo but just in case here it is:
sessionInfo()
R version 2.15.1 (2012-06-22)
Platform: x86_64-apple-darwin9.8.0/x86_64 (64-bit)
locale:
[1] en_GB.UTF-8/en_GB.UTF-8/en_GB.UTF-8/C/en_GB.UTF-8/en_GB.UTF-8
attached base packages:
[1] stats graphics grDevices utils datasets methods base
other attached packages:
[1] httr_0.2
loaded via a namespace (and not attached):
[1] digest_0.5.2 plyr_1.7.1 RCurl_1.95-1.1 stringr_0.6.1 tools_2.15.1
EDIT:
So in a further attempt to follow the documentation as suggested by @hadley this is what I tried and got...any more helpful hints on where im going wrong would be greatly appreciated... :
require(httr)
aws.key <- "xxxxxxx"
aws.secret <- "xxxxxxxxxxxx"
verb <- "GET"
zone <- "ec2.amazonaws.com"
func <- "DescribeImages"
ami.number <- "ami-xxxxxxxxx"
params <- list(paste0("ImageId.1=",ami.number),
"Version=2012-10-01",
"Expires=2012-11-20T12%3A00%3A00Z")
# adding in method and key parameters for creation of string to sign
orig.len.params <- length(params)
params.w.method.key <- params
params.w.method.key[[orig.len.params+1]] <- "SignatureVersion=2"
params.w.method.key[[orig.len.params+2]] <- "SignatureMethod=HmacSHA1"
params.w.method.key[[orig.len.params+3]] <- paste0("AWSAccessKeyId=",aws.key)
# String to sign (s2s)
s2s <- paste(c(paste0(verb,"\n",zone,"\n","/\n","AWSAccessKeyId=",aws.key),paste0("Action=",func),paste(sort(unlist(params.w.method.key)),collapse="&")),collapse="&")
# Signature(sig)
sig <- hmac_sha1(aws.secret, s2s)
# adding in signature, method and key parameters for signed request url generation
params.w.sig.method.key <- params
params.w.sig.method.key[[orig.len.params+1]] <- paste0("Signature=",sig)
params.w.sig.method.key[[orig.len.params+2]] <- "SignatureVersion=2"
params.w.sig.method.key[[orig.len.params+3]] <- "SignatureMethod=HmacSHA1"
params.w.sig.method.key[[orig.len.params+4]] <- paste0("AWSAccessKeyId=",aws.key)
# Signed request (sr)
sr <- paste(c(paste0("https://",zone,paste0("?Action=",func)),paste(unlist(params.w.sig.method.key),collapse="&")),collapse="&")
# GET signed request
GET(sr)
to which I get the response:
Response [https://ec2.amazonaws.com?Action=DescribeImages&ImageId.1=[ami.number.from.before]&Version=2012-10-01&Expires=2012-11-20T12%3A00%3A00Z&Signature=[sig.value.from.before]&SignatureVersion=2&SignatureMethod=HmacSHA1&AWSAccessKeyId=[aws.key.from.before]/]
Status: 401
Content-type:
<?xml version="1.0" encoding="UTF-8"?>
<Response><Errors><Error><Code>AuthFailure</Code><Message>AWS was not able to validate the provided access credentials</Message></Error></Errors><RequestID>5e10fb0b-f304-4677-9c64-98b4537c659a</RequestID></Response>
Here's my attempt at a step by step conversion of the algorithm to R code. In my experience you really want to do each step separately so that you can check at each stage that the results are correct.
require("httr")
require("RCurl")
require("stringr")
# 0: get key and secret from envvars, and set up request parameters
aws.key <- Sys.getenv("AWS_KEY")
aws.secret <- Sys.getenv("AWS_SECRET_KEY")
verb <- "GET"
zone <- "ec2.amazonaws.com"
ami.number <- "ami-xxxxxxxxx"
params <- list(
Action = "DescribeImages",
ImageId.1 = ami.number,
Version = "2012-10-01",
Expires = "2012-11-20T12:00:00Z",
SignatureVersion = 2,
SignatureMethod = "HmacSHA1",
AWSAccessKeyId = aws.key)
# 1a: Sort the UTF-8 query string components by parameter name
params <- params[order(names(params))]
# 1b: URL encode the parameter name and values
params_e <- lapply(params, curlEscape)
names(params_e) <- curlEscape(names(params_e))
params_str <- str_c(names(params_e), "=", unlist(params_e), collapse = "&")
params_str <- gsub("%2E",".",gsub("%2D","-",params_str))
# 2: Create the string to sign
string_to_sign <- str_c(
toupper(verb), "\n",
tolower(zone), "\n",
"/", "\n",
params_str
)
# 3: Calculate an RFC 2104-compliant HMAC
# 4: Convert the resulting value to base64.
hmac <- hmac_sha1(aws.secret, string_to_sign)
params$Signature <- hmac
GET(paste0("https://",zone),query=params)
这篇关于R + httr和EC2 API身份验证问题的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
