使用Node.js将数据加载到Redshift中 [英] Load data into Redshift using Node.js
本文介绍了使用Node.js将数据加载到Redshift中的处理方法,对大家解决问题具有一定的参考价值,需要的朋友们下面随着小编来一起学习吧!
问题描述
使用node.js将数据插入Amazon Redshift的几种方法是什么?
What are some ways of inserting data into Amazon Redshift using node.js?
这应该很简单,但是我找不到有效的具体示例
This should be pretty straightforward, but I was unable to find any concrete example for efficient loading.
推荐答案
一种方法是使用AWS node.js SDK (其中有一个示例),然后使用 node-pg 来将数据复制到Redshift中:
One way of doing that would be to load the data into S3 using AWS node.js SDK (there's an example in the documentation), then use node-pg to COPY the data into Redshift :
var pg = require('pg');
var conString = "postgres://user:password@db-endpoint:port/schema";
var client = new pg.Client(conString);
client.connect(function(err) {
if(err) {
return console.error('could not connect to postgres', err);
}
//assuming credentials are exported as enviornment variables,
//both CLI- and S3cmd-style are supported here.
//Also, you may want to specify the file's format (e.g. CSV),
//max errors, etc.
var copyCmd = 'copy my_redshift_table from \'s3://your_bucket/your_file\' credentials \'aws_access_key_id='
+ (process.env.AWS_ACCESS_KEY || process.env.AWS_ACCESS_KEY_ID)
+ ';aws_secret_access_key='
+ (process.env.AWS_SECRET_KEY || process.env.AWS_SECRET_ACCESS_KEY)
+ '\'';
client.query(copyCmd, function(err, result) {
if(err) {
return console.error('error running query', err);
}
logger.info("redhshift load: no errors, seem to be successful!");
client.end();
});
});
请注意,您不需要任何特殊的驱动程序即可运行。
Note that you don't need any special drivers for this to run.
这篇关于使用Node.js将数据加载到Redshift中的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!
查看全文