Google App Engine的速度有多快？ [英] How fast is Google App Engine?
问题描述
我正在试用最近2个月的GAE。
我通过上传CSV文件将记录保存到bigtable。
我的测试文件大小为300 KB。
这里是我发现的
本地系统
在Google Sandbox上
-
上传需要5-7秒钟。
/ p> -
它只保存60-180条记录。
我的问题是
li>为什么需要太多时间？ - 有没有办法缩短这个时间？
- Google将这个处理计算在CPU使用中。他们不公开h / w，因此他们在内部使用什么样的CPU？我的意思是我得到一个CPU是否等于或大于PIII？
$ p
$ b编辑为 @Drew Sears 的答案。
目前我在做什么
- 将文件上传至GAE
- 获取上传的数据字节。
- 有一个唯一的字段id，我的记录。
- 现在，我创建了队列
int x = linesCount / 50;
{
x = i * 50;
队列队列= QueueFactory.getQueue（test-queue）;
queue.add（TaskOptions.Builder.url（/ TestQueue）
.param（id，id.toString（））
.param（startIdx，String.valueOf （x））
.param（totRec，String.valueOf（50））
）;
}
int y = linesCount％50;
if（y> 0）
{
x =（linesCount / 50）* 50;
队列队列= QueueFactory.getQueue（test-queue）;
queue.add（TaskOptions.Builder.url（/ TestQueue）
.param（id，id.toString（））
.param（startIdx，String.valueOf （x））
.param（totRec，String.valueOf（y））
）;
$ b处理servlet的任务从存储中读取文件并使用totRec和startIdx处理文件并关闭它。
解决方案这实在不是测试App Engine可伸缩性的好方法。 b
- 如果需要7秒后才能发布300KB，那么瓶颈几乎就是您的上游带宽，而不是谷歌的下游带宽，或者与App Engine有关。我经常获得更快的上传速度。
- 如果您希望请求更快完成，请尽量减少RPC调用。每个数据存储区的获取，放入或查询都是到外部服务器的往返。如果循环数百行并在每次循环迭代中进行put操作，则会产生大量不必要的开销。使用一个数据存储放置所有实体，您将获得更快的结果。 Guido的 AppStats框架是寻找RPC优化机会的绝佳工具。
I am experimenting with GAE for last 2 Months.
I am saving records to the bigtable by uploading CSV file.
My Test File's size is 300 KB.
Here what i found
Local system
- Upload take less than 1 second
- Process 2500 records in 3 seconds
On Google Sandbox
Upload takes 5-7 seconds.
Processing file gives timeout.
It only save 60-180 records.
My questions are
- Why it takes too much time?
- Is there a way to reduce this time?
- Google counts this processing towards CPU uses. They do not disclose h/w so what CPU internally they use? I mean do i get a CPU euquivalent or greater than PIII?
Edited for @Drew Sears's answer.
What i am doing at present
- Upload the file to GAE
- Get uploaded data bytes. By stream, count lines , save it into bigtable.
- There is a unique field, id, my Record.
- Now, i create queue
int x = linesCount/ 50;
for(int i<0;i=x;i++)
{
x = i * 50;
Queue queue = QueueFactory.getQueue("test-queue");
queue.add(TaskOptions.Builder.url("/TestQueue")
.param("id", id.toString())
.param("startIdx",String.valueOf(x))
.param("totRec",String.valueOf(50))
);
}
int y = linesCount % 50;
if( y > 0 )
{
x = (linesCount / 50) * 50;
Queue queue = QueueFactory.getQueue("test-queue");
queue.add(TaskOptions.Builder.url("/TestQueue")
.param("id", id.toString())
.param("startIdx",String.valueOf(x))
.param("totRec",String.valueOf(y))
);
}
The task processing servlet read file from storage and using totRec and startIdx process the file and close it..
This is really not a great way to test App Engine's scalability.
- If it's taking you 7 seconds to post 300KB, the bottleneck is almost certainly your upstream bandwidth, not Google's downstream bandwidth, or anything to do with App Engine. I routinely get much faster upload speeds.
- If you want requests to finish faster, minimize your RPC calls. Every datastore get, put, or query is a round-trip to an external server. If you're looping over hundreds of rows and doing a put inside each loop iteration, you're incurring a massive amount of unnecessary overhead. Save all of your entities using one datastore put and you will get much faster results. Guido's AppStats framework is a great tool for finding RPC optimization opportunities.
这篇关于Google App Engine的速度有多快？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
