将大数据从服务器发送到另一个服务器 [英] Sending large data from a server to another

问题描述

我使用CURL在服务器之间发送大量数据，我使用POST，这是OK还是有任何更好/标准的方式发送大的序列化数据与curl？

问题是在max-post-size中的php设置，我必须更改它（默认2MB）。我没有遇到任何问题，但是，当系统将在线时，可能的数据大于50MB将每次发送！

任何想法？
谢谢。

编辑：

而不是FILES，则一旦接收到的数据应该由第二个服务器处理并保存到数据库/文件/执行某些操作，并且可能需要在处理数据后发送响应。

我只是想知道，我会面对任何其他问题，除了max-post-size？ （忘记curl和php的超时），有没有使服务器不看max_post_size？也许通过使用PUSH？或PUT？这post_size影响PUSH或PUT？和如何通过curl使用它？
这么多问题！

解决方案

使用cURL是完全正常的。

就个人而言，我宁愿不必通过web服务器（例如Apache）来做，因为一路上可能有太多的潜在故障，例如。 PHP超时，Web服务器超时，内存限制，无写入权限，仅限于web根目录等。

我希望通过为文件传输设计的机制： / p>

• FTP


• scp


• Amazon S3（简单API， PHP库）


• 等。

I am using CURL to send large amounts of data between servers , am using POST , is this OK or is there any better/standard way to send large serialized data with curl ?

the problem is in the max-post-size in the php settings , i have to change it (default 2MB) . i didn't encounter any problems with this yet , but when the system will be online it is possible that data larger than 50MB will be sent each time !

Any ideas ? Thank you .

EDIT :

I am sending DATA and not FILES , a data that once received should be processed by the second server and saved to database/file/do some action and might need to send a response after processing the data .

I just would like to know , will i face any other problem except max-post-size ? (forget about timeouts of both curl and php) , and is there anyway to make the server not look at max_post_size ? maybe by using PUSH ? or PUT ? does that post_size affect the PUSH or PUT ?? and how to use it via curl ? so many questions !

解决方案

Using cURL is perfectly fine.

Personally, I would prefer to not having to do it through web server (eg. Apache) as there can be too many potential faults along the way, eg. PHP timeout, web server timeout, memory limit, no write privileges, limited to web root, etc.

I would prefer to do it through mechanisms designed for file transfers:

• FTP
• scp (generally FTP over SSH)
• Dropbox (there are APIs)
• Amazon S3 (simple API with PHP library)
• etc.

这篇关于将大数据从服务器发送到另一个服务器的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！