FTP是可靠的用于自动数据交换应用中使用？ [英] Is FTP reliable for use with an automated data-exchange application?
问题描述
我被要求来设计从外部供应商批处理应用程序，将检索数据（具体地，详细的交易数据的列表）在周期性基础上。我们已同意使用XML的数据交换，但正在调查不同的方法/协议，以便实际的数据传输。厂商建议的电子邮件或FTP作为传输数据的装置，但我们拒绝第一个选项出右由于物流和可靠性问题。
至于第二个，FTP，我一直犹豫不决，使用FTP在生产环境中的可靠性是一个问题。一个设计其中一个供应商发布文件到FTP进行周期性的拉下似乎不可靠，而且容易出错。我最初的反应是将倾向于对像web服务（此特定供应商可能会或可能甚至不能或不愿提供），其中数据可被查询，根据需要，对于特定的时间段。
在一般情况下，是什么情况下使用，例如这一点的最好办法？是FTP（或SFTP）通常被认为是一个可以接受的选择，或者是有什么好？是一个Web服务矫枉过正的数据，这样一个简单的交流？是否有我完全可以俯瞰？
其他可行方案文件传输presents了一些并发症的发生。
我会preFER Web服务，或只是HTTPS与消化/基本身份验证的文件访问，但对于非常大的文件，这可能不切实际他们。
另一个答案可能是使用共享水桶在Amazon S3上，在这里您已经阅读访问，他们有写入权限。我已经使用了几次，作为一个可怜的人的安全的文件传输。
我已经以这种方式使用FTP的口味，这里有一些提示，如果你做的：
1）使用安全的版本一样SFTP - FTP是不固定的凭据或数据
。2）使用信号文件以指示最新的文件是完整和可用，或确保当他们编写的文件到FTP目录，他们将它整体，所以你不要访问不完整的文件。
3）确保每个文件都有一个唯一的文件名（时间戳，序列号等），所以你可以跟踪哪些你处理，你没有。不要重复使用的文件名，因为你不知道什么时候你已经处理，可以得到该文件的竞争状态更新为你访问它。
4）使用散列值来检查成功转让。他们可以提供一个MD5哈希的文件，然后你可以对你的版本检查这个曾经完成复制它。我经常使用的MD5文件作为一个信号量为好，都表明一个文件可用，并提供一种手段来检查转移是完整和正确的。
I've been asked to design a batch application that would retrieve data (specifically, a detailed list of transactions) from an external vendor on a periodic basis. We have agreed to use XML for the data exchange, but are investigating different methods/protocols to facilitate the actual data transfer. The vendor suggested email or FTP as a means to transfer the data, but we rejected the first option out-right due to logistics and reliability concerns.
As for the second, FTP, I have always been hesitant to utilize FTP in a production environment where reliability is a concern. A design whereby a vendor publishes files to an FTP to be periodically pulled down seems unreliable and error prone. My initial reaction would be to gravitate towards something like a web service (which this particular vendor may or may not even be able or willing to provide), where the data could be queried, as needed, for a specific time period.
In general, what is the best approach to use in a situation such as this? Is FTP (or SFTP) generally considered to be an acceptable option, or is there something better? Is a web-service overkill for such a simple exchange of data? Are there other viable options that I am completely overlooking?
File transfer presents a number of complications.
I would prefer a webservice, or just https access to the file with digest/basic auth, but for very large files, that may not be practical for them.
Another answer could be to use a shared bucket on amazon s3, where you have read access, and they have write access. I have used that a couple of times as a poor man's secure file transfer.
I have used flavors of FTP in this way, here are some tips if you do:
1) Use a secure version like sftp - ftp is just not secure for the credentials or data.
2) Use a semaphore file to indicate when the latest file is complete and available, or make sure that when they write the file to the FTP dir, they move it in whole, so you do not access incomplete files.
3) Make sure each file has a unique file name (timestamp, sequence number, etc.) so you can keep track of which you have processed and which you haven't. Do not reuse the file name, as you do not know when you have processed already, and could get a race condition of the file is updated as you are accessing it.
4) Use a hash value to check for successful transfer. They could provide an md5 hash for the file, and then you could check this against your version once you have completed copying it. I have often used the md5 file as a semaphore as well, to both indicate a file is available, and provide a means to check the transfer was complete and correct.
这篇关于FTP是可靠的用于自动数据交换应用中使用？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
