Spring Cloud Stream应用程序VS Spring Cloud Task应用程序 [英] Spring Cloud Stream App VS Spring Cloud Task App
问题描述
我正在探索SCDF，以为我的系统编排现有的批处理任务.我注意到SC Stream App Starters已经提供了一些OOTB准备使用流媒体应用程序，这适合我的用例:
I'm exploring the the SCDF for orchestrating the existing batch task for my system. I noticed that the SC Stream App Starters already provides some OOTB ready to use streaming applications, which suits my use case of:
- 从FTP提取xml文件-> ftp源
- 自定义转换
- 将转换后的xml文件上传到另一个FTP-> ftp接收器
但是，我的系统仅要求上述批处理作业每隔10分钟执行一次.使用streams应用程序实现它很好，但是它需要stream应用程序运行24-7(将SCDF与Kubernetes结合使用).我担心的是，我有20多个上述批处理任务，每个流应用程序都需要运行"才能每隔10分钟触发一次.
But, my system only requires the above mentioned batch job to be executed at every 10 minutes interval. Implementing it using streams app is fine but it requires the stream app to be running 24-7 (using SCDF with Kubernetes). My concern is that I have 20+ of the above batch tasks, and each of the stream app will needs to be "running" to get triggered at every 10 minutes interval.
在阅读Spring Cloud Task后，我注意到它更适合我的用法，不需要流应用程序持续运行(短命).但是，它没有OOTB准备使用我需要的任务应用程序:ftp-source和ftp-sink.
After reading on the Spring Cloud Task, I noticed that it better suits my usage, which does not require the stream apps to be constantly running (short-lived). But, it does not have the OOTB ready to use task apps that I need: ftp-source and ftp-sink.
此外，使用Spring Cloud Task不允许我将XML文件从上一个任务应用程序传递到下一个任务应用程序.似乎是根据
Besides that, using Spring Cloud Task does not allow me to pass the XML file from previous task app to the next one. It seems like it is a code smell to pass data from task to task according to comment in this question.
然后，为了实现上述用例，task-app-1将下载xml文件并输出到某个存储中，然后task-app-2从存储中拾取该xml文件并执行自定义转换并上传到另一个存储.然后，task-app-3拾取xml文件并上传到ftp.这合适吗?
Then, in order to implement the above use case, task-app-1 will download the xml file and output to some storage, then task-app-2 pick up the xml file from the storage and perform customize transformation and upload to another storage. Then, task-app-3 picks up the xml file and upload to ftp. Will this be appropriate??
除此之外，我还想在上述用例中注册多个流，并通过外部源(SCDF REST API)触发流的部署?这是否会破坏现有任务计划功能的目的?
Besides that, I also have the idea of registering multiple streams with the above use case and trigger the deployment of the stream through external source (SCDF REST API)? Will this defeat the purpose of existing task schedule feature?
有人可以对此提出建议吗?
Anyone can advise on this?
推荐答案
对于您的情况，如您所知，您需要创建一组执行要求的批处理作业.您可以查看此示例作为参考. 同样，您可以潜在地使用Composed Task Runner来驱动此特定情况.有关组合任务的更多信息，您可以参考文档和示例
For your case, as you learnt you need to create a set of batch jobs that perform your requirement. You can check this example as a reference. Also, you can potentially use Composed Task Runner to drive this specific case. For more information on Composed Tasks, you can refer to the documentation and an example here
这篇关于Spring Cloud Stream应用程序VS Spring Cloud Task应用程序的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
