建议在Azure上运行数据处理工作流的体系结构 [英] Suggested architecture to run data processing workflows on Azure
问题描述
我想要一个通用的基础架构来扩展云上的数据处理工作流
我想知道是否有满足以下要求的现有服务/基础设施，或者您认为构建一个服务/基础设施需要什么.
最重要的要求是:
-可以从现有处理节点快速组装工作流程
-可以使用新逻辑快速创建/部署节点
-可以快速将工作流包装到服务中(以创建特定于客户的解决方案)
-可以直观地设计工作流程图"，并在本地对其进行调试(类似于"Windows工作流基础所做的工作")
-开发时间仅用于开发节点逻辑，每个节点都是他自己的可调试实体
-工作流可视化组装和部署是基础架构的通用功能.
-可以监视门户中的工作流程及其阶段
听起来像是"Windows工作流基础"从旧的角度来看在云上运行，而从新的角度讲带有微服务开发的服务结构，但是我找不到关于云中的工作流管理的最新信息，其中包括 以上所有要点.
您是否知道满足这些要求的任何Azure服务/基础结构?
您建议采用哪种Azure架构来实现这样的框架?
谢谢
伊曼纽尔
Emanuel Shalev
根据您的描述，我认为Service Fabric将是一个不错的选择. /p>
我在Service Fabric中没有看到的一件事是像Windows WorkFlow Foundation这样的流.当然，您可以使用多种方法从Visual Studio，模板和VSTS部署Service Fabric.因此，您可以实施某种工作流基础，但 它不是SF本身特有的.相反，您可以插入混合路线.
此外，根据您的用户名标签，我看到您是MSFT员工.如果您愿意，可以离线与我联系，我可以为您提供一些其他资源，以帮助您进行研究.
I would like to have a generic infrastructure to scale data processing workflows on the cloud
I am interested in knowing if there is an existing service/infrastructure that answer to the following requirements, or what in your opinion is needed to build one.
The most important requirements are:
- Can quickly assemble a workflow from existing processing nodes
- Can quickly create/deploy nodes with new logic
- Can quickly wrap the workflow in a service (to create customer specific solutions)
- Can "visually design the workflow graph" and debug it locally (similar to what "windows workflow foundation was doing")
- Development time is spent on developing the nodes logic only, and each node is his own debugable entity
- Workflow visual assembly and deployment are a generic capability of the infrastructure.
- Can monitor the workflow and its stages in a portal
its sounds like a "windows workflow foundation" running on the cloud in old terms, and service fabric with micro services development in newer terms, but I couldn't find anything up to date about workflows management in the cloud that would include all the above points.
Do you know of any Azure service/infrastructure that would fit these requirements?
What Azure architecture would you suggest to implement such a framework?
thanks,
Emanuel
Emanuel Shalev
From you description I would say that Service Fabric would be a good choice to go with.
One thing I have not seen in Service Fabric is a flow like Windows WorkFlow foundation. You can of course use multiple methods of deploying Service Fabric from Visual Studio, templates and VSTS. So you could implement some sort of workflow foundation but it would not be something that is specific to SF itself. Rather another route you could plug into the mix.
In addition, based on your username tag I see you are an MSFT employee. If you like you can reach out to me offline and I can provide you with some additional resources to help you conduct your research.
这篇关于建议在Azure上运行数据处理工作流的体系结构的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
