加入源系统上的数据与加入SSIS上的数据 [英] Joining the data on source system vs Join on the SSIS

问题描述

我用过必须从MySQL和Extract&读取数据的情况。使用SSIS进行转换并存储到SQL Server中。
$


我正在通过加入Fact表从多个维度表中读取数据。源数据在MySQL上。一个事实和一个维度表。



现在我担心的是，在源上加入来自所有（fct&维度）的数据将是更好的选择或使用多个读取器和放大器读取数据;使用Join / Merge组件加入SSIS会是明智之举吗？。$


我希望每15分钟在Fact Table上有3000行，在维度上有4000行表。该ETL每5分钟持续运行。

Hi,

I've use case where have to read the data from MySQL and Extract & Transformation using the SSIS and store into the SQL Server.

I'm reading data from multiple dimension tables with Joining to the Fact table. Source data is on MySQL. One fact and one dimension table.

Now my worry is Joining data from all (fct & dimension) on source would be better option or read data using multiple reader & Join in SSIS using Join/Merge component would be wise idea ?.

I'm expecting the 3000 rows on the Fact Table every 15 minutes and 4000 rows on the dimension table. This ETL is continuouslygoing to run every 5 minutes.

推荐答案

我并不完全熟悉MySQL性能调优，但是根据我在SQL Server中的了解，在源（DB）级别执行连接（以及任何其他转换，最好）总是更好，
之前将数据输出到SSIS。否则，你只是将你的工作卸载给一个"中间人"。对大型数据集上的此类操作进行了较少优化。

I'm not entirely familiar with MySQL performance tuning, but from what I know in SQL Server, it would always be better to perform Joins (and any other Transformations, as much as possible) at the source (DB) level, before the data is outputted to SSIS. Otherwise, you're simply offloading your work to a "middle-man" that's less optimized for such operations on large data sets.

这篇关于加入源系统上的数据与加入SSIS上的数据的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！