(ADF v2)Oracle DB源/Azure Blob接收器吞吐量非常慢 [英] (ADF v2) Oracle DB Source/Azure Blob Sink Throughput very slow
问题描述
我正在从Oracle数据库视图读取数据并写入Azure Blob接收器.
尽管尝试使用不同的接收器方案场景并评估了网络连接性，但吞吐量却异常糟糕.
IR客户端(Azure VM)与源(Azure VM)以及正在写入的Azure存储网络位于相同的区域和VNet中. IR客户端几乎没有被使用，并且IR进程的CPU使用率通常不会超过10％.总内存为 大约25％.
有任何想法要解决此问题吗? Blob格式是JSON还是Text是否重要?在其他情况下，我们看不到与数据库交互的问题.
事件日志条目:
报告作业进度:[MaxMemoryLimit，1073741824]; [Source.WorkingDuration，118.8008801]; [Source.WorkerCount，1]; [Source.PeakWorkerCount，1]; [Source.StartTime，04/21/2018 14:10 :41.939]; [Serialization.WorkingDuration，0.0428152]; [Serialization.WorkerCount， 0]; [Serialization.PeakWorkerCount，1]; [Compression.WorkingDuration，0]; [Compression.WorkerCount，0]; [Compression.PeakWorkerCount，0]; [Sink.WorkingDuration，0]; [Sink.WorkerCount，0] ; [Sink.PeakWorkerCount，0]; [AvailableMemoryLimit，1065353215]; [DataSizeOutbound， 0]; [SinkFileCount，0]; [Source.ReadFirstByteTime，04/21/2018 14:11:08.278]; [DataSizeInbound，31392]; [SourceInboundRowCount，40]; [RowCount，39]; [RuntimeStageNames，["源"，序列化"，压缩"，接收器"]]; [Source.Type， Oracle]; [Serialization.Type，JsonFormat]; [Compression.Type，GZip]; [Sink.Type，AzureBlob]; [TaskPickupTime，04/21/2018 14:10:38.714]; [PreProcess.Duration，0.07]; [Memory.TotalMemoryInMB，8191.00]; [GC.IsServerGC，False]; [Memory.ProcessPrivateWorkingSetInMB，62.66]; [Memory.ProcessWorkingSetInMB， 129.99]； [Memory.ProcessWorkingSetPeakInMB，134.45]； [Memory.ProcessVirtualBytesInMB，1019.20]； [Memory.ProcessVirtualBytesPeakInMB，1028.07]； [Memory.ProcessPageFileBytesInMB，233.05]； [Memory.ProcessPageFileBytesPeakInMB，235.67]； [Memory.ProcessPrivate 233.30]; [Processor.ProcessCpuUtilizationRatio，0.10]; [Memory.AvailableMemoryInMB，6321.00]; [Memory.CommitLimitInMB，9471.55]; [Memory.CommittedBytesInMB，2165.11]; [Memory.CommittedBytesInUseRatio，22.86]; [Network.NetworkRecieved，1. ; [Network.NetworkSent， 1.81]; [Processor.CpuUtilizationRatio，0.14]; [GC.GCRatio，4.06]; [GC.ProcessGCRatio，0.00]; [LogicalDisk.CFreeSpaceInMB，101333.00]; [LogicalDisk.TotalFreeSpaceInMB，151150.00]; [GatewayNodeName，udfg]
IR Client版本为3.5.6639.1，并说明它是最新版本.
嗨big_z，
您是否已通过DBA检查了Oracle DB的工作量?
谢谢
David
I'm working on reading from an Oracle DB view and writing to an Azure Blob sink.
Though attempting different scenarios for the sink option and evaluating network connectivity, I am getting horrible throughput.
The IR client (Azure VM) is in the same region and VNet as the source (Azure VM) as well as the Azure Storage network that is being written to. The IR Client is barely utilized and the IR processes are not often above 10% CPU utilization. Total memory is around 25%.
Any thoughts to troubleshoot this? Does it matter significantly whether the Blob format is JSON vs Text? We aren't seeing issues interacting with the DB in other scenarios.
Event Log Entry:
Report job progress: [MaxMemoryLimit, 1073741824];[Source.WorkingDuration, 118.8008801];[Source.WorkerCount, 1];[Source.PeakWorkerCount, 1];[Source.StartTime, 04/21/2018 14:10:41.939];[Serialization.WorkingDuration, 0.0428152];[Serialization.WorkerCount, 0];[Serialization.PeakWorkerCount, 1];[Compression.WorkingDuration, 0];[Compression.WorkerCount, 0];[Compression.PeakWorkerCount, 0];[Sink.WorkingDuration, 0];[Sink.WorkerCount, 0];[Sink.PeakWorkerCount, 0];[AvailableMemoryLimit, 1065353215];[DataSizeOutbound, 0];[SinkFileCount, 0];[Source.ReadFirstByteTime, 04/21/2018 14:11:08.278];[DataSizeInbound, 31392];[SourceInboundRowCount, 40];[RowCount, 39];[RuntimeStageNames, ["Source","Serialization","Compression","Sink"]];[Source.Type, Oracle];[Serialization.Type, JsonFormat];[Compression.Type, GZip];[Sink.Type, AzureBlob];[TaskPickupTime, 04/21/2018 14:10:38.714];[PreProcess.Duration, 0.07];[Memory.TotalMemoryInMB, 8191.00];[GC.IsServerGC, False];[Memory.ProcessPrivateWorkingSetInMB, 62.66];[Memory.ProcessWorkingSetInMB, 129.99];[Memory.ProcessWorkingSetPeakInMB, 134.45];[Memory.ProcessVirtualBytesInMB, 1019.20];[Memory.ProcessVirtualBytesPeakInMB, 1028.07];[Memory.ProcessPageFileBytesInMB, 233.05];[Memory.ProcessPageFileBytesPeakInMB, 235.67];[Memory.ProcessPrivateBytesInMB, 233.30];[Processor.ProcessCpuUtilizationRatio, 0.10];[Memory.AvailableMemoryInMB, 6321.00];[Memory.CommitLimitInMB, 9471.55];[Memory.CommittedBytesInMB, 2165.11];[Memory.CommittedBytesInUseRatio, 22.86];[Network.NetworkRecieved, 1.91];[Network.NetworkSent, 1.81];[Processor.CpuUtilizationRatio, 0.14];[GC.GCRatio, 4.06];[GC.ProcessGCRatio, 0.00];[LogicalDisk.CFreeSpaceInMB, 101333.00];[LogicalDisk.TotalFreeSpaceInMB, 151150.00];[GatewayNodeName, udfg]
IR Client version is 3.5.6639.1 and stating it is current.
Hi big_z,
Have you checked the workload of your Oracle DB with the DBA?
Thanks,
David
这篇关于(ADF v2)Oracle DB源/Azure Blob接收器吞吐量非常慢的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
