如何在不同机器上计算python程序的性能 [英] How to compute the performance of a python program on different machines

问题描述

我想知道可以用来在2个不同系统上查找python代码性能的不同性能特征是什么.还可以在其他机器上扩展其性能吗?这种东西有可能吗?

I would like to know what are the different performance characteristics that can be used to find the performance of a python code on 2 different systems. Also is it possible to extend about its performance on a different machine? Is this kind of stuff possible?

让我们假设两个系统之一是在GPU上进行计算，另一个系统是在CPU上进行计算我想在启用CPU的其他系统上扩展python代码的性能.

Lets assume that one of the two systems is computation on GPU and other is on a CPU I want to extend the python code's performance on a CPU enabled different system.

这也可以通过分析得出吗?

Can this be also derived analytically?

推荐答案

以我的经验为基础，基于动手性能分析做出的假设足以识别初始实例的大小/需求，然后使用实时遥测和仪器进行紧密监控这些解决方案.

In my experiences making assumptions based on hands on performance analysis has been sufficient for identifying initial instance sizes/requirements, and then using real time telemetry and instrumentation to closely monitor those solutions.

我已经使用了几种方式来降低性能(我所用的术语是胡言乱语的):

There are a couple ways, I've used, to commute performance (the terms are gibberish i've made up):

• 瓶颈的非正式表征这涉及非正式地了解您的应用程序的瓶颈可能在哪里，从而对容量/机器要求有一个非常粗略的了解.如果您要在几乎没有网络的情况下执行CPU限制的计算，则可以绕开从网络优化的实例开始.另外，如果您要对文件系统进行处理，并且内存开销很小或有限，那么就不需要高内存实例.

• Informal Characterization of Bottlenecks This has involved informally understanding where the bottlenecks of your application are likely to be, to give a very rough idea of capacity/machine requirements. If you're performing CPU bound calculations with little to no network, then could bypassing starting with a network optimized instance. Also if you're materializing processing to filesystem, and memory overhead is pretty small or bounded then you don't need a high memory instance.

外部效果实验这涉及创建性能测试工具以建立基准线实验，从而允许您更改计算机变量，以确定它们对程序性能产生何种影响.我喜欢通过吞吐量测试来设置基于队列的系统，例如@ 10k请求/秒什么是队列饱和度，什么是服务时间.它涉及将日志记录/遥测添加到代码中以记录这些数字.还要设置一个积压，以了解单个实例处理积压的速度.

External Performance Experiments This involves creating performance test harnesses to establish base line experiments allowing you to change computer variables to determine what sort of effect they have on your program performance. I like to setup queue based systems with throughput tests ie @10k requests / second what is the queue saturation, what is the service time. It involves adding logging/telemetry to code to log those numbers. Also setup a backlog to understand how fast a single instance can process a backlog.

对于HTTP，有许多工具可以生成负载.

For HTTP there are many tools to generate load.

希望有一个自动化的工具来支持您的输入格式，但如果没有，您可能必须编写自己的输入格式.

Hopefully there is an automated tool to support your input format but if not you may have to write your own.

• 性能分析

我认为这是使用低级"工具科学地(与非正式分析相反)来确定您的代码在哪里花费时间的.通常涉及使用 Python探查器来确定您要花时间在哪些例程上，然后尝试优化它们. http://www.brendangregg.com/linuxperf.html

I consider this using "low level" tools to scientifically (opposed to the informal analysis) determine where your code is spending its time. Usually involves using python profiler to determine which routines you're spending time in, and then try to optimize them. http://www.brendangregg.com/linuxperf.html

对于此步骤，如果性能测试工具具有可接受的性能，则可以忽略:p

For this step if the performance test harness has acceptable performance then this can be ignored :p

• 实时遥测确定可接受的性能和实例大小后，实时遥测对于了解程序如何实时执行到现实的工作负载至关重要.
  我发现吞吐量，处理计数，错误等对于维护高性能系统都是至关重要的: http://www.brendangregg.com/usemethod.html

• Real time telemetry After acceptable performance and instance size has been determined, real time telemetry is critical to see how program is perform in real-time-ish to real life workloads.
  I've found Throughput, processing counts, errors, etc to all be critical to maintaining high performance systems: http://www.brendangregg.com/usemethod.html

这篇关于如何在不同机器上计算python程序的性能的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！