Python 上下文中的运行时是什么?它由什么组成? [英] What is runtime in context of Python? What does it consist of?

问题描述

在这个问题的上下文中什么是运行时"?(https://stackoverflow.com/questions/3900549/what-is-runtime/3900561)

In context to this question What is "runtime"? (https://stackoverflow.com/questions/3900549/what-is-runtime/3900561)

我试图了解 Python 运行时由什么组成.我的猜测是:

I am trying to understand what would a python runtime be made of. My guess is:

1. 包含所有运行时变量的 Python 进程.
2. GIL
3. 底层解释器代码(CPython 等).

现在，如果这是正确的，我们可以说 python 中的多处理创建了多个运行时并且一个 python 进程是我们可以直接与运行时相关的东西吗?(我认为这是正确的选择)

Now if this is right, can we say that multiprocessing in python creates multiple runtimes and a python process is something we can directly relate to the runtime? (I think this is the right option)

或者，每个 python 线程都有自己的堆栈，与父进程在相同的 GIL 和内存空间上工作，可以被称为具有单独的运行时?

Or, every python thread with its own stack which works on the same GIL and memory space as the parent process can be called as having a separate runtime?

或者，不管有多少线程或进程正在运行，它都会在一个运行时下运行?

Or, doesn't matter how many threads or processes are running, it will all come under a single runtime?

简单的说，Python上下文中runtime的定义是什么?

Simply put, what is the definition of runtime in the context of Python?

PS:我理解线程和进程之间的区别.GIL:我理解这些影响，但我不理解.

PS: I understand the difference between threads and processes. GIL: I understand the impacts but I do not grok it.

推荐答案

您在谈论计算机科学中两个不同(但相似)的概念；多进程和多线程.以下是一些可能有用的问题/答案汇编:

You are talking about two different (yet similar) concepts in computer science; multiprocess, and multithreading. Here is some compilation of questions/answers that might be useful:

• 多处理 -- 维基百科

多处理是在单个计算机系统中使用两个或多个中央处理器 (CPU).该术语还指系统支持多个处理器的能力或在它们之间分配任务的能力.

Multiprocessing is the use of two or more central processing units (CPUs) within a single computer system.The term also refers to the ability of a system to support more than one processor or the ability to allocate tasks between them.

• 多线程 -- 维基百科

在计算机体系结构中，多线程是指中央处理单元 (CPU)(或多核处理器中的单核)在操作系统支持下同时提供多个执行线程的能力.这种方法不同于多处理.在多线程应用中，线程共享单个或多个内核的资源，包括计算单元、CPU 缓存和转换后备缓冲区 (TLB).

In computer architecture, multithreading is the ability of a central processing unit (CPU) (or a single core in a multi-core processor) to provide multiple threads of execution concurrently, supported by the operating system. This approach differs from multiprocessing. In a multithreaded application, the threads share the resources of a single or multiple cores, which include the computing units, the CPU caches, and the translation lookaside buffer (TLB).

• 进程和进程有什么区别线?-- StackOverflow

每个进程提供执行程序所需的资源.进程具有虚拟地址空间、可执行代码、系统对象的开放句柄、安全上下文、唯一进程标识符、环境变量、优先级类、最小和最大工作集大小以及至少一个执行线程.每个进程都以单个线程启动，通常称为主线程，但可以从其任何线程创建其他线程.

Process

Each process provides the resources needed to execute a program. A process has a virtual address space, executable code, open handles to system objects, a security context, a unique process identifier, environment variables, a priority class, minimum and maximum working set sizes, and at least one thread of execution. Each process is started with a single thread, often called the primary thread, but can create additional threads from any of its threads.

线程是进程中可以调度执行的实体.进程的所有线程共享其虚拟地址空间和系统资源.此外，每个线程都维护异常处理程序、调度优先级、线程本地存储、唯一的线程标识符以及系统将用来保存线程上下文直到被调度的一组结构.线程上下文包括线程的机器寄存器集、内核堆栈、线程环境块和线程进程地址空间中的用户堆栈.线程也可以有自己的安全上下文，可用于模拟客户端.

A thread is an entity within a process that can be scheduled for execution. All threads of a process share its virtual address space and system resources. In addition, each thread maintains exception handlers, a scheduling priority, thread local storage, a unique thread identifier, and a set of structures the system will use to save the thread context until it is scheduled. The thread context includes the thread's set of machine registers, the kernel stack, a thread environment block, and a user stack in the address space of the thread's process. Threads can also have their own security context, which can be used for impersonating clients.

---

• 运行时环境"和软件框架"的含义?-- StackOverflow

运行时环境基本上是一个运行在机器之上的虚拟机——提供机器抽象.它通常比图书馆低.框架可以包含运行时环境，但通常与库相关联.

A runtime environment basically is a virtual machine that runs on top of a machine - provides machine abstraction. It is generally lower level than a library. A framework can contain a runtime environment, but is generally tied to a library.

• 运行时系统 -- 维基百科

在计算机编程中，运行时系统(也称为运行时环境)主要实现部分执行模型.大多数语言都有某种形式的运行时系统，提供程序运行的环境.这种环境可以解决许多问题，包括应用程序内存的布局、程序如何访问变量、在过程之间传递参数的机制、与操作系统的接口等.通常，运行时系统将负责设置和管理堆栈和堆，并且可能包括诸如垃圾收集、线程或其他内置于语言中的动态功能等功能.

In computer programming, a runtime system, also called runtime environment, primarily implements portions of an execution model. Most languages have some form of runtime system that provides an environment in which programs run. This environment may address a number of issues including the layout of application memory, how the program accesses variables, mechanisms for passing parameters between procedures, interfacing with the operating system, and otherwise. Typically the runtime system will have some responsibility for setting up and managing the stack and heap, and may include features such as garbage collection, threads or other dynamic features built into the language.

• 全局解释器锁 -- Python 文档

CPython 解释器用来确保一次只有一个线程执行 Python 字节码的机制.这通过使对象模型(包括关键的内置类型，如 dict)隐式安全地防止并发访问来简化 CPython 实现.锁定整个解释器使解释器更容易成为多线程，但代价是多处理器机器提供的大部分并行性.

The mechanism used by the CPython interpreter to assure that only one thread executes Python bytecode at a time. This simplifies the CPython implementation by making the object model (including critical built-in types such as dict) implicitly safe against concurrent access. Locking the entire interpreter makes it easier for the interpreter to be multi-threaded, at the expense of much of the parallelism afforded by multi-processor machines.

然而，一些扩展模块，无论是标准的还是第三方的，都被设计为在执行压缩或散列等计算密集型任务时释放 GIL.此外，在进行 I/O 时总是释放 GIL.

However, some extension modules, either standard or third-party, are designed so as to release the GIL when doing computationally-intensive tasks such as compression or hashing. Also, the GIL is always released when doing I/O.

过去创建自由线程"解释器(以更精细的粒度锁定共享数据的解释器)的努力并未成功，因为在常见的单处理器情况下性能受到影响.相信克服这个性能问题会使实现更加复杂，因此维护成本更高.

Past efforts to create a "free-threaded" interpreter (one which locks shared data at a much finer granularity) have not been successful because performance suffered in the common single-processor case. It is believed that overcoming this performance issue would make the implementation much more complicated and therefore costlier to maintain.

• 什么是Python全局解释器锁(吉尔)?--真正的Python

有关 GIL 的更多信息的有用来源.

Useful source for more info on GIL.

• python os.fork 是否使用相同的 python 解释器?-- StackOverflow

每当您 fork 时，整个 Python 进程都会在内存中复制(包括 Python 解释器、您的代码和任何库、当前堆栈等)以创建第二个进程 - 一个原因 fork 进程比创建进程要昂贵得多一个线程.

Whenever you fork, the entire Python process is duplicated in memory (including the Python interpreter, your code and any libraries, current stack etc.) to create a second process - one reason why forking a process is much more expensive than creating a thread.

这会创建一个新的 Python 解释器副本.

This creates a new copy of the python interpreter.

运行两个 Python 解释器的一个优势是您现在拥有两个 GIL(全局解释器锁)，因此可以在多核系统上进行真正的多处理.

One advantage of having two python interpreters running is that you now have two GIL's (Global Interpreter Locks), and therefore can have true multi-processing on a multi-core system.

一个进程中的线程共享相同的 GIL，这意味着在给定时刻只有一个线程运行，只会给人一种并行的错觉.

Threads in one process share the same GIL, meaning only one runs at a given moment, giving only the illusion of parallelism.

• 内存管理 -- Python 文档

Python 中的内存管理涉及一个包含所有 Python 对象和数据结构的私有堆.这个私有堆的管理是由 Python 内存管理器在内部确保的.Python 内存管理器具有处理各种动态存储管理方面的不同组件，例如共享、分段、预分配或缓存.

Memory management in Python involves a private heap containing all Python objects and data structures. The management of this private heap is ensured internally by the Python memory manager. The Python memory manager has different components which deal with various dynamic storage management aspects, like sharing, segmentation, preallocation or caching.

---

当您通过 threading 库生成线程时，您实际上是在单个 Python 运行时内生成作业.该运行时确保线程拥有共享内存，并通过全局解释器锁来管理这些线程的运行顺序:

---

When you spawn a thread via the threading library, you are effectively spawning jobs inside a single Python runtime. This runtime ensures the threads have a shared memory and manages the running sequence of these threads via the global interpreter lock:

• 了解 Python GIL -- dabeaz

当您通过 multiprocessing 库生成进程时，您生成的新进程包含运行指定代码的新 Python 解释器(新运行时).如果你想共享内存，你必须使用 multiprocessing.shared_memory:

When you spawn a process via the multiprocessing library, you are spawning a new process that contains a new Python interpreter (a new runtime) that runs the designated code. If you want to share memory you have to use multiprocessing.shared_memory:

• multiprocessing.shared_memory -- Python 文档
• multiprocessing.shared_memory -- Python Docs

该模块提供了一个类 SharedMemory，用于分配和管理共享内存，供多核或对称多处理器 (SMP) 机器上的一个或多个进程访问.为了协助共享内存的生命周期管理，尤其是跨不同进程的共享内存，在 multiprocessing.managers 模块中还提供了 BaseManager 子类 SharedMemoryManager.

This module provides a class, SharedMemory, for the allocation and management of shared memory to be accessed by one or more processes on a multicore or symmetric multiprocessor (SMP) machine. To assist with the life-cycle management of shared memory especially across distinct processes, a BaseManager subclass, SharedMemoryManager, is also provided in the multiprocessing.managers module.

---

我们可以说python中的多处理创建了多个运行时并且一个python进程是我们可以直接与运行时相关联的东西吗?

是的.不同的 GIL，不同的内存空间，不同的运行时间.

---

Can we say that multiprocessing in python creates multiple runtimes and a python process is something we can directly relate to the runtime?

Yes. Different GIL, different memory space, different runtime.

取决于您所说的堆栈"是什么意思.相同的 GIL，共享内存空间，相同的运行时间.

Depends what you mean by "stack". Same GIL, shared memory space, same runtime.

取决于是否是多线程/多进程.

Depends if multithreading/multiprocess.

运行时环境实际上是python.exe或/usr/bin/python.它是 Python 可执行文件，它将通过将 Python 代码转换为 CPU 可读的字节码来解释它.当您使用多线程时，您只会运行一个 python.当您 multiprocess 时，您有多个 python 正在运行.

The runtime environment is literally python.exe or /usr/bin/python. It's the Python executable that will interpret your Python code by transforming it into CPU-readable bytecode. When you multithread, you only have one python running. When you multiprocess you have multiple pythons running.

我希望核心开发人员可以进来并更详细地对此进行更多讨论.目前，以上只是一个来源汇编，供您开始理解/看到更大的图景.

I hope that a core dev can come in and speak more to this in greater detail. For now the above is simply just a compilation of sources for you to start understanding/seeing the bigger picture.

这篇关于Python 上下文中的运行时是什么?它由什么组成?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！