PCIe 总线上的写入是原子的吗? [英] Are writes on the PCIe bus atomic?

问题描述

我是 PCIe 的新手，所以这可能是一个愚蠢的问题.这似乎是询问有关 PCIe 接口的相当基本的信息，但我很难找到答案，所以我猜我遗漏了一些使答案显而易见的信息.

I am a newbie to PCIe, so this might be a dumb question. This seems like fairly basic information to ask about PCIe interfaces, but I am having trouble finding the answer so I am guessing that I am missing some information which makes the answer obvious.

我有一个系统，其中有一个 ARM 处理器(主机)通过 PCIe(设备)与 Xilinx SoC 通信.SoC 内的端点也是 ARM 处理器.

I have a system in which I have an ARM processor (host) communicating to a Xilinx SoC via PCIe (device). The endpoint within the SoC is an ARM processor as well.

外部 ARM 处理器(主机)将通过 PCIe 写入 SoC 的 ARM 处理器(设备)的寄存器空间.这将命令 SoC 执行各种操作.该寄存器空间对于 SoC(设备)将是只读的.外部 ARM 处理器(主机)将写入该寄存器空间，然后向 SoC 发出中断信号，以指示 SoC 已写入新参数并应对其进行处理.

The external ARM processor (host) is going to be writing to the register space of the SoC's ARM processor (device) via PCIe. This will command the SoC to do various things. That register space will be read-only with respect to the SoC (device). The external ARM processor (host) will make a write to this register space, and then signal an interrupt to indicate to the SoC that new parameters have been written and it should process them.

我的问题是:外部 ARM(主机)的写入是否保证相对于 SoC(设备)的读取是原子的?在传统的共享内存情况下，对单个字节的写入保证是原子操作(即，您永远不会处于读取器已读取字节的前 2 位，但在读取后 6 位之前writer 用新值替换它们，导致垃圾数据).PCIe 也是这种情况吗?如果是这样，原子性的单位"是什么?单个事务中的所有字节是否相对于整个事务都是原子的，还是每个字节仅与自身相关?

My question is: are the writes made by the external ARM (host) guaranteed to be atomic with respect to the reads by the SoC (device)? In conventional shared memory situations, a write to a single byte is guaranteed to be an atomic operation (i.e. you can never be in a situation where the reader had read the first 2 bits of the byte, but before it reads the last 6 bits the writer replace them with a new value, leading to garbage data). Is this the case in PCIe as well? And if so, what is the "unit" of atomic-ness? Are all bytes in a single transaction atomic with respect to the entire transaction, or is each byte atomic only in relation to itself?

这个问题有意义吗?

基本上我想知道在我的情况下需要多大程度的内存保护.如果可能的话，我想避免锁定内存区域，因为两个处理器都运行 RTOS，避免内存锁定会使设计更简单.

Basically I want to know to what extent memory protection is necessary in my situation. If at all possible, I would like to avoid locking memory regions as both processors are running RTOSes and avoiding memory locks would make design simpler.

推荐答案

所以关于原子性的问题 PCIe 3.0 规范(我只有一个)被提到了几次.

So on the question of atomicity the PCIe 3.0 specification (only one I have) is mentioned a few times.

首先，您有第 6.5 节锁定交易，这可能不是您需要的，但无论如何我想记录下来.基本上，这是您之前描述的最糟糕的情况.

First you have SECTION 6.5 Locked Transactions this is likely not what you need but I want to document it anyway. Basically it's the worst case scenario of what you were describing earlier.

需要锁定事务支持以防止使用遗留软件的系统中出现死锁这会导致对 I/O 设备的访问

Locked Transaction support is required to prevent deadlock in systems that use legacy software which causes the accesses to I/O devices

但是您需要按照说明正确检查使用它.

But you need to properly check using this anyway as it notes.

如果与锁定序列关联的任何读取未成功完成，则请求者必须假设不再保证锁的原子性，并且锁之间的路径请求者和完成者不再被锁定

If any read associated with a locked sequence is completed unsuccessfully, the Requester must assume that the atomicity of the lock is no longer assured, and that the path between the Requester and Completer is no longer locked

因此，第 6.15 节原子操作 (AtomicOps) 更像您感兴趣的内容.您可以使用 AtomicOps 指令执行 3 种类型的操作.

With that said Section 6.15 Atomic Operations (AtomicOps) is much more like what you are interested in. There are 3 types of operations you can perform with the AtomicOps instruction.

FetchAdd(获取和添加):请求包含单个操作数，即添加"值

FetchAdd (Fetch and Add): Request contains a single operand, the "add" value

Swap(无条件交换):请求包含单个操作数，即交换"值

Swap (Unconditional Swap): Request contains a single operand, the "swap" value

CAS(比较和交换):请求包含两个操作数，一个比较"值和一个交换"值

CAS (Compare and Swap): Request contains two operands, a "compare" value and a "swap" value

阅读第 6.15.1 节，我们看到提到这些指令主要用于单个总线上存在多个生产者/消费者的情况.

Reading Section 6.15.1 we see mention that these instructions are largely implemented for cases where multiple producers/consumers exist on a singular bus.

AtomicOps 启用高级同步机制，当有以下情况时特别有用需要以非阻塞方式同步的多个生产者和/或多个消费者.例如，多个生产者可以安全地加入一个公共队列，而无需任何显式锁定.

AtomicOps enable advanced synchronization mechanisms that are particularly useful when there are multiple producers and/or multiple consumers that need to be synchronized in a non-blocking fashion. For example, multiple producers can safely enqueue to a common queue without any explicit locking.

搜索规范的其余部分，我发现在与这些 AtomicOps 相关的部分之外几乎没有提到原子性.这对我来说意味着规范仅在使用这些操作时确保这种行为，但是围绕为什么实施的上下文表明他们只在存在多生产者/消费者环境时才期望这些问题，而您的环境显然不存在.

Searching the rest of the specification I find little mention of atomicity outside of the sections pertaining to these AtomicOps. That would imply to me that the spec only insures such behavior when these operations are used however the context around why this was implemented suggests that they only expect such questions when a multi producer/consumer environment exists which yours clearly does not.

我建议寻找回答您问题的最后一个地方是第 2.4 节交易排序 要注意的是，我相当确定交易传递"其他人的想法只有在中间的开关才有意义，因为这些交换机可以做出这样的决定，一旦你把位放在你的情况下的总线上，就没有回头路了.因此，这可能仅适用于您在其中放置开关的情况.

The last place I would suggest looking to answer your question is Section 2.4 Transaction Ordering To note I am fairly sure the idea of transactions "passing" others only makes sense with switches in the middle as these switches can make such decisions, once your put bits on the bus in your case there is no going back. So this likely only applies if you place a switch in there.

您关心的是写入是否可以绕过读取.写是贴，读是不贴.

Your concern is can a write bypass a read. Write being posted, read being non-posted.

A3, A4 A Posted Request must be able to pass Non-Posted Requests to avoid deadlocks.

所以通常允许写绕过读以避免死锁.

So in general the write is allowed to bypass the read to avoid deadlocks.

提出这个问题后，我认为写入不可能绕过系统上的读取，因为总线上没有设备可以执行此事务重新排序.由于您有 RTOS，我非常怀疑他们是否在发送之前对 PCIe 事务进行了查询和重新排序，尽管我没有亲自研究过.

With that concern raised I do not believe it is possible for the write to bypass the read on your system since there is no device on the bus to do this transaction reordering. Since you have RTOSes I highly doubt they are enquing the PCIe transactions and reordering them before sending although I have not looked into that personally.

这篇关于PCIe 总线上的写入是原子的吗?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！