没有事务时 GAE 上的 TransactionFailedError [英] TransactionFailedError on GAE when no transaction
问题描述
我收到此错误:
TransactionFailedError:对这些数据存储实体的争用过多.请再试一次.即使我没有进行任何交易.导致错误的代码行是
ndb.put_multi(entity_list) # entity_list 是 100 个实体的列表这个错误并不经常发生，所以没什么大不了的，但我很好奇为什么会出现这个错误.有什么想法吗?
这里是大部分的回溯:
回溯(最近一次调用最后一次):...文件/base/data/home/runtimes/python27_experiment/python27_lib/versions/1/google/appengine/ext/deferred/deferred.py"，第 318 行，在帖子中self.run_from_request()run_from_request 中的文件/base/data/home/runtimes/python27_experiment/python27_lib/versions/1/google/appengine/ext/deferred/deferred.py"，第 313 行运行(self.request.body)运行中的文件/base/data/home/runtimes/python27_experiment/python27_lib/versions/1/google/appengine/ext/deferred/deferred.py"，第 155 行返回 func(*args, **kwds)文件/base/data/home/apps/s~opavote/2017-09-15.404125237783169549/tasks.py"，第70行，在start_electionmodels.Voter.create(e.eid，块)文件/base/data/home/apps/s~opavote/2017-09-15.404125237783169549/models.py"，第2426行，在创建ndb.put_multi(选民+ vbs)文件/base/data/home/runtimes/python27_experiment/python27_lib/versions/1/google/appengine/ext/ndb/model.py"，第 3958 行，在 put_multi 中未来在 put_multi_async(entities, **ctx_options)]文件/base/data/home/runtimes/python27_experiment/python27_lib/versions/1/google/appengine/ext/ndb/tasklets.py"，第 383 行，在 get_result 中self.check_success()文件/base/data/home/runtimes/python27_experiment/python27_lib/versions/1/google/appengine/ext/ndb/tasklets.py"，第 427 行，在 _help_tasklet_alongvalue = gen.throw(exc.__class__, exc, tb)文件/base/data/home/runtimes/python27_experiment/python27_lib/versions/1/google/appengine/ext/ndb/context.py"，第 824 行，输入key = yield self._put_batcher.add(entity, options)文件/base/data/home/runtimes/python27_experiment/python27_lib/versions/1/google/appengine/ext/ndb/tasklets.py"，第 427 行，在 _help_tasklet_alongvalue = gen.throw(exc.__class__, exc, tb)_put_tasklet 中的文件/base/data/home/runtimes/python27_experiment/python27_lib/versions/1/google/appengine/ext/ndb/context.py"，第 358 行键 = 产量 self._conn.async_put(options, datastore_entities)文件/base/data/home/runtimes/python27_experiment/python27_lib/versions/1/google/appengine/ext/ndb/tasklets.py"，第 513 行，_on_rpc_completion结果 = rpc.get_result()get_result 中的文件/base/data/home/runtimes/python27_experiment/python27_lib/versions/1/google/appengine/datastore/datastore_rpc.py"，第 928 行结果 = rpc.get_result()get_result 中的文件/base/data/home/runtimes/python27_experiment/python27_lib/versions/1/google/appengine/api/apiproxy_stub_map.py"，第 613 行返回 self.__get_result_hook(self)文件/base/data/home/runtimes/python27_experiment/python27_lib/versions/1/google/appengine/datastore/datastore_rpc.py"，第 1893 行，在 __put_hookself.check_rpc_success(rpc)check_rpc_success 中的文件/base/data/home/runtimes/python27_experiment/python27_lib/versions/1/google/appengine/datastore/datastore_rpc.py"，第 1385 行引发 _ToDatastoreError(err)TransactionFailedError:这些数据存储实体的争用过多.请再试一次.请注意，错误实际上是从数据存储本身接收到的，在 RPC 响应中:self.check_rpc_success(rpc).>
这让我怀疑在数据存储方面，为了确保支持它的冗余基础设施之间的操作一致性/可靠性，每个写操作实际上都使用与事务操作相同/相似的机制.不同之处在于，它们在客户端、RPC 交换之前/之后也有一些事务检查，可能还有数据存储的显式 RPC 事务开始/结束触发器.
来自 数据存储写入的生命，引述了一些常见机制无论操作是否是事务性的都被使用(强调我的):
<块引用>如果提交阶段成功但应用阶段失败，则数据存储将前滚以将更改应用于两个下的索引情况:
- 下次您对该实体组执行读取或写入或启动事务时，数据存储区将首先滚动转发并完全应用此已提交但未应用的写入，基于日志中的数据.
失败的可能原因之一就是对同一实体的并行访问过多，即使它们只是只读的.请参阅 Google App Engine 中的争用问题，但在这种情况下它们用于客户端的交易.
请注意，这只是一个理论;)
I got this error:
TransactionFailedError: too much contention on these datastore entities. please try again.
Even though I'm not doing any transactions. The line of my code that causes the error is
ndb.put_multi(entity_list) # entity_list is a list of 100 entities
This error doesn't happen often so it isn't a big deal, but I'm curious why I get this error. Any ideas?
Here is most of the traceback:
Traceback (most recent call last):
...
File "/base/data/home/runtimes/python27_experiment/python27_lib/versions/1/google/appengine/ext/deferred/deferred.py", line 318, in post
self.run_from_request()
File "/base/data/home/runtimes/python27_experiment/python27_lib/versions/1/google/appengine/ext/deferred/deferred.py", line 313, in run_from_request
run(self.request.body)
File "/base/data/home/runtimes/python27_experiment/python27_lib/versions/1/google/appengine/ext/deferred/deferred.py", line 155, in run
return func(*args, **kwds)
File "/base/data/home/apps/s~opavote/2017-09-15.404125237783169549/tasks.py", line 70, in start_election
models.Voter.create(e.eid, chunk)
File "/base/data/home/apps/s~opavote/2017-09-15.404125237783169549/models.py", line 2426, in create
ndb.put_multi(voters + vbs)
File "/base/data/home/runtimes/python27_experiment/python27_lib/versions/1/google/appengine/ext/ndb/model.py", line 3958, in put_multi
for future in put_multi_async(entities, **ctx_options)]
File "/base/data/home/runtimes/python27_experiment/python27_lib/versions/1/google/appengine/ext/ndb/tasklets.py", line 383, in get_result
self.check_success()
File "/base/data/home/runtimes/python27_experiment/python27_lib/versions/1/google/appengine/ext/ndb/tasklets.py", line 427, in _help_tasklet_along
value = gen.throw(exc.__class__, exc, tb)
File "/base/data/home/runtimes/python27_experiment/python27_lib/versions/1/google/appengine/ext/ndb/context.py", line 824, in put
key = yield self._put_batcher.add(entity, options)
File "/base/data/home/runtimes/python27_experiment/python27_lib/versions/1/google/appengine/ext/ndb/tasklets.py", line 427, in _help_tasklet_along
value = gen.throw(exc.__class__, exc, tb)
File "/base/data/home/runtimes/python27_experiment/python27_lib/versions/1/google/appengine/ext/ndb/context.py", line 358, in _put_tasklet
keys = yield self._conn.async_put(options, datastore_entities)
File "/base/data/home/runtimes/python27_experiment/python27_lib/versions/1/google/appengine/ext/ndb/tasklets.py", line 513, in _on_rpc_completion
result = rpc.get_result()
File "/base/data/home/runtimes/python27_experiment/python27_lib/versions/1/google/appengine/datastore/datastore_rpc.py", line 928, in get_result
result = rpc.get_result()
File "/base/data/home/runtimes/python27_experiment/python27_lib/versions/1/google/appengine/api/apiproxy_stub_map.py", line 613, in get_result
return self.__get_result_hook(self)
File "/base/data/home/runtimes/python27_experiment/python27_lib/versions/1/google/appengine/datastore/datastore_rpc.py", line 1893, in __put_hook
self.check_rpc_success(rpc)
File "/base/data/home/runtimes/python27_experiment/python27_lib/versions/1/google/appengine/datastore/datastore_rpc.py", line 1385, in check_rpc_success
raise _ToDatastoreError(err)
TransactionFailedError: too much contention on these datastore entities. please try again.
Note that the error is actually received from the datastore itself, in the RPC response: self.check_rpc_success(rpc).
Which makes me suspect that on the datastore side, to ensure operation consistency/reliability across the redundant pieces of infra supporting it, every write operation is actually using the same/similar mechanisms as for transactional operations. The difference would be that those also have some transactional checks on the client side, before/after the RPC exchange and maybe explicit RPC transaction start/end triggers for the datastore.
From Life of a Datastore Write, a quote suggesting that some common mechanisms are being used regardless of the operations being transactional or not (emphasis mine):
If the commit phase has succeeded but the apply phase failed, the datastore will roll forward to apply the changes to indexes under two circumstances:
- The next time you execute a read or write or start a transaction on this entity group, the datastore will first roll forward and fully apply this committed but unapplied write, based on the data in the log.
And one of the possible reasons for failures would be simply too many parallel accesses to the same entities, even if they're just read-only. See Contention problems in Google App Engine, though in that case they're for transactions on the client side.
Note that this is just a theory ;)
这篇关于没有事务时 GAE 上的 TransactionFailedError的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
