没有事务时GAE上的TransactionFailedError [英] TransactionFailedError on GAE when no transaction

问题描述

  TransactionFailedError：对这些数据存储实体的争用太多。请再试一次。 
  

即使我没有做任何交易。导致错误的代码行是：

pre $ ndb.put_multi（entity_list）＃entity_list是一个包含100个实体的列表


这个错误不会经常发生，所以这不是什么大问题，但我很好奇为什么我得到这个错误。任何想法？

以下是大部分回溯：

  Traceback （最近的最后一次呼叫）：
 ... 
文件/base/data/home/runtimes/python27_experiment/python27_lib/versions/1/google/appengine/ext/deferred/deferred.py，第318行，后
 self.run_from_request（）
文件/base/data/home/runtimes/python27_experiment/python27_lib/versions/1/google/appengine/ext/deferred/deferred.py，第313行，在run_from_request 
 run（self.request.body）
文件/ base / data / home / runtimes / python27_experiment / python27_lib / versions / 1 / google / appengine / ext / deferred / deferred。 py，第155行，运行
返回func（* args，** kwds）
文件/base/data/home/apps/s~opavote/2017-09-15.404125237783169549/tasks.py ，第70行，在start_election 
 models.Voter.create（e.eid，chunk）
文件/base/data/home/apps/s~opavote/2017-09-15.404125237783169549/models。 py，第2426行，创建
 ndb.put_multi（选项+ vbs）
文件/ base / data / home / runtimes / python27_experiment / python27_lib / versions / 1 / google / appengine / ext / ndb / model.py，第3958行，在put_multi 
中用于将来的put_multi_async（实体，** ctx_options）] 
文件/base/data/home/runtimes/python27_experiment/python27_lib/versions/1/google/appengine/ext/ndb/tasklets.py，第383行，在get_result中
 self.check_success（）
文件/base/data/home/runtimes/python27_experiment/python27_lib/versions/1/google/appengine/ext/ndb/tasklets.py，第427行，在_help_tasklet_along 
 value = gen.throw（exc .__ class__ ，exc，tb）
文件/base/data/home/runtimes/python27_experiment/python27_lib/versions/1/google/appengine/ext/ndb/context.py，第824行，放入
 key = yield self._put_batcher.add（entity，options）
文件/base/data/home/runtimes/python27_experiment/python27_lib/versions/1/google/appengine/ext/ndb/tasklets.py，行427，在_help_tasklet_along 
 value = gen.throw（exc .__ class__，exc，tb）
文件/ base / data / home / runtim es / python27_experiment / python27_lib / versions / 1 / google / appengine / ext / ndb / context.py，第358行，位于_put_tasklet 
 keys = yield self._conn.async_put（options，datastore_entities）
 File /base/data/home/runtimes/python27_experiment/python27_lib/versions/1/google/appengine/ext/ndb/tasklets.py，第513行，在_on_rpc_completion中
 result = rpc.get_result（）
文件/base/data/home/runtimes/python27_experiment/python27_lib/versions/1/google/appengine/datastore/datastore_rpc.py，第928行，在get_result中
 result = rpc.get_result（）
在get_result 
中返回文件/base/data/home/runtimes/python27_experiment/python27_lib/versions/1/google/appengine/api/apiproxy_stub_map.py，第613行返回self .__ get_result_hook（self）
文件/base/data/home/runtimes/python27_experiment/python27_lib/versions/1/google/appengine/datastore/datastore_rpc.py，行1893，在__put_hook 
 self.check_rpc_success（rpc）
文件/ base / data / home / runtimes / python27_experiment /python27_lib/versions/1/google/appengine/datastore/datastore_rpc.py，第1385行，在check_rpc_success中
 raise _ToDatastoreError（err）
 TransactionFailedError：这些数据存储实体上的争用太多。请再试一次。 
  

解决方案

请注意，错误实际上是从数据存储本身接收的，在RPC响应中： self.check_rpc_success（rpc）。

这让我怀疑在数据存储区为了确保支持它的冗余冗余部分的运行一致性/可靠性，每个写操作实际上都使用与事务操作相同/类似的机制。区别在于那些在RPC交换之前/之后在客户端也具有一些事务检查，并且可能是数据存储的明确的RPC事务开始/结束触发器。

从数据存储的生活写入，这个引用表明一些常见的机制正在被使用，而不管这些操作是否是事务性的（我的重点）：

如果提交阶段已成功，但申请阶段失败，
数据存储将前滚以将更改应用于两个
情况下的索引：

1. 下一次在此实体组上执行读取或写入或启动事务时，数据存储将首先向前滚动
   ，并完全应用此已提交但未应用的写入，基于
   日志中的数据。
   

其中一个可能的原因是即使它们只是只读的，失败对同一个实体的并行访问也会太多。尽管在这种情况下，请参阅 Google App Engine中的争用问题它们用于客户端的交易。

请注意，这只是一个理论;）

I got this error:

TransactionFailedError: too much contention on these datastore entities. please try again.

Even though I'm not doing any transactions. The line of my code that causes the error is

ndb.put_multi(entity_list) # entity_list is a list of 100 entities

This error doesn't happen often so it isn't a big deal, but I'm curious why I get this error. Any ideas?

Here is most of the traceback:

Traceback (most recent call last):
  ...
  File "/base/data/home/runtimes/python27_experiment/python27_lib/versions/1/google/appengine/ext/deferred/deferred.py", line 318, in post
    self.run_from_request()
  File "/base/data/home/runtimes/python27_experiment/python27_lib/versions/1/google/appengine/ext/deferred/deferred.py", line 313, in run_from_request
    run(self.request.body)
  File "/base/data/home/runtimes/python27_experiment/python27_lib/versions/1/google/appengine/ext/deferred/deferred.py", line 155, in run
    return func(*args, **kwds)
  File "/base/data/home/apps/s~opavote/2017-09-15.404125237783169549/tasks.py", line 70, in start_election
    models.Voter.create(e.eid, chunk)
  File "/base/data/home/apps/s~opavote/2017-09-15.404125237783169549/models.py", line 2426, in create
    ndb.put_multi(voters + vbs)
  File "/base/data/home/runtimes/python27_experiment/python27_lib/versions/1/google/appengine/ext/ndb/model.py", line 3958, in put_multi
    for future in put_multi_async(entities, **ctx_options)]
  File "/base/data/home/runtimes/python27_experiment/python27_lib/versions/1/google/appengine/ext/ndb/tasklets.py", line 383, in get_result
    self.check_success()
  File "/base/data/home/runtimes/python27_experiment/python27_lib/versions/1/google/appengine/ext/ndb/tasklets.py", line 427, in _help_tasklet_along
    value = gen.throw(exc.__class__, exc, tb)
  File "/base/data/home/runtimes/python27_experiment/python27_lib/versions/1/google/appengine/ext/ndb/context.py", line 824, in put
    key = yield self._put_batcher.add(entity, options)
  File "/base/data/home/runtimes/python27_experiment/python27_lib/versions/1/google/appengine/ext/ndb/tasklets.py", line 427, in _help_tasklet_along
    value = gen.throw(exc.__class__, exc, tb)
  File "/base/data/home/runtimes/python27_experiment/python27_lib/versions/1/google/appengine/ext/ndb/context.py", line 358, in _put_tasklet
    keys = yield self._conn.async_put(options, datastore_entities)
  File "/base/data/home/runtimes/python27_experiment/python27_lib/versions/1/google/appengine/ext/ndb/tasklets.py", line 513, in _on_rpc_completion
    result = rpc.get_result()
  File "/base/data/home/runtimes/python27_experiment/python27_lib/versions/1/google/appengine/datastore/datastore_rpc.py", line 928, in get_result
    result = rpc.get_result()
  File "/base/data/home/runtimes/python27_experiment/python27_lib/versions/1/google/appengine/api/apiproxy_stub_map.py", line 613, in get_result
    return self.__get_result_hook(self)
  File "/base/data/home/runtimes/python27_experiment/python27_lib/versions/1/google/appengine/datastore/datastore_rpc.py", line 1893, in __put_hook
    self.check_rpc_success(rpc)
  File "/base/data/home/runtimes/python27_experiment/python27_lib/versions/1/google/appengine/datastore/datastore_rpc.py", line 1385, in check_rpc_success
    raise _ToDatastoreError(err)
TransactionFailedError: too much contention on these datastore entities. please try again.

解决方案

Note that the error is actually received from the datastore itself, in the RPC response: self.check_rpc_success(rpc).

Which makes me suspect that on the datastore side, to ensure operation consistency/reliability across the redundant pieces of infra supporting it, every write operation is actually using the same/similar mechanisms as for transactional operations. The difference would be that those also have some transactional checks on the client side, before/after the RPC exchange and maybe explicit RPC transaction start/end triggers for the datastore.

From Life of a Datastore Write, a quote suggesting that some common mechanisms are being used regardless of the operations being transactional or not (emphasis mine):

If the commit phase has succeeded but the apply phase failed, the datastore will roll forward to apply the changes to indexes under two circumstances:

1. The next time you execute a read or write or start a transaction on this entity group, the datastore will first roll forward and fully apply this committed but unapplied write, based on the data in the log.

And one of the possible reasons for failures would be simply too many parallel accesses to the same entities, even if they're just read-only. See Contention problems in Google App Engine, though in that case they're for transactions on the client side.

Note that this is just a theory ;)

这篇关于没有事务时GAE上的TransactionFailedError的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！