尝试批量插入多个实体时出现App Engine错误，条件是属性为“UserNo”每次插入实体时增加1 [英] App Engine Errors when trying to insert many entities in bulk with condition that property "UserNo" increases by 1 everytime an entity's inserted
问题描述
我想批量插入许多用户实体。用户实体拥有名称&的 UserNo 即可。要求是，每次插入用户实体时， UserNo 属性会自动增加1。
如果我尝试下面的代码，一次插入1条记录。
public void insertUser（String name）{
Entity userEntity = new Entity 用户）;
long maxID = Utility.getPropertyMaxID（User，UserNo）+ 1;
userEntity.setProperty（Name，name）;
datastore.put（userEntity）;
$ b $ public static long getPropertyMaxID（String entityName，String propertyName）{
//按字母顺序排序按属性名称排序：
Query q = new Query（entityName）
.addSort（propertyName，SortDirection.DESCENDING）;
列表< Entity> results = datastore.prepare（q）
.asList（FetchOptions.Builder.withDefaults（））;
if（results.size（）> 0）{
Entity e = results.get（0）;
long maxID =（Long）e.getProperty（propertyName）;
返回maxID;
}
else {
return 0;
}
}
但是，如果我尝试插入（int i = 0; i <10; i ++）{
insertUser（names [$ p
$ b
一世]）;
}
然后，在检查数据时，我可以看到名称已正确插入。但是， UserNo 弄糟了，因为它不会正确地增加1。
所以，我认为这可能是 Google App Engine数据存储的最终一致性，因为数据需要插入时间才能使程序错误地执行 maxUserNo 。 那么，我该怎么做？不应该不使用 UserNo ？
我曾尝试使用Transaction，但无法使用
public void insertUser（String name）{
Transaction tx = datastore.beginTransaction（）;
尝试{
实体userEntity =新实体（用户）;
long maxID = Utility.getPropertyMaxID（User，UserNo）+ 1;
userEntity.setProperty（Name，name）;
datastore.put（userEntity）;
tx.commit（）;}
catch（Exception ex）{ex.printStackTrace（）};
}
那么，最佳解决方案是什么？
注意：我听说非祖先查询总是最终一致。因此， getPropertyMaxID 方法中的查询不是非常一致。
Extra ：关于如果我制作假人并强制所有用户成为该人的子女，然后在所有用户上进行Ancestor查询。尽管所有用户循环查看电子邮件，例如？看到下面的代码：
Entity personEntity = new Entity（Person，Tom）;
Key personKey = personEntity.getKey（）;
实体userEntity =新实体（User，userName）;
userEntity.setProperty（电子邮件，电子邮件）;
datastore.put（userEntity）;
然后创建一个函数来检查唯一的电子邮件地址
public static boolean checkUniqueEmail（String email）{
Entity personEntity = new Entity（Person，Tom）;
Key personKey = personEntity.getKey（）;
查询查询=新查询（用户）
.setAncestor（personKey）;
列表< Entity> results = datastore.prepare（query）
.asList（FetchOptions.Builder.withDefaults（））;
（实体e：结果）{
String em =（String）e.getProperty（Email）;
if（e.equals（email））返回false;
}
}
I找到答案
检查此链接
若要了解如何构建强大的一致性数据，请比较App Engine入门练习中的留言簿示例应用程序的两种不同方法。第一种方法为每个问候语创建一个新的根实体：
import com.google.appengine.api.datastore.Entity;
实体问候=新实体（问候）;
//没有指定父键，所以Greeting是一个根实体。
greeting.setProperty（user，user）;
greeting.setProperty（date，date）;
greeting.setProperty（content，content）;
然后它会查询最近十次问候的实体种类Greeting。
import com.google.appengine.api.datastore.DatastoreService;
import com.google.appengine.api.datastore.DatastoreServiceFactory;
import com.google.appengine.api.datastore.Entity;
DatastoreService datastore = DatastoreServiceFactory.getDatastoreService（）;
Query query = new Query（Greeting）
.addSort（date，Query.SortDirection.DESCENDING）;
列表< Entity> greetings = datastore.prepare（query）
.asList（FetchOptions.Builder.withLimit（10））;
但是，因为我们使用的是非祖先查询，所以用于在此执行查询的副本方案在执行查询时可能没有看到新的问候语。尽管如此，几乎所有的写操作都会在提交后的几秒钟内用于非祖先查询。对于许多应用程序来说，在当前用户自己的更改的上下文中提供非祖先查询结果的解决方案通常足以使此类复制延迟完全可接受。
如果强一致性对于应用程序很重要，则另一种方法是编写具有祖先路径的实体，以便在所有实体中标识必须在单个一致的祖先查询中读取的同一根实体：
import com.google.appengine.api.datastore.DatastoreService;
import com.google.appengine.api.datastore.DatastoreServiceFactory;
import com.google.appengine.api.datastore.Entity;
字符串guestbookName = req.getParameter（guestbookName）;
Key guestbookKey = KeyFactory.createKey（Guestbook，guestbookName）;
String content = req.getParameter（content）;
日期日期=新日期（）;
//将问候语放置在与留言簿相同的实体组中
实体问候语=新实体（问候语，guestbookKey）;
greeting.setProperty（user，user）;
greeting.setProperty（date，date）;
greeting.setProperty（content，content）;
然后，您将能够在通用根实体标识的实体组中执行一致的祖先查询：
import com.google.appengine.api.datastore.DatastoreService;
import com.google.appengine.api.datastore.DatastoreServiceFactory;
import com.google.appengine.api.datastore.Entity;
DatastoreService datastore = DatastoreServiceFactory.getDatastoreService（）;
Key guestbookKey = KeyFactory.createKey（Guestbook，guestbookName）;
Query query = new Query（Greeting，guestbookKey）
.setAncestor（guestbookKey）
.addSort（date，Query.SortDirection.DESCENDING）;
列表< Entity> greetings = datastore.prepare（query）
.asList（FetchOptions.Builder.withLimit（10））;
这种方法通过向每个留言簿写入单个实体组来实现强大的一致性，该留言簿每秒不超过1次写入（实体组支持的限制）。如果您的应用程序可能遇到较重的写入使用情况，则可能需要考虑使用其他方式：例如，您可能会将最近发布的内容放在具有到期日期的内存缓存中，并显示来自内存缓存和数据存储区的近期帖子，或者您可能会将它们缓存到Cookie中，或者在URL中放置一些状态，或者完全是其他内容。我们的目标是找到一个缓存解决方案，该解决方案在用户发布到应用程序的时间段内为当前用户提供数据。请记住，如果您执行get，祖先查询或事务中的任何操作，您将始终看到最近写入的数据。
I want to insert many user entities in bulk. User entity has property Name & UserNo. The requirement is that the property UserNo will automatically increase by 1 every time a User entity was inserted.
The following code works fine if I try to insert 1 record at a time.
public void insertUser(String name){
Entity userEntity=new Entity("User");
long maxID=Utility.getPropertyMaxID("User", "UserNo")+1;
userEntity.setProperty("Name",name);
datastore.put(userEntity);
}
public static long getPropertyMaxID(String entityName, String propertyName){
// Order alphabetically by property name:
Query q = new Query(entityName)
.addSort(propertyName, SortDirection.DESCENDING);
List<Entity> results = datastore.prepare(q)
.asList(FetchOptions.Builder.withDefaults());
if(results.size()>0){
Entity e=results.get(0);
long maxID=(Long)e.getProperty(propertyName);
return maxID;
}
else{
return 0;
}
}
However, if I try to insert many entities at once
for(int i=0; i<10; i++){
insertUser(names[i]);
}
Then, when checking data I can see the name got inserted correctly. However, the UserNo got messed up as it did not increase 1 by 1 correctly.
So, I think this could be the "Eventual consistency" of Google App Engine datastore because the data takes time to be inserted so the program may take the maxUserNo incorrectly.
So, what should I do? Should not not to use "UserNo"?
I did try Transaction, but it didn't work anyway
public void insertUser(String name){
Transaction tx=datastore.beginTransaction();
try{
Entity userEntity=new Entity("User");
long maxID=Utility.getPropertyMaxID("User", "UserNo")+1;
userEntity.setProperty("Name",name);
datastore.put(userEntity);
tx.commit();}
catch (Exception ex){ ex.printStackTrace()};
}
So, what is the best solution for this?
Note: I heard that "Non-ancestor queries are always eventually consistent". SO the query inside getPropertyMaxID method is not Strongly consistent.
Extra: what about if I make a fake Person and force all Users to be children of that person & then make Ancestor query on all Users. Loop though all users to check the email for example? See this code:
Entity personEntity=new Entity("Person", "Tom");
Key personKey=personEntity.getKey();
Entity userEntity=new Entity("User", userName);
userEntity.setProperty("Email", email);
datastore.put(userEntity);
Then make a function to check the unique email
public static boolean checkUniqueEmail(String email){
Entity personEntity = new Entity("Person", "Tom");
Key personKey = personEntity.getKey();
Query query = new Query("User")
.setAncestor(personKey);
List<Entity> results = datastore.prepare(query)
.asList(FetchOptions.Builder.withDefaults());
for(Entity e : results){
String em=(String)e.getProperty("Email");
if(e.equals(email)) return false;
}
}
I found the answer
Check this link
To understand how to structure your data for strong consistency, compare two different approaches for the guestbook example application from the App Engine Getting Started exercise. The first approach creates a new root entity for each greeting:
import com.google.appengine.api.datastore.Entity;
Entity greeting = new Entity("Greeting");
// No parent key specified, so Greeting is a root entity.
greeting.setProperty("user", user);
greeting.setProperty("date", date);
greeting.setProperty("content", content);
It then queries on the entity kind Greeting for the ten most recent greetings.
import com.google.appengine.api.datastore.DatastoreService;
import com.google.appengine.api.datastore.DatastoreServiceFactory;
import com.google.appengine.api.datastore.Entity;
DatastoreService datastore = DatastoreServiceFactory.getDatastoreService();
Query query = new Query("Greeting")
.addSort("date", Query.SortDirection.DESCENDING);
List<Entity> greetings = datastore.prepare(query)
.asList(FetchOptions.Builder.withLimit(10));
However, because we are using a non-ancestor query, the replica used to perform the query in this scheme may not have seen the new greeting by the time the query is executed. Nonetheless, nearly all writes will be available for non-ancestor queries within a few seconds of commit. For many applications, a solution that provides the results of a non-ancestor query in the context of the current user's own changes will usually be sufficient to make such replication latencies completely acceptable.
If strong consistency is important to your application, an alternate approach is to write entities with an ancestor path that identifies the same root entity across all entities that must be read in a single, strongly-consistent ancestor query:
import com.google.appengine.api.datastore.DatastoreService;
import com.google.appengine.api.datastore.DatastoreServiceFactory;
import com.google.appengine.api.datastore.Entity;
String guestbookName = req.getParameter("guestbookName");
Key guestbookKey = KeyFactory.createKey("Guestbook", guestbookName);
String content = req.getParameter("content");
Date date = new Date();
// Place greeting in same entity group as guestbook
Entity greeting = new Entity("Greeting", guestbookKey);
greeting.setProperty("user", user);
greeting.setProperty("date", date);
greeting.setProperty("content", content);
You will then be able to perform a strongly-consistent ancestor query within the entity group identified by the common root entity:
import com.google.appengine.api.datastore.DatastoreService;
import com.google.appengine.api.datastore.DatastoreServiceFactory;
import com.google.appengine.api.datastore.Entity;
DatastoreService datastore = DatastoreServiceFactory.getDatastoreService();
Key guestbookKey = KeyFactory.createKey("Guestbook", guestbookName);
Query query = new Query("Greeting", guestbookKey)
.setAncestor(guestbookKey)
.addSort("date", Query.SortDirection.DESCENDING);
List<Entity> greetings = datastore.prepare(query)
.asList(FetchOptions.Builder.withLimit(10));
This approach achieves strong consistency by writing to a single entity group per guestbook, but it also limits changes to the guestbook to no more than 1 write per second (the supported limit for entity groups). If your application is likely to encounter heavier write usage, you may need to consider using other means: for example, you might put recent posts in a memcache with an expiration and display a mix of recent posts from the memcache and the Datastore, or you might cache them in a cookie, put some state in the URL, or something else entirely. The goal is to find a caching solution that provides the data for the current user for the period of time in which the user is posting to your application. Remember, if you do a get, an ancestor query, or any operation within a transaction, you will always see the most recently written data.
这篇关于尝试批量插入多个实体时出现App Engine错误，条件是属性为“UserNo”每次插入实体时增加1的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
