Google Datastore结合（联合）多组实体结果以实现OR条件 [英] Google Datastore combine (union) multiple sets of entity results to achieve OR condition

问题描述

我正在使用Google App Engine上的NodeJS和Datastore数据库。

由于数据存储不支持OR运算符，我需要运行多个查询并合并结果。 / p>

我计划运行多个查询，然后将结果合并到单个实体对象数组中。我已经有一个单一的查询了。

问题：将Datastore返回的两个（或更多）实体组合包括重复数据删除的合理有效方法是什么？我相信这将是集合论中的一个联合操作。

以下是基本查询大纲，它将使用一些不同的过滤器多次运行以实现OR条件。

  //设置请求者用户名
 const requester = req.user.userName; 
 //在Transfer Request类表上创建数据存储查询
 const task_history = datastore.createQuery（'Task'）; 
 //设置查询条件
 task_history.filter（'requester'，requester）; 
 //运行数据存储查询
 datastore.runQuery（task_history，function（err，entities）{
 if（err）{
 console.log（'任务历史记录JSON无法返回数据结果错误信息：'，err）; 
 return; 
 //如果查询工作并返回任何实体
} else if（entities [0]）{
 //否则如果查询有效但不返回任何实体返回空JSON响应
 res.json（entities）; //如何有效地结合（UNION）多个实体集合
 return; 
} 
}）; 
  

这是我原来的帖子：带有OR条件的Google数据存储过滤器

解决方案

恕我直言，最有效的方法是在第一阶段使用仅限键的查询，然后将获得的密钥组合成单个列表（包括重复数据删除），然后通过简单查找获得实体。从投影查询：

仅键查询

仅键查询（这是一种投影查询）仅返回
结果实体的键而不是实体本身，比检索整个实体的延迟和成本低
。

执行然后
从结果中获取实体的一个子集，而不是执行
的一般查询，这可能会获取比实际需要的更多实体。

下面介绍如何创建一个仅用于键的查询：

  const query = datastore.createQuery（）
 .select（'__ key__'）
 .limit（1）; 
  

此方法解决了您在尝试直接合并时可能遇到的几个问题通过常规的，非关键字查询获得的实体列表：

• 您无法正确地进行重复删除，因为您无法分辨具有相同值的不同实体之间的差异以及出现在乘法查询结果中的相同实体
  
• 通过属性值比较实体可能比较棘手，并且肯定比仅比较实体键


• 如果您无法在单个请求中处理所有结果，那么您在读取它们时会产生不必要的数据存储成本，而无法实际使用它们。


• 在处理实体键时，通过多个请求（例如通过任务队列）拆分实体的处理非常简单

以及：

• 它可能有点慢因为你需要两次访问数据存储区：一次是密钥，一次是获取实际实体


• ，因此无法通过非密钥获取所需的属性，只有投影查询

I am working with NodeJS on Google App Engine with the Datastore database.

Due to the fact that Datastore does not have support the OR operator, I need to run multiple queries and combine the results.

I am planning to run multiple queries and then combine the results into a single array of entity objects. I have a single query working already.

Question: What is a reasonably efficient way to combine two (or more) sets of entities returned by Datastore including de-duplication? I believe this would be a "union" operation in terms of set theory.

Here is the basic query outline that will be run multiple times with some varying filters to achieve the OR conditions required.

  //Set requester username
  const requester = req.user.userName;
  //Create datastore query on Transfer Request kind table
  const task_history = datastore.createQuery('Task');
  //Set query conditions
  task_history.filter('requester', requester);
  //Run datastore query
  datastore.runQuery(task_history, function(err, entities) {
    if(err) {
      console.log('Task History JSON unable to return data results. Error message: ', err);
      return;
      //If query works and returns any entities
    } else if (entities[0]) {
      //Else if query works but does not return any entities return empty JSON response
      res.json(entities); //HOW TO COMBINE (UNION) MULTIPLE SETS OF ENTITIES EFFICIENTLY?
      return;
    }
  });

Here is my original post: Google Datastore filter with OR condition

解决方案

IMHO the most efficient way would be to use Keys-only queries in the 1st stage, then perform the combination of the keys obtained into a single list (including deduplication), followed by obtaining the entities simply by key lookup. From Projection queries:

Keys-only queries

A keys-only query (which is a type of projection query) returns just the keys of the result entities instead of the entities themselves, at lower latency and cost than retrieving entire entities.

It is often more economical to do a keys-only query first, and then fetch a subset of entities from the results, rather than executing a general query which may fetch more entities than you actually need.

Here's how to create a keys-only query:

const query = datastore.createQuery()
  .select('__key__')
  .limit(1);

This method addresses several problems you may encounter when trying to directly combine lists of entities obtained through regular, non-keys-only queries:

• you can't de-duplicate properly because you can't tell the difference between different entities with identical values and the same entity appearing in multiply query results
• comparing entities by property values can be tricky and is definitely slower/more computing expensive than comparing just entity keys
• if you can't process all the results in a single request you're incurring unnecessary datastore costs for reading them without actually using them
• it is much simpler to split processing of entities in multiple requests (via task queues, for example) when handling just entity keys

There are some disadvantages as well:

• it may be a bit slower because you're going to the datastore twice: once for the keys and once to get the actual entities
• you can't take advantage of getting just the properties you need via non-keys-only projection queries

这篇关于Google Datastore结合（联合）多组实体结果以实现OR条件的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！