使用Facebook的DataLoader传递参数 [英] Passing down arguments using Facebook's DataLoader

问题描述

我正在使用 DataLoader 将请求/查询一起批处理. 在我的加载程序函数中，我需要知道请求的字段，以避免出现SELECT * FROM query而不是SELECT field1, field2, ... FROM query ...

I'm using DataLoader for batching the requests/queries together. In my loader function I need to know the requested fields to avoid having a SELECT * FROM query but rather a SELECT field1, field2, ... FROM query...

使用DataLoader传递所需的resolveInfo的最佳方法是什么? (我使用resolveInfo.fieldNodes获取请求的字段)

What would be the best approach using DataLoader to pass down the resolveInfo needed for it? (I use resolveInfo.fieldNodes to get the requested fields)

此刻，我正在做这样的事情:

At the moment, I'm doing something like this:

await someDataLoader.load({ ids, args, context, info });

，然后在实际的loaderFn中:

and then in the actual loaderFn:

const loadFn = async options => {
const ids = [];
let args;
let context;
let info;
options.forEach(a => {
    ids.push(a.ids);
    if (!args && !context && !info) {
        args = a.args;
        context = a.context;
        info = a.info;
    }
});

return Promise.resolve(await new DataProvider().get({ ...args, ids}, context, info));};

但是正如您所看到的，它很hacky，感觉并不好...

but as you can see, it's hacky and doesn't really feel good...

有人知道我如何实现这一目标吗?

Does anyone have an idea how I could achieve this?

推荐答案

我不确定这个问题是否有一个很好的答案，这仅仅是因为Dataloader不是为此用例制作的，但是我已经与Dataloader进行了广泛的合作，编写了类似的实现并探索了其他编程语言中的类似概念.

I am not sure if there is a good answer to this question simply because Dataloader is not made for this usecase but I have worked extensively with Dataloader, written similar implementations and explored similar concepts on other programming languages.

让我们理解为什么不为该用例创建Dataloader，以及我们如何使它能够正常工作(大致与您的示例类似).

Let's understand why Dataloader is not made for this usecase and how we could still make it work (roughly like in your example).

数据加载器用于简单的键值查找.这意味着给定 key 这样的ID，它将在其后加载一个值.为此，它假定ID后面的对象在失效之前始终是相同的.这是启用数据加载器功能的单一假设.没有它，Dataloader的三个关键功能将不再起作用:

Dataloader is made for simple key-value-lookups. That means given a key like an ID it will load a value behind it. For that it assumes that the object behind the ID will always be the same until it is invalidated. This is the single assumption that enables the power of dataloader. Without it the three key features of Dataloader won't work anymore:

1. 批处理请求(多个请求在一个查询中一起完成)
2. 重复数据删除(对同一密钥的两次请求导致一次查询)
3. 缓存(相同键的连续请求不会导致多次查询)

如果要最大化Dataloader的功能，这将导致以下两个重要规则:

This leads us to the following two important rules if we want to maximise the power of Dataloader:

两个不同的实体不能共享同一密钥，否则我们可能会返回错误的实体.这听起来微不足道，但在您的示例中却并非如此.假设我们要加载ID为1且字段为id和name的用户.稍后(或同时)，我们要加载ID为1且字段为id和email的用户.从技术上讲，这是两个不同的实体，它们需要具有不同的密钥.

Two different entities cannot share the same key, othewise we might return the wrong entity. This sounds trivial but it is not in your example. Let's say we want to load a user with ID 1 and the fields id and name. A little bit later (or at the same time) we want to load user with ID 1 and fields id and email. These are technically two different entities and they need to have a different key.

同一实体应该始终具有相同的密钥.听起来似乎微不足道，但实际上不在示例中.标识为1且字段为id和name的用户应与标识为1且字段为name和id的用户相同(请注意顺序).

The same entity should have the same key all the time. Again sounds trivial but really is not in the example. User with ID 1 and fields id and name should be the same as user with ID 1 and fields name and id (notice the order).

简而言之，密钥需要具有唯一标识一个实体所需的全部信息，但不能超过.

await someDataLoader.load({ ids, args, context, info });

在您的问题中，您还为Dataloader提供了一些关键的东西.首先，我不会将args和context放入键中.当上下文改变时(例如，您现在正在查询其他数据库)，您的实体会改变吗?可能是的，但是您想在数据加载器的实现中考虑到这一点吗?相反，我建议按照 docs 中的说明为每个请求创建新的数据加载器.

In your question you have provided a few more things to your Dataloader as a key. First I would not put in args and context into the key. Does your entity change when the context changes (e.g. you are querying a different database now)? Probably yes, but do you want to account for that in your dataloader implementation? I would instead suggest to create new dataloaders for each request as described in the docs.

整个请求信息都应该在密钥中吗?不，但是我们需要所要求的字段.除此之外，您提供的实现是错误的，并且在调用带有两个不同解析信息的加载器时会中断.您仅从第一个调用设置了解析信息，但实际上每个对象上的解析信息可能有所不同(请考虑上面的第一个用户示例).最终，我们可以实现数据加载器的以下实现:

Should the whole request info be in the key? No, but we need the fields that are requested. Apart from that your provided implementation is wrong and would break when the loader is called with two different resolve infos. You only set the resolve info from the first call but really it might be different on each object (think about the first user example above). Ultimately we could arrive at the following implementation of a dataloader:

// This function creates unique cache keys for different selected
// fields
function cacheKeyFn({ id, fields }) {
  const sortedFields = [...(new Set(fields))].sort().join(';');
  return `${id}[${sortedFields}]`;
}

function createLoaders(db) {
  const userLoader = new Dataloader(async keys => {
    // Create a set with all requested fields
    const fields = keys.reduce((acc, key) => {
      key.fields.forEach(field => acc.add(field));
      return acc;
    }, new Set());
    // Get all our ids for the DB query
    const ids = keys.map(key => key.id);
    // Please be aware of possible SQL injection, don't copy + paste
    const result = await db.query(`
      SELECT
        ${fields.entries().join()}
      FROM
        user
      WHERE
        id IN (${ids.join()})
    `);
  }, { cacheKeyFn });

  return { userLoader };
}

// now in a resolver
resolve(parent, args, ctx, info) {
  // https://www.npmjs.com/package/graphql-fields
  return ctx.userLoader.load({ id: args.id, fields: Object.keys(graphqlFields(info)) });
}

这是一个可靠的实现，但有一些缺点.首先，如果在同一个批处理请求中有不同的字段要求，那么我们将提取过多的字段.其次，如果我们从缓存键函数中获取了键为1[id,name]的实体，我们也可以使用该对象(至少在JavaScript中)应答键1[id]和1[name].在这里，我们可以构建一个可以提供给Dataloader的自定义地图实现.足够了解我们的缓存这些知识就足够了.

This is a solid implementation but it has a few weaknesses. First, we are overfetching a lot of fields if we have different field requiements in the same batch request. Second, if we have fetched an entity with key 1[id,name] from cache key function we could also answer (at least in JavaScript) keys 1[id] and 1[name] with that object. Here we could build a custom map implementation that we could supply to Dataloader. It would be smart enough to know these things about our cache.

我们看到这确实是一件复杂的事情.我知道它通常被列为GraphQL的一个优点，您不必为每个查询都从数据库中获取所有字段，但事实是，在实践中，这很少值得您解决. 不要优化不慢的内容.甚至很慢，这是瓶颈吗?

We see that this is really a complicated matter. I know it is often listed as a benefit of GraphQL that you don't have to fetch all fields from a database for every query, but the truth is that in practice this is seldomly worth the hassle. Don't optimise what is not slow. And even is it slow, is it a bottleneck?

我的建议是:编写简单的Dataloader，以简单地获取所有(所需)字段.如果您有一个客户端，则对于大多数实体而言，客户端很可能无论如何都会获取所有字段，否则它们将不属于您的API，对吗?然后使用类似查询解释的方法来衡量慢速查询，然后找出哪个字段恰好是慢速.然后，您仅优化最慢的事情(例如，请参见我的答案

My suggestion is: Write trivial Dataloaders that simply fetch all (needed) fields. If you have one client it is very likely that for most entities the client fetches all fields anyways, otherwise they would not be part of you API, right? Then use something like query introsprection to measure slow queries and then find out which field exactly is slow. Then you optimise only the slow thing (see for example my answer here that optimises a single use case). And if you are a big ecomerce platform please don't use Dataloader for this. Build something smarter and don't use JavaScript.

这篇关于使用Facebook的DataLoader传递参数的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！