使用 Facebook 的 DataLoader 传递参数 [英] Passing down arguments using Facebook's DataLoader
问题描述
我正在使用 DataLoader 将请求/查询一起批处理.在我的加载器函数中，我需要知道请求的字段以避免使用 SELECT * FROM query 而不是 SELECT field1, field2, ... FROM query...
I'm using DataLoader for batching the requests/queries together.
In my loader function I need to know the requested fields to avoid having a SELECT * FROM query but rather a SELECT field1, field2, ... FROM query...
使用 DataLoader 传递所需的 resolveInfo 的最佳方法是什么?(我使用 resolveInfo.fieldNodes 来获取请求的字段)
What would be the best approach using DataLoader to pass down the resolveInfo needed for it? (I use resolveInfo.fieldNodes to get the requested fields)
目前，我正在做这样的事情:
At the moment, I'm doing something like this:
await someDataLoader.load({ ids, args, context, info });
然后在实际的 loaderFn 中:
and then in the actual loaderFn:
const loadFn = async options => {
const ids = [];
let args;
let context;
let info;
options.forEach(a => {
ids.push(a.ids);
if (!args && !context && !info) {
args = a.args;
context = a.context;
info = a.info;
}
});
return Promise.resolve(await new DataProvider().get({ ...args, ids}, context, info));};
但是正如你所看到的，它很笨拙，感觉不太好......
but as you can see, it's hacky and doesn't really feel good...
有人知道我如何实现这一目标吗?
Does anyone have an idea how I could achieve this?
推荐答案
我不确定这个问题是否有好的答案，因为 Dataloader 不是为此用例而设计的，但我已经广泛使用 Dataloader，编写了类似的实现并探索了其他编程语言的类似概念.
I am not sure if there is a good answer to this question simply because Dataloader is not made for this usecase but I have worked extensively with Dataloader, written similar implementations and explored similar concepts on other programming languages.
让我们了解为什么 Dataloader 不是为这个用例而设计的，以及我们如何仍然使它工作(大致就像你的例子一样).
Let's understand why Dataloader is not made for this usecase and how we could still make it work (roughly like in your example).
Dataloader 用于简单的键值查找.这意味着给定一个 key 就像一个 ID，它会在它后面加载一个值.为此，它假定 ID 后面的对象将始终相同，直到它失效.这是启用数据加载器功能的单一假设.没有它，Dataloader 的三个关键功能 将不再起作用:
Dataloader is made for simple key-value-lookups. That means given a key like an ID it will load a value behind it. For that it assumes that the object behind the ID will always be the same until it is invalidated. This is the single assumption that enables the power of dataloader. Without it the three key features of Dataloader won't work anymore:
- 批处理请求(在一个查询中同时完成多个请求)
- 重复数据删除(对同一键的两次请求会导致一次查询)
- 缓存(同一键的连续请求不会导致多个查询)
如果我们想最大限度地发挥 Dataloader 的功能，这将引导我们遵循以下两个重要规则:
This leads us to the following two important rules if we want to maximise the power of Dataloader:
两个不同的实体不能共享同一个键，否则我们可能会返回错误的实体.这听起来微不足道，但它不在您的示例中.假设我们要加载一个 ID 为 1 以及字段 id 和 name 的用户.稍后(或同时)我们想要加载用户 ID 1 和字段 id 和 email.从技术上讲，它们是两个不同的实体，它们需要具有不同的密钥.
Two different entities cannot share the same key, othewise we might return the wrong entity. This sounds trivial but it is not in your example. Let's say we want to load a user with ID 1 and the fields id and name. A little bit later (or at the same time) we want to load user with ID 1 and fields id and email. These are technically two different entities and they need to have a different key.
同一个实体应该始终拥有相同的密钥.再次听起来微不足道，但实际上不在示例中.ID 1 和 id 和 name 字段的用户应该与 ID 1 和 字段的用户相同>name 和 id(注意顺序).
The same entity should have the same key all the time. Again sounds trivial but really is not in the example. User with ID 1 and fields id and name should be the same as user with ID 1 and fields name and id (notice the order).
简而言之一个键需要具有唯一标识一个实体所需的所有信息，但不能更多.
await someDataLoader.load({ ids, args, context, info });
在您的问题中，您已向 Dataloader 提供了更多内容作为关键.首先，我不会将 args 和 context 放入键中.当上下文发生变化时，您的实体是否会发生变化(例如，您现在正在查询不同的数据库)?可能是的，但是你想在你的数据加载器实现中考虑到这一点吗?我建议为每个请求创建新的数据加载器，如 docs 中所述.
In your question you have provided a few more things to your Dataloader as a key. First I would not put in args and context into the key. Does your entity change when the context changes (e.g. you are querying a different database now)? Probably yes, but do you want to account for that in your dataloader implementation? I would instead suggest to create new dataloaders for each request as described in the docs.
整个请求信息是否应该在密钥中?不，但我们需要请求的字段.除此之外，您提供的实现是错误的，并且在使用两个不同的解析信息调用加载程序时会中断.您只在第一次调用时设置解析信息，但实际上每个对象上的解析信息可能不同(想想上面的第一个用户示例).最终，我们可以实现数据加载器的以下实现:
Should the whole request info be in the key? No, but we need the fields that are requested. Apart from that your provided implementation is wrong and would break when the loader is called with two different resolve infos. You only set the resolve info from the first call but really it might be different on each object (think about the first user example above). Ultimately we could arrive at the following implementation of a dataloader:
// This function creates unique cache keys for different selected
// fields
function cacheKeyFn({ id, fields }) {
const sortedFields = [...(new Set(fields))].sort().join(';');
return `${id}[${sortedFields}]`;
}
function createLoaders(db) {
const userLoader = new Dataloader(async keys => {
// Create a set with all requested fields
const fields = keys.reduce((acc, key) => {
key.fields.forEach(field => acc.add(field));
return acc;
}, new Set());
// Get all our ids for the DB query
const ids = keys.map(key => key.id);
// Please be aware of possible SQL injection, don't copy + paste
const result = await db.query(`
SELECT
${fields.entries().join()}
FROM
user
WHERE
id IN (${ids.join()})
`);
}, { cacheKeyFn });
return { userLoader };
}
// now in a resolver
resolve(parent, args, ctx, info) {
// https://www.npmjs.com/package/graphql-fields
return ctx.userLoader.load({ id: args.id, fields: Object.keys(graphqlFields(info)) });
}
这是一个可靠的实现，但它有一些弱点.首先，如果我们在同一个批处理请求中有不同的字段需求，我们就会过度获取很多字段.其次，如果我们从缓存键函数中获取了一个键为 1[id,name] 的实体，我们还可以回答(至少在 JavaScript 中)键 1[id] 和1[name] 与该对象.在这里，我们可以构建可以提供给 Dataloader 的自定义地图实现.了解有关我们缓存的这些信息就足够聪明了.
This is a solid implementation but it has a few weaknesses. First, we are overfetching a lot of fields if we have different field requiements in the same batch request. Second, if we have fetched an entity with key 1[id,name] from cache key function we could also answer (at least in JavaScript) keys 1[id] and 1[name] with that object. Here we could build a custom map implementation that we could supply to Dataloader. It would be smart enough to know these things about our cache.
我们看到这确实是一个复杂的问题.我知道它经常被列为 GraphQL 的一个好处，即您不必为每个查询从数据库中获取所有字段，但事实是，实际上这很少值得麻烦.不要优化不慢的东西.甚至慢，是不是瓶颈?
We see that this is really a complicated matter. I know it is often listed as a benefit of GraphQL that you don't have to fetch all fields from a database for every query, but the truth is that in practice this is seldomly worth the hassle. Don't optimise what is not slow. And even is it slow, is it a bottleneck?
我的建议是:编写简单的 Dataloaders 来简单地获取所有(需要的)字段.如果您有一个客户端，很可能对于大多数实体，客户端无论如何都会获取所有字段，否则它们将不会成为您 API 的一部分，对吗?然后使用诸如查询内省之类的东西来衡量慢查询，然后找出到底哪个字段是慢的.然后你只优化缓慢的事情(例如参见我的回答 此处 优化了单个用例).如果您是大型电子商务平台，请不要为此使用 Dataloader.构建更智能的东西，不要使用 JavaScript.
My suggestion is: Write trivial Dataloaders that simply fetch all (needed) fields. If you have one client it is very likely that for most entities the client fetches all fields anyways, otherwise they would not be part of you API, right? Then use something like query introsprection to measure slow queries and then find out which field exactly is slow. Then you optimise only the slow thing (see for example my answer here that optimises a single use case). And if you are a big ecomerce platform please don't use Dataloader for this. Build something smarter and don't use JavaScript.
这篇关于使用 Facebook 的 DataLoader 传递参数的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
