如何使用提取处理流数据? [英] How to handle streaming data using fetch?

问题描述

我已经使用async在处理node.js进程的输出流方面取得了巨大的成功，但是我一直在努力获取希望通过浏览器 fetch 工作"的东西.API.

I have used async for with great success in handling output streams from processes with node.js, but I'm struggling to get something that I was hoping could "just work" with the browser fetch API.

这非常适合异步处理来自流程的输出流的大块:

This works great to async'ly handle chunks of output streaming from a process:

for await (const out of proc.child.stdout) {
  ...
}

(当然是在异步函数的上下文中)

(in an async function context here of course)

我试图在浏览器中执行类似的操作，在该浏览器中，我想在服务器将数据发送给我时获得对数据的访问权限.

I tried to do something similar in a browser where I want to gain access to the data while it is being sent to me from the server.

for await (const chunk of (await fetch('/data.jsonl')).body) {
  console.log('got', chunk);
}

这在Chrome中不起作用(未捕获的TypeError :(中间值).body不可异步迭代).

This does not work in Chrome (Uncaught TypeError: (intermediate value).body is not async iterable).

对于我的用例来说，这不是必需的，因此我现在仅在客户端代码中使用 let data = await(await fetch(datapath)).text(); .这类似于在等待的提取操作上通常使用 .json()而不是 .text()，因此，直到接收到整个响应后，才能开始处理.浏览器.由于明显的原因，这是不理想的.

For my use case this is not necessary, so I am simply using let data = await (await fetch(datapath)).text(); in my client code for now. This is analogous to the typical use of .json() instead of .text() on the awaited fetch, so no processing can begin until the entire response is received by the browser. This is not ideal for obvious reasons.

我正在查看 Oboe.js (我认为相关的展示位于

I was looking at Oboe.js (I think the relevant impl is somewhere near here) which pretty much deals with this but its internals are fairly ugly so it looks like that might be the only way to do this for now?

如果未实现异步迭代(意味着尚不能使用异步)，是否还有另一种以实际方式使用ReadableStream的方法?

If async iteration isn't implemented (meaning async for cannot be used yet) isn't there another way to use the ReadableStream in a practical way?

推荐答案

不幸的是，尽管已在规范中，但尚未实现异步可迭代支持.相反，您可以手动进行迭代，如规格说明中的示例所示.(在此答案中，我将为您将示例转换为async/await.)

Unfortunately async iterable support is not yet implemented, despite being in the spec. Instead you can manually iterate, as shown in this example from the spec. (I'll convert examples to async/await for you in this answer.)

const reader = response.body.getReader();
const { value, done } = reader.read();

if (done) {
  console.log("The stream was already closed!");
} else {
  console.log(value);
}

您可以使用递归或循环重复执行此操作，如另一个例子:

You can use recursion or a loop to do this repeatedly, as in this other example:

async function readAllChunks(readableStream) {
  const reader = readableStream.getReader();
  const chunks = [];

  let done, value;
  while (!done) {
    ({ value, done } = await reader.read());
    if (done) {
      return chunks;
    }
    chunks.push(value);
  }
}

console.log(await readAllChunks(response.body));

这篇关于如何使用提取处理流数据?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！