特效和异步IO的细节 [英] Cats-effect and asynchronous IO specifics

问题描述

几天来，我一直将注意力集中在猫效应和IO上.而且我觉得我对这种效果有误解，或者只是错过了要点.

For few days I have been wrapping my head around cats-effect and IO. And I feel I have some misconceptions about this effect or simply I missed its point.

1. 首先-如果IO可以取代Scala的Future，我们如何创建异步IO任务?使用IO.shift?使用IO.async吗? IO.delay是同步还是异步?我们可以使用类似Async[F].delay(...)的代码来执行通用异步任务吗?还是当我们使用unsafeToAsync或unsafeToFuture调用IO时发生异步?
2. 猫效应中异步和并发的意义是什么?他们为什么分开?
3. IO是绿色线程吗?如果是，为什么有猫效果的纤维物体?据我了解，光纤是绿色线程，但是文档声称我们可以将IO视为绿色线程.

1. First of all - if IO can replace Scala's Future, how can we create an async IO task? Using IO.shift? Using IO.async? Is IO.delay sync or async? Can we make a generic async task with code like this Async[F].delay(...)? Or async happens when we call IO with unsafeToAsync or unsafeToFuture?
2. What's the point of Async and Concurrent in cats-effect? Why they are separated?
3. Is IO a green thread? If yes, why is there a Fiber object in cats-effect? As I understand the Fiber is the green thread, but docs claim we can think of IOs as green threads.

我希望对其中的任何内容进行一些澄清，因为我未能理解这些内容上的cats-effect文档，并且互联网没有那么大的帮助...

I would appreciate some clarifing on any of this as I have failed comprehending cats-effect docs on those and internet was not that helpfull...

推荐答案

如果IO可以取代Scala的Future，我们如何创建异步IO任务

if IO can replace Scala's Future, how can we create an async IO task

首先，我们需要弄清什么是异步任务.通常 async 的意思是不阻塞OS线程"，但是由于您提到的是Future，所以有点模糊.说，如果我写了:

First, we need to clarify what is meant as an async task. Usually async means "does not block the OS thread", but since you're mentioning Future, it's a bit blurry. Say, if I wrote:

Future { (1 to 1000000).foreach(println) }

它不是 async ，因为它是一个阻塞循环并阻塞了输出​​，但是它可能会在由隐式ExecutionContext管理的另一个OS线程上执行.等效的cats-effect代码为:

it would not be async, as it's a blocking loop and blocking output, but it would potentially execute on a different OS thread, as managed by an implicit ExecutionContext. The equivalent cats-effect code would be:

for {
  _ <- IO.shift
  _ <- IO.delay { (1 to 1000000).foreach(println) }
} yield ()

(不是较短的版本)

所以

• IO.shift用于更改线程/线程池. Future会在每次操作中执行此操作，但这并不是从性能上考虑.
• IO.delay {...}(又称IO { ... })确实使 NOT 保持异步，并且 NOT 不会切换线程.它用于从同步副作用API创建简单的IO值

• IO.shift is used to maybe change thread / thread pool. Future does it on every operation, but it's not free performance-wise.
• IO.delay { ... } (a.k.a. IO { ... }) does NOT make anything async and does NOT switch threads. It's used to create simple IO values from synchronous side-effecting APIs

现在，让我们回到真正的异步.这里要了解的是:

Now, let's get back to true async. The thing to understand here is this:

    每个异步计算都可以表示为采用回调的函数.

     Every async computation can be represented as a function taking callback.

无论您使用的是返回Future或Java的CompletableFuture的API，还是NIO CompletionHandler之类的东西，都可以转换为回调.这就是IO.async的作用:您可以将任何带回调的功能转换为IO.并以此类推:

Whether you're using API that returns Future or Java's CompletableFuture, or something like NIO CompletionHandler, it all can be converted to callbacks. This is what IO.async is for: you can convert any function taking callback to an IO. And in case like:

for {
  _ <- IO.async { ... }
  _ <- IO(println("Done"))
} yield ()

Done仅在(以及如果)...中的计算回调时才打印.您可以将其视为阻塞绿色线程，而不是OS线程.

Done will be only printed when (and if) the computation in ... calls back. You can think of it as blocking the green thread, but not OS thread.

所以

• IO.async用于将任何已经异步的计算转换为IO.
• IO.delay用于将任何完全同步计算转换为IO.
• 具有真正异步计算的代码的行为就像阻塞了一个绿色线程.

• IO.async is for converting any already asynchronous computation to IO.
• IO.delay is for converting any completely synchronous computation to IO.
• The code with truly asynchronous computations behaves like it's blocking a green thread.

使用Future时，最接近的类比是创建scala.concurrent.Promise并返回p.future.

The closest analogy when working with Futures is creating a scala.concurrent.Promise and returning p.future.

当我们使用unsafeToAsync或unsafeToFuture调用IO时，还是发生异步?

Or async happens when we call IO with unsafeToAsync or unsafeToFuture?

排序.使用IO，什么都不会发生，除非您调用其中之一(或使用IOApp).但是IO不能保证您将在不同的OS线程上执行，甚至不能异步执行，除非您通过IO.shift或IO.async明确要求这样做.

Sort of. With IO, nothing happens unless you call one of these (or use IOApp). But IO does not guarantee that you would execute on a different OS thread or even asynchronously unless you asked for this explicitly with IO.shift or IO.async.

例如，您可以保证随时进行线程切换. (IO.shift *> myIO).unsafeRunAsyncAndForget().这完全有可能是因为myIO直到被询问时才执行，无论您是val myIO还是def myIO.

You can guarantee thread switching any time with e.g. (IO.shift *> myIO).unsafeRunAsyncAndForget(). This is possible exactly because myIO would not be executed until asked for it, whether you have it as val myIO or def myIO.

但是，您不能神奇地将阻塞操作转换为非阻塞操作.既不能使用Future也不能使用IO.

You cannot magically transform blocking operations into non-blocking, however. That's not possible neither with Future nor with IO.

猫效应"中的异步和并发"有什么意义?他们为什么分开?

What's the point of Async and Concurrent in cats-effect? Why they are separated?

Async和Concurrent(和Sync)是类型类.它们的设计使程序员可以避免被锁定在cats.effect.IO上，并且可以为您提供支持您选择的内容的API，例如monix Task或Scalaz 8 ZIO，甚至是monad转换器类型，例如OptionT[Task, *something*]. fs2，monix和http4s之类的库利用它们来为您提供更多使用它们的选择.

Async and Concurrent (and Sync) are type classes. They are designed so that programmers can avoid being locked to cats.effect.IO and can give you API that supports whatever you choose instead, such as monix Task or Scalaz 8 ZIO, or even monad transformer type such as OptionT[Task, *something*]. Libraries like fs2, monix and http4s make use of them to give you more choice of what to use them with.

Concurrent在Async之上添加了其他内容，其中最重要的是.cancelable和.start.它们与Future没有直接的类比，因为它根本不支持取消.

Concurrent adds extra things on top of Async, most important of them being .cancelable and .start. These do not have a direct analogy with Future, since that does not support cancellation at all.

.cancelable是.async的一个版本，允许您还指定一些逻辑来取消要包装的操作.一个常见的示例是网络请求-如果您不再对结果感兴趣，则可以在不等待服务器响应的情况下中止它们，而不会在读取响应时浪费任何套接字或处理时间.您可能永远都不会直接使用它，但是它有它的位置.

.cancelable is a version of .async that allows you to also specify some logic to cancel the operation you're wrapping. A common example is network requests - if you're not interested in results anymore, you can just abort them without waiting for server response and don't waste any sockets or processing time on reading the response. You might never use it directly, but it has it's place.

但是，如果您不能取消可取消的操作，那么有什么好处呢?这里的主要观察结果是您不能从内部取消操作.其他人必须做出该决定，并且这会与操作本身(类型类获得其名称的地方)同时发生.这就是.start出现的地方.简而言之，

But what good are cancelable operations if you can't cancel them? Key observation here is that you cannot cancel an operation from within itself. Somebody else has to make that decision, and that would happen concurrently with the operation itself (which is where the type class gets its name). That's where .start comes in. In short,

     .start是绿色线程的显式分支.

      .start is an explicit fork of a green thread.

执行someIO.start类似于执行val t = new Thread(someRunnable); t.start()，除了现在是绿色.而且Fiber本质上是Thread API的精简版本:您可以执行.join，就像Thread#join()一样，但是它不会阻塞OS线程.和.cancel，这是.interrupt()的安全版本.

Doing someIO.start is akin to doing val t = new Thread(someRunnable); t.start(), except it's green now. And Fiber is essentially a stripped down version of Thread API: you can do .join, which is like Thread#join(), but it does not block OS thread; and .cancel, which is safe version of .interrupt().

请注意，还有其他分叉绿色线程的方法.例如，执行并行操作:

Note that there are other ways to fork green threads. For example, doing parallel operations:

val ids: List[Int] = List.range(1, 1000)
def processId(id: Int): IO[Unit] = ???
val processAll: IO[Unit] = ids.parTraverse_(processId)

将把所有ID分叉到绿色线程，然后将它们全部加入.或使用.race:

will fork processing all IDs to green threads and then join them all. Or using .race:

val fetchFromS3: IO[String] = ???
val fetchFromOtherNode: IO[String] = ???

val fetchWhateverIsFaster = IO.race(fetchFromS3, fetchFromOtherNode).map(_.merge)

将并行执行提取，使您的第一个结果完成，并自动取消较慢的提取.因此，执行.start并使用Fiber并不是派生更多绿色线程的唯一方法，而是最明确的方法.答案是:

will execute fetches in parallel, give you first result completed and automatically cancel the fetch that is slower. So, doing .start and using Fiber is not the only way to fork more green threads, just the most explicit one. And that answers:

IO是绿色线程吗?如果是，为什么有猫效果的纤维物体?据我了解，光纤是绿色线程，但是文档声称我们可以将IO视为绿色线程.

Is IO a green thread? If yes, why is there a Fiber object in cats-effect? As I understand the Fiber is the green thread, but docs claim we can think of IOs as green threads.

• IO就像一个绿色线程，这意味着您可以让许多线程并行运行而没有OS线程的开销，并且理解代码的行为就像是在阻塞要计算的结果一样.

  • IO is like a green thread, meaning you can have lots of them running in parallel without overhead of OS threads, and the code in for-comprehension behaves as if it was blocking for the result to be computed.

    Fiber是用于控制显式分叉(等待完成或取消)的绿色线程的工具.

    Fiber is a tool for controlling green threads explicitly forked (waiting for completion or cancelling).

    这篇关于特效和异步IO的细节的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！