在 Haskell 中使用 GNU/Linux 系统调用`splice`进行零拷贝 Socket 到 Socket 数据传输 [英] Using GNU/Linux system call `splice` for zero-copy Socket to Socket data transfers in Haskell
问题描述
更新:尼莫先生的回答帮助解决了这个问题！下面的代码包含修复！请参阅下面的 nb False 和 nb True 调用.
Update: Mr. Nemo's answer helped solve the problem! The code below contains the fix! See the nb False and nb True calls below.
还有一个名为 splice(它具有最知名的套接字到套接字数据传输循环的操作系统特定和可移植实现).
There is also a new Haskell package called splice (, which has OS-specific and portable implementations of best known socket to socket data transfer loops).
我有以下(Haskell)代码:
I have the following (Haskell) code:
#ifdef LINUX_SPLICE
#include <fcntl.h>
{-# LANGUAGE CPP #-}
{-# LANGUAGE ForeignFunctionInterface #-}
#endif
module Network.Socket.Splice (
Length
, zeroCopy
, splice
#ifdef LINUX_SPLICE
, c_splice
#endif
) where
import Data.Word
import Foreign.Ptr
import Network.Socket
import Control.Monad
import Control.Exception
import System.Posix.Types
import System.Posix.IO
#ifdef LINUX_SPLICE
import Data.Int
import Data.Bits
import Unsafe.Coerce
import Foreign.C.Types
import Foreign.C.Error
import System.Posix.Internals
#else
import System.IO
import Foreign.Marshal.Alloc
#endif
zeroCopy :: Bool
zeroCopy =
#ifdef LINUX_SPLICE
True
#else
False
#endif
type Length =
#ifdef LINUX_SPLICE
(#type size_t)
#else
Int
#endif
-- | The 'splice' function pipes data from
-- one socket to another in a loop.
-- On Linux this happens in kernel space with
-- zero copying between kernel and user spaces.
-- On other operating systems, a portable
-- implementation utilizes a user space buffer
-- allocated with 'mallocBytes'; 'hGetBufSome'
-- and 'hPut' are then used to avoid repeated
-- tiny allocations as would happen with 'recv'
-- 'sendAll' calls from the 'bytestring' package.
splice :: Length -> Socket -> Socket -> IO ()
splice l (MkSocket x _ _ _ _) (MkSocket y _ _ _ _) = do
let e = error "splice ended"
#ifdef LINUX_SPLICE
(r,w) <- createPipe
print ('+',r,w)
let s = Fd x -- source
let t = Fd y -- target
let c = throwErrnoIfMinus1 "Network.Socket.Splice.splice"
let u = unsafeCoerce :: (#type ssize_t) -> (#type size_t)
let fs = sPLICE_F_MOVE .|. sPLICE_F_MORE
let nb v = do setNonBlockingFD x v
setNonBlockingFD y v
nb False
finally
(forever $ do
b <- c $ c_splice s nullPtr w nullPtr l fs
if b > 0
then c_splice r nullPtr t nullPtr (u b) fs)
else e
(do closeFd r
closeFd w
nb True
print ('-',r,w))
#else
-- ..
#endif
#ifdef LINUX_SPLICE
-- SPLICE
-- fcntl.h
-- ssize_t splice(
-- int fd_in,
-- loff_t* off_in,
-- int fd_out,
-- loff_t* off_out,
-- size_t len,
-- unsigned int flags
-- );
foreign import ccall "splice"
c_splice
:: Fd
-> Ptr (#type loff_t)
-> Fd
-> Ptr (#type loff_t)
-> (#type size_t)
-> Word
-> IO (#type ssize_t)
sPLICE_F_MOVE :: Word
sPLICE_F_MOVE = (#const "SPLICE_F_MOVE")
sPLICE_F_MORE :: Word
sPLICE_F_MORE = (#const "SPLICE_F_MORE")
#endif
注意: 上面的代码现在可以正常工作了！感谢 Nemo，下面的代码不再有效！
Note: The code above now just works! Below is no longer valid thanks to Nemo!
我调用 splice 如上面定义的那样，使用两个打开和连接的套接字(它们已经用于使用套接字 API send 和 recv 调用或转换为句柄并与 hGetLine 和 hPut 一起使用)，我不断得到:
I call splice as defined above with two open and connected sockets (which are already used to transmit minimal amount of handshake data using either the sockets API send and recv calls or converted to handles and used with hGetLine and hPut) and I keep getting:
Network.Socket.Splice.splice: resource exhausted (Resource temporarily unavailable)
在第一个 c_splice 调用点:c_splice 返回 -1 并将一些 errno 设置为一个值 (可能是 EAGAIN) 读取 resource 已用完 |查找时资源暂时不可用.
at the first c_splice call site: c_splice returns -1 and sets some errno to a value (probably EAGAIN) that reads resource exhausted | resource temporarily unavailable when looked up.
我测试了使用不同 Length 值调用 splice:1024、8192.
I tested calling splice with different Length values: 1024, 8192.
推荐答案
我不知道Haskell，但是资源暂时不可用"是EAGAIN.
I don't know Haskell, but "resource temporarily unavailable" is EAGAIN.
它看起来像 Haskell 默认将其套接字设置为非阻塞模式.因此，如果您在没有数据时尝试从其中读取，或者在缓冲区已满时尝试写入，您将失败并返回 EAGAIN.
And it looks like Haskell sets its sockets to non-blocking mode by default. So if you try to read from one when there is no data, or try to write to one when its buffer is full, you will fail with EAGAIN.
弄清楚如何将套接字更改为阻塞模式，我打赌你会解决你的问题.
Figure out how to change the sockets to blocking mode, and I bet you will solve your problem.
[更新]
或者，在尝试读取或写入套接字之前调用 select 或 poll.但是您仍然需要处理 EAGAIN，因为在极少数情况下，Linux select 会指示套接字已准备好，而实际上它并没有.
Alternatively, call select or poll before attempting to read or write the socket. But you still need to handle EAGAIN, because there are rare corner cases where Linux select will indicate a socket is ready when actually it isn't.
这篇关于在 Haskell 中使用 GNU/Linux 系统调用`splice`进行零拷贝 Socket 到 Socket 数据传输的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
