Spring Stomp over Websocket:流式传输大文件 [英] Spring Stomp over Websocket: Stream large files

问题描述

我在网页中的 SockJs 客户端，发送帧大小为 16K 的消息.消息大小限制决定了我可以传输的文件的最大大小.

My SockJs client in webpage, sends message with a frame size of 16K. The message size limit is what determines the max size of the file that I can transfer.

以下是我在文档中找到的内容.

Below is what I found in the doc.

/**
 * Configure the maximum size for an incoming sub-protocol message.
 * For example a STOMP message may be received as multiple WebSocket messages
 * or multiple HTTP POST requests when SockJS fallback options are in use.
 *
 * <p>In theory a WebSocket message can be almost unlimited in size.
 * In practice WebSocket servers impose limits on incoming message size.
 * STOMP clients for example tend to split large messages around 16K
 * boundaries. Therefore a server must be able to buffer partial content
 * and decode when enough data is received. Use this property to configure
 * the max size of the buffer to use.
 *
 * <p>The default value is 64K (i.e. 64 * 1024).
 *
 * <p><strong>NOTE</strong> that the current version 1.2 of the STOMP spec
 * does not specifically discuss how to send STOMP messages over WebSocket.
 * Version 2 of the spec will but in the mean time existing client libraries
 * have already established a practice that servers must handle.
 */
public WebSocketTransportRegistration setMessageSizeLimit(int messageSizeLimit) {
    this.messageSizeLimit = messageSizeLimit;
    return this;
}

我的问题:我可以设置部分消息传递，以便文件逐个传输，而不是像现在那样作为单个消息传输吗?

MY QUESTION: Can I setup a partial messaging so that a file is transferred part by part and is not getting transferred as a single message as it is been done now?

更新:仍在寻找部分消息传递的解决方案同时现在使用 HTTP 处理大消息(这是我的应用程序中的文件上传/下载).

Update: Still looking for a solution with partial messaging Meanwhile using HTTP now for large messages (which is file uploads/downloads in my application).

推荐答案

我是否可以设置部分消息传递，以便文件逐个传输，而不是像现在那样作为单个消息传输?

Can I setup a partial messaging so that a file is transferred part by part and is not getting transferred as a single message as it is been done now?

是的.这是我的 Spring boot 实验项目中的相关配置 - 基本上 UploadWSHandler 已注册并且 WebSocketTransportRegistration.setMessageSizeLimit 已设置.

Yes. Here is the relevant Config from my Spring boot experimental project - basically UploadWSHandler is registered and WebSocketTransportRegistration.setMessageSizeLimit is set.

@Configuration
@EnableWebSocket
public class WebSocketConfig extends AbstractWebSocketMessageBrokerConfigurer implements WebSocketConfigurer {
    public void registerWebSocketHandlers(WebSocketHandlerRegistry registry) {
        registry.addHandler(new UploadWSHandler(), "/binary");
    }
    
    @Override
    public void configureWebSocketTransport(WebSocketTransportRegistration registration) {
        registration.setMessageSizeLimit(50 * 1024 * 1024);
    }
}

UploadWShandler 如下.抱歉这里代码太多 - 要点

The UploadWShandler is as follows. Sorry too much code here - Key points

• supportsPartialMessage 返回 true.
• handleBinaryMessage 将使用部分消息多次调用，因此我们需要组装字节.所以 afterConnectionEstablished 使用 websocket URL 查询建立一个身份.但是您不必使用这种机制.我选择这种机制的原因是为了保持客户端简单，以便我只调用一次 webSocket.send(files[0]) ，即我没有在 javascript 端切片文件 blob 对象.(旁注:我想在客户端使用普通的 websocket - 没有 stomp/socks)
• 内部客户端分块机制提供message.isLast()最后一条消息
• 出于演示目的，我将其写入文件系统并使用 FileUploadInFlight 在内存中累积这些字节，但您不必这样做，并且可以随时在其他地方流式传输.

• supportsPartialMessage which returns true.
• handleBinaryMessage will be called multiple times with partial message so we need to assemble the bytes. So afterConnectionEstablished establishes an identity using the websocket URL query. But you don't have to use this mechanism. The reason why I choose this mechanism is to keep the client side simple so that I call webSocket.send(files[0]) only once i.e. I am not slicing the file blob object on the javascript side. (side point: I want to use plain websocket on client side - no stomp/socks)
• The iternal client side chunking mechanism provides message.isLast() last message
• For demo purposes I am writing this to file system and accumulating those bytes in memory with FileUploadInFlight but you don't have to do this and can stream somewhere else as you go.

public class UploadWSHandler extends BinaryWebSocketHandler {

    Map<WebSocketSession, FileUploadInFlight> sessionToFileMap = new WeakHashMap<>();

    @Override
    public boolean supportsPartialMessages() {
        return true;
    }

    @Override
    protected void handleBinaryMessage(WebSocketSession session, BinaryMessage message) throws Exception {
        ByteBuffer payload = message.getPayload();
        FileUploadInFlight inflightUpload = sessionToFileMap.get(session);
        if (inflightUpload == null) {
            throw new IllegalStateException("This is not expected");
        }
        inflightUpload.append(payload);

        if (message.isLast()) {
            Path basePath = Paths.get(".", "uploads", UUID.randomUUID().toString());
            Files.createDirectories(basePath);
            FileChannel channel = new FileOutputStream(
                    Paths.get(basePath.toString() ,inflightUpload.name).toFile(), false).getChannel();
            channel.write(ByteBuffer.wrap(inflightUpload.bos.toByteArray()));
            channel.close();
            session.sendMessage(new TextMessage("UPLOAD "+inflightUpload.name));
            session.close();
            sessionToFileMap.remove(session);
        }
        String response = "Upload Chunk: size "+ payload.array().length;
        System.out.println(response);

    }

    @Override
    public void afterConnectionEstablished(WebSocketSession session) throws Exception {
        sessionToFileMap.put(session, new FileUploadInFlight(session));
    }

    static class FileUploadInFlight {
        String name;
        String uniqueUploadId;
        ByteArrayOutputStream bos = new ByteArrayOutputStream();
        /**
         * Fragile constructor - beware not prod ready
         * @param session
         */
        FileUploadInFlight(WebSocketSession session) {
            String query = session.getUri().getQuery();
            String uploadSessionIdBase64 = query.split("=")[1];
            String uploadSessionId = new String(Base64Utils.decodeUrlSafe(uploadSessionIdBase64.getBytes()));
            System.out.println(uploadSessionId);
            List<String> sessionIdentifiers = Splitter.on("\\").splitToList(uploadSessionId);
            String uniqueUploadId = session.getRemoteAddress().toString()+sessionIdentifiers.get(0);
            String fileName = sessionIdentifiers.get(1);
            this.name = fileName;
            this.uniqueUploadId = uniqueUploadId;
        }
        public void append(ByteBuffer byteBuffer) throws IOException{
            bos.write(byteBuffer.array());
        }
    }
}

顺便说一句，一个工作项目也是 sprint-boot-with-websocked-chunking-assembly-and-fetch 在 with-websocked-chunking-assembly-and-fetch 分支

BTW a working project is also sprint-boot-with-websocked-chunking-assembly-and-fetch in with-websocked-chunking-assembly-and-fetch branch

这篇关于Spring Stomp over Websocket:流式传输大文件的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！