Java TCP 套接字:数据传输很慢 [英] Java TCP socket: data transfer is slow
问题描述
我使用 ServerSocket 设置了一个服务器，并使用客户端计算机连接到它.它们通过交换机直接联网，ping 时间<1ms.
I set up a server with a ServerSocket, connect to it with a client machine. They're directly networked through a switch and the ping time is <1ms.
现在，我尝试通过套接字的输出流将大量"数据从客户端推送到服务器.传输 0.6Gb 需要 23 分钟.我可以通过 scp 在几秒钟内推送一个更大的文件.
Now, I try to push a "lot" of data from the client to the server through the socket's output stream. It takes 23 minutes to transfer 0.6Gb. I can push a much larger file in seconds via scp.
知道我做错了什么吗?我基本上只是在套接字上循环并调用 writeInt .速度问题与数据来自何处无关，即使我只是发送一个常量整数而不是从磁盘读取.
Any idea what I might be doing wrong? I'm basically just looping and calling writeInt on the socket. The speed issue doesn't matter where the data is coming from, even if I'm just sending a constant integer and not reading from disk.
我尝试将两侧的发送和接收缓冲区设置为 4Mb，没有骰子.我为读取器和写入器使用缓冲流，没有骰子.
I tried setting the send and receive buffer on both sides to 4Mb, no dice. I use a buffered stream for the reader and writer, no dice.
我错过了什么吗?
代码
这是我制作插座的地方
System.out.println("Connecting to " + hostname);
serverAddr = InetAddress.getByName(hostname);
// connect and wait for port assignment
Socket initialSock = new Socket();
initialSock.connect(new InetSocketAddress(serverAddr, LDAMaster.LDA_MASTER_PORT));
int newPort = LDAHelper.readConnectionForwardPacket(new DataInputStream(initialSock.getInputStream()));
initialSock.close();
initialSock = null;
System.out.println("Forwarded to " + newPort);
// got my new port, connect to it
sock = new Socket();
sock.setReceiveBufferSize(RECEIVE_BUFFER_SIZE);
sock.setSendBufferSize(SEND_BUFFER_SIZE);
sock.connect(new InetSocketAddress(serverAddr, newPort));
System.out.println("Connected to " + hostname + ":" + newPort + " with buffers snd=" + sock.getSendBufferSize() + " rcv=" + sock.getReceiveBufferSize());
// get the MD5s
try {
byte[] dataMd5 = LDAHelper.md5File(dataFile),
indexMd5 = LDAHelper.md5File(indexFile);
long freeSpace = 90210; // ** TODO: actually set this **
output = new DataOutputStream(new BufferedOutputStream(sock.getOutputStream()));
input = new DataInputStream(new BufferedInputStream(sock.getInputStream()));
这里是我进行服务器端连接的地方:
Here's where I do the server-side connection:
ServerSocket servSock = new ServerSocket();
servSock.setSoTimeout(SO_TIMEOUT);
servSock.setReuseAddress(true);
servSock.bind(new InetSocketAddress(LDA_MASTER_PORT));
int currPort = LDA_START_PORT;
while (true) {
try {
Socket conn = servSock.accept();
System.out.println("Got a connection. Sending them to port " + currPort);
clients.add(new MasterClientCommunicator(this, currPort));
clients.get(clients.size()-1).start();
Thread.sleep(500);
LDAHelper.sendConnectionForwardPacket(new DataOutputStream(conn.getOutputStream()), currPort);
currPort++;
} catch (SocketTimeoutException e) {
System.out.println("Done listening. Dispatching instructions.");
break;
}
catch (IOException e) {
e.printStackTrace();
}
catch (Exception e) {
e.printStackTrace();
}
}
好的，这是我要发送超过 0.6Gb 数据的地方.
Alright, here's where I'm shipping over ~0.6Gb of data.
public static void sendTermDeltaPacket(DataOutputStream out, TIntIntHashMap[] termDelta) throws IOException {
long bytesTransferred = 0, numZeros = 0;
long start = System.currentTimeMillis();
out.write(PACKET_TERM_DELTA); // header
out.flush();
for (int z=0; z < termDelta.length; z++) {
out.writeInt(termDelta[z].size()); // # of elements for each term
bytesTransferred += 4;
}
for (int z=0; z < termDelta.length; z++) {
for (int i=0; i < termDelta[z].size(); i++) {
out.writeInt(1);
out.writeInt(1);
}
}
到目前为止看起来很简单......
It seems pretty straightforward so far...
推荐答案
当您传输大量数据时，您不想想写入单个字节.
You do not want to write single bytes when you are transferring large amounts of data.
import java.io.FileInputStream;
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
import java.net.ServerSocket;
import java.net.Socket;
public class Transfer {
public static void main(String[] args) {
final String largeFile = "/home/dr/test.dat"; // REPLACE
final int BUFFER_SIZE = 65536;
new Thread(new Runnable() {
public void run() {
try {
ServerSocket serverSocket = new ServerSocket(12345);
Socket clientSocket = serverSocket.accept();
long startTime = System.currentTimeMillis();
byte[] buffer = new byte[BUFFER_SIZE];
int read;
int totalRead = 0;
InputStream clientInputStream = clientSocket.getInputStream();
while ((read = clientInputStream.read(buffer)) != -1) {
totalRead += read;
}
long endTime = System.currentTimeMillis();
System.out.println(totalRead + " bytes read in " + (endTime - startTime) + " ms.");
} catch (IOException e) {
}
}
}).start();
new Thread(new Runnable() {
public void run() {
try {
Thread.sleep(1000);
Socket socket = new Socket("localhost", 12345);
FileInputStream fileInputStream = new FileInputStream(largeFile);
OutputStream socketOutputStream = socket.getOutputStream();
long startTime = System.currentTimeMillis();
byte[] buffer = new byte[BUFFER_SIZE];
int read;
int readTotal = 0;
while ((read = fileInputStream.read(buffer)) != -1) {
socketOutputStream.write(buffer, 0, read);
readTotal += read;
}
socketOutputStream.close();
fileInputStream.close();
socket.close();
long endTime = System.currentTimeMillis();
System.out.println(readTotal + " bytes written in " + (endTime - startTime) + " ms.");
} catch (Exception e) {
}
}
}).start();
}
}
这在我的机器上在短短 19 秒内复制了 1 GiB 的数据.这里的关键是使用 InputStream.read 和 OutputStream.write 方法接受一个字节数组作为参数.缓冲区的大小并不重要，它应该比 5 大一点. 用上面的 BUFFER_SIZE 试验看看它如何影响速度，但也要记住它可能因您正在运行的每台机器而异这个程序上.64 KiB 似乎是一个不错的折衷方案.
This copies 1 GiB of data in short over 19 seconds on my machine. The key here is using the InputStream.read and OutputStream.write methods that accept a byte array as parameter. The size of the buffer is not really important, it just should be a bit larger than, say, 5. Experiment with BUFFER_SIZE above to see how it effects the speed but also keep in mind that it probably is different for every machine you are running this program on. 64 KiB seem to be a good compromise.
这篇关于Java TCP 套接字:数据传输很慢的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
