Cassandra .csv导入错误：批量太大 [英] Cassandra .csv import error:batch too large

问题描述

我试图从.csv文件导入数据到Cassandra 3.2.1通过复制命令。在文件只有299行与14列。我收到错误：

无法导入299行：InvalidRequest - code = 2200 [无效的查询] message =批量太大 / p>

我使用以下复制comand并试图增加批量大小：

table（Col1，Col2，...）from'file.csv'
with delimiter =';'and header = true and MAXBATCHSIZE = 5000;

我认为299行没有太多导入到cassandra或我错了？

解决方案

您遇到的错误是一个服务器端错误消息，称批量插入的大小（以字节计数）过大。

此批处理大小在 cassandra.yaml 文件中定义：

 ＃对超过此值的任何批处理大小进行日志警告。默认情况下为每批5kb。 
＃应该注意增加此阈值的大小，因为它可能导致节点不稳定。 
 batch_size_warn_threshold_in_kb：5 
 
＃超过此值的任何批次失败。默认为50kb（10x警告阈值）。 
 batch_size_fail_threshold_in_kb：50 
  

如果插入很多大列迅速达到这个阈值。尝试将 MAXBATCHSIZE 减少到200。

有关COPY选项的更多信息 此处

I'm trying to import data from a .csv file to Cassandra 3.2.1 via copy command.In the file are only 299 rows with 14 columns. I get the Error:

Failed to import 299 rows: InvalidRequest - code=2200 [Invalid query] message="Batch too large"

I used the following copy comand and tryied to increase the batch size:

copy table (Col1,Col2,...)from 'file.csv' with delimiter =';' and header = true and MAXBATCHSIZE = 5000;

I think 299 rows are not too much to import to cassandra or i am wrong?

解决方案

The error you're encountering is a server-side error message, saying that the size (in term of bytes count) of your batch insert is too large.

This batch size is defined in the cassandra.yaml file:

# Log WARN on any batch size exceeding this value. 5kb per batch by default.
# Caution should be taken on increasing the size of this threshold as it can lead to node instability.
batch_size_warn_threshold_in_kb: 5

# Fail any batch exceeding this value. 50kb (10x warn threshold) by default.
batch_size_fail_threshold_in_kb: 50

If you insert a lot of big columns (in size) you may reach quickly this threshold. Try to reduce MAXBATCHSIZE to 200.

More info on COPY options here

这篇关于Cassandra .csv导入错误：批量太大的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！