Cassandra CQL3复合键不是由Hadoop还原器编写的 [英] Cassandra CQL3 composite key not written by Hadoop reducer

问题描述

我使用Cassandra 1.2.8，并且有几个Hadoop MapReduce作业，从一些CQL3表读取行，并将结果写回另一个CQL3表。

I'm using Cassandra 1.2.8, and have several Hadoop MapReduce jobs, that read rows from some CQL3 tables and write result back to another CQL3 tables.

如果输出CQL3表包含复合键，则复合键字段的值不会由reducer写入 - 而是在cqlsh中执行select查询时，会看到这些字段的空值。

If output CQL3 tables contain composite key, values of the composite key fields are not written by reducer - instead I see empty values for those fields, while performing select query in cqlsh. If the primary key is not composite, everything works correctly.

输出CQL3表与复合键的示例：

Example of the output CQL3 table with composite key:

CREATE TABLE events_by_type_with_source (
    event_type_id ASCII,
    period ASCII,
    date TIMESTAMP,
    source_name ASCII,
    events_number COUNTER,
    PRIMARY KEY((event_type_id, period), date, source_name)
);

我的输出查询是：
UPDATE events_by_type_with_source SET events_number = events_number +？

My output query is: UPDATE events_by_type_with_source SET events_number = events_number + ?

我的Reducer函数看起来像这样：

My Reducer function looks like this:

public void reduce(BytesWritable key, Iterable<BytesWritable> values, Context context) {
     ...
    private final Map<String, ByteBuffer> keys = new HashMap<>();
    ...
    keys.put(COLUMN_EVENT_TYPE_ID, eventTypeIdByteBuffer);
    keys.put(COLUMN_SOURCE_NAME, sourceNameByteBuffer);
    keys.put(COLUMN_DATE, dateByteBuffer);
    keys.put(COLUMN_PERIOD, periodByteBuffer);
    ...
    context.write(keys, Arrays.asList(countByteBuffer));

}

结果是： / p>

The result is:

cqlsh:keyspace1> select * from dd_events_by_type_with_source ;

 event_type_id | period | date                     | source_name | events_number
---------------+--------+--------------------------+-------------+---------------
               |        | 2013-08-01 00:00:00+0000 |           A |            24
               |        | 2013-08-26 00:00:00+0000 |           A |            24
               |        | 2013-08-27 00:00:00+0000 |           A |            24
               |        | 2013-08-27 08:00:00+0000 |           A |            24

正如你所看到的，event_type_id和period字段是空的，即使我把非空有效ASCII字符串在reducer中。

As you can see, event_type_id and period fields are empty, even though I put not-empty valid ASCII strings in the reducer.

任何想法如何解决这个问题？

Any idea how to fix this?

推荐答案

在1.2.2之前的已知问题Cassandra： https://issues.apache.org/jira / browse / CASSANDRA-5949

This is a known issue in pre-1.2.10 Cassandra: https://issues.apache.org/jira/browse/CASSANDRA-5949

根据以前的发布计划，我预计1.2.10将在2013年9月底前可用。似乎不存在于Cassandra 2.0中。

Based on previous release schedule, I would expect 1.2.10 to be available near the end of September 2013. This issue does not appear to exist in Cassandra 2.0.

这篇关于Cassandra CQL3复合键不是由Hadoop还原器编写的的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！