分片 [shardId-000000000000] 未关闭.如果我们在重新分片操作正在进行时构建了分片列表，就会发生这种情况 [英] Shard [shardId-000000000000] is not closed. This can happen if we constructed the list of shards while a reshard operation was in progress

问题描述

我在从 Amazon kinesis Stream 获取数据时收到此错误.我正在做以下步骤

I am getting this error while fetching data from Amazon kinesis Stream. I am doing below steps

1. 创建亚马逊 kinesis Steam
2. 使用AmazonKinesisClient 的putRecord api 放入数据.
3. 然后使用 Worker Of KCL 库从流中获取数据.

1. creating amazon kinesis Steam
2. put the data using putRecord api of AmazonKinesisClient.
3. Then using Worker Of KCL library to get the data from stream.

推荐答案

有几种可能性.

1. 在您下令创建流后，您是否等待完成的时间足够长?有时，创建分片可能需要 10 分钟.由于您设法使用了 putRecord 方法，因此流和分片应该处于活动状态.
2. 您是否正确配置了 DynamoDB?我假设您将它用于 Kinesis 应用程序同步.查看DynamoDB 表的内部记录.所有列(即leaseKey、checkpoint、leaseOwner、parentShardId 等)是否正确填写?也许数据库中的数据是旧的，所以您可能正在访问一个关闭的分片?

1. After you ordered to create the stream, did you wait long enough for completion? Sometimes, it may took 10 minutes for a shard to be created. Since you managed to use putRecord method, the stream and shard should be active.
2. Did you configure the DynamoDB correctly? I assume you are using it for your Kinesis Application synchronization. See inside the records of the DynamoDB table. Are all columns (ie. leaseKey, checkpoint, leaseOwner, parentShardId etc.) correctly filled? Maybe the data in the database is old, so you may be accessing a closed shard?

你提到的例外在这里:https://github.com/awslabs/amazon-kinesis-client/blob/master/src/main/java/com/amazonaws/services/kinesis/clientlibrary/lib/worker/ShardSyncer.java#L193

尝试删除 DynamoDB 表并让 Kinesis 应用程序 (KCL) 创建一个全新的表.链接:https://console.aws.amazon.com/dynamodb/

Try deleting the DynamoDB table and let the Kinesis Application (KCL) create a brand new one. Link: https://console.aws.amazon.com/dynamodb/

这篇关于分片 [shardId-000000000000] 未关闭.如果我们在重新分片操作正在进行时构建了分片列表，就会发生这种情况的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！