MongoDB区域副本集-每个区域中的主节点? [英] MongoDb regional replica set - primary node in each region?
问题描述
给一个具有区域用户(区域为北美，欧洲，Aisa)的应用程序，您如何创建一个副本集，以允许HK用户写入最近的节点?
Given an application with regional users (regions being North America, Europe, Aisa), how do you create a replica set that allows HK users to write to the nearest nodes?
目标是香港用户可以读写本地节点，而不会遇到写入美国的延迟.我的假设是，我可以在每个区域中建立一个应用程序堆栈，通过副本集在所有区域之间共享数据，并且用户可以连接到其最近的提供程序(全局负载平衡).
The goal is that users in HK read and write to their local nodes and not suffer latencies writing to the US. My assumption was that I could set up an application stack in each region, share data between all regions through the replica set and users could connect to their closest provider (global load balancing).
问题在于副本集仅具有一个主节点，并且所有写操作都必须到达该节点.读取数据具有辅助功能和NEAREST提示是很简单的，但是我想不出一种写解决方案.
The problem is that a replica set has exactly one primary node and all writes must go to that node. Reading data is straight-forward with secondaries and NEAREST hint, but I can't figure out a solution for writing.
这似乎是一个非常可靠的用例，应该加以处理，但不能取得任何进展.解决方案，想法，指向某些来源的指针?
This seems like a pretty solid use case and that it should be handled but cannot make any headway. Solutions, Ideas, Pointers to some source?
推荐答案
感谢@ avanti，@ MarkusWMalhberg-思考如何回应评论将我推向正确的方向.这花了一点时间，所以我会稍微冗长地解释配置.
Thanks @avanti, @MarkusWMalhberg - pondering how to respond to the comments pushed me in the right direction. This took a bit to pull together so I will be a bit verbose explaining the config.
基于用户体验，我们要创建一个Mongo数据库配置，该配置允许读写操作最接近用户.
Focusing on user experience, we want to create a Mongo database configuration that allows reads and writes to occur closest to the user.
假设
- 用户几乎总是在其所在区域读写文档，并且不介意偶尔读取其他区域的数据是否较慢.
- 每个文档都包含一个用于指示其区域的键(为了简化/清晰起见)
很多分片文档都集中在HA/DR上.凭借用户体验和区域合规性，重点放在本地而不是负载分配上.
Much of the sharding documentation focuses on HA/DR. With user experience and regional compliance the focus is on locality instead of load distribution.
此示例将完全忽略HA/DR，读取首选项和写入问题，但是如果POC成熟，则需要解决这些问题.该示例忽略了这些内容，而是为了实现目标:本地读/写.
This example will completely ignore HA/DR, read preferences, and write concerns but these need to be addressed if the POC is matured. The example ignores these in favor of clarity meeting the goal: local reads/writes.
- Operational Segmentation
- Managing Shard Tags
- ReplicaSet Configuration
- Different mongos config db errors
我们知道
- 我们想要一个应用程序数据库，以便所有数据都可用
- 我们希望用户在本地进行读取/写入，因此我们需要在每个用户组附近建立一个数据库；我们需要一个副本集
- 只能对主要副本集节点进行写操作，因此，要使主要节点紧挨每个用户组，我们需要多个副本；分片集群
在标准ReplicaSet和分片知识中，此配置有2个键:
Within standard ReplicaSet and Sharding knowledge, there are 2 keys to this configuration:
- 为区域本地ReplicaSet节点分配优先级，以确保它成为主节点.
- 使用位置感知的分片键标记来确保将数据写入本地分片
分片键可以是任何东西:与有效的负载共享相比，我们只关心用户能够在本地读取/写入数据.
Shard keys can be anything: we are only concerned with users being able to read/write locally in contrast to effective load sharing.
每个集合都必须被分片，否则写入将被分片为零.
Each collection will have to be sharded, or writes will go to shard zero.
#!/usr/bin/env bash
echo ">>> Clean up processes and files from previous runs"
echo ">>> killAll mongod mongos"
killall mongod mongos
echo ">>> Remove db files and logs"
rm -rf data
rm -rf log
# Create the common log directory
mkdir log
echo ">>> Start replica set for shard US-East"
mkdir -p data/shard-US-East/rsMemberEast data/shard-US-East/rsMemberWest
mongod --replSet shard-US-East --logpath "log/shard-US-East-rsMemberEast.log" --dbpath data/shard-US-East/rsMemberEast --port 37017 --fork --shardsvr --smallfiles
mongod --replSet shard-US-East --logpath "log/shard-US-East-rsMemberWest.log" --dbpath data/shard-US-East/rsMemberWest --port 37018 --fork --shardsvr --smallfiles
echo ">>> Sleep 15s to allow US-East replica set to start"
sleep 15
# The US-East replica set member is assigned priority 2 so that it becomes primary
echo ">>> Configure replica set for shard US-East"
mongo --port 37017 << 'EOF'
config = { _id: "shard-US-East", members:[
{ _id : 0, host : "localhost:37017", priority: 2 },
{ _id : 1, host : "localhost:37018" }]};
rs.initiate(config)
EOF
echo ">>> Start replica set for shard-US-West"
mkdir -p data/shard-US-West/rsMemberEast data/shard-US-West/rsMemberWest
mongod --replSet shard-US-West --logpath "log/shard-US-West-rsMemberEast.log" --dbpath data/shard-US-West/rsMemberEast --port 47017 --fork --shardsvr --smallfiles
mongod --replSet shard-US-West --logpath "log/shard-US-West-rsMemberWest.log" --dbpath data/shard-US-West/rsMemberWest --port 47018 --fork --shardsvr --smallfiles
echo ">>> Sleep 15s to allow US-West replica set to start"
sleep 15
# The US-West replica set member is assigned priority 2 so that it becomes primary
echo ">>> Configure replica set for shard-US-West"
mongo --port 47017 << 'EOF'
config = { _id: "shard-US-West", members:[
{ _id : 0, host : "localhost:47017" },
{ _id : 1, host : "localhost:47018", priority: 2 }]};
rs.initiate(config)
EOF
# Shard config servers: should be 3 and all must be up to deploy a shard cluster
# These are the mongos backing store for routing information
echo ">>> Start config servers"
mkdir -p data/config/config-us-east data/config/config-us-west data/config/config-redundant
mongod --logpath "log/cfg-us-east.log" --dbpath data/config/config-us-east --port 57040 --fork --configsvr --smallfiles
mongod --logpath "log/cfg-us-west.log" --dbpath data/config/config-us-west --port 57041 --fork --configsvr --smallfiles
mongod --logpath "log/cfg-redundant.log" --dbpath data/config/config-redundant --port 57042 --fork --configsvr --smallfiles
echo ">>> Sleep 5 to allow config servers to start and stabilize"
sleep 5
# All mongos's must point at the same config server, a coordinator dispatches writes to each
echo ">>> Start mongos"
mongos --logpath "log/mongos-us-east.log" --configdb localhost:57040,localhost:57041,localhost:57042 --port 27017 --fork
mongos --logpath "log/mongos-us-west.log" --configdb localhost:57040,localhost:57041,localhost:57042 --port 27018 --fork
echo ">>> Wait 60 seconds for the replica sets to stabilize"
sleep 60
# Enable sharding on the 'sales' database and 'sales.users' collection
# Every collection in 'sales' must be sharded or the writes will go to shard 0
# Add a shard tag so we can associate shard keys with the tag (region)
# Shard tag range main and max cannot be the same so we use a region id for US-East = 1
# and US-West = 2. sh.addTagRange() is inclusive of minKey and exclusive of maxKey.
# We only need to configure one mongos - config will be propogated to all mongos through
# the config server
echo ">>> Add shards to mongos"
mongo --port 27017 <<'EOF'
db.adminCommand( { addshard : "shard-US-East/"+"localhost:37017" } );
db.adminCommand( { addshard : "shard-US-West/"+"localhost:47017" } );
db.adminCommand({enableSharding: "sales"})
db.adminCommand({shardCollection: "sales.users", key: {region:1}});
sh.addShardTag("shard-US-East", "US-East")
sh.addShardTag("shard-US-West", "US-West")
sh.addTagRange("sales.users", { region: 1 }, { region: 2 }, "US-East")
sh.addTagRange("sales.users", { region: 2 }, { region: 3 }, "US-West")
EOF
测试
使用sh.status()验证我们的配置正确.请注意，分片已正确分配，标记已正确分配，区域分片键也已正确分配.
Testing
Verify our configuration is correct with sh.status(). Note shards are correctly assigned and tags, and regional shard keys are correctly assigned.
[starver@rakshasa RegionalSharding 14:38:50]$ mongo --port 27017 sales
...
rakshasa(mongos-3.0.5)[mongos] sales> sh.status()
sharding version: {
"_id": 1,
"minCompatibleVersion": 5,
"currentVersion": 6,
"clusterId": ObjectId("55fdddc5746e30dc3651cda4")
}
shards:
{ "_id": "shard-US-East", "host": "shard-US-East/localhost:37017,localhost:37018", "tags": [ "US-East" ] }
{ "_id": "shard-US-West", "host": "shard-US-West/localhost:47017,localhost:47018", "tags": [ "US-West" ] }
balancer:
Currently enabled: yes
Currently running: no
Failed balancer rounds in last 5 attempts: 0
Migration Results for the last 24 hours:
1 : Success
databases:
{ "_id": "admin", "partitioned": false, "primary": "config" }
{ "_id": "test", "partitioned": false, "primary": "shard-US-East" }
{ "_id": "sales", "partitioned": true, "primary": "shard-US-East" }
sales.users
shard key: { "region": 1 }
chunks:
shard-US-East: 2
shard-US-West: 1
{ "region": { "$minKey" : 1 } } -> { "region": 1 } on: shard-US-East Timestamp(2, 1)
{ "region": 1 } -> { "region": 2 } on: shard-US-East Timestamp(1, 3)
{ "region": 2 } -> { "region": { "$maxKey" : 1 } } on: shard-US-West Timestamp(2, 0)
tag: US-East {
"region": 1
} -> {
"region": 2
}
tag: US-West {
"region": 2
} -> {
"region": 3
}
验证是否对正确的分片和主数据进行写操作. 在每个区域创建一条记录
Verify writes are made to the right shard and primary. Create a record in each region
db.users.insert({region:1, name:"us east user"})
db.users.insert({region:2, name:"us west user"})
您可以登录到每个副本集的每个成员，并仅在US-East分片上看到东部用户，而仅在US-West分片上看到西部用户.
You can log on to each member of each replica set and see the east user only on the US-East shard and the west user only on the US-West shard.
这篇关于MongoDB区域副本集-每个区域中的主节点?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
