在云模式下设置Apache Solr [英] Setting Up Apache Solr in Cloud Mode
问题描述
我必须执行以下操作:
- 我必须在2个服务器/节点上部署Solr.
- 在另一台服务器上部署Zookeeper.
- 将自定义配置上传到Zookeeper
- 使用2个分片和2个副本创建一个自定义集合
Solr 7.4.0及更高版本动物园管理员: 3.4.12
我已经执行以下操作:
设置Zookeeper:
- 创建了Zookeeper数据文件夹&制作了一个zoo.conf&将dataDir放在此处.
- 使用
./zkServer.sh start
启动了Zookeeper
设置Solr:
-
使用以下命令启动Solr:
./solr start -cloud -s /home/demo/LocalFolder/Downloads/SolrHome -p 8987 -z localhost:2181
-
尝试使用以下方法在Zookeeper中上载配置:
./solr create -c mycollection -d /media/sf_VM/Dump/conf
这给了我一个例外:
Caused by: javax.servlet.UnavailableException: Error processing the request. CoreContainer is either not initialized or shutting down.
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:341)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:323)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1634)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:533)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:146)
at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:257)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595)
at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1253)
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1564)
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1155)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144)
at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:219)
at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:126)
... 17 more
</pre>
我搜索了许多页面&看过Solr教程,但在那里他们使用了默认示例.我只是对
没有任何循序渐进的想法- 如何在Zookeeper中上载配置?
- 然后我需要做的就是创建一个指向该配置的集合.我希望该集合具有2个碎片& 2个副本.
- solr.xml在哪里.如果应该在Zookeeper中,该如何上传到那里
- 我如何在Zookeeper中看到配置已上传?
我知道这个问题可能是重复的.我已经阅读了几篇文章,但无法提出解决方案.请帮忙.
好了,我弄清楚了如何设置.请注意,我已经从其他站点和网站阅读了这些步骤.一步一步聚集在这里:
1.设置ZooKeeper:
这用于将特定于集合的配置文件存储在中央空间&中.用名称映射它.稍后,我们可以使用该名称来创建指向此配置的集合.此配置与您在以下文件夹中找到的配置相同:
solr-7.4.0\server\solr\configsets\sample_techproducts_configs\conf
1.1下载最新的Zookeeper.我用过3.4.12
1.2解压缩下载的存档并将conf/zoo_sample.cfg复制到conf/zoo.cfg
1.3修改zoo.cfg:
1.3.1将dataDir更改为要保留所有群集配置数据的目录.
dataDir=/var/zookeeper/data
1.3.2添加有关所有Zookeeper服务器的信息:我仅使用了1个ZooKeeper服务器,因此这不是必需的.如果要添加更多服务器,请通过以下链接:
1.3.3使用(在zookeeper-3.4.12/之后)启动ZooKeeper:
./bin/zkServer.sh start-foreground conf/zoo.cfg
OR
./bin/zkServer.sh start conf/zoo.cfg
注意:您可以使用以下命令停止ZooKeeper:
bin/zkServer.sh stop
1.3.4 ZooKeeper状态:
点击以下内容:
bin/zkServer.sh status
或执行telnet localhost 2181
并在连接后按stats
.
2.设置Solr
2.1下载Solr
2.2从.tar文件中提取install_solr_service.sh. Solr包含服务安装脚本(bin/install_solr_service.sh),可帮助您在Linux上将Solr作为服务安装.有关更多信息,请单击此处.>
tar -xzf solr-7.4.0.tgz solr-7.4.0/bin/install_solr_service.sh --strip-components=2
2.3使用上述脚本将Solr作为服务安装:
sudo bash ./install_solr_service.sh solr-7.4.0.tgz
这还将把solr-7.4.0.tgz提取到/opt/solr
2.4转到/opt/solr
并执行以下操作:
mkdir solr/server/solr2
mkdir solr/server/solr3
mkdir solr/server/solr4
cp solr/server/solr/solr.xml solr/server/solr2
cp solr/server/solr/solr.xml solr/server/solr3
cp solr/server/solr/solr.xml solr/server/solr4
2.5在solr.xml中更改码头端口.对上面步骤中提到的所有3 solr.xml都执行此操作:
vi solr/server/solr2/solr.xml
搜索端口8983&更改为8984(对于solr2),更改为8985(对于solr3),更改为8986(对于solr4)
2.6在8983处停止Solr运行
root@dev-base:/opt/solr/bin# ./solr stop -p 8983
2.7启动所有solr实例:
root@dev-base:/opt/solr# bin/solr start -c -s server/solr -p 8983 -z localhost:2181 -noprompt -force
root@dev-base:/opt/solr# bin/solr start -c -s server/solr2 -p 8984 -z localhost:2181 -noprompt -force
root@dev-base:/opt/solr# bin/solr start -c -s server/solr3 -p 8985 -z localhost:2181 -noprompt -force
root@dev-base:/opt/solr# bin/solr start -c -s server/solr4 -p 8986 -z localhost:2181 -noprompt -force
注意:出于安全原因,不建议将solr作为root用户运行.
2.8请参阅Solr状态:
root@dev-base:/opt/solr# bin/solr status
3进行自定义配置
3.1将conf
目录从solr-7.4.0\server\solr\configsets\sample_techproducts_configs\conf
复制到另一个位置(In my case it is /media/sf_VM/Dump/new/conf )
.
3.2更改conf
中的managed-schema
文件以指定您正在使用的字段.
4将配置上传到ZooKeeper:
root@dev-base:/opt/solr# bin/solr zk -z localhost:2181 upconfig -d /media/sf_VM/Dump/new/conf -n myConf6
我上传的配置名称为myConf6
5使用此自定义配置创建Solr集合
root@dev-base:/opt/solr# bin/solr create -c myNewCollection -n myConf6 -shards 2 -replicationFactor 2 -force
点击 Solr管理员URL
6使用JSON的POST API使用索引数据
URL: http://localhost:8983/solr/myNewCollection/update >
方法:开机自检
身体:
[{
"_id": "99999999999999999999",
"author": [
"New Inserted 9000"
],
"authorLastName": [
"New Inserted 9000"
],
"impn": "New Inserted 9000",
"isbn10": "9999999999",
"isbn13": "9999999999999",
"title": "New Inserted 9000",
"publisher": "New Inserted 9000",
"sales_a": 5.0,
"sales_t": 5.0,
"haveImage": 1,
"pages": "76",
"image": "http://ip.ip.com/is/image/",
"format": "Paper",
"mtc_id": "99999999999",
"subjects" : [
"9000"
]
"rating": 0,
"description_long": "Snahashis call me in your marriage."
}
在Zookeeper中删除配置:
如果要删除已经上传到ZooKeeper的旧/错误配置,请运行以下命令:
./server/scripts/cloud-scripts/zkcli.sh -cmd clear -z "<ZK_HOST>:<ZK_PORT>" /configs/AAA
配置路径为/configs/< name of the configset >
要删除特定文件,请执行以下操作:
zkcli.sh --zkhost <ZK_HOST>:<ZK_PORT> -cmd clear /configs/<MY_COLLECTION>/solrconfig.xml
上传更新的文件:
zkcli.sh --zkhost <ZK_HOST>:<ZK_PORT> -cmd putfile /configs/<MY_COLLECTION>/solrconfig.xml /<MY_UPDATED_FILE_LOCAL_FOLDER>/solrconfig.xml
然后,我们需要重新启动solr节点.
要通过API删除集合,请执行以下操作:
- 首先删除在此集合上创建的别名(如果有的话)
http://localhost:8983/solr/admin/collections ?action = DELETEALIAS& name = aliasName
- 删除收藏集:
http://localhost:8983/solr/admin/collections ?action = DELETE& name = collectionName
I have to do the following:
- I have to deploy Solr on 2 servers/nodes.
- Deploy Zookeeper on another server.
- Upload a custom config to Zookeeper
- Create a custom collection with 2 shards and 2 replicas
Version of Solr 7.4.0 & Zookeeper: 3.4.12
I have done the following:
Set Up Zookeeper:
- Created a Zookeeper data folder & made a zoo.conf & put the dataDir there.
- Started zookeeper using
./zkServer.sh start
Set up Solr:
Started Solr using:
./solr start -cloud -s /home/demo/LocalFolder/Downloads/SolrHome -p 8987 -z localhost:2181
Trying to upload config in Zookeeper using:
./solr create -c mycollection -d /media/sf_VM/Dump/conf
It is giving me an exception:
Caused by: javax.servlet.UnavailableException: Error processing the request. CoreContainer is either not initialized or shutting down.
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:341)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:323)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1634)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:533)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:146)
at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:257)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595)
at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1253)
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1564)
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1155)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144)
at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:219)
at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:126)
... 17 more
</pre>
I have searched many pages & seen Solr tutorials but there they have used the default examples. I just dont have any step by step idea to
- How to Upload a config in Zookeeper?
- Then what I need to do to create a collection pointing to that config. I want that collection to have 2 shards & 2 replicas.
- Where will be the solr.xml. If it should be in Zookeeper how do I upload it there
- How do I see in Zookeeper that the config has been uploaded?
I know this question might be a duplicate. I have read several of posts but not able to come up a solution. Please help.
Well I figured out how to setup. Please note that I have read these steps from different site & gathered here step by step:
1. Set Up ZooKeeper:
This is used to keep the collection specific configuration files to store in a central space & map it with a name. Later we can use this name to create collection which would point to this config. This config is the same as you find in the below folder:
solr-7.4.0\server\solr\configsets\sample_techproducts_configs\conf
1.1 Download latest Zookeeper. I have used 3.4.12
1.2 Unpack downloaded archive and copy conf/zoo_sample.cfg to conf/zoo.cfg
1.3 Modify zoo.cfg:
1.3.1 Change dataDir to directory where you want to hold all cluster configuration data.
dataDir=/var/zookeeper/data
1.3.2 Add information about all Zookeeper servers: I have used only 1 ZooKeeper server so this is not required. If you want to add more servers then please go through the below links:
1.3.3 Start ZooKeeper using (after going to zookeeper-3.4.12/):
./bin/zkServer.sh start-foreground conf/zoo.cfg
OR
./bin/zkServer.sh start conf/zoo.cfg
Note: You can stop the ZooKeeper using the below command:
bin/zkServer.sh stop
1.3.4 ZooKeeper Status:
Hit the below:
bin/zkServer.sh status
Or do telnet localhost 2181
and hit stats
when connected.
2. Set Up Solr
2.1 Download the Solr
2.2 Extract install_solr_service.sh from the .tar file. Solr includes a service installation script (bin/install_solr_service.sh) to help you install Solr as a service on Linux. For more info click here.
tar -xzf solr-7.4.0.tgz solr-7.4.0/bin/install_solr_service.sh --strip-components=2
2.3 Install Solr as a service using the above script:
sudo bash ./install_solr_service.sh solr-7.4.0.tgz
This will also extracting solr-7.4.0.tgz to /opt/solr
2.4 Go to /opt/solr
and do the following:
mkdir solr/server/solr2
mkdir solr/server/solr3
mkdir solr/server/solr4
cp solr/server/solr/solr.xml solr/server/solr2
cp solr/server/solr/solr.xml solr/server/solr3
cp solr/server/solr/solr.xml solr/server/solr4
2.5 Change the jetty port in solr.xml. Do this for all the 3 solr.xml mentioned in the above step:
vi solr/server/solr2/solr.xml
Search for the port 8983 & change it 8984 (for solr2), change it 8985 (for solr3), change it 8986 (for solr4)
2.6 Stop the Solr running at 8983
root@dev-base:/opt/solr/bin# ./solr stop -p 8983
2.7 Start all the solr instances:
root@dev-base:/opt/solr# bin/solr start -c -s server/solr -p 8983 -z localhost:2181 -noprompt -force
root@dev-base:/opt/solr# bin/solr start -c -s server/solr2 -p 8984 -z localhost:2181 -noprompt -force
root@dev-base:/opt/solr# bin/solr start -c -s server/solr3 -p 8985 -z localhost:2181 -noprompt -force
root@dev-base:/opt/solr# bin/solr start -c -s server/solr4 -p 8986 -z localhost:2181 -noprompt -force
Note: Running solr as a root is not recommended for security reason.
2.8 See Solr Status:
root@dev-base:/opt/solr# bin/solr status
3 Make a Custom Config
3.1 Copy the conf
directory from solr-7.4.0\server\solr\configsets\sample_techproducts_configs\conf
to another location (In my case it is /media/sf_VM/Dump/new/conf )
.
3.2 Change the managed-schema
file inside conf
to specify the fields you are using.
4 Upload the config to ZooKeeper:
root@dev-base:/opt/solr# bin/solr zk -z localhost:2181 upconfig -d /media/sf_VM/Dump/new/conf -n myConf6
The name of the config I have uploaded is myConf6
5 Create a Solr Collection using this custom config
root@dev-base:/opt/solr# bin/solr create -c myNewCollection -n myConf6 -shards 2 -replicationFactor 2 -force
Hit Solr Admin URL
6 Index Data Using POST API using Json
URL: http://localhost:8983/solr/myNewCollection/update
Method: POST
Body:
[{
"_id": "99999999999999999999",
"author": [
"New Inserted 9000"
],
"authorLastName": [
"New Inserted 9000"
],
"impn": "New Inserted 9000",
"isbn10": "9999999999",
"isbn13": "9999999999999",
"title": "New Inserted 9000",
"publisher": "New Inserted 9000",
"sales_a": 5.0,
"sales_t": 5.0,
"haveImage": 1,
"pages": "76",
"image": "http://ip.ip.com/is/image/",
"format": "Paper",
"mtc_id": "99999999999",
"subjects" : [
"9000"
]
"rating": 0,
"description_long": "Snahashis call me in your marriage."
}
Delete a configuration in Zookeeper:
If you want to delete an old/wrong configuration already uploaded to ZooKeeper run the below command:
./server/scripts/cloud-scripts/zkcli.sh -cmd clear -z "<ZK_HOST>:<ZK_PORT>" /configs/AAA
The path of the configuration is /configs/< name of the configset >
To delete a specific file:
zkcli.sh --zkhost <ZK_HOST>:<ZK_PORT> -cmd clear /configs/<MY_COLLECTION>/solrconfig.xml
Upload an updated file:
zkcli.sh --zkhost <ZK_HOST>:<ZK_PORT> -cmd putfile /configs/<MY_COLLECTION>/solrconfig.xml /<MY_UPDATED_FILE_LOCAL_FOLDER>/solrconfig.xml
Then we need to restart the solr nodes.
To Delete a Collection Via API:
- First delete alias created on this collection (if any)
http://localhost:8983/solr/admin/collections?action=DELETEALIAS&name=aliasName
- Delete Collection:
http://localhost:8983/solr/admin/collections?action=DELETE&name=collectionName
这篇关于在云模式下设置Apache Solr的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!