Solrcloud单机伪集群部署

线上有一套双节点的Solrcloud节点，因机器性能较老，环境搭建于2013年，原节点有数百个已经被unload的collections，考虑以后可能还会需要，所以搭建一套和原节点相同的solrcloud环境，因为目前只有一台新机器，所以在一台机器上搭建Solrcloud伪双节点集群，

软件版本

JDK：jdk1.6.0_12 http://download.oracle.com/otn/java/jdk/6u12-b04/jdk-6u12-linux-x64.bin
Tomcat：apache-tomcat-6.0.20 https://archive.apache.org/dist/tomcat/tomcat-6/v6.0.20/bin/apache-tomcat-6.0.20.tar.gz
Zookeeper：zookeeper-3.4.5 http://archive.apache.org/dist/zookeeper/zookeeper-3.4.5/zookeeper-3.4.5.tar.gz
Solr：solr-4.2.1 https://archive.apache.org/dist/lucene/solr/4.2.1/solr-4.2.1.tgz

原节点环境信息

主机名(IP)	ec2d-cntprocdb-01(10.189.100.128)	ec2d-cntprocdb-02(10.189.100.128)
User	tnuser
JDK	/usr/java/jdk1.6.0_12
tomcat（port）	/usr/local/apache-tomcat-6.0.20 (8983)	/usr/local/apache-tomcat-6.0.20 (8983)
zookeeper	/usr/local/contentplatform/zookeeper-3.4.5	/usr/local/contentplatform/zookeeper-3.4.5
solr(port)	/usr/local/contentplatform/solr/solr	/usr/local/contentplatform/solr/solr

新节点的安装路径

主机名（IP）	ec2d-newcntprocdb-01（10.189.100.75）
User	tnuser
JDK	/usr/java/jdk1.6.0_12
tomcat（port）	/usr/local/apache-tomcat-6.0.20-8983 (8983)	/usr/local/apache-tomcat-6.0.20-8984 (8984)
Zookeeper	/usr/local/contentplatform/zookeeper-3.4.5
Solr(port)	/usr/local/contentplatform/solr8983/solr (8983)	/usr/local/contentplatform/solr8984/solr (8984)

1. 系统参数修改

1.1 内核参数

$ tail -n  /etc/sysctl.conf

vm.overcommit_memory=

vm.overcommit_ratio=

fs.aio-max-nr=

fs.file-max=

net.ipv4.ip_local_port_range=

net.core.rmem_default=

net.core.rmem_max=

net.core.wmem_default=

net.core.wmem_max=

kernel.sem=

kernel.shmall=

kernel.shmmax=

1.2. 资源使用

$ tail -n  /etc/security/limits.conf

tnuser soft nproc

tnuser hard nproc

tnuser soft nofile

tnuser hard nofile

1.3. 环境变量

$ cat ~/.bash_profile

# .bash_profile

# Get the aliases and functions

if [ -f ~/.bashrc ]; then

    . ~/.bashrc

fi

# User specific environment and startup programs

JAVA_HOME=/usr/java/jdk1..0_12

ZOOKEEPER_HOME=/usr/local/contentplatform/zookeeper-3.4.

JRE_HOME=$JAVA_HOME/jre

CLASSPATH=.:$JAVA_HOME/lib/tools.jar:$JAVA_HOME/lib/dt.jar:$JRE_HOME/lib

PATH=$JAVA_HOME/bin:$ZOOKEEPER_HOME/bin:$PATH:$HOME/.local/bin:$HOME/bin

export PATH

2. Zookeeper安装

$ mkdir /usr/local/contentplatform

$ cd /usr/local/contentplatform

$ wget http://archive.apache.org/dist/zookeeper/zookeeper-3.4.5/zookeeper-3.4.5.tar.gz

$ tar -zxf zookeeper-3.4..tar.gz

$ mkdir zookeeper-3.4./data

$ cat zookeeper-3.4./conf/zoo.cfg

tickTime=

initLimit=

syncLimit=

dataDir=/usr/local/contentplatform/zookeeper-3.4./data

clientPort=

$ zookeeper-3.4./bin/zkServer.sh start

$ zookeeper-3.4./bin/zkServer.sh status

JMX enabled by default

Using config: /usr/local/contentplatform/zookeeper-3.4./bin/../conf/zoo.cfg

Mode: standalone

3. solr安装

$ cd /usr/local/contentplatform

$ wget https://archive.apache.org/dist/lucene/solr/4.2.1/solr-4.2.1.tgz

$ tar -zxf solr-4.2..tgz

$ mkdir {solr8983,solr8984}

$ cp -r solr-4.2./example/* solr8983/

$ cat solr8983/solr/solr.xml

<?xml version="1.0" encoding="UTF-8" ?>

<solr persistent="true">

  <cores defaultCoreName="tnpoi" adminPath="/admin/cores" zkClientTimeout="${zkClientTimeout:60000}" hostPort="8983" hostContext="solr">

    <core loadOnStartup="true" shard="shard1" instanceDir="tifbase/" transient="false" name="tnpoi"/>

  </cores>

</solr>

默认的collection为collection1，我将这里改成我实际的collection名tnpoi，注意hostPort的值一定要改成和第一个tomcat的端口号对应

$ cp -r solr8983/* solr8984/

$ cat solr8984/solr/solr.xml

<?xml version="1.0" encoding="UTF-8" ?>

<solr persistent="true">

  <cores defaultCoreName="tnpoi" adminPath="/admin/cores" zkClientTimeout="${zkClientTimeout:60000}" hostPort="8984" hostContext="solr">

    <core loadOnStartup="true" shard="shard2" instanceDir="tifbase/" transient="false" name="tnpoi"/>

  </cores>

</solr>

注意第二个solr的hostPort的值一定要改成和第二个tomcat的端口号对应

4. tomcat安装

$ cd /usr/local

$ https://archive.apache.org/dist/tomcat/tomcat-6/v6.0.20/bin/apache-tomcat-6.0.20.tar.gz

$ tar -zxf apache-tomcat-6.0..tar.gz

$ mkdir {apache-tomcat-6.0.-,apache-tomcat-6.0.-}

$ rm -rf apache-tomcat-6.0.-/webapps/*

$ unzip contentplatform/solr-4.2.1/example/webapps/solr.war -d apache-tomcat-6.0.20-8983/webapps/solr

$ cp -r /usr/local/contentplatform/solr-4.2.1/dist/*.jar /usr/local/apache-tomcat-6.0.20-8983/webapps/solr/WEB-INF/lib/

$ cp -r /usr/local/contentplatform/solr-4.2.1/contrib/velocity/lib/*.jar /usr/local/apache-tomcat-6.0.20-8983/webapps/solr/WEB-INF/lib/

$ cp -r /usr/local/contentplatform/solr-4.2.1/contrib/extraction/lib/*.jar /usr/local/apache-tomcat-6.0.20-8983/webapps/solr/WEB-INF/lib/

$ cp -r /usr/local/contentplatform/solr-4.2.1/example/lib/*.jar /usr/local/apache-tomcat-6.0.20-8983/webapps/solr/WEB-INF/lib/

修改bin/catalina.sh文件

sh /usr/local/apache-tomcat-6.0.-/bin/createSymbolicConf.sh

JAVA_HOME=/usr/java/jdk1..0_12/

JAVA_OPTS="$JAVA_OPTS -Xms8000M -Xmx8000M -Duser.language=en -Duser.country=US -Dsolr.solr.home=/usr/local/contentplatform/solr8983/solr -DhostContext=solr -Dbootstrap_conf=true -Dhostport=8983 -DzkHost=ec2d-newcntprocdb-01:9983 -DnumShards=2"

echo "Java Options: $JAVA_OPTS"

createSymbolicConf.sh脚本为每次启动tomcat时添加成conf目录软连接到指定的collection下

$ cat apache-tomcat-6.0.-/bin/createSymbolicConf.sh

#!/bin/sh

baseDir=/usr/local/contentplatform/solr8983/solr

confDir=/usr/local/contentplatform/solr8983/solr/tifbase/conf

for subDir in $baseDir/*

do

if [ !  -d $subDir/conf ] && [[ "$subDir" == *poi* ]]

  then

    echo "$subDir don't have conf dir, creating symbolic link"

    ln -sb $confDir $subDir/conf

  fi

done

修改conf/server.xml文件

<?xml version='1.0' encoding='utf-8'?>

<Server port="8005" shutdown="SHUTDOWN">

  <Listener className="org.apache.catalina.core.AprLifecycleListener" SSLEngine="on" />

  <Listener className="org.apache.catalina.core.JasperListener" />

  <Listener className="org.apache.catalina.mbeans.ServerLifecycleListener" />

  <Listener className="org.apache.catalina.mbeans.GlobalResourcesLifecycleListener" />

  <GlobalNamingResources>

    <Resource name="UserDatabase" auth="Container"

              type="org.apache.catalina.UserDatabase"

              description="User database that can be updated and saved"

              factory="org.apache.catalina.users.MemoryUserDatabaseFactory"

              pathname="conf/tomcat-users.xml" />

  </GlobalNamingResources>

  <Service name="Catalina">

    <Connector port="8983" protocol="HTTP/1.1"

               connectionTimeout="20000"

               redirectPort="8443" />

    <Connector port="8009" protocol="AJP/1.3" redirectPort="8443" />

    <Engine name="Catalina" defaultHost="localhost">

      <Realm className="org.apache.catalina.realm.UserDatabaseRealm"

             resourceName="UserDatabase"/>

      <Host name="localhost"  appBase="webapps"

            unpackWARs="true" autoDeploy="true"

            xmlValidation="false" xmlNamespaceAware="false">

      </Host>

    </Engine>

  </Service>

</Server>

修改tomcat下的solr的web.xml文件

    <env-entry>

       <env-entry-name>solr/home</env-entry-name>

       <env-entry-value>/usr/local/contentplatform/solr8983/solr</env-entry-value>

       <env-entry-type>java.lang.String</env-entry-type>

    </env-entry>

将tomcat目录复制一份为第二个tomcat目录

$ cp -r apache-tomcat-6.0.-/* apache-tomcat-6.0.20-8984/

修改第二个tomcat下的bin/catalina.sh文件

sh /usr/local/apache-tomcat-6.0.-/bin/createSymbolicConf.sh

JAVA_HOME=/usr/java/jdk1..0_12/

JAVA_OPTS="$JAVA_OPTS -Xms8000M -Xmx8000M -Duser.language=en -Duser.country=US -Dsolr.solr.home=/usr/local/contentplatform/solr8984/solr -DhostContext=solr -Dbootstrap_conf=true -Dhostport=8984 -DzkHost=ec2d-newcntprocdb-01:9983 -DnumShards=2"

echo "Java Options: $JAVA_OPTS"

修改第二个tomcat的conf/server.xml文件

<?xml version='1.0' encoding='utf-8'?>

<Server port="8006" shutdown="SHUTDOWN">

  <Listener className="org.apache.catalina.core.AprLifecycleListener" SSLEngine="on" />

  <Listener className="org.apache.catalina.core.JasperListener" />

  <Listener className="org.apache.catalina.mbeans.ServerLifecycleListener" />

  <Listener className="org.apache.catalina.mbeans.GlobalResourcesLifecycleListener" />

  <GlobalNamingResources>

    <Resource name="UserDatabase" auth="Container"

              type="org.apache.catalina.UserDatabase"

              description="User database that can be updated and saved"

              factory="org.apache.catalina.users.MemoryUserDatabaseFactory"

              pathname="conf/tomcat-users.xml" />

  </GlobalNamingResources>

  <Service name="Catalina">

    <Connector port="8984" protocol="HTTP/1.1"

               connectionTimeout="20000"

               redirectPort="8443" />

    <Connector port="8010" protocol="AJP/1.3" redirectPort="8443" />

    <Engine name="Catalina" defaultHost="localhost">

      <Realm className="org.apache.catalina.realm.UserDatabaseRealm"

             resourceName="UserDatabase"/>

      <Host name="localhost"  appBase="webapps"

            unpackWARs="true" autoDeploy="true"

            xmlValidation="false" xmlNamespaceAware="false">

      </Host>

    </Engine>

  </Service>

</Server>

修改第二个tomcat下的solr的web.xml文件

    <env-entry>

       <env-entry-name>solr/home</env-entry-name>

       <env-entry-value>/usr/local/contentplatform/solr8984/solr</env-entry-value>

       <env-entry-type>java.lang.String</env-entry-type>

    </env-entry>

依次启动第一个和第二个tomcat

$ /usr/local/apache-tomcat-6.0.-/bin/catalina.sh start

$ /usr/local/apache-tomcat-6.0.-/bin/catalina.sh start

查看

http://10.189.100.75:8983/solr/#/

http://10.189.100.75:8984/solr/#/

创建collection

http://10.189.100.75:8983/solr/admin/collections?action=CREATE&name=poi_EDBSTORE&numShards=2&collection.configName=tnpoi

Solrcloud单机伪集群部署的更多相关文章

zookeeper的单实例和伪集群部署
原文链接: http://gudaoyufu.com/?p=1395 zookeeper工作方式 ZooKeeper 是一个开源的分布式协调服务,由雅虎创建,是 Google Chubby 的开源实现 ...
Zookeeper单机伪集群
Zookeeper单机伪集群 1.配置 zookeeper下载地址:http://apache.mirrors.lucidnetworks.net/zookeeper/ 可以选择需要的版本,我下载的是 ...
Zookeeper集群搭建（多节点，单机伪集群，Docker集群）
Zookeeper介绍原理简介 ZooKeeper是一个分布式的.开源的分布式应用程序协调服务.它公开了一组简单的原语,分布式应用程序可以在此基础上实现更高级别的同步.配置维护.组和命名服务.它的设 ...
redis在Windows下以后台服务一键搭建集群(单机--伪集群)
redis在Windows下以后台服务一键搭建集群(单机--伪集群) 一.概述此教程介绍如何在windows系统中同一台机器上布置redis伪集群,同时要以后台服务的模式运行.布置以脚本的形式,一键 ...
备忘zookeeper(单机+伪集群+集群)
#下载: #单机模式解压到合适目录. 进入zookeeper目录下的conf子目录, 复制zoo_sample.cfg-->zoo.cfg(如果没有data和logs就新建):tickTime ...
ActiveMQ伪集群部署
本文借鉴http://www.cnblogs.com/guozhen/p/5984915.html,在此基础上进行了完善,使之成为一个完整版的伪分布式部署说明,在此记录一下! 一.本文目的介绍如何在 ...
zookeeper安装和配置(单机+伪集群+集群)
#单机模式解压到合适目录. 进入zookeeper目录下的conf子目录, 复制zoo_sample.cfg-->zoo.cfg(如果没有data和logs就新建):tickTime=2000 ...
Redis学习笔记之Redis单机,伪集群,Sentinel主从复制的安装和配置
0x00 Redis简介 Redis是一款开源的.高性能的键-值存储(key-value store).它常被称作是一款数据结构服务器(data structure server). Redis的键值 ...
Kafka入门初探+伪集群部署
Kafka是目前非常流行的消息队列中间件,常用于做普通的消息队列.网站的活性数据分析(PV.流量.点击量等).日志的搜集(对接大数据存储引擎做离线分析). 全部内容来自网络,可信度有待考证!如有问题, ...

随机推荐

【leetcode】Department Top Three Salaries
The Employee table holds all employees. Every employee has an Id, and there is also a column for the ...
20. ClustrixDB explain参数解读
ClustrixDB有一个流模型,它从容器(表和索引)开始并通过操作符图对行进行流处理.ClustrixDB有一组丰富的操作符,随着更多功能和优化的添加,这些操作符偶尔会增加.我们使用一个地方来记录操 ...
Clojure的引用类型：var,ref,agent和atom
作为其他值的容器,也都可以解引用. deref不会阻塞. add-watch (add-watch reference key fn) 可以定义引用值发生改变时的回调,fn是4个参数 :key (at ...
【杂题】[CodeForces 1172F] Nauuo and Bug【数据结构】【线段树】
Description 给出一个长度为n的序列a和一个整数p 有m组询问,每组询问给出一个区间\([l,r]\) 你需要给出下面这个过程的结果 ans = 0 for i from l to r { ...
洛谷 P2820 局域网x
题目背景某个局域网内有n(n<=100)台计算机,由于搭建局域网时工作人员的疏忽,现在局域网内的连接形成了回路,我们知道如果局域网形成回路那么数据将不停的在回路内传输,造成网络卡的现象.因为连 ...
max函数结合lambda使用
说明:d.keys() 以及列表可以看做lambda函数的实参,max的判断对象是key的值.最终返回的是使得key的值最大的那个实参.
python下selenium模拟浏览器基础操作
1.安装及下载 selenium安装: pip install selenium 即可自动安装selenium geckodriver下载:https://github.com/mozilla/ge ...
JS框架_(JQuery.js)上传进度条
百度云盘传送门密码: 1pou 纯CSS上传进度条效果: <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN ...
Java并发编程的艺术笔记（四）——ThreadLocal的使用
ThreadLocal,即线程变量,是一个以ThreadLocal对象为键.任意对象为值的存储结构.这个结构被附带在线程上,也就是说一个线程可以根据一个ThreadLocal对象查询到绑定在这个线程上 ...
关于Jdk7与Jdk8对Collections进行分组的区别
先准备一点数据: public class User { private Integer id; private String type; private String name; ...

Solrcloud单机伪集群部署

Solrcloud单机伪集群部署的更多相关文章

随机推荐

热门专题