不多说，直接上干货！

　　我这里，采取的是CentOS6.5，当然大家也可以在ubuntu 16.04系统里，这些都是小事

CentOS 6.5的安装详解

hadoop-2.6.0.tar.gz + spark-1.5.2-bin-hadoop2.6.tgz的集群搭建（单节点）（Ubuntu系统）

　　我这里，采取的是CentOS6.5，而且Spark on YARN模式

Spark on YARN模式的安装（spark-1.6.1-bin-hadoop2.6.tgz + hadoop-2.6.0.tar.gz）（master、slave1和slave2）（博主推荐）

　　系统环境变量配置文件

#java

export JAVA_HOME=/home/hadoop/app/jdk

export JRE_HOME=$JAVA_HOME/jre

export CLASSPATH=.:$JAVA_HOME/lib:$JRE_HOME/lib

export PATH=$PATH:$JAVA_HOME/bin


#scala

export SCALA_HOME=/home/hadoop/app/scala

export PATH=$PATH:$SCALA_HOME/bin



#zookeeper export ZOOKEEPER_HOME=/home/hadoop/app/zookeeper 
export PATH=$PATH:$ZOOKEEPER_HOME/bin #hadoop 
export HADOOP_HOME=/home/hadoop/app/hadoop-2.6. 
export PATH=$PATH:$HADOOP_HOME/bin:$HADOOP_HOME/sbin 

#spark 
export SPARK_HOME=/home/hadoop/app/spark 
export PATH=$PATH:$SPARK_HOME/bin:$SPARK_HOME/sbin




　　我的路径是在/home/hadoop/app下






 Hadoop的配置文件


slaves




slave1

slave2




hdfs-site.xml




<?xml version="1.0" encoding="UTF-8"?>

<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>

<!--

  Licensed under the Apache License, Version 2.0 (the "License");

  you may not use this file except in compliance with the License.

  You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

  Unless required by applicable law or agreed to in writing, software

  distributed under the License is distributed on an "AS IS" BASIS,

  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.

  See the License for the specific language governing permissions and

  limitations under the License. See accompanying LICENSE file.

-->

<!-- Put site-specific property overrides in this file. -->

<configuration>

    <property>

                <name>dfs.namenode.rpc-address</name>

                <value>master:</value>

        </property>

             <property>

                <name>dfs.namenode.http-address</name>

                <value>master:</value>

        </property>

        <property>

                <name>dfs.replication</name>

                <value></value>

        <description>Set to  for pseudo-distributed mode,Set to  for distributed mode,Set to  for distributed mode.</description>

        </property>

        <property>

                <name>dfs.permissions</name>

                <value>false</value>

        </property>

        <property>

                <name>dfs.permissions.enabled</name>

                <value>false</value>

       </property>

</configuration>




core-site.xml




<?xml version="1.0" encoding="UTF-8"?>

<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>

<!--

  Licensed under the Apache License, Version 2.0 (the "License");

  you may not use this file except in compliance with the License.

  You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

  Unless required by applicable law or agreed to in writing, software

  distributed under the License is distributed on an "AS IS" BASIS,

  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.

  See the License for the specific language governing permissions and

  limitations under the License. See accompanying LICENSE file.

-->

<!-- Put site-specific property overrides in this file. -->

<configuration>

     <property>

                <name>fs.default.name</name>

                <value>hdfs://master:9000</value>

                <description>The name of the default file system, using  port.</description>

        </property>

        <property>

                <name>hadoop.tmp.dir</name>

                <value>/home/hadoop/data/tmp</value>

                <description>A base for other temporary directories.</description>

        </property>

        <property>

            <name>ha.zookeeper.quorum</name>

               <value>master:,slave1:,slave2:</value>

    </property>

</configuration>




mapred-site.xml




<?xml version="1.0"?>

<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>

<!--

  Licensed under the Apache License, Version 2.0 (the "License");

  you may not use this file except in compliance with the License.

  You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

  Unless required by applicable law or agreed to in writing, software

  distributed under the License is distributed on an "AS IS" BASIS,

  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.

  See the License for the specific language governing permissions and

  limitations under the License. See accompanying LICENSE file.

-->

<!-- Put site-specific property overrides in this file. -->

<configuration>

    <property>

                <name>mapred.job.tracker</name>

                <value>master:</value>

        </property>

        <property>

                <name>mapreduce.framework.name</name>

                <value>yarn</value>

        </property>

</configuration>




yarn-site.xml




<?xml version="1.0"?>

<!--

  Licensed under the Apache License, Version 2.0 (the "License");

  you may not use this file except in compliance with the License.

  You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

  Unless required by applicable law or agreed to in writing, software

  distributed under the License is distributed on an "AS IS" BASIS,

  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.

  See the License for the specific language governing permissions and

  limitations under the License. See accompanying LICENSE file.

-->

<configuration>

    <property>

        <name>yarn.resourcemanager.hostname</name>

        <value>master</value>

    </property>

    <property>

        <name>yarn.nodemanager.aux-services</name>

        <value>mapreduce_shuffle</value>

    </property>

    <property>

        <name>yarn.log-aggregation-enable</name>

        <value>true</value>

    </property>

    <property>

        <name>yarn.log-aggregation.retain-seconds</name>

        <value></value>

    </property>

</configuration>




hadoop-env.sh




# Licensed to the Apache Software Foundation (ASF) under one

# or more contributor license agreements.  See the NOTICE file

# distributed with this work for additional information

# regarding copyright ownership.  The ASF licenses this file

# to you under the Apache License, Version 2.0 (the

# "License"); you may not use this file except in compliance

# with the License.  You may obtain a copy of the License at

#

#     http://www.apache.org/licenses/LICENSE-2.0

#

# Unless required by applicable law or agreed to in writing, software

# distributed under the License is distributed on an "AS IS" BASIS,

# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.

# See the License for the specific language governing permissions and

# limitations under the License.

# Set Hadoop-specific environment variables here.

# The only required environment variable is JAVA_HOME.  All others are

# optional.  When running a distributed configuration it is best to

# set JAVA_HOME in this file, so that it is correctly defined on

# remote nodes.

# The java implementation to use.

export JAVA_HOME=/home/hadoop/app/jdk1..0_79

# The jsvc implementation to use. Jsvc is required to run secure datanodes

# that bind to privileged ports to provide authentication of data transfer

# protocol.  Jsvc is not required if SASL is configured for authentication of

# data transfer protocol using non-privileged ports.

#export JSVC_HOME=${JSVC_HOME}

export HADOOP_CONF_DIR=${HADOOP_CONF_DIR:-"/etc/hadoop"}

# Extra Java CLASSPATH elements.  Automatically insert capacity-scheduler.

for f in $HADOOP_HOME/contrib/capacity-scheduler/*.jar; do

  if [ "$HADOOP_CLASSPATH" ]; then

    export HADOOP_CLASSPATH=$HADOOP_CLASSPATH:$f

  else

    export HADOOP_CLASSPATH=$f

  fi

done

# The maximum amount of heap to use, in MB. Default is 1000.

#export HADOOP_HEAPSIZE=

#export HADOOP_NAMENODE_INIT_HEAPSIZE=""

# Extra Java runtime options.  Empty by default.

export HADOOP_OPTS="$HADOOP_OPTS -Djava.net.preferIPv4Stack=true"

# Command specific options appended to HADOOP_OPTS when specified

export HADOOP_NAMENODE_OPTS="-Dhadoop.security.logger=${HADOOP_SECURITY_LOGGER:-INFO,RFAS} -Dhdfs.audit.logger=${HDFS_AUDIT_LOGGER:-INFO,NullAppender} $HADOOP_NAMENODE_OPTS"

export HADOOP_DATANODE_OPTS="-Dhadoop.security.logger=ERROR,RFAS $HADOOP_DATANODE_OPTS"

export HADOOP_SECONDARYNAMENODE_OPTS="-Dhadoop.security.logger=${HADOOP_SECURITY_LOGGER:-INFO,RFAS} -Dhdfs.audit.logger=${HDFS_AUDIT_LOGGER:-INFO,NullAppender} $HADOOP_SECONDARYNAMENODE_OPTS"

export HADOOP_NFS3_OPTS="$HADOOP_NFS3_OPTS"

export HADOOP_PORTMAP_OPTS="-Xmx512m $HADOOP_PORTMAP_OPTS"

# The following applies to multiple commands (fs, dfs, fsck, distcp etc)

export HADOOP_CLIENT_OPTS="-Xmx512m $HADOOP_CLIENT_OPTS"

#HADOOP_JAVA_PLATFORM_OPTS="-XX:-UsePerfData $HADOOP_JAVA_PLATFORM_OPTS"

# On secure datanodes, user to run the datanode as after dropping privileges.

# This **MUST** be uncommented to enable secure HDFS if using privileged ports

# to provide authentication of data transfer protocol.  This **MUST NOT** be

# defined if SASL is configured for authentication of data transfer protocol

# using non-privileged ports.

export HADOOP_SECURE_DN_USER=${HADOOP_SECURE_DN_USER}

# Where log files are stored.  $HADOOP_HOME/logs by default.

#export HADOOP_LOG_DIR=${HADOOP_LOG_DIR}/$USER

# Where log files are stored in the secure data environment.

export HADOOP_SECURE_DN_LOG_DIR=${HADOOP_LOG_DIR}/${HADOOP_HDFS_USER}

###

# HDFS Mover specific parameters

###

# Specify the JVM options to be used when starting the HDFS Mover.

# These options will be appended to the options specified as HADOOP_OPTS

# and therefore may override any similar flags set in HADOOP_OPTS

#

# export HADOOP_MOVER_OPTS=""

###

# Advanced Users Only!

###

# The directory where pid files are stored. /tmp by default.

# NOTE: this should be set to a directory that can only be written to by

#       the user that will run the hadoop daemons.  Otherwise there is the

#       potential for a symlink attack.

export HADOOP_PID_DIR=${HADOOP_PID_DIR}

export HADOOP_SECURE_DN_PID_DIR=${HADOOP_PID_DIR}

# A string representing this instance of hadoop. $USER by default.

export HADOOP_IDENT_STRING=$USER




Spark的配置文件


spark-env.sh




# - HADOOP_CONF_DIR, to point Spark towards Hadoop configuration files

# - SPARK_EXECUTOR_INSTANCES, Number of executors to start (Default: )

# - SPARK_EXECUTOR_CORES, Number of cores for the executors (Default: ).

# - SPARK_YARN_DIST_ARCHIVES, Comma separated list of archives to be distributed with the job.

# Options for the daemons used in the standalone deploy mode

# - SPARK_MASTER_IP, to bind the master to a different IP address or hostname

# - SPARK_MASTER_PORT / SPARK_MASTER_WEBUI_PORT, to use non-default ports for the master

# - SPARK_MASTER_OPTS, to set config properties only for the master (e.g. "-Dx=y")

# - SPARK_WORKER_CORES, to set the number of cores to use on this machine

# - SPARK_WORKER_MEMORY, to set how much total memory workers have to give executors (e.g. 1000m, 2g)

# - SPARK_WORKER_PORT / SPARK_WORKER_WEBUI_PORT, to use non-default ports for the worker

# - SPARK_WORKER_INSTANCES, to set the number of worker processes per node

# - SPARK_WORKER_DIR, to set the working directory of worker processes

# - SPARK_WORKER_OPTS, to set config properties only for the worker (e.g. "-Dx=y")

# - SPARK_DAEMON_MEMORY, to allocate to the master, worker and history server themselves (default: 1g).

# - SPARK_HISTORY_OPTS, to set config properties only for the history server (e.g. "-Dx=y")

# - SPARK_SHUFFLE_OPTS, to set config properties only for the external shuffle service (e.g. "-Dx=y")

# - SPARK_DAEMON_JAVA_OPTS, to set config properties for all daemons (e.g. "-Dx=y")

# - SPARK_PUBLIC_DNS, to set the public dns name of the master or workers

# Generic options for the daemons used in the standalone deploy mode

# - SPARK_CONF_DIR      Alternate conf dir. (Default: ${SPARK_HOME}/conf)

# - SPARK_LOG_DIR       Where log files are stored.  (Default: ${SPARK_HOME}/logs)

# - SPARK_PID_DIR       Where the pid file is stored. (Default: /tmp)

# - SPARK_IDENT_STRING  A string representing this instance of spark. (Default: $USER)

# - SPARK_NICENESS      The scheduling priority for daemons. (Default: )

export JAVA_HOME=/home/hadoop/app/jdk

export SCALA_HOME=/home/hadoop/app/scala

export HADOOP_HOME=/home/hadoop/app/hadoop

export HADOOP_CONF_DIR=/home/hadoop/app/hadoop/etc/hadoop

export SPARK_MASTER_IP=192.168.80.145

export SPARK_WORKER_MERMORY=1G




slaves




#

# Licensed to the Apache Software Foundation (ASF) under one or more

# contributor license agreements.  See the NOTICE file distributed with

# this work for additional information regarding copyright ownership.

# The ASF licenses this file to You under the Apache License, Version 2.0

# (the "License"); you may not use this file except in compliance with

# the License.  You may obtain a copy of the License at

#

#    http://www.apache.org/licenses/LICENSE-2.0

#

# Unless required by applicable law or agreed to in writing, software

# distributed under the License is distributed on an "AS IS" BASIS,

# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.

# See the License for the specific language governing permissions and

# limitations under the License.

#

# A Spark Worker will be started on each of the machines listed below.

slave1

slave2




大家一定要去看我的前期博客。


　　下面，我是接着直接来下载并安装配置Zeppelin


　　首先一点想说的是版本问题，为什么本博文的标题我会写清楚版本号呢！原因就是版本不对真的很会坑人。


　  博主我也尝试过hadoop-2.7.3.tar.gz + spark-2.1.0-bin-hadoop2.7.tgz。


　　但是呢，对于zeppelin的版本，目前官网开发团队确实做的不太好，控制版本有待完善。


　　坑人的地方在哪里呢，Zeppelin0.5.6不支持Spark2.1.0这个版本


　　于是我又仔细的查看了官网的教程：


　　得出的结论是我必须要装老版本的，还好的是支持Spark2.0，于是我又安装了Spark1.6.1


　　不过呢如果你没有任何的集群环境，上面这篇是值得参考的，只不过你得自己改下版本号，从2.1.0到1.6.1，其他完全一样。


　　


　　当然，庆幸的是，教大家，灵活会用软连接，实现多版本的使用。


大数据各子项目的环境搭建之建立与删除软连接（博主推荐）






1、上传zeppelin-0.5.6-incubating-bin-all.tgz压缩包










[hadoop@master app]$ pwd

/home/hadoop/app

[hadoop@master app]$ ll

total

drwxrwxr-x    hadoop hadoop  May  : apache-storm-0.9.

drwxrwxr-x   hadoop hadoop  May  : apache-storm-1.0.

drwxrwxr-x    hadoop hadoop  May   : azkaban

drwxrwxr-x    hadoop hadoop  Apr  : elasticsearch-2.4.

drwxrwxr-x    hadoop hadoop  Apr  : elasticsearch-2.4.

lrwxrwxrwx    hadoop hadoop    Apr  : es -> elasticsearch-2.4./

lrwxrwxrwx    hadoop hadoop    Apr  : flume -> flume-1.6.

drwxrwxr-x    hadoop hadoop  Apr  : flume-1.6.

drwxrwxr-x    hadoop hadoop  Apr  : flume-1.7.

lrwxrwxrwx.   hadoop hadoop    Apr  : hadoop -> hadoop-2.6.

drwxr-xr-x.  hadoop hadoop  Apr  : hadoop-2.6.

lrwxrwxrwx.   hadoop hadoop    Apr  : hbase -> hbase-0.98.

drwxrwxr-x.   hadoop hadoop  Apr  : hbase-0.98.

lrwxrwxrwx.   hadoop hadoop    Apr  : hive -> hive-1.0.

drwxrwxr-x.   hadoop hadoop  May  : hive-1.0.

lrwxrwxrwx.   hadoop hadoop    Apr  : jdk -> jdk1..0_79

drwxr-xr-x.   hadoop hadoop  Apr    jdk1..0_79

drwxr-xr-x.   hadoop hadoop  Aug     jdk1..0_60

lrwxrwxrwx    hadoop hadoop    May   : kafka -> kafka_2.-0.8.2.2

drwxr-xr-x    hadoop hadoop  May   : kafka_2.-0.8.2.2

lrwxrwxrwx    hadoop hadoop    Apr  : kibana -> kibana-4.6.-linux-x86_64/

drwxrwxr-x   hadoop hadoop  Nov     kibana-4.6.-linux-x86_64

lrwxrwxrwx    hadoop hadoop    Jun   : scala -> scala-2.11./

drwxrwxr-x    hadoop hadoop  Feb    scala-2.10.

drwxrwxr-x    hadoop hadoop  Mar     scala-2.11.

lrwxrwxrwx    hadoop hadoop    May   : snappy -> snappy-1.1.

drwxr-xr-x    hadoop hadoop  May   : snappy-1.1.

lrwxrwxrwx    hadoop hadoop    Jun   : spark -> spark-1.6.-bin-hadoop2./

drwxr-xr-x   hadoop hadoop  Feb    spark-1.6.-bin-hadoop2.

lrwxrwxrwx.   hadoop hadoop    Apr  : sqoop -> sqoop-1.4.

drwxr-xr-x.   hadoop hadoop  May  : sqoop-1.4.

lrwxrwxrwx    hadoop hadoop    May  : storm -> apache-storm-1.0./

lrwxrwxrwx.   hadoop hadoop    Apr  : zookeeper -> zookeeper-3.4.

drwxr-xr-x.  hadoop hadoop  Apr  : zookeeper-3.4.

[hadoop@master app]$ rz

[hadoop@master app]$ ll

total

drwxrwxr-x    hadoop hadoop       May  : apache-storm-0.9.

drwxrwxr-x   hadoop hadoop       May  : apache-storm-1.0.

drwxrwxr-x    hadoop hadoop       May   : azkaban

drwxrwxr-x    hadoop hadoop       Apr  : elasticsearch-2.4.

drwxrwxr-x    hadoop hadoop       Apr  : elasticsearch-2.4.

lrwxrwxrwx    hadoop hadoop         Apr  : es -> elasticsearch-2.4./

lrwxrwxrwx    hadoop hadoop         Apr  : flume -> flume-1.6.

drwxrwxr-x    hadoop hadoop       Apr  : flume-1.6.

drwxrwxr-x    hadoop hadoop       Apr  : flume-1.7.

lrwxrwxrwx.   hadoop hadoop         Apr  : hadoop -> hadoop-2.6.

drwxr-xr-x.  hadoop hadoop       Apr  : hadoop-2.6.

lrwxrwxrwx.   hadoop hadoop         Apr  : hbase -> hbase-0.98.

drwxrwxr-x.   hadoop hadoop       Apr  : hbase-0.98.

lrwxrwxrwx.   hadoop hadoop         Apr  : hive -> hive-1.0.

drwxrwxr-x.   hadoop hadoop       May  : hive-1.0.

lrwxrwxrwx.   hadoop hadoop         Apr  : jdk -> jdk1..0_79

drwxr-xr-x.   hadoop hadoop       Apr    jdk1..0_79

drwxr-xr-x.   hadoop hadoop       Aug     jdk1..0_60

lrwxrwxrwx    hadoop hadoop         May   : kafka -> kafka_2.-0.8.2.2

drwxr-xr-x    hadoop hadoop       May   : kafka_2.-0.8.2.2

lrwxrwxrwx    hadoop hadoop         Apr  : kibana -> kibana-4.6.-linux-x86_64/

drwxrwxr-x   hadoop hadoop       Nov     kibana-4.6.-linux-x86_64

lrwxrwxrwx    hadoop hadoop         Jun   : scala -> scala-2.11./

drwxrwxr-x    hadoop hadoop       Feb    scala-2.10.

drwxrwxr-x    hadoop hadoop       Mar     scala-2.11.

lrwxrwxrwx    hadoop hadoop         May   : snappy -> snappy-1.1.

drwxr-xr-x    hadoop hadoop       May   : snappy-1.1.

lrwxrwxrwx    hadoop hadoop         Jun   : spark -> spark-1.6.-bin-hadoop2./

drwxr-xr-x   hadoop hadoop       Feb    spark-1.6.-bin-hadoop2.

lrwxrwxrwx.   hadoop hadoop         Apr  : sqoop -> sqoop-1.4.

drwxr-xr-x.   hadoop hadoop       May  : sqoop-1.4.

lrwxrwxrwx    hadoop hadoop         May  : storm -> apache-storm-1.0./

-rw-r--r--    hadoop hadoop  Jun   : zeppelin-0.5.-incubating-bin-all.tgz

lrwxrwxrwx.   hadoop hadoop         Apr  : zookeeper -> zookeeper-3.4.

drwxr-xr-x.  hadoop hadoop       Apr  : zookeeper-3.4.

[hadoop@master app]$ 




2、解压并为创建软连接




[hadoop@master app]$ tar -zxvf zeppelin-0.5.-incubating-bin-all.tgz 












[hadoop@master app]$ ll

total

drwxrwxr-x    hadoop hadoop       May  : apache-storm-0.9.

drwxrwxr-x   hadoop hadoop       May  : apache-storm-1.0.

drwxrwxr-x    hadoop hadoop       May   : azkaban

drwxrwxr-x    hadoop hadoop       Apr  : elasticsearch-2.4.

drwxrwxr-x    hadoop hadoop       Apr  : elasticsearch-2.4.

lrwxrwxrwx    hadoop hadoop         Apr  : es -> elasticsearch-2.4./

lrwxrwxrwx    hadoop hadoop         Apr  : flume -> flume-1.6.

drwxrwxr-x    hadoop hadoop       Apr  : flume-1.6.

drwxrwxr-x    hadoop hadoop       Apr  : flume-1.7.

lrwxrwxrwx.   hadoop hadoop         Apr  : hadoop -> hadoop-2.6.

drwxr-xr-x.  hadoop hadoop       Apr  : hadoop-2.6.

lrwxrwxrwx.   hadoop hadoop         Apr  : hbase -> hbase-0.98.

drwxrwxr-x.   hadoop hadoop       Apr  : hbase-0.98.

lrwxrwxrwx.   hadoop hadoop         Apr  : hive -> hive-1.0.

drwxrwxr-x.   hadoop hadoop       May  : hive-1.0.

lrwxrwxrwx.   hadoop hadoop         Apr  : jdk -> jdk1..0_79

drwxr-xr-x.   hadoop hadoop       Apr    jdk1..0_79

drwxr-xr-x.   hadoop hadoop       Aug     jdk1..0_60

lrwxrwxrwx    hadoop hadoop         May   : kafka -> kafka_2.-0.8.2.2

drwxr-xr-x    hadoop hadoop       May   : kafka_2.-0.8.2.2

lrwxrwxrwx    hadoop hadoop         Apr  : kibana -> kibana-4.6.-linux-x86_64/

drwxrwxr-x   hadoop hadoop       Nov     kibana-4.6.-linux-x86_64

lrwxrwxrwx    hadoop hadoop         Jun   : scala -> scala-2.11./

drwxrwxr-x    hadoop hadoop       Feb    scala-2.10.

drwxrwxr-x    hadoop hadoop       Mar     scala-2.11.

lrwxrwxrwx    hadoop hadoop         May   : snappy -> snappy-1.1.

drwxr-xr-x    hadoop hadoop       May   : snappy-1.1.

lrwxrwxrwx    hadoop hadoop         Jun   : spark -> spark-1.6.-bin-hadoop2./

drwxr-xr-x   hadoop hadoop       Feb    spark-1.6.-bin-hadoop2.

lrwxrwxrwx.   hadoop hadoop         Apr  : sqoop -> sqoop-1.4.

drwxr-xr-x.   hadoop hadoop       May  : sqoop-1.4.

lrwxrwxrwx    hadoop hadoop         May  : storm -> apache-storm-1.0./

drwxr-xr-x    hadoop hadoop       Jan    zeppelin-0.5.-incubating-bin-all

-rw-r--r--    hadoop hadoop  Jun   : zeppelin-0.5.-incubating-bin-all.tgz

lrwxrwxrwx.   hadoop hadoop         Apr  : zookeeper -> zookeeper-3.4.

drwxr-xr-x.  hadoop hadoop       Apr  : zookeeper-3.4.

[hadoop@master app]$ rm zeppelin-0.5.-incubating-bin-all.tgz

[hadoop@master app]$ ln -s zeppelin-0.5.-incubating-bin-all/ zeppelin

[hadoop@master app]$ ll

total

drwxrwxr-x    hadoop hadoop  May  : apache-storm-0.9.

drwxrwxr-x   hadoop hadoop  May  : apache-storm-1.0.

drwxrwxr-x    hadoop hadoop  May   : azkaban

drwxrwxr-x    hadoop hadoop  Apr  : elasticsearch-2.4.

drwxrwxr-x    hadoop hadoop  Apr  : elasticsearch-2.4.

lrwxrwxrwx    hadoop hadoop    Apr  : es -> elasticsearch-2.4./

lrwxrwxrwx    hadoop hadoop    Apr  : flume -> flume-1.6.

drwxrwxr-x    hadoop hadoop  Apr  : flume-1.6.

drwxrwxr-x    hadoop hadoop  Apr  : flume-1.7.

lrwxrwxrwx.   hadoop hadoop    Apr  : hadoop -> hadoop-2.6.

drwxr-xr-x.  hadoop hadoop  Apr  : hadoop-2.6.

lrwxrwxrwx.   hadoop hadoop    Apr  : hbase -> hbase-0.98.

drwxrwxr-x.   hadoop hadoop  Apr  : hbase-0.98.

lrwxrwxrwx.   hadoop hadoop    Apr  : hive -> hive-1.0.

drwxrwxr-x.   hadoop hadoop  May  : hive-1.0.

lrwxrwxrwx.   hadoop hadoop    Apr  : jdk -> jdk1..0_79

drwxr-xr-x.   hadoop hadoop  Apr    jdk1..0_79

drwxr-xr-x.   hadoop hadoop  Aug     jdk1..0_60

lrwxrwxrwx    hadoop hadoop    May   : kafka -> kafka_2.-0.8.2.2

drwxr-xr-x    hadoop hadoop  May   : kafka_2.-0.8.2.2

lrwxrwxrwx    hadoop hadoop    Apr  : kibana -> kibana-4.6.-linux-x86_64/

drwxrwxr-x   hadoop hadoop  Nov     kibana-4.6.-linux-x86_64

lrwxrwxrwx    hadoop hadoop    Jun   : scala -> scala-2.11./

drwxrwxr-x    hadoop hadoop  Feb    scala-2.10.

drwxrwxr-x    hadoop hadoop  Mar     scala-2.11.

lrwxrwxrwx    hadoop hadoop    May   : snappy -> snappy-1.1.

drwxr-xr-x    hadoop hadoop  May   : snappy-1.1.

lrwxrwxrwx    hadoop hadoop    Jun   : spark -> spark-1.6.-bin-hadoop2./

drwxr-xr-x   hadoop hadoop  Feb    spark-1.6.-bin-hadoop2.

lrwxrwxrwx.   hadoop hadoop    Apr  : sqoop -> sqoop-1.4.

drwxr-xr-x.   hadoop hadoop  May  : sqoop-1.4.

lrwxrwxrwx    hadoop hadoop    May  : storm -> apache-storm-1.0./

lrwxrwxrwx    hadoop hadoop    Jun   : zeppelin -> zeppelin-0.5.-incubating-bin-all/

drwxr-xr-x    hadoop hadoop  Jan    zeppelin-0.5.-incubating-bin-all

lrwxrwxrwx.   hadoop hadoop    Apr  : zookeeper -> zookeeper-3.4.

drwxr-xr-x.  hadoop hadoop  Apr  : zookeeper-3.4.

[hadoop@master app]$ 




　　


  3、修改环境变量配置






[hadoop@master app]$ su root

Password:

[root@master app]# vim /etc/profile






#zeppelin

export ZEPPELIN_HOME=/home/hadoop/app/zeppelin

export PATH=$PATH:$ZEPPELIN_HOME/bin








[hadoop@master app]$ su root

Password:

[root@master app]# vim /etc/profile

[root@master app]# source /etc/profile

[root@master app]# 




4.安装并修改配置文件：


（1）安装


　　网络安装版需要运行下面的命令：




./bin/install-interpreter.sh --all




　　　


　　而完整版不需要，直接进入到zeppelin的根目录修改配置文件即可。（我这里）




cd zeppelin-0.5.6-incubating-bin-all




（2）/home/hadoop/app/zeppelin-0.5.6-incubating-bin-all/conf/zeppelin-env.sh




export JAVA_HOME=/home/hadoop/app/jdk

export SPARK_HOME=/home/hadoop/app/spark

export HADOOP_HOME=/home/hadoop/app/hadoop

export HADOOP_CONF_DIR=$HADOOP_HOME/etc/hadoop

export ZEPPELIN_HOME_INTP_JAVA_OPTS="-XX:PermSize=512M -XX:MaxPermSize=1024M"








[hadoop@master conf]$ pwd

/home/hadoop/app/zeppelin-0.5.-incubating-bin-all/conf

[hadoop@master conf]$ ll

total

-rw-r--r--  hadoop hadoop  Jan    configuration.xsl

-rw-r--r--  hadoop hadoop  Jan    log4j.properties

-rw-r--r--  hadoop hadoop  Jan    README.md

-rw-r--r--  hadoop hadoop  Jan    zeppelin-env.sh.template

-rwxr-xr-x  hadoop hadoop  Jan    zeppelin-site.xml.template

[hadoop@master conf]$ mv zeppelin-env.sh.template zeppelin-env.sh

[hadoop@master conf]$ vi zeppelin-env.sh








export JAVA_HOME=/home/hadoop/app/jdk

export SPARK_HOME=/home/hadoop/app/spark

export HADOOP_HOME=/home/hadoop/app/hadoop

export HADOOP_CONF_DIR=$HADOOP_HOME/etc/hadoop

export ZEPPELIN_HOME_INTP_JAVA_OPTS="-XX:PermSize=512M -XX:MaxPermSize=1024M"




　以上，主要是设置Java、Spark、Hadoop的安装路径，并且设置Java内存的使用量。


（3）/home/hadoop/app/zeppelin-0.5.6-incubating-bin-all/conf/zeppelin-site.xml


　　这里，我跟大家普及一个大数据领域的端口知识。（以下是我习惯用的端口情况）


YARN web ui 是8088


HBase web ui是60010、60030


Hadoop web ui是50070


Spark web ui是8081


Ooize web ui是11000


Kibana web ui是5601


Hue web ui是8888


Elasticsearch/Kopf/Head web ui是9200


Resourcemanager web ui是23188


History web ui是19888


Azakban web ui是8443


Storm web ui是9999






　　基于以上考虑，


为了防止端口冲突我将zeppelin的默认端口8080改为8099，具体情况要看自己的机子决定：




<property>

  <name>zeppelin.server.port</name>

  <value>8099</value>

  <description>Server port.</description>

</property>








[hadoop@master conf]$ pwd

/home/hadoop/app/zeppelin-0.5.-incubating-bin-all/conf

[hadoop@master conf]$ ll

total

-rw-r--r--  hadoop hadoop  Jan    configuration.xsl

-rw-r--r--  hadoop hadoop  Jan    log4j.properties

-rw-r--r--  hadoop hadoop  Jan    README.md

-rw-r--r--  hadoop hadoop  Jun   : zeppelin-env.sh

-rwxr-xr-x  hadoop hadoop  Jan    zeppelin-site.xml.template

[hadoop@master conf]$ mv zeppelin-site.xml.template zeppelin-site.xml

[hadoop@master conf]$ vi zeppelin-site.xml








4.启动或关闭：


修改之前的启动脚本为:（当然大家，也可不修改）




#!/bin/bash

echo -e "\033[31m ========Start The Cluster======== \033[0m"

echo -e "\033[31m Starting Hadoop Now !!! \033[0m"

/opt/hadoop-2.7./sbin/start-all.sh

echo -e "\033[31m Starting Spark Now !!! \033[0m"

/opt/spark-2.0.-bin-hadoop2./sbin/start-all.sh

echo -e "\033[31m Starting Zeppelin Now !!! \033[0m"

/opt/zeppelin-0.6.-bin-all/bin/zeppelin-daemon.sh start

echo -e "\033[31m The Result Of The Command \"jps\" :  \033[0m"

jps

echo -e "\033[31m ========END======== \033[0m"




修改之前的关闭脚本为：（当然大家，也可不修改）




#!/bin/bash

echo -e "\033[31m ===== Stoping The Cluster ====== \033[0m"

echo -e "\033[31m Stoping Zeppelin Now !!! \033[0m"

/opt/zeppelin-0.6.-bin-all/bin/zeppelin-daemon.sh stop

echo -e "\033[31m Stoping Spark Now !!! \033[0m"

/opt/spark-2.0.-bin-hadoop2./sbin/stop-all.sh

echo -e "\033[31m Stopting Hadoop Now !!! \033[0m"

/opt/hadoop-2.7./sbin/stop-all.sh

echo -e "\033[31m The Result Of The Command \"jps\" :  \033[0m"

jps

echo -e "\033[31m ======END======== \033[0m"




启动Apache Zeppelin


　　在zeppelin_home目录下执行如下命令：




[hadoop@master zeppelin-0.5.6-incubating-bin-all]$ bin/zeppelin-daemon.sh start










[hadoop@master zeppelin-0.5.-incubating-bin-all]$ pwd

/home/hadoop/app/zeppelin-0.5.-incubating-bin-all

[hadoop@master zeppelin-0.5.-incubating-bin-all]$ ll

total

drwxr-xr-x   hadoop hadoop     Jun   : bin

drwxr-xr-x   hadoop hadoop     Jun   : conf

-rw-r--r--   hadoop hadoop      Jan    DISCLAIMER

drwxr-xr-x  hadoop hadoop     Jun   : interpreter

drwxr-xr-x   hadoop hadoop     Jun   : lib

-rw-r--r--   hadoop hadoop    Jan    LICENSE

drwxr-xr-x   hadoop hadoop     Jun   : licenses

drwxr-xr-x   hadoop hadoop     Jun   : notebook

-rw-r--r--   hadoop hadoop     Jan    NOTICE

-rw-r--r--   hadoop hadoop     Jan    README.md

-rw-r--r--   hadoop hadoop    Jan    zeppelin-server-0.5.-incubating.jar

-rw-r--r--   hadoop hadoop  Jan    zeppelin-web-0.5.-incubating.war

[hadoop@master zeppelin-0.5.-incubating-bin-all]$ bin/zeppelin-daemon.sh start

Log dir doesn't exist, create /home/hadoop/app/zeppelin/logs

Pid dir doesn't exist, create /home/hadoop/app/zeppelin/run

Zeppelin start                                             [  OK  ]

[hadoop@master zeppelin-0.5.-incubating-bin-all]$ ls

bin  conf  DISCLAIMER  interpreter  lib  LICENSE  licenses  logs  notebook  NOTICE  README.md  run  webapps  zeppelin-server-0.5.-incubating.jar  zeppelin-web-0.5.-incubating.war

[hadoop@master zeppelin-0.5.-incubating-bin-all]$ 




　其启动/停止命令： bin/zeppelin-daemon.sh start/stop。


　启动之后，打开localhost:8099访问zepplin主页。




http://localhost:8099/#/




我这里是




在打开Zeppelin Web UI界面之后，然后如下




　　　　单击，会显示出Zeppelin Tutorial Notebook现成的范例供用户参考


然后，可以看到










后续博客


Zeppelin的入门使用系列之创建新的Notebook（一）
												


											
hadoop-2.6.0.tar.gz + spark-1.6.1-bin-hadoop2.6.tgz + zeppelin-0.5.6-incubating-bin-all.tgz（master、slave1和slave2）（博主推荐）（图文详解）的更多相关文章
	

    
						
  1. [转]protobuf-2.5.0.tar.gz的下载与安装
		
    protobuf-2.5.0.tar.gz的下载与安装 原文地址:http://blog.csdn.net/tdmyl/article/details/31811317 版权声明:本文为博主原创文章, ...
    
		
    2. 
						
  2. hadoop-2.6.0.tar.gz + spark-1.6.1-bin-hadoop2.6.tgz的集群搭建（单节点）（CentOS系统）
		
    福利 => 每天都推送 欢迎大家,关注微信扫码并加入我的4个微信公众号:   大数据躺过的坑      Java从入门到架构师      人工智能躺过的坑         Java全栈大联盟   ...
    
		
    3. 
						
  3. linux下安装nginx（nginx（nginx-1.8.0.tar.gz），openssl（openssl-fips-2.0.9.tar.gz） ，zlib（zlib-1.2.11.tar.gz），pcre（pcre-8.39.tar.gz））
		
    :要按顺序安装: 1:先检查是否安装 gcc ,没有先安装:通过yum install gcc-c++完成安 2:openssl : tar -zxf  openssl-fips-2.0.9.tar. ...
    
		
    4. 
						
  4. Spark on YARN模式的安装（spark-1.6.1-bin-hadoop2.6.tgz    +  hadoop-2.6.0.tar.gz）（master、slave1和slave2）（博主推荐）
		
    说白了 Spark on YARN模式的安装,它是非常的简单,只需要下载编译好Spark安装包,在一台带有Hadoop YARN客户端的的机器上运行即可.  Spark on YARN简介与运行wor ...
    
		
    5. 
						
  5. Hadoop-2.2.0 （传 hadoop-2.2.0.tar.gz）
		
    配置hadoop 2.1 上传hadoop包 2.2 解压hadoop包 首先在根目录下创建一个cloud目录 mkdir /cloud tar -zxvf hadoop-2.2.0.tar.gz - ...
    
		
    6. 
						
  6. hadoop-2.6.0.tar.gz + spark-1.5.2-bin-hadoop2.6.tgz的集群搭建（单节点）
		
    前言 本人呕心沥血所写,经过好一段时间反复锤炼和整理修改.感谢所参考的博友们!同时,欢迎前来查阅赏脸的博友们收藏和转载,附上本人的链接.http://www.cnblogs.com/zlslch/p/ ...
    
		
    7. 
						
  7. hadoop-2.6.0.tar.gz + spark-1.5.2-bin-hadoop2.6.tgz  的集群搭建（3节点和5节点皆适用）
		
    本人呕心沥血所写,经过好一段时间反复锤炼和整理修改.感谢所参考的博友们!同时,欢迎前来查阅赏脸的博友们收藏和转载,附上本人的链接.http://www.cnblogs.com/zlslch/p/584 ...
    
		
    8. 
						
  8. hadoop-2.6.0.tar.gz + spark-1.5.2-bin-hadoop2.6.tgz的集群搭建（单节点）（Ubuntu系统）
		
    前言 本人呕心沥血所写,经过好一段时间反复锤炼和整理修改.感谢所参考的博友们!同时,欢迎前来查阅赏脸的博友们收藏和转载,附上本人的链接.http://www.cnblogs.com/zlslch/p/ ...
    
		
    9. 
						
  9. hadoop-2.7.3.tar.gz + spark-2.0.2-bin-hadoop2.7.tgz + zeppelin-0.6.2-incubating-bin-all.tgz（master、slave1和slave2）（博主推荐）（图文详解）
		
    不多说,直接上干货! 我这里,采取的是ubuntu 16.04系统,当然大家也可以在CentOS6.5里,这些都是小事 CentOS 6.5的安装详解 hadoop-2.6.0.tar.gz + sp ...
    
		
    10. 
				

	


随机推荐
	

    
									
  1. 利用闭包特性改写addEventListener的回调函数
			
    var numClicks = 0; document.addEventListener("click",function(){ alert( ++numClicks); },fa ...
    
			
    2. 
						
  2. 快速的熟悉一个angular的项目从run看起
			
    config之类的都会注入到controller或者run里边
    
			
    3. 
						
  3. Javascript-我对作用链、闭包、原型及原型链的理解
			
    Javascript-基础概念总结(2) 最近学习一些javascript基础知识,也解决了很多之前的疑惑,记得第一次被问及怎样理解闭包时,我的回答是:就是类似于封装吧!现在想想是有多白痴,学习技术是 ...
    
			
    4. 
						
  4. 继续吐槽XE5中的boost::thread问题：找不到CC32130MT.DLL
			
    在XE5中,随便创建一个HelloWorld程序,加入boost::thread的功能,编译链接后,无法运行,无法调试,提示找不到CC32130MT.DLL 这个DLL名字看上去好像是跟多线程有关的, ...
    
			
    5. 
						
  5. 20165210 Java第二次实验报告
			
    20165210 实验二 Java面向对象程序设计 一.面向对象程序设计1--单元测试和TDD 实验要求 参考 http://www.cnblogs.com/rocedu/p/6371315.html ...
    
			
    6. 
						
  6. SQLAlchemyの增删改查
			
    用a*my写原味sql from sqlalchemy.ext.declarative import declarative_base from sqlalchemy import Column, I ...
    
			
    7. 
						
  7. php是如何工作的
			
    a:前提条件: apache服务器启动正常工作 b:客户端浏览器在地址栏输入一个程序地栏 按回车发送请求 {请求}http://127.0.0.1/day03/1.php c:apache接收请求,并 ...
    
			
    8. 
						
  8. LeetCode 315. Count of Smaller Numbers After Self
			
    原题链接在这里:https://leetcode.com/problems/count-of-smaller-numbers-after-self/ 题目: You are given an inte ...
    
			
    9. 
						
  9. 洛谷 P3052 [USACO12MAR]摩天大楼里的奶牛Cows in a Skyscraper
			
    题目描述 A little known fact about Bessie and friends is that they love stair climbing races. A better k ...
    
			
    10. 
						
  10. 在CentOS上安装Java开发环境：使用yum安装jdk
			
    请参考百度经验:http://jingyan.baidu.com/article/4853e1e51d0c101909f72607.html 如果您阅读过此文章有所收获,请为我顶一个,如果文章中有错误 ...
    
			
    11.