官网:

http://archive.cloudera.com/cdh5/cdh/5/hue-3.7.0-cdh5.3.6/

一、Hue环境搭建

1. 下载

http://archive.cloudera.com/cdh5/cdh/5/hue-3.7.0-cdh5.3.6.tar.gz

2. 解压

tar -zxvf hue-3.7.-cdh5.3.6.tar.gz -C /opt/modules/cdh/

3. 安装依赖包

sudo yum -y install ant asciidoc cyrus-sasl-devel cyrus-sasl-gssapi gcc gcc-c++ krb5-devel libtidy libxml2-devel libxslt-devel openldap-devel python-devel sqlite-devel openssl-devel mysql-devel gmp-devel

4. 编译安装

cd /opt/modules/cdh/hue-3.7.-cdh5.3.6/

make apps

5. 启动

build/env/bin/supervisor

首次登陆需要设置用户名和密码,为了方便,建议使用hdfs有权限的用户

二、集成

1. [desktop]

  # Set this to a random string, the longer the better.
# This is used for secure hashing in the session store.
secret_key=hue_session_store_secret_key_30_60_character # Webserver listens on this address and port
http_host=beifeng-hadoop-
http_port= # Time zone name
time_zone=Asia/Shanghai

2. 集成hdfs,yarn

1)配置hue.ini中的hdfs

[hadoop]

  # Configuration for HDFS NameNode
# ------------------------------------------------------------------------
[[hdfs_clusters]]
# HA support by using HttpFs [[[default]]]
# Enter the filesystem uri
fs_defaultfs=hdfs://beifeng-hadoop-02:9000 # NameNode logical name.
## logical_name= # Use WebHdfs/HttpFs as the communication mechanism.
# Domain should be the NameNode or HttpFs host.
# Default port is for HttpFs.
webhdfs_url=http://beifeng-hadoop-02:50070/webhdfs/v1
# webhdfs_url=http://beifeng-hadoop-02:14000/webhdfs/v1 # Change this if your HDFS cluster is Kerberos-secured
## security_enabled=false # Default umask for file and directory creation, specified in an octal value.
## umask= # Directory of the Hadoop configuration
hadoop_conf_dir=/opt/modules/cdh/hadoop-2.5.-cdh5.3.6/etc/hadoop

2)配置hdfs-site.xml

<configuration>

        <!-- 数据副本数,副本数等于所有datanode的总和 -->
<property>
<name>dfs.replication</name>
<value>1</value>
</property> <property>
<name>dfs.namenode.secondary.http-address</name>
<value>beifeng-hadoop-02:50090</value>
</property> <property>
<name>dfs.permissions.enabled</name>
<value>false</value>
</property> <property>
<name>dfs.webhdfs.enabled</name>
<value>true</value>
</property> </configuration>

3)配置core-site.xml

   <!-- HUI -->
<property>
<name>hadoop.proxyuser.hue.hosts</name>
<value>*</value>
</property>
<property>
<name>hadoop.proxyuser.hue.groups</name>
<value>*</value>
</property>

4)配置httpfs-site.xml

<configuration>

   <!-- HUI -->
<property>
<name>hadoop.proxyuser.hue.hosts</name>
<value>*</value>
</property>
<property>
<name>hadoop.proxyuser.hue.groups</name>
<value>*</value>
</property> </configuration>

5)配置hue.ini中的yarn

  [[yarn_clusters]]

    [[[default]]]
# Enter the host on which you are running the ResourceManager
resourcemanager_host=beifeng-hadoop- # The port where the ResourceManager IPC listens on
resourcemanager_port= # Whether to submit jobs to this cluster
submit_to=True # Resource Manager logical name (required for HA)
## logical_name= # Change this if your YARN cluster is Kerberos-secured
## security_enabled=false # URL of the ResourceManager API
resourcemanager_api_url=http://beifeng-hadoop-02:8088 # URL of the ProxyServer API
proxy_api_url=http://beifeng-hadoop-02:8088 # URL of the HistoryServer API
istory_server_api_url=http://beifeng-hadoop-02:19888

5)重启hdfs集群

6)启动httpfs

sbin/httpfs.sh start

3. 集成hive

1)修改hui中的beeswax

  # Host where HiveServer2 is running.
# If Kerberos security is enabled, use fully-qualified domain name (FQDN).
hive_server_host=beifeng-hadoop- # Port where HiveServer2 Thrift server runs on.
hive_server_port= # Hive configuration directory, where hive-site.xml is located
hive_conf_dir=/opt/modules/cdh/hive-0.13.-cdh5.3.6/conf # Timeout in seconds for thrift calls to Hive service
server_conn_timeout=

2)修改hive-site.xml

<property>
<name>hive.server2.authentication</name>
<value>NOSASL</value>
<description>
Client authentication types.
NONE: no authentication check
LDAP: LDAP/AD based authentication
KERBEROS: Kerberos/GSSAPI authentication
CUSTOM: Custom authentication provider
(Use with property hive.server2.custom.authentication.class)
PAM: Pluggable authentication module.
</description>
</property>

3)重新启动hiveserver2

nohup hive --service metastore > ~/hive_metastore.run.log >& &
nohup hive --service hiveserver2 > ~/hiveserver2.run.log >& &

4)使用hue检验hive

4. 集成oozie

1)在oozie-site.xml添加以下配置

    <!-- Default proxyuser configuration for Hue -->
<property>
<name>oozie.service.ProxyUserService.proxyuser.hue.hosts</name>
<value>*</value>
</property> <property>
<name>oozie.service.ProxyUserService.proxyuser.hue.groups</name>
<value>*</value>
</property>

2)在hue.ini中启用oozie的配置

[liboozie]
# The URL where the Oozie service runs on. This is required in order for
# users to submit jobs. Empty value disables the config check.
oozie_url=http://beifeng-hadoop-02:11000/oozie # Requires FQDN in oozie_url if enabled
## security_enabled=false # Location on HDFS where the workflows/coordinator are deployed when submitted.
remote_deployement_dir=/user/hue/oozie/deployments ###########################################################################
# Settings to configure the Oozie app
########################################################################### [oozie]
# Location on local FS where the examples are stored.
## local_data_dir=..../examples # Location on local FS where the data for the examples is stored.
## sample_data_dir=...thirdparty/sample_data # Location on HDFS where the oozie examples and workflows are stored.
remote_data_dir=/user/hue/oozie/workspaces # Maximum of Oozie workflows or coodinators to retrieve in one API call.
oozie_jobs_count= # Use Cron format for defining the frequency of a Coordinator instead of the old frequency number/unit.
enable_cron_scheduling=true

3)解决问题/user/oozie/share/lib Oozie 分享库 (Oozie Share Lib) 无法安装到默认位置。

(1)修改oozie-site.xml

    <property>
<name>oozie.service.WorkflowAppService.system.libpath</name>
<value>/user/ooozie/share/lib</value>
<description>
System library path to use for workflow applications.
This path is added to workflow application if their job properties sets
the property 'oozie.use.system.libpath' to true.
</description>
</property>

(2)将共享依赖包解压上传hdfs的/user/oozie/share/lib

bin/oozie-setup.sh sharelib create -fs hdfs://beifeng-hadoop-02:9000/ -locallib oozie-sharelib-4.0.0-cdh5.3.6-yarn.tar.gz

(3)重新启动oozie

(4)重新启动hue

5. 集成HBase

1)修改hue.ini中HBase相关配置

[hbase]
# Comma-separated list of HBase Thrift servers for clusters in the format of '(name|host:port)'.
# Use full hostname with security.
hbase_clusters=(HBaseCluster|beifeng-hadoop-:) # HBase configuration directory, where hbase-site.xml is located.
hbase_conf_dir=/opt/modules/cdh/hbase-0.98.-cdh5.3.6/conf # Hard limit of rows or columns per row fetched before truncating.
## truncate_limit = # 'buffered' is the default of the HBase Thrift Server and supports security.
# 'framed' can be used to chunk up responses,
# which is useful when used in conjunction with the nonblocking server in Thrift.
## thrift_transport=buffered

2)启动HBase

bin/start-hbase.sh

3)启动thrift server

bin/hbase-daemon.sh start thrift

《OD大数据实战》Hue环境搭建的更多相关文章

  1. 《OD大数据实战》环境整理

    一.关机后服务重新启动 1. 启动hadoop服务 sbin/hadoop-daemon.sh start namenode sbin/hadoop-daemon.sh start datanode ...

  2. 《OD大数据实战》Hive环境搭建

    一.搭建hadoop环境 <OD大数据实战>hadoop伪分布式环境搭建 二.Hive环境搭建 1. 准备安装文件 下载地址: http://archive.cloudera.com/cd ...

  3. 《OD大数据实战》HDFS入门实例

    一.环境搭建 1.  下载安装配置 <OD大数据实战>Hadoop伪分布式环境搭建 2. Hadoop配置信息 1)${HADOOP_HOME}/libexec:存储hadoop的默认环境 ...

  4. 《OD大数据实战》驴妈妈旅游网大型离线数据电商分析平台

    一.环境搭建 1. <OD大数据实战>Hadoop伪分布式环境搭建 2. <OD大数据实战>Hive环境搭建 3. <OD大数据实战>Sqoop入门实例 4. &l ...

  5. 《OD大数据实战》Hadoop伪分布式环境搭建

    一.安装并配置Linux 8. 使用当前root用户创建文件夹,并给/opt/下的所有文件夹及文件赋予775权限,修改用户组为当前用户 mkdir -p /opt/modules mkdir -p / ...

  6. 《OD大数据实战》Storm环境搭建

    一.环境搭建 1. 下载 http://www.apache.org/dyn/closer.lua/storm/apache-storm-0.9.6/apache-storm-0.9.6.tar.gz ...

  7. 《OD大数据实战》MongoDB环境搭建

    一.MongonDB环境搭建 1. 下载 https://fastdl.mongodb.org/linux/mongodb-linux-x86_64-3.0.6.tgz 2. 解压 tar -zxvf ...

  8. 《OD大数据实战》HBase环境搭建

    一.环境搭建 1. 下载 hbase-0.98.6-cdh5.3.6.tar.gz 2. 解压 tar -zxvf hbase-0.98.6-cdh5.3.6.tar.gz -C /opt/modul ...

  9. 《OD大数据实战》Oozie环境搭建

    一.Oozie环境搭建 1. 下载oozie-4.0.0-cdh5.3.6.tar.gz 下载地址:http://archive.cloudera.com/cdh5/cdh/5/ 2. 解压 tar ...

随机推荐

  1. 功率单位mW 和 dBm 的换算

    无线电发射机输出的射频信号,通过馈线(电缆)输送到天线,由天线以电磁波形式辐射出去.电磁波到达接收地点后,由天线接收下来(仅仅接收很小很小一部分功率),并通过馈线送到无线电接收机.因此在无线网络的工程 ...

  2. HDOJ1050

    Moving Tables Time Limit: 2000/1000 MS (Java/Others)    Memory Limit: 65536/32768 K (Java/Others)Tot ...

  3. ajax原理总结附简单实例及其优点

    在工作中用了Ajax N多次了,也看过一些相关方面的书籍,也算是认识了它,但是一直没有认真总结和整理过相关的东东,失败! 近有闲情,将之总结如下: [名称] Ajax是Asynchronous Jav ...

  4. Ckeditor 中一些核心的对象的作用

    1.CKEditor CKEditor对象用于掌管全局,他是一个单例对象,管理着所有实例化了的编辑框. 通过replace方法创建编辑框实例. 2.CKEditor.editor 表示一个编辑框实例, ...

  5. rpc 小例子

    RpcFramework /* * Copyright 2011 Alibaba.com All right reserved. This software is the * confidential ...

  6. 转载 Memcached BinaryProtocol incr指令内存泄露的bug

    缘起 最近有个分布式限速的需求.支付宝的接口双11只允许每秒调用10次. 单机的限速,自然是用google guava的RateLimiter. http://docs.guava-libraries ...

  7. js的正则表达式

    正则表达式(regular expression)是一中描述字符模式的对象,js的RegExp类表示正则表达式,String与RegExp都定义了相应的方法来操作正则表达式,比如模式匹配,文本检索和替 ...

  8. java编译做了哪些事?

    Javac编译器,主要做了如下的事情:1.解析与填充符号表:    2.注解处理器:    3.语义分析与字节码生成:        3.1.标注检查        3.2.数据及控制流分析      ...

  9. libevent功能使用简介

    http://blog.csdn.net/happyanger6/article/details/7272324 1. 介绍 libevent是一个用来开发可扩展的网络服务器的事件通知函数库.当一个文 ...

  10. linux 中permission denied的问题:

    执行安装命令的时候 ./install 遇到 permission denied, bash: ./install: Permission denied另外,在 root下也是同样的问题, 请教该如何 ...