hue下载地址:https://github.com/cloudera/hue

hue学习文档地址:http://archive.cloudera.com/cdh5/cdh/5/hue-3.7.0-cdh5.3.6/manual.html

我目前使用的是hue-3.7.0-cdh5.3.6

hue(HUE=Hadoop User Experience)

Hue是一个开源的Apache Hadoop UI系统,由Cloudera Desktop演化而来,最后Cloudera公司将其贡献给Apache基金会的Hadoop社区,它是基于Python Web框架Django实现的。

通过使用Hue我们可以在浏览器端的Web控制台上与Hadoop集群进行交互来分析处理数据,例如操作HDFS上的数据,运行MapReduce Job,执行Hive的SQL语句,浏览HBase数据库等等。

hue特点:

  能够支持各个版本的hadoop
  hue默认数据库:sql lite
  文件浏览器:对数据进行增删改查
  hue下载src包进行一次编译,二次编译,在这用的是已经一次编译

hue部署:

1、下载依赖包:yum源安装

sudo yum -y install ant asciidoc cyrus-sasl-devel cyrus-sasl-gssapi gcc gcc-c++ krb5-devel libtidy libxml2-devel libxslt-devel mvn mysql mysql-devel openldap-devel python-devel sqlite-devel openssl-devel  

sudo yum install ant asciidoc cyrus-sasl-devel cyrus-sasl-gssapi cyrus-sasl-plain gcc gcc-c++ krb5-devel libffi-devel libxml2-devel libxslt-devel make mysql mysql-devel openldap-devel python-devel sqlite-devel gmp-devel

2、解压hue tar包

  tar -zxvf hue-3.7.0-cdh5.3.6.tar.gz -C /指定的目录

3、二次编译

  进入hue目录:执行make apps  会出现个build目录

错误:(centos 7会有)
error: static declaration of ‘X509_REVOKED_dup’ follows non-static declaration
static X509_REVOKED * X509_REVOKED_dup(X509_REVOKED *orig) {
^
In file included from /usr/include/openssl/ssl.h:156:0,
from OpenSSL/crypto/x509.h:17,
from OpenSSL/crypto/crypto.h:30,
from OpenSSL/crypto/crl.c:3:
/usr/include/openssl/x509.h:751:15: note: previous declaration of ‘X509_REVOKED_dup’ was here
X509_REVOKED *X509_REVOKED_dup(X509_REVOKED *rev);
^
error: command 'gcc' failed with exit status 1

给下面两个删掉:/usr/include/openssl/x509.h -》751、752行
X509_REVOKED *X509_REVOKED_dup(X509_REVOKED *rev);
X509_REQ *X509_REQ_dup(X509_REQ *req);
##必须删掉,注释不行

4、进入到hue-3.7.0-cdh5.3.6/desktop/conf

配置hue.ini文件:

secret_key=jFE93j;2[290-eiw.KEiwN2s3['d;/.q[eIW^y#e=+Iei*@Mn<qW5o
http_host=hadoop01.xningge.com
http_port=8888
time_zone=Asia/Shanghai

5、启动hue

  两种方式

1-->cd build/env/bin---》./supervisor
2-->build/env/bin/supervisor

 6、浏览器访问hue

  主机名+8888
  创建用户名和密码

hue和hadoop的组件配置

1、hdfs的配置

  在hdfs-site.xml中配置

<property>
  <name>dfs.webhdfs.enabled</name>
  <value>true</value>
</property>

  在core-site.xml中配置

  

<property>
  <name>hadoop.proxyuser.hue.hosts</name>
  <value>*</value>
</property>
<property>
  <name>hadoop.proxyuser.hue.groups</name>
  <value>*</value>
</property>

2、重新启动hdfs进程

  sbin/start-yarn.sh

3、hue配置

  [[hdfs_clusters]]
  # HA support by using HttpFs

[[[default]]]
# Enter the filesystem uri
fs_defaultfs=hdfs://hadoop01.xningge.com:8020

# NameNode logical name.
## logical_name=

# Use WebHdfs/HttpFs as the communication mechanism.
# Domain should be the NameNode or HttpFs host.
# Default port is 14000 for HttpFs.
webhdfs_url=http://hadoop01.xningge.com:50070/webhdfs/v1

# This is the home of your Hadoop HDFS installation
hadoop_hdfs_home=/opt/modules/cdh/hadoop-2.5.0-cdh5.3.6

# Use this as the HDFS Hadoop launcher script
hadoop_bin=/opt/modules/cdh/hadoop-2.5.0-cdh5.3.6/bin

# Change this if your HDFS cluster is Kerberos-secured
## security_enabled=false

# Default umask for file and directory creation, specified in an octal value.
## umask=022

# Directory of the Hadoop configuration
hadoop_conf_dir=/opt/modules/cdh/hadoop-2.5.0-cdh5.3.6/etc/hadoop

[[yarn_clusters]]

[[[default]]]
# Enter the host on which you are running the ResourceManager
resourcemanager_host=hadoop01.xningge.com

# The port where the ResourceManager IPC listens on
resourcemanager_port=8032

# Whether to submit jobs to this cluster
submit_to=True

# Resource Manager logical name (required for HA)
## logical_name=

# Change this if your YARN cluster is Kerberos-secured
## security_enabled=false

# URL of the ResourceManager API
resourcemanager_api_url=http://hadoop01.xningge.com:8088

# URL of the ProxyServer API
proxy_api_url=http://hadoop01.xningge.com:8088

# URL of the HistoryServer API
history_server_api_url=http://hadoop01.xningge.com:19888

eg:此配置都是伪分布式模式

4、启动hue服务

  build/env/bin/supervisor

hue与hive配置

1、hive配置

  在hive-site.xml配置

<property>
  <name>hive.server2.thrift.bind.host</name>
  <value>hadoop01.xningge.com</value>
</property>
<property>
  <name>hive.metastore.uris</name>
  <value>hadoop01.xningge.com:9083</value>
</property>

2、启动hive服务

  bin/hiveserver2 &

  bin/hive --service metastore &

3 、hue配置

  修改hue.ini文件

[beeswax]

# Host where HiveServer2 is running.
# If Kerberos security is enabled, use fully-qualified domain name (FQDN).
hive_server_host=hadoop01.xningge.com

# Port where HiveServer2 Thrift server runs on.
hive_server_port=10000

# Hive configuration directory, where hive-site.xml is located
hive_conf_dir=/opt/modules/cdh/hive-0.13.1-cdh5.3.6/conf

# Timeout in seconds for thrift calls to Hive service
server_conn_timeout=120

# Choose whether Hue uses the GetLog() thrift call to retrieve Hive logs.
# If false, Hue will use the FetchResults() thrift call instead.
## use_get_log_api=true

# Set a LIMIT clause when browsing a partitioned table.
# A positive value will be set as the LIMIT. If 0 or negative, do not set any limit.
## browse_partitioned_table_limit=250

# A limit to the number of rows that can be downloaded from a query.
# A value of -1 means there will be no limit.
# A maximum of 65,000 is applied to XLS downloads.
## download_row_limit=1000000

# Hue will try to close the Hive query when the user leaves the editor page.
# This will free all the query resources in HiveServer2, but also make its results inaccessible.
## close_queries=false

# Thrift version to use when communicating with HiveServer2
## thrift_version=5

hue与关系型数据库配置

[librdbms]
# The RDBMS app can have any number of databases configured in the databases
# section. A database is known by its section name
# (IE sqlite, mysql, psql, and oracle in the list below).

[[databases]]
# sqlite configuration.
[[[sqlite]]] //注意这里一定要取消注释
# Name to show in the UI.
nice_name=SQLite

# For SQLite, name defines the path to the database.
name=/opt/modules/hue-3.7.0-cdh5.3.6/desktop/desktop.db

# Database backend to use.
engine=sqlite

# Database options to send to the server when connecting.
# https://docs.djangoproject.com/en/1.4/ref/databases/
## options={}

# mysql, oracle, or postgresql configuration.

  ##注意:这里的数据不能改动,默认是hue的数据库

[[[mysql]]] //注意这里一定要取消注释
# Name to show in the UI.
nice_name="My SQL DB"

# For MySQL and PostgreSQL, name is the name of the database.
# For Oracle, Name is instance of the Oracle server. For express edition
# this is 'xe' by default.
name=sqoop//这个name是数据库表名

# Database backend to use. This can be:
# 1. mysql
# 2. postgresql
# 3. oracle
engine=mysql

# IP or hostname of the database to connect to.
host=hadoop01.xningge.com

# Port the database server is listening to. Defaults are:
# 1. MySQL: 3306
# 2. PostgreSQL: 5432
# 3. Oracle Express Edition: 1521
port=3306

# Username to authenticate with when connecting to the database.
user=xningge

# Password matching the username to authenticate with when
# connecting to the database.
password=???

# Database options to send to the server when connecting.
# https://docs.djangoproject.com/en/1.4/ref/databases/
## options={}

hue与zookeeper配置

  只需修改hue.ini文件
  host_ports=hadoop01.xningge.com:2181
  启动zookeeper:

hue与oozie的配置

修改:hue.ini文件
[liboozie]
oozie_url=http://hadoop01.xningge.com:11000/oozie

  如果没有出来的:
  修改:oozie-site.xml
  <property>
    <name>oozie.service.WorkflowAppService.system.libpath</name>
    <value>/user/oozie/share/lib</value>
  </property>

  到oozie目录下重新创建sharelib库:
  bin/oozie-setup.sh sharelib create -fs hdfs://hadoop01.xningge.com:8020 -locallib oozie-sharelib-4.0.0-cdh5.3.6-yarn.tar.gz
  启动oozie:bin/oozied.sh start

hue与hbase的配置

修改hue.ini文件:
hbase_clusters=(Cluster|hadoop01.xningge.com:9090)
hbase_conf_dir=/opt/cdh_5.3.6/hbase-0.98.6-cdh5.3.6/conf
修改hbase-site.xml,添加以下配置:
<property>
  <name>hbase.regionserver.thrift.http</name>
  <value>true</value>
</property>
<property>
  <name>hbase.thrift.support.proxyuser</name>
  <value>true</value>
</property>
启动hbase:
bin/start-hbase.sh
bin/hbase-daemon.sh start thrift

##hbase完全分布式
hbase_clusters=(Cluster1|hostname:9090,Cluster2|hostname:9090,Cluster3|hostname:9090)

cdh版本的hue安装配置部署以及集成hadoop hbase hive mysql等权威指南的更多相关文章

  1. cdh版本的sqoop安装以及配置

    sqoop安装需要提前安装好sqoop依赖:hadoop .hive.hbase.zookeeper hadoop安装步骤请访问:http://www.cnblogs.com/xningge/arti ...

  2. CDH版本的oozie安装执行bin/oozie-setup.sh prepare-war,没生成oozie.war?

    不多说,直接上干货! 前期博客 Oozie安装部署 问题描述 bin/oozie-setup.sh prepare-war 解决办法 [hadoop@bigdatamaster bin]$ pwd / ...

  3. hbase安装配置(整合到hadoop)

    hbase安装配置(整合到hadoop) 如果想详细了解hbase的安装:http://abloz.com/hbase/book.html 和官网http://hbase.apache.org/ 1. ...

  4. hue安装与部署

    运行环境 centOS 6.6 hadoop 2.4.0 hive 1.2.0 spark 1.4.1 HUE 3.9 介绍: Hue是一个开源的Apache Hadoop UI系统,最早是由Clou ...

  5. hbase单机及集群安装配置,整合到hadoop

    问题导读:1.配置的是谁的目录conf/hbase-site.xml,如何配置hbase.rootdir2.如何启动hbase?3.如何进入hbase shell?4.ssh如何达到互通?5.不安装N ...

  6. Phoenix |安装配置| 命令行操作| 与hbase的映射| spark对其读写

    Phoenix Phoenix是HBase的开源SQL皮肤.可以使用标准JDBC API代替HBase客户端API来创建表,插入数据和查询HBase数据. 1.特点 1) 容易集成:如Spark,Hi ...

  7. Hue集成Hadoop和Hive

    一.环境准备 1.下载Hue:https://dl.dropboxusercontent.com/u/730827/hue/releases/3.12.0/hue-3.12.0.tgz 2.安装依赖 ...

  8. cdh版本的zookeeper安装以及配置(伪分布式模式)

    需要的软件包:zookeeper-3.4.5-cdh5.3.6.tar.gz  1.将软件包上传到Linux系统指定目录下: /opt/softwares/cdh 2.解压到指定的目录:/opt/mo ...

  9. hadoop cdh 4.5的安装配置

    春节前用的shark,是从github下载的源码,自己编译.shark的master源码仅支持hive 0.9,支持hive 0.11的shark只是个分支,不稳定,官方没有发布release版,在使 ...

随机推荐

  1. POJ1635:Subway tree systems

    链接:http://poj.org/problem?id=1635 填坑树同构 题目给出的是除根外的括号序列表示. 其实只要跟你说hash大家都能写得出来…… hash函数取个效果别太差的就行了吧 # ...

  2. zoj 3494:BCD Code

    Description Binary-coded decimal (BCD) is an encoding for decimal numbers in which each digit is rep ...

  3. oracle erp 表结构

    BOM模块常用表结构 表名: bom.bom_bill_of_materials 说明: BOM清单父项目 BILL_SEQUENCE_ID NUMBER 清单序号(关键字)ASSEMBLY_ITEM ...

  4. java与C++变量初始化的对比

    java尽力保证:所有变量在使用前都能得到恰当的初始化 ①函数/方法局部变量的初始化 在C/C++中,变量的初始化还是得依赖于程序员的自觉性.对于函数局部变量,编译器不会为基本类型赋予默认初始值,新手 ...

  5. IOS safari浏览器登陆时Cookie无法保存的问题

    近期完成了一个儿童的测评项目,测试到最后的时候发现在ipad mini上登陆成功之后无法跳转页面,而安卓和pc端都可以,找了大半天bug,发现其他的苹果设备都没问题,只能一个一个的调试,结果发现设备的 ...

  6. Dora.Interception, 一个为.NET Core度身打造的AOP框架[3]:Interceptor的注册

    在<不一样的Interceptor>中我们着重介绍了Dora.Interception中最为核心的对象Interceptor,以及定义Interceptor类型的一些约定.由于Interc ...

  7. sizeof与strlen的不同

    sizeof操作符的结果类型是size_t,它在头文件中typedef为unsigned int类型. 该类型保证能容纳实现所建立的最大对象的字节大小. sizeof是算符,strlen是函数. si ...

  8. JavaScript函数声明提升

    首先,JavaScript中函数有两种创建方式,即函数声明.函数表达式两种. 1.函数声明. function boo(){ console.log(123); } boo() 2.函数表达式. va ...

  9. PHP开发api接口安全验证

    php的api接口 在实际工作中,使用PHP写api接口是经常做的,PHP写好接口后,前台就可以通过链接获取接口提供的数据,而返回的数据一般分为两种情况,xml和json,在这个过程中,服务器并不知道 ...

  10. Screen命令安装使用教程

    在安装lnmp之前,我们一般先运行一下Screen程序,因为screen好像一个容器一样,把lnmp的安装过程保护了起来.以CentOS中安装lnmp为例,程序下载.编译都需要比较长的时间,如果中途遇 ...