环境:

hadoop 3.1.1

hive 3.1.0

mysql 8.0.11

安装前准备:

准备好mysql-connector-java-8.0.12.jar驱动包

上传hive的tar包并解压

第一步:

进入hive/conf,拷贝hive-env.sh.template 为hive-env.sh,修改部分为第48,51,54行

 # Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License. # Set Hive and Hadoop environment variables here. These variables can be used
# to control the execution of Hive. It should be used by admins to configure
# the Hive installation (so that users do not have to set environment variables
# or set command line parameters to get correct behavior).
#
# The hive service being invoked (CLI etc.) is available via the environment
# variable SERVICE # Hive Client memory usage can be an issue if a large number of clients
# are running at the same time. The flags below have been useful in
# reducing memory usage:
#
# if [ "$SERVICE" = "cli" ]; then
# if [ -z "$DEBUG" ]; then
# export HADOOP_OPTS="$HADOOP_OPTS -XX:NewRatio=12 -Xms10m -XX:MaxHeapFreeRatio=40 -XX:MinHeapFreeRatio=15 -XX:+UseParNewGC -XX:-UseGCOverheadLimit"
# else
# export HADOOP_OPTS="$HADOOP_OPTS -XX:NewRatio=12 -Xms10m -XX:MaxHeapFreeRatio=40 -XX:MinHeapFreeRatio=15 -XX:-UseGCOverheadLimit"
# fi
# fi # The heap size of the jvm stared by hive shell script can be controlled via:
#
# export HADOOP_HEAPSIZE=
#
# Larger heap size may be required when running queries over large number of files or partitions.
# By default hive shell scripts use a heap size of (MB). Larger heap size would also be
# appropriate for hive server. # Set HADOOP_HOME to point to a specific hadoop install directory
HADOOP_HOME=/opt/module/hadoop-3.1. # Hive Configuration Directory can be controlled by:
export HIVE_CONF_DIR=/opt/module/hive/conf # Folder containing extra libraries required for hive compilation/execution can be controlled by:
export HIVE_AUX_JARS_PATH=/opt/module/hive/lib

第二步,拷贝hive-default.xml.template为hive-site.xml,主要是一些连接数据库的信息,包括用户名,密码,注意把hive.metastore.schema.verification设置为false

 <?xml version="1.0" encoding="UTF-8" standalone="no"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?><!--
Licensed to the Apache Software Foundation (ASF) under one or more
contributor license agreements. See the NOTICE file distributed with
this work for additional information regarding copyright ownership.
The ASF licenses this file to You under the Apache License, Version 2.0
(the "License"); you may not use this file except in compliance with
the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
<configuration>
<property>
<name>javax.jdo.option.ConnectionURL</name>
<value>jdbc:mysql://127.0.0.1:3306/hive?useUnicode=true&amp;characterEncoding=utf8&amp;useSSL=false&amp;serverTimezone=GMT</value>
<description>JDBC connect string for a JDBC metastore</description>
</property> <property>
<name>hive.metastore.schema.verification</name>
<value>false</value>
</property>
<property>
<name>javax.jdo.option.ConnectionDriverName</name>
<value>com.mysql.cj.jdbc.Driver</value>
<description>Driver class name for a JDBC metastore</description>
</property> <property>
<name>javax.jdo.option.ConnectionUserName</name>
<value>root</value>
<description>username to use against metastore database</description>
</property> <property>
<name>javax.jdo.option.ConnectionPassword</name>
<value></value>
<description>password to use against metastore database</description>
</property>
<property>
<name>datanucleus.schema.autoCreateAll</name>
<value>true</value>
</property>
</configuration>

第三步:上传mysql驱动包到hive/lib目录下

第四步:在mysql中创建hive数据库

create database hive;

第五步:进入bin目录执行(指定元数据库并进行初始化)

./schematool -dbType mysql -initSchema

第六步:启动hive(要先启动hadoop)

./hive

启动完成后show databases;

注意事项:

1.hive-site.xml中 mysql的驱动名称为com.mysql.cj.jdbc.Driver

2.xml文档中javax.jdo.option.ConnectionURL中&要用&amp;替代,一定要指定字符集,时区

3.我已经提前对数据库的root用户进行授权

4.如果测试hive插入数据,要在hdfs上创建/user/hive/warehouse路径

常见问题:

1../schematool -dbType mysql -initSchema提示server code 255之类的是连接数据库的字符集没指定,failed的话删除hive数据库,重新创建,再次执行此命令即可

2.The server time zone value 'PDT' is unrecognized or represents more than one

时区问题.写成jdbc:mysql://127.0.0.1:3306/hive?useUnicode=true&amp;characterEncoding=utf8&amp;useSSL=false&amp;serverTimezone=GMT即可

参考https://blog.csdn.net/m0_37520980/article/details/80364884

hive 3.1.0 安装配置的更多相关文章

  1. CentOS 7.0安装配置Vsftp服务器

    一.配置防火墙,开启FTP服务器需要的端口 CentOS 7.0默认使用的是firewall作为防火墙,这里改为iptables防火墙. 1.关闭firewall: systemctl stop fi ...

  2. CentOS 7.0安装配置LAMP服务器(Apache+PHP+MariaDB)

    CentOS 7.0默认使用的是firewall作为防火墙,这里改为iptables防火墙. 1.关闭firewall: systemctl stop firewalld.service #停止fir ...

  3. CentOS 7.0安装配置Vsftp服务器步骤详解

    安装Vsftp讲过最多的就是在centos6.x版本中了,这里小编看到有朋友写了一篇非常不错的CentOS 7.0安装配置Vsftp服务器教程,下面整理分享给各位. 一.配置防火墙,开启FTP服务器需 ...

  4. CentOS 7.0 安装配置LAMP服务器方法(Apache+PHP+MariaDB)(转)

    转自:http://www.jb51.net/os/188488.html 作者:佚名 字体:[增加 减小] 来源:osyunwei  准备篇: CentOS 7.0系统安装配置图解教程 http:/ ...

  5. Win10 64位+VS2015+Opencv3.3.0安装配置

    Win10 64位+VS2015+Opencv3.3.0安装配置 1.我们首先下载VS2015.OpenCV3.3.0. 1.1 VS2015下载 在官网https://visualstudio.mi ...

  6. 实战:ADFS3.0单点登录系列-ADFS3.0安装配置

    本文为系列第三章,主要讲下ADFS3.0的安装和配置.本文和前面的文章是一个系列,因此有些地方是有前后关联,比如本文中使用的通配符证书就是第二篇讲解的,因此需要连贯的进行阅读. 全文目录如下: 实战: ...

  7. 【转发】【linux】【ftp】CentOS 7.0安装配置Vsftp服务器

    adduser -d /var/www/android -g ftp -s /sbin/nologin ftp2 一.配置防火墙,开启FTP服务器需要的端口 CentOS 7.0默认使用的是firew ...

  8. Hive 2.1.1安装配置

    ##前期工作 安装JDK 安装Hadoop 安装MySQL ##安装Hive ###下载Hive安装包 可以从 Apache 其中一个镜像站点中下载最新稳定版的 Hive, apache-hive-2 ...

  9. Linux Tomcat 6.0安装配置实践总结

    系统环境: Red Hat Enterprise Linux Server release 5.7 (Tikanga)  64位 Tomcat下载 从官方网站 http://tomcat.apache ...

随机推荐

  1. Servlet的异常处理机制

    一 声明式异常处理   在web.xml中对声明对各种异常的处理方法. 通过 <error-page>元素来声明. 此元素的结构如下:                    +------ ...

  2. Scala具体解释---------数组、元组、映射

    一.数组 1.定长数组 声明数组的两种形式: 声明指定长度的数组 val 数组名= new Array[类型](数组长度) 提供数组初始值的数组,无需newkeyword Scala声明数组时.须要带 ...

  3. 解决Keystore was tampered with, or password was incorrect

    使用签名文件keystore查看生成的数字签名中报错解决 Keystore was tampered with, or password was incorrect 这是由于android规定自己定义 ...

  4. 过滤input框中的特殊字符

    两种方式,我觉得是一样的效果,请看: var charFilter1 = function(str) { var pattern = new RegExp("[`~!@#$^&*() ...

  5. tensorflow 的 Batch Normalization 实现(tf.nn.moments、tf.nn.batch_normalization)

    tensorflow 在实现 Batch Normalization(各个网络层输出的归一化)时,主要用到以下两个 api: tf.nn.moments(x, axes, name=None, kee ...

  6. POJ 1511 Invitation Cards (ZOJ 2008) 使用优先队列的dijkstra

    传送门: http://poj.org/problem?id=1511 http://acm.zju.edu.cn/onlinejudge/showProblem.do?problemId=1008 ...

  7. C++项目參考解答:累加求圆周率

    [项目-累加求圆周率] 用例如以下公式求π的近似值(计算直到最后一项的绝对值小于10−5) π4=1−13+15−17+... [參考解答] #include <iostream> usi ...

  8. 【u125】最大子树和

    Time Limit: 1 second Memory Limit: 128 MB [问题描述] 小明对数学饱有兴趣,并且是个勤奋好学的学生,总是在课后留在教室向老师请教一些问题.一天他早晨骑车去上课 ...

  9. Spring Boot+Mybatis+Pagehelper分页

    Spring Boot 集成MyBatis和Pagehelper分页插件 mybatis-spring-boot-starter依赖树如下: pom配置 <project xmlns=" ...

  10. 特征点提取之Harris角点提取法

    1. 特征点提取的意义 2.角点 3. Harris角点检測的基本原理 4.Harris角点检測算法的步骤 5.Harris角点提取算法设计 <span style="font-siz ...