Java连接hive进行操作的方式有多种,接触到了两种:

首先,hive要起动远程服务接口,命令:

hive --service hiveserver -p 50000 &

1. 通过jdbc驱动连接hive

当然还有其他的连接方式,比如ODBC等,这种方式很常用。

不稳定,经常会被大数据量冲挂,不建议使用。

package cn.ac.iscas.hiveclient;

import java.io.IOException;
import java.io.InputStream;
import java.sql.Connection;
import java.sql.DriverManager;
import java.sql.ResultSet;
import java.sql.SQLException;
import java.sql.Statement;
import java.util.Properties; public class HiveJDBCClient { private static String driverName;
private static String url;
private static String user;
private static String password; private Connection conn = null;
private Statement stmt = null; static {
Properties prop = new Properties();
InputStream in = HiveJDBCClient.class.getResourceAsStream("hiveCli.properties");
try{
prop.load(in);
driverName = prop.getProperty("driverName");
url = prop.getProperty("url");
user = prop.getProperty("user");
password = prop.getProperty("password");
}catch (IOException e){
e.printStackTrace();
}
} public boolean execute(String sql){
boolean rs = false;
try {
conn = getConn();
stmt = conn.createStatement();
rs = stmt.execute(sql);
} catch (ClassNotFoundException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (SQLException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}finally{
try{
if( null != conn){
conn.close();
conn = null;
}
if( null != stmt){
stmt.close();
stmt = null;
}
}catch (SQLException e){
e.printStackTrace();
}
}
return rs;
} public ResultSet executeQuery(String sql){
ResultSet rs = null;
try {
conn = getConn();
stmt = conn.createStatement();
rs = stmt.executeQuery(sql);
} catch (ClassNotFoundException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (SQLException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}finally{
try{
if( null != conn){
conn.close();
conn = null;
}
if( null != stmt){
stmt.close();
stmt = null;
}
}catch (SQLException e){
e.printStackTrace();
}
}
return rs;
} private static Connection getConn() throws ClassNotFoundException,
SQLException{
Class.forName(driverName);
Connection conn = DriverManager.getConnection(url,user,password);
return conn;
} public static void main(String[] args){
HiveJDBCClient hc = new HiveJDBCClient();
ResultSet rs = hc.executeQuery("desc carsrecord");
try {
while(rs.next()){
System.out.println(rs.getString(1));
}
} catch (SQLException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
}

2. 通过hive thrift连接

package cn.ac.iscas.hiveclient;
import java.io.IOException;
import java.io.InputStream;
import java.util.ArrayList;
import java.util.List;
import java.util.Properties; import org.apache.hadoop.hive.service.HiveClient;
import org.apache.hadoop.hive.service.HiveServerException;
import org.apache.thrift.TException;
import org.apache.thrift.protocol.TBinaryProtocol;
import org.apache.thrift.protocol.TProtocol;
import org.apache.thrift.transport.TSocket;
import org.apache.thrift.transport.TTransport;
import org.apache.thrift.transport.TTransportException; public class HiveThreatClient {
static TTransport transport;
static TProtocol protocol;
static HiveClient client ;
static String ip;
static int port; static {
Properties prop = new Properties();
InputStream in = HiveJDBCClient.class.getResourceAsStream("hiveCli.properties");
try{
prop.load(in);
ip = prop.getProperty("ip");
port = Integer.valueOf(prop.getProperty("port"));
}catch (IOException e){
e.printStackTrace();
}
} public static List<String> execute(String query) throws HiveServerException,
TException,TTransportException{
List<String> result = new ArrayList<String>();
transport = new TSocket(ip,port);
protocol = new TBinaryProtocol(transport);
client = new HiveClient(protocol);
transport.open();
client.send_execute(query);
client.recv_execute();
//client.send_commit_txn(rqst);
//client.execute(query);
List<String> list = client.fetchN(10);
while(null!=list && list.size()>0){
for(String r :list){
System.out.println(r);
result.add(r);
}
list = client.fetchN(10);
}
client.shutdown();
transport.close();
return result;
} public static void main(String[] args){
try {
//HiveThreatClient.execute("desc carsrecord");
//HiveThreatClient.execute("select distinct addressname from carsrecord where collectiontime='2015-02-02'");
//load data inpath '/2015/02/2015-02-01.dat' overwrite into table carsrecord partition(collectiontime='2015-02-01')
for(int i = 10; i < 29; i++){
String day = i > 9 ? ""+i:"0"+i;
String stat = "load data inpath '/2015/02/2015-02-"+day+".dat' overwrite into table carsrecord partition(collectiontime='2015-02-"+day+"')";
System.out.println(stat);
HiveThreatClient.execute(stat);
}
//HiveThreatClient.execute("select * from carsrecord where collectiondate>='2014-01-01' and collectiondate<'2014-01-03'");
} catch (HiveServerException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (TTransportException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (TException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
}

需要引入的jar包:

  • <classpathentry kind="lib" path="hive-service-0.13.1.jar"/>
        <classpathentry kind="lib" path="hive-exec-0.13.1.jar"/>

    <classpathentry kind="lib" path="slf4j-api-1.6.6.jar"/>

    <classpathentry kind="lib" path="hive-metastore-0.13.1.jar"/>

    <classpathentry kind="lib" path="libfb303-0.9.0.jar"/>

hive Java API的更多相关文章

  1. 使用hive客户端java api读写hive集群上的信息

    上文介绍了hdfs集群信息的读取方式,本文说hive 1.先解决依赖 <properties> <hive.version>1.2.1</hive.version> ...

  2. hive-通过Java API操作

    通过Java API操作hive,算是测试hive第三种对外接口 测试hive 服务启动 package org.admln.hive; import java.sql.SQLException; i ...

  3. Sqoop Java API 导入应用案例

    环境信息: Linux+JDK1.7 Sqoop 1.4.6-cdh5.5.2 hadoop-core 2.6.0-mr1-cdh5.5.2 hadoop-common 2.6.0-cdh5.5.2 ...

  4. 使用JAVA API 解析ORC File

    使用JAVA API 解析ORC File orc File 的解析过程中,使用FileInputFormat的getSplits(conf, 1)函数, 然后使用 RecordReaderreade ...

  5. Hbase框架原理及相关的知识点理解、Hbase访问MapReduce、Hbase访问Java API、Hbase shell及Hbase性能优化总结

    转自:http://blog.csdn.net/zhongwen7710/article/details/39577431 本blog的内容包含: 第一部分:Hbase框架原理理解 第二部分:Hbas ...

  6. hbase java API跟新数据,创建表

    package hbaseCURD; import java.io.IOException; import org.apache.hadoop.conf.Configuration; import o ...

  7. _00017 Kafka的体系结构介绍以及Kafka入门案例(0基础案例+Java API的使用)

    博文作者:妳那伊抹微笑 itdog8 地址链接 : http://www.itdog8.com(个人链接) 博客地址:http://blog.csdn.net/u012185296 博文标题:_000 ...

  8. Phoenix简介概述,Phoenix的Java API 相关操作优秀案例

    Phoenix简介概述,Phoenix的Java API 相关操作优秀案例 一.Phoenix概述简介 二.Phoenix实例一:Java API操作 2.1 phoenix.properties 2 ...

  9. Atitit 图像处理 调用opencv 通过java  api   attilax总结

    Atitit 图像处理 调用opencv 通过java  api   attilax总结 1.1. Opencv java api的支持 opencv2.4.2 就有了对java api的支持1 1. ...

随机推荐

  1. Page Object Model (Selenium, Python)

    时间 2015-06-15 00:11:56  Qxf2 blog 原文  http://qxf2.com/blog/page-object-model-selenium-python/ 主题 Sel ...

  2. 判断图片的类型(图片是data类型 )

    + (NSString *)typeForImageData:(NSData *)data { uint8_t c; [data getBytes:&c length:1]; switch ( ...

  3. Android自学指导

    如果想自学Android,以下的文章可以作为参考: 如何自学Android(Gityuan) 那两年炼就的Android内功修养(老罗的Android之旅)

  4. CLR via C#(12)-委托Delegate

    本来按照进度应该学习事件了,可总觉得应该委托在前,事件在后,才好理解. 委托是一个类,它提供了回调函数机制,而且是类型安全的.使用委托可以将方法当作另一个方法的参数来进行传递,这种将方法动态地赋给参数 ...

  5. CLR via C#(03)- 对象创建和类型转换

    一. 创建对象 CLR要求用new操作符创建对象,这个操作符在编译时产生的IL指令为newobj.例如: Student XiaoJing=new Student(“XiaoJing”,”1986”) ...

  6. MYSQL的增删改查语句样码

    慢慢来,慢慢来.. 增: INSERT INTO person (person_id, fname, lname, gender, birth_date) VALUES (null, 'William ...

  7. .Learning.Python.Design.Patterns.2nd.Edition之单实例模式

    可以慢慢理解.. 对照JAVA class Singleton(object): def __new__(cls): if not hasattr(cls, 'instance'): cls.inst ...

  8. Shell脚本获取C语言可执行程序返回值

    #!/bin/sh #./test是c程序,该程序 返回0 ./test OP_MODE=$? echo $OP_MODE # $? 显示最后命令的退出状态.0表示没有错误,其他任何值表明有错误.

  9. Android之TabHost布局(转)

    1.概念 盛放Tab的容器就是TabHost.TabHost的实现有两种方式: 第一种继承TabActivity,从TabActivity中用getTabHost()方法获取TabHost.各个Tab ...

  10. Android:dimen尺寸资源文件的使用(转)

    为了适配不同的分辨率. dimen.xml在values文件夹下面 <resources> <!-- Default screen margins, per the Android ...