Hive导出表数据
法一:
hive (stuchoosecourse) > insert overwrite local directory '/home/landen/文档/exportDir'
> select * from hiddenipinfo;
Total MapReduce jobs = 1
Launching Job 1 out of 1
Number of reduce tasks is set to 0 since there's no reduce operator
Starting Job = job_201312042044_0026, Tracking URL = http://Master:50030/jobdetails.jsp?jobid=job_201312042044_0026
Kill Command = /home/landen/UntarFile/hadoop-1.0.4/libexec/../bin/hadoop job -kill job_201312042044_0026
Hadoop job information for Stage-1: number of mappers: 1; number of reducers: 0
2013-12-09 19:33:35,962 Stage-1 map = 0%, reduce = 0%
2013-12-09 19:33:41,937 Stage-1 map = 100%, reduce = 0%, Cumulative CPU 0.4 sec
2013-12-09 19:33:43,008 Stage-1 map = 100%, reduce = 0%, Cumulative CPU 0.4 sec
2013-12-09 19:33:44,093 Stage-1 map = 100%, reduce = 0%, Cumulative CPU 0.4 sec
2013-12-09 19:33:45,146 Stage-1 map = 100%, reduce = 0%, Cumulative CPU 0.4 sec
2013-12-09 19:33:46,233 Stage-1 map = 100%, reduce = 0%, Cumulative CPU 0.4 sec
2013-12-09 19:33:47,271 Stage-1 map = 100%, reduce = 0%, Cumulative CPU 0.4 sec
MapReduce Total cumulative CPU time: 400 msec
Ended Job = job_201312042044_0026
Copying data to local directory /home/landen/文档/exportDir
Copying data to local directory /home/landen/文档/exportDir
3 Rows loaded to /home/landen/文档/exportDir
MapReduce Jobs Launched:
Job 0: Map: 1 Cumulative CPU: 0.4 sec HDFS Read: 490 HDFS Write: 233 SUCCESS
Total MapReduce CPU Time Spent: 400 msec
OK
ip countrycode countryname region regionname city latitude longitude timezone
Time taken: 80.784 seconds
But Hive使用 ^A 符号作为域的分隔符,导出后内容如下:
221.12.10.218CNChina02ZhejiangHangzhou30.293594120.16141Asia/Shanghai
60.180.248.201CNChina02ZhejiangWenzhou27.999405120.66681Asia/Shanghai
125.111.251.118CNChina02ZhejiangNingbo29.878204121.5495Asia/Shanghai
故可以使用sed命令将其替换为所需的域分隔符,命令如下:
landen@Master:~/文档/exportDir$ sed -e 's/\x01/\t/g (后缀/g意味着sed会替换每一处匹配)' 000000_0
此时只会显示执行后的文件内容,但000000_0文件内容still时原文件,故需进行重定向到新文件,如下:
sed -e 's/\x01/\t/g' 000000_0 > (重定向到新文件) ipInfo.txt
/Ng会忽略前N处匹配,并从第N+1出开始替换.
landen@Master:~/文档/exportDir$ cat ipInfo.txt
221.12.10.218 CN China 02 Zhejiang Hangzhou 30.293594 120.16141 Asia/Shanghai
60.180.248.201 CN China 02 Zhejiang Wenzhou 27.999405 120.66681 Asia/Shanghai
125.111.251.118 CN China 02 Zhejiang Ningbo 29.878204 121.5495 Asia/Shanghai
法 二:
landen@Master:~/UntarFile/hive-0.10.0$ bin/hive --database 'stuchoosecourse' -e 'select * from hiddenipinfo' >> /home/landen/文档/exportDir/ip.tsv
WARNING: org.apache.hadoop.metrics.jvm.EventCounter is deprecated. Please use org.apache.hadoop.log.metrics.EventCounter in all the log4j.properties files.
Logging initialized using configuration in jar:file:/home/landen/UntarFile/hive-0.10.0/lib/hive-common-0.10.0.jar!/hive-log4j.properties
Hive history file=/home/landen/UntarFile/hive-0.10.0/logs/hive_job_log_landen_201312091934_46210224.txt
OK
Time taken: 17.15 seconds
OK
Time taken: 6.904 seconds
ip.tsv内容如下(包含表列名):
ip countrycode countryname region regionname city latitude longitude timezone
221.12.10.218 CN China 02 Zhejiang Hangzhou 30.293594 120.16141 Asia/Shanghai
60.180.248.201 CN China 02 Zhejiang Wenzhou 27.999405 120.66681 Asia/Shanghai
125.111.251.118 CN China 02 Zhejiang Ningbo 29.878204 121.5495 Asia/Shanghai
法 三:
landen@Master:~/UntarFile/hive-0.10.0$ bin/hive --database 'stuchoosecourse' -f '/home/landen/文档/testSql.q >> ~/ip.tsv'
WARNING: org.apache.hadoop.metrics.jvm.EventCounter is deprecated. Please use org.apache.hadoop.log.metrics.EventCounter in all the log4j.properties files.
Logging initialized using configuration in jar:file:/home/landen/UntarFile/hive-0.10.0/lib/hive-common-0.10.0.jar!/hive-log4j.properties
Hive history file=/home/landen/UntarFile/hive-0.10.0/logs/hive_job_log_landen_201312091450_505292945.txt
OK
Time taken: 4.939 seconds
Total MapReduce jobs = 1
Launching Job 1 out of 1
Number of reduce tasks is set to 0 since there's no reduce operator
Starting Job = job_201312042044_0024, Tracking URL = http://Master:50030/jobdetails.jsp?jobid=job_201312042044_0024
Kill Command = /home/landen/UntarFile/hadoop-1.0.4/libexec/../bin/hadoop job -kill job_201312042044_0024
Hadoop job information for Stage-1: number of mappers: 1; number of reducers: 0
2013-12-09 14:51:19,055 Stage-1 map = 0%, reduce = 0%
2013-12-09 14:51:25,127 Stage-1 map = 100%, reduce = 0%, Cumulative CPU 1.21 sec
2013-12-09 14:51:26,133 Stage-1 map = 100%, reduce = 0%, Cumulative CPU 1.21 sec
2013-12-09 14:51:27,156 Stage-1 map = 100%, reduce = 0%, Cumulative CPU 1.21 sec
2013-12-09 14:51:28,160 Stage-1 map = 100%, reduce = 0%, Cumulative CPU 1.21 sec
2013-12-09 14:51:29,164 Stage-1 map = 100%, reduce = 0%, Cumulative CPU 1.21 sec
2013-12-09 14:51:30,168 Stage-1 map = 100%, reduce = 0%, Cumulative CPU 1.21 sec
2013-12-09 14:51:31,172 Stage-1 map = 100%, reduce = 100%, Cumulative CPU 1.21 sec
MapReduce Total cumulative CPU time: 1 seconds 210 msec
Ended Job = job_201312042044_0024
MapReduce Jobs Launched:
Job 0: Map: 1 Cumulative CPU: 1.21 sec HDFS Read: 306 HDFS Write: 188 SUCCESS
Total MapReduce CPU Time Spent: 1 seconds 210 msec
OK
_c0
CN China 02 Zhejiang Hangzhou 30.293594 120.16141 Asia/Shanghai
CN China 02 Zhejiang Wenzhou 27.999405 120.66681 Asia/Shanghai
CN China 02 Zhejiang Ningbo 29.878204 121.5495 Asia/Shanghai
Time taken: 47.517 seconds
OK
ip countrycode countryname region regionname city latitude longitude timezone
221.12.10.218 CN China 02 Zhejiang Hangzhou 30.293594 120.16141 Asia/Shanghai
60.180.248.201 CN China 02 Zhejiang Wenzhou 27.999405 120.66681 Asia/Shanghai
125.111.251.118 CN China 02 Zhejiang Ningbo 29.878204 121.5495 Asia/Shanghai
Time taken: 0.441 seconds
Hive导出表数据的更多相关文章
- Hive[4] 数据定义 HiveQL
HiveQL 是 Hive 查询语言,它不完全遵守任一种 ANSI SQL 标准的修订版,但它与 MySQL 最接近,但还有显著的差异,Hive 不支持行级插入,更新和删除的操作,也不支持事务,但 H ...
- Mysql数据库备份和按条件导出表数据
Mysql数据库备份和按条件导出表数据 一.备份数据库 # mysqldump -u root -p dbcurr>/home/20090219.sql mysqldum为备份命令,- ...
- 使用Sqoop从mysql向hdfs或者hive导入数据时出现的一些错误
1.原表没有设置主键,出现错误提示: ERROR tool.ImportTool: Error during import: No primary key could be found for tab ...
- hadoop笔记之Hive的数据存储(视图)
Hive的数据存储(视图) Hive的数据存储(视图) 视图(view) 视图是一种虚表,是一个逻辑概念:可以跨越多张表 既然视图是一种虚表,那么也就是说用操作表的方式也可以操作视图 但是视图是建立在 ...
- hadoop笔记之Hive的数据存储(桶表)
Hive的数据存储(桶表) Hive的数据存储(桶表) 桶表 桶表是对数据进行哈希取值,然后放到不同文件中存储. 比如说,创建三个桶,而创建桶的原则可以按照左边表中学生的名字来创建对应的桶.这样子把左 ...
- hadoop笔记之Hive的数据存储(外部表)
Hive的数据存储(外部表) Hive的数据存储(外部表) 外部表 指向已经在HDFS中存在的数据,可以创建Partition 它和内部表在元数据的组织上是相同的,而实际数据的存储则有较大的差异 外部 ...
- hadoop笔记之Hive的数据存储(分区表)
Hive的数据存储(分区表) Hive的数据存储(分区表) 分区表 Partition对应于数据库的Partition列的密集索引 在Hive中,表中的一个Partition对应于表下的一个目录,所有 ...
- hadoop笔记之Hive的数据存储(内部表)
Hive的数据存储(内部表) Hive的数据存储(内部表) 基于HDFS 可使用hadoop给我们提供的web管理工具查看数据.打开管理工具localhost:9000–>Utilities下的 ...
- PL/SQL DEVELOPER 导出表数据
http://jingyan.baidu.com/album/fcb5aff78e6a48edab4a7146.html?picindex=4 1. 导出表数据 打开pl/sql客户端 在左侧 点击t ...
随机推荐
- 2018.09.09 cogs693. Antiprime数(搜索)
传送门 看完题发现很sb. 前10个质数乘起来已经超出题目范围了. 因此只用搜索前几个质数每个的次数比较谁的因数的就行了. 代码: #include<iostream> #define l ...
- 41 Pain and Pain Management 疼痛与疼痛管理
Pain and Pain Management 疼痛与疼痛管理 ①Years ago,doctors often said that pain was a normal part of life.I ...
- Java 继承关系中:static,构造函数,成员变量的加载顺序
首先看下面的例子: package simple.demo; /** * @author Administrator * @date 2019/01/03 */ public class ClassA ...
- LA 3026 && POJ 1961 Period (KMP算法)
题意:给定一个长度为n字符串s,求它每个前缀的最短循环节.也就是对于每个i(2<=i<=n),求一个最大整数k>1(如果存在),使得s的前i个字符组成的前缀是某个字符串重复得k次得到 ...
- DataFrame按行读取:DataFrame之values
http://blog.csdn.net/u014607457/article/details/51290582 def fill_core(self): RatingTable=pd.read_cs ...
- hdu3333 Turing Tree 2016-09-18 20:53 42人阅读 评论(0) 收藏
Turing Tree Time Limit: 6000/3000 MS (Java/Others) Memory Limit: 32768/32768 K (Java/Others) Tota ...
- spark ml 的例子
一.关于spark ml pipeline与机器学习 一个典型的机器学习构建包含若干个过程 1.源数据ETL 2.数据预处理 3.特征选取 4.模型训练与验证 以上四个步骤可以抽象为一个包括多个步骤的 ...
- java web开发过程中的“\”指的是什么,如何区分
- Postgres重置自增长id列(reset sequence)
简单的两个方法,个人比较喜欢第一个 ①ALTER SEQUENCE seq RESTART WITH 1;② SELECT setval('sequence_name', 0); 参考自http:// ...
- 微软儿童编程技术,kodu(酷豆)为儿童创造一个游戏世界
受微软青年星火项目组的邀请,我和微软项目组的朋友参加了一场针对儿童编程的技术指导. 儿童和编程,本来是两个完全不靠边的词.在大家的常规思维中,编程一直是软件开发人员的事情,是高科技类型的工作,高不可攀 ...