Sqoop 1.99.4 安装
1.安装准备工作:
已经装好的 hadoop 环境是 hadoop-2.5.1 64位
下载的sqoop安装包(注意是hadoop200)
http://www.us.apache.org/dist/sqoop/1.99.4/sqoop-1.99.4-bin-hadoop200.tar.gz
2.解压文件到工作目录:
tar -xvf sqoop-1.99.3-bin-hadoop200.tar.gz
mv sqoop-1.99.4-bin-hadoop200 /usr/local/software/
3.修改环境变量:
vim /etc/profile
添加如下内容:
#sqoop
export SQOOP_HOME/usr/local/software/sqoop-1.99.4-bin-hadoop200
export PATH=$SQOOP_HOME/bin:$PATH
export CATALINA_HOME=$SQOOP_HOME/server
export LOGDIR=$SQOOP_HOME/logs
保存退出即时生效:
source /etc/profile
4.修改sqoop配置:
vim /usr/local/sqoop-1.99.4-bin-hadoop200/server/conf/sqoop.properties
#修改指向我的hadoop安装目录
org.apache.sqoop.submission.engine.mapreduce.configuration.directory=/usr/local/software/hadoop-2.5.1-64/
#把hadoop目录下的jar包都引进来
vim /usr/local/software/sqoop-1.99.4-bin-hadoop200/server/conf/catalina.properties
common.loader=/usr/local/software/hadoop-2.5.1-64/share/hadoop/common/*.jar,/usr/local/software/hadoop-2.5.1-64/share/hadoop/common/lib/*.jar,/usr/local/software/hadoop-2.5.1-64/share/hadoop/hdfs/*.jar,/usr/local/software/hadoop-2.5.1-64/share/hadoop/hdfs/lib/*.jar,/usr/local/software/hadoop-2.5.1-64/share/hadoop/mapreduce/*.jar,/usr/local/software/hadoop-2.5.1-64/share/hadoop/mapreduce/lib/*.jar,/usr/local/software/hadoop-2.5.1-64/share/hadoop/tools/*.jar,/usr/local/software/hadoop-2.5.1-64/share/hadoop/tools/lib/*.jar,/usr/local/software/hadoop-2.5.1-64/share/hadoop/yarn/*.jar,/usr/local/software/hadoop-2.5.1-64/share/hadoop/yarn/lib/*.jar,/usr/local/software/hadoop-2.5.1-64/share/hadoop/httpfs/tomcat/lib/*.jar
5.下载mysql驱动包
mysql-connector-java-5.1.16-bin.jar
并放到 /usr/local/software/sqoop-1.99.4-bin-hadoop200/server/lib/ 目录下
[注意:下载的是 mysql-5.6.13.tar.gz 只需要把里面的 mysql-connector-java.5.1.34-bin.jar 考出来即可,这是个坑啊]
6.启动/停止sqoop200
/usr/local/software/sqoop-1.99.4-bin-hadoop200/bin/sqoop.sh server start/stop
查看启动日志:
vim usr/local/software/sqoop-1.99.4-bin-hadoop200/server/logs/catalina.out
7.进入客户端交互目录
/usr/local/software/sqoop-1.99.4-bin-hadoop200/bin/sqoop.sh client
Sqoop home directory: /usr/local/software/sqoop-1.99.4-bin-hadoop200
Sqoop Shell: Type 'help' or '\h' for help.
sqoop:000>
为客户端配置服务器:set server --host hadoopMaster --port 12000 --webapp sqoop 【红色部分为 本机hostname 主机名】
Server is set successfully
sqoop:000> show version --all 查看版本信息
client version:
Sqoop 1.99.4 source revision 2475a76ef70a0660f381c75c3d47d0d24f00b57f
Compiled by gshapira on Sun Nov 16 02:50:00 PST 2014
server version:
Sqoop 1.99.4 source revision 2475a76ef70a0660f381c75c3d47d0d24f00b57f
Compiled by gshapira on Sun Nov 16 02:50:00 PST 2014
API versions:
[v1]
sqoop:000> show connector --all 显示连接器 【会显示很长一串。。。】
【注意:以下是 1.99.4版本用的命令,要是 1.99.3以下版本请看最底部链接】
sqoop:000> create link --cid 1 创建数据库连接
Creating link for connector with id 1
Please fill following values to create new link object Name: mysql --输入名称
Link configuration
JDBC Driver Class: com.mysql.jdbc.Driver --输入
JDBC Connection String: jdbc:mysql://localhost:3306/widgets
Username: root
Password: ***
JDBC Connection Properties:
There are currently 0 values in the map:
entry#
sqoop:001> create job --from 1 --to 1
To database configuration
Schema name: traceweb
Table name: widgets
Table SQL statement:
Table column names:
Stage table name:
Should clear stage table:
Throttling resources
Extractors:
Loaders:
New job was successfully created with validation status OK and persistent id 1
sqoop:001> show link 显示所有链接
sqoop:001> show job 显示所有job
sqoop:000> start job --jid 1 启动job
sqoop:000> status job --jid 1 查看导入状态
ll /usr/local/software/sqoop-1.99.4-bin-hadoop200/sqoop_output/
下面命令一看就知道是什么意思了:
sqoop:000> show job
sqoop:000> delete job --jid 1
sqoop:000> show job
sqoop:000> delete job --jid 2
sqoop:000> show job
sqoop:000> show connection
报错:The specified function "connection" is not recognized.
原因是版本不一样,SQOOP 1.99.4以后命令就变了: 具体看这里:https://sqoop.apache.org/docs/1.99.4/CommandLineClient.html#create-link-function
SQOOP 1.99.3 安装 参考页面:http://houshangxiao.iteye.com/blog/2070057
Sqoop 1.99.4 安装的更多相关文章
- Sqoop 1.99.6 安装和使用
安装 1.安装准备工作: 下载的sqoop安装包 http://mirrors.hust.edu.cn/apache/sqoop/1.99.6/sqoop-1.99.6.tar.gz ...
- sqoop 1.99.7 安装及配置
一 下载sqoop 1.99.7 http://mirror.bit.edu.cn/apache/sqoop/1.99.7/ 二 解压安装文件 三 配置Sqoop 环境变量 最后把mysql的驱动j ...
- [sqoop1.99.7] sqoop入门-下载、安装、运行和常用命令
一.简介 Apache Sqoop is a tool designed for efficiently transferring data betweeen structured, semi-str ...
- Hadoop2.2.0环境下Sqoop1.99.3安装
本文转载自http://blog.csdn.net/liuwenbo0920/article/details/40504045 1.安装准备工作: 已经装好的hadoop环境是hadoop 2.2.0 ...
- 安装sqoop 1.99.4
参考http://sqoop.apache.org/docs/1.99.4/Installation.html 1.简介 sqoop2分为server和client两部分.server作为maprde ...
- sqoop1.99.4安装与简介
1.什么是sqoop clouder公司开发的一个在关系数据库和hdfs,hive之间数据导入导出的一个工具 2.版本及其区别 现在分为sqoop1和sqoop2 1)架构图(sqoop1和hadoo ...
- HIVE之 Sqoop 1.4.6 安装、hive与oracle表互导
1. sqoop数据迁移 1.1 概述 sqoop是apache旗下一款“Hadoop和关系数据库服务器之间传送数据”的工具. 导入数据:MySQL,Oracle导入数据到Hadoop的HDFS.HI ...
- Sqoop之 Sqoop 1.4.6 安装
1. sqoop数据迁移 1.1 概述 sqoop是apache旗下一款“Hadoop和关系数据库服务器之间传送数据”的工具. 导入数据:MySQL,Oracle导入数据到Hadoop的HDFS.HI ...
- Sqoop 1.4.6 安装配置
配置环境变量 # SQOOP SQOOP_HOME=/home/hadoop/development/src/sqoop-1.4.6-cdh5.6.0 PATH=$PATH:$SQOOP_HOME/b ...
随机推荐
- Linux 学习之路:read,array,declare
一.read 键盘读取变量 用法:read -p "PLS keyin your name:" -t 60 name -p :后面接提示符,-t 后面接可以等待的时间,其中nam ...
- 3.4.2内核下的I2C驱动
1. 框架1.1 硬件协议简介1.2 驱动框架1.3 bus-drv-dev模型及写程序a. 设备的4种构建方法a.1 定义一个i2c_board_info, 里面有:名字, 设备地址 然后i2c_r ...
- 【技巧性(+递归运用)】UVa 1596 - Bug Hunt
In this problem, we consider a simple programming language that has only declarations of onedimensio ...
- 浅谈 Data URI 与 BASE 64 编码
前言(废话):鼓捣 Stylish 的时候发现了这么个奇怪的代码行: Data:image/gif;BASE64,R0lGODlhEAAQAKEAAEKF9NPi/AAAAAAAACH5BAEAAAI ...
- Delphi 中的MD5实现方法(转)
在Delphi自带的Indy控件中其实是提供了MD2,MD4,MD5对象的,我们可以直接使用它们来完成MD5的签名算法.而不需要再去找其它的DLL或是Pas了. 在Uses单元中引用 IdHashMe ...
- Linux 命令 - umask: 显示或设置文件模式掩码值
umask 命令控制着创建文件时指定给文件的默认权限.它使用八进制表示法从文件模式属性中删除一个位掩码. 参见下面的例子: [huey@huey-K42JE cmdline]$ rm -f foo.t ...
- 仿QQ注册验证码的实现。
最近发现一些网站的验证码全部换成了“极验”和“点触”的,发现QQ的注册也是与“点触”的相似.就想尝试实现一个. 先上效果图: 下面贴上主要思路及代码: 第一步:得到常用汉字列表 public stat ...
- 使用Xpath从网页中获取数据
/// <summary> /// 从官方网站中抓取产品信息存放在本地数据库中 /// </summary> /// <returns></returns&g ...
- 感受函数式编程-scala
/** * Created by jx_luo on 2015/3/18. */object test03 { def main(Args:Array[String]): Unit ={ val st ...
- centos6.5下磁盘创建交换分区
1.创建磁盘交换分区 2.创建文件交换分区