hadoop hadoop install (1)
vmuser@vmuser-VirtualBox:~$ sudo useradd -m hadoop -s /bin/bash
[sudo] vmuser 的密码:
vmuser@vmuser-VirtualBox:~$ sudo passwd hadoop
输入新的 UNIX 密码:
重新输入新的 UNIX 密码:
passwd:已成功更新密码
vmuser@vmuser-VirtualBox:~$ sudo adduser hadoop sudo
正在添加用户"hadoop"到"sudo"组...
正在将用户“hadoop”加入到“sudo”组中
完成。
vmuser@vmuser-VirtualBox:~$ sudo apt-get install openssh-server
正在读取软件包列表... 完成
正在分析软件包的依赖关系树
正在读取状态信息... 完成
将会同时安装下列软件:
ncurses-term openssh-client openssh-sftp-server ssh-import-id
建议安装:
keychain libpam-ssh monkeysphere ssh-askpass molly-guard rssh
下列【新】软件包将被安装:
ncurses-term openssh-server openssh-sftp-server ssh-import-id
下列软件包将被升级:
openssh-client
升级了 1 个软件包,新安装了 4 个软件包,要卸载 0 个软件包,有 386 个软件包未被升级。
需要下载 637 kB/1,248 kB 的归档。
解压缩后会消耗 5,316 kB 的额外空间。
您希望继续执行吗? [Y/n] y
获取:1 http://mirrors.aliyun.com/ubuntu bionic-updates/main amd64 ncurses-term all 6.1-1ubuntu1.18.04 [248 kB]
获取:2 http://mirrors.aliyun.com/ubuntu bionic-updates/main amd64 openssh-sftp-server amd64 1:7.6p1-4ubuntu0.1 [45.5 kB]
获取:3 http://mirrors.aliyun.com/ubuntu bionic-updates/main amd64 openssh-server amd64 1:7.6p1-4ubuntu0.1 [332 kB]
获取:4 http://mirrors.aliyun.com/ubuntu bionic-updates/main amd64 ssh-import-id all 5.7-0ubuntu1.1 [10.9 kB]
已下载 637 kB,耗时 0秒 (3,215 kB/s)
正在预设定软件包 ...
(正在读取数据库 ... 系统当前共安装有 140635 个文件和目录。)
正准备解包 .../openssh-client_1%3a7.6p1-4ubuntu0.1_amd64.deb ...
正在将 openssh-client (1:7.6p1-4ubuntu0.1) 解包到 (1:7.6p1-4) 上 ...
正在选中未选择的软件包 ncurses-term。
正准备解包 .../ncurses-term_6.1-1ubuntu1.18.04_all.deb ...
正在解包 ncurses-term (6.1-1ubuntu1.18.04) ...
正在选中未选择的软件包 openssh-sftp-server。
正准备解包 .../openssh-sftp-server_1%3a7.6p1-4ubuntu0.1_amd64.deb ...
正在解包 openssh-sftp-server (1:7.6p1-4ubuntu0.1) ...
正在选中未选择的软件包 openssh-server。
正准备解包 .../openssh-server_1%3a7.6p1-4ubuntu0.1_amd64.deb ...
正在解包 openssh-server (1:7.6p1-4ubuntu0.1) ...
正在选中未选择的软件包 ssh-import-id。
正准备解包 .../ssh-import-id_5.7-0ubuntu1.1_all.deb ...
正在解包 ssh-import-id (5.7-0ubuntu1.1) ...
正在设置 ncurses-term (6.1-1ubuntu1.18.04) ...
正在处理用于 ufw (0.35-5) 的触发器 ...
正在处理用于 ureadahead (0.100.0-20) 的触发器 ...
正在处理用于 systemd (237-3ubuntu10.3) 的触发器 ...
正在处理用于 man-db (2.8.3-2) 的触发器 ...
正在设置 openssh-client (1:7.6p1-4ubuntu0.1) ...
正在设置 ssh-import-id (5.7-0ubuntu1.1) ...
正在设置 openssh-sftp-server (1:7.6p1-4ubuntu0.1) ...
正在设置 openssh-server (1:7.6p1-4ubuntu0.1) ...
Creating config file /etc/ssh/sshd_config with new version
Creating SSH2 RSA key; this may take some time ...
2048 SHA256:+UzKAYam4shz8eFVxXfjzNthoobWMAQk/ncyjqOwnAY root@vmuser-VirtualBox (RSA)
Creating SSH2 ECDSA key; this may take some time ...
256 SHA256:Y2lX2D5tkCgVFTgJ6inqzO0v5o07tgEhD2sgea8w24M root@vmuser-VirtualBox (ECDSA)
Creating SSH2 ED25519 key; this may take some time ...
256 SHA256:ZcP+F60Vdlb/ddaHog4J/KhZFAgufj4IRyxnnqrUtgM root@vmuser-VirtualBox (ED25519)
Created symlink /etc/systemd/system/sshd.service → /lib/systemd/system/ssh.service.
Created symlink /etc/systemd/system/multi-user.target.wants/ssh.service → /lib/systemd/system/ssh.service.
正在处理用于 ureadahead (0.100.0-20) 的触发器 ...
正在处理用于 systemd (237-3ubuntu10.3) 的触发器 ...
hadoop@vmuser-VirtualBox:/home/vmuser$ sudo su hadoop
[sudo] hadoop 的密码:
hadoop@vmuser-VirtualBox:/home/vmuser$ cd ~
hadoop@vmuser-VirtualBox:~$ pwd
/home/hadoop
hadoop@vmuser-VirtualBox:~$ ls
examples.desktop
hadoop@vmuser-VirtualBox:~$ ls
examples.desktop
hadoop@vmuser-VirtualBox:~$ mkdir .ssh
hadoop@vmuser-VirtualBox:~$ cd .ssh/
hadoop@vmuser-VirtualBox:~/.ssh$ ssh-keygen -t rsa
Generating public/private rsa key pair.
Enter file in which to save the key (/home/hadoop/.ssh/id_rsa):
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /home/hadoop/.ssh/id_rsa.
Your public key has been saved in /home/hadoop/.ssh/id_rsa.pub.
The key fingerprint is:
SHA256:VjU4i/ypqi/K4xRQXmYN1wX2JyE0o23ehNR3W+4PSf8 hadoop@vmuser-VirtualBox
The key's randomart image is:
+---[RSA 2048]----+
| . =o.oO+o.o |
| o + ..=.*+o... .|
|. . ..+.=+.. + |
| . oo+.o o .|
| . S... . + |
| . . o o o|
| . . .o|
| o. . . E|
| .+o.+o. |
+----[SHA256]-----+
hadoop@vmuser-VirtualBox:~/.ssh$ cat id_rsa.pub >> authorized_keys
hadoop@vmuser-VirtualBox:~/.ssh$ sudo apt install openjdk-8-jdk openjdk-8-jre
正在读取软件包列表... 完成
正在分析软件包的依赖关系树
正在读取状态信息... 完成
update-alternatives: 使用 /usr/lib/jvm/java-8-openjdk-amd64/bin/jconsole 来在自动模式中提供 /usr/bin/jconsole (jconsole)
正在处理用于 libc-bin (2.27-3ubuntu1) 的触发器 ...
正在处理用于 ca-certificates (20180409) 的触发器 ...
Updating certificates in /etc/ssl/certs...
0 added, 0 removed; done.
Running hooks in /etc/ca-certificates/update.d...
done.
done.
hadoop@vmuser-VirtualBox:~/.ssh$ dpkg -L openjdk-8-jdk
/.
/usr
/usr/lib
/usr/lib/jvm
/usr/lib/jvm/java-8-openjdk-amd64
/usr/lib/jvm/java-8-openjdk-amd64/bin
/usr/lib/jvm/java-8-openjdk-amd64/bin/appletviewer
/usr/lib/jvm/java-8-openjdk-amd64/bin/jconsole
/usr/lib/jvm/java-8-openjdk-amd64/include
/usr/lib/jvm/java-8-openjdk-amd64/include/jawt.h
/usr/lib/jvm/java-8-openjdk-amd64/include/linux
/usr/lib/jvm/java-8-openjdk-amd64/include/linux/jawt_md.h
/usr/lib/jvm/java-8-openjdk-amd64/lib
/usr/lib/jvm/java-8-openjdk-amd64/lib/jconsole.jar
/usr/lib/jvm/java-8-openjdk-amd64/man
/usr/lib/jvm/java-8-openjdk-amd64/man/ja_JP.UTF-8
/usr/lib/jvm/java-8-openjdk-amd64/man/ja_JP.UTF-8/man1
/usr/lib/jvm/java-8-openjdk-amd64/man/ja_JP.UTF-8/man1/appletviewer.1.gz
/usr/lib/jvm/java-8-openjdk-amd64/man/ja_JP.UTF-8/man1/jconsole.1.gz
/usr/lib/jvm/java-8-openjdk-amd64/man/man1
/usr/lib/jvm/java-8-openjdk-amd64/man/man1/appletviewer.1.gz
/usr/lib/jvm/java-8-openjdk-amd64/man/man1/jconsole.1.gz
/usr/share
/usr/share/doc
/usr/share/doc/openjdk-8-jre-headless
/usr/share/doc/openjdk-8-jre-headless/test-amd64
/usr/share/doc/openjdk-8-jre-headless/test-amd64/check-hotspot-hotspot.log.gz
/usr/share/doc/openjdk-8-jre-headless/test-amd64/check-jdk-hotspot.log.gz
/usr/share/doc/openjdk-8-jre-headless/test-amd64/check-langtools-hotspot.log.gz
/usr/share/doc/openjdk-8-jre-headless/test-amd64/failed_tests-hotspot.tar.gz
/usr/share/doc/openjdk-8-jre-headless/test-amd64/jtreg-summary-hotspot.log.gz
/usr/share/doc/openjdk-8-jre-headless/test-amd64/jtreport-hotspot.tar.gz
/usr/share/doc/openjdk-8-jdk
hadoop@vmuser-VirtualBox:~/.ssh$ which javac
/usr/bin/javac
hadoop@vmuser-VirtualBox:~/.ssh$ ls -al /usr/bin/javac
lrwxrwxrwx 1 root root 23 1月 20 13:29 /usr/bin/javac -> /etc/alternatives/javac
hadoop@vmuser-VirtualBox:~/.ssh$ ls -al /etc/alternatives/javac
lrwxrwxrwx 1 root root 43 1月 20 13:29 /etc/alternatives/javac -> /usr/lib/jvm/java-8-openjdk-amd64/bin/javac
hadoop@vmuser-VirtualBox:~/.ssh$
add in ~/.bashrc
export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
hadoop@vmuser-VirtualBox:~/.ssh$ source ~/.bashrc
hadoop@vmuser-VirtualBox:~/.ssh$ echo $JAVA_HOME
/usr/lib/jvm/java-8-openjdk-amd64
hadoop@vmuser-VirtualBox:~/.ssh$ java -version
openjdk version "1.8.0_191"
OpenJDK Runtime Environment (build 1.8.0_191-8u191-b12-0ubuntu0.18.04.1-b12)
OpenJDK 64-Bit Server VM (build 25.191-b12, mixed mode)
hadoop@vmuser-VirtualBox:~/.ssh$ $JAVA_HOME/bin/java -version
openjdk version "1.8.0_191"
OpenJDK Runtime Environment (build 1.8.0_191-8u191-b12-0ubuntu0.18.04.1-b12)
OpenJDK 64-Bit Server VM (build 25.191-b12, mixed mode)
hadoop@vmuser-VirtualBox:~/.ssh$ wget http://mirrors.hust.edu.cn/apache/hadoop/common/hadoop-2.7.6/hadoop-2.7.6.tar.gz
--2019-01-20 13:38:58-- http://mirrors.hust.edu.cn/apache/hadoop/common/hadoop-2.7.6/hadoop-2.7.6.tar.gz
正在解析主机 mirrors.hust.edu.cn (mirrors.hust.edu.cn)... 202.114.18.160
正在连接 mirrors.hust.edu.cn (mirrors.hust.edu.cn)|202.114.18.160|:80... 已连接。
已发出 HTTP 请求,正在等待回应... 200 OK
长度: 216745683 (207M) [application/octet-stream]
正在保存至: “hadoop-2.7.6.tar.gz”
hadoop-2.7.6.tar.gz 100%[===================>] 206.70M 8.81MB/s 用时 30s
2019-01-20 13:39:28 (6.96 MB/s) - 已保存 “hadoop-2.7.6.tar.gz” [216745683/216745683])
hadoop@vmuser-VirtualBox:/usr/local$ sudo tar zvxf hadoop-2.7.6.tar.gz -C /usr/local
hadoop@vmuser-VirtualBox:~/.ssh$ cd /usr/local/
hadoop@vmuser-VirtualBox:/usr/local$ ls
bin etc games hadoop-2.7.6 include lib man sbin share src
hadoop@vmuser-VirtualBox:/usr/local$ sudo mv ./hadoop-2.7.6 ./hadoop
hadoop@vmuser-VirtualBox:/usr/local$ sudo chown -R hadoop ./hadoop
hadoop@vmuser-VirtualBox:/usr/local$ vim hadoop/etc/hadoop/core-site.xml
hadoop@vmuser-VirtualBox:/usr/local$ vim hadoop/etc/hadoop/hdfs-site.xml
hadoop@vmuser-VirtualBox:/usr/local$ cd hadoop/
hadoop@vmuser-VirtualBox:/usr/local/hadoop$ ./bin/hdfs namenode -format
19/01/20 13:46:18 INFO namenode.FSImageFormatProtobuf: Saving image file /usr/local/hadoop/tmp/dfs/name/current/fsimage.ckpt_0000000000000000000 using no compression
19/01/20 13:46:18 INFO namenode.FSImageFormatProtobuf: Image file /usr/local/hadoop/tmp/dfs/name/current/fsimage.ckpt_0000000000000000000 of size 323 bytes saved in 0 seconds.
19/01/20 13:46:18 INFO namenode.NNStorageRetentionManager: Going to retain 1 images with txid >= 0
19/01/20 13:46:18 INFO util.ExitUtil: Exiting with status 0
19/01/20 13:46:18 INFO namenode.NameNode: SHUTDOWN_MSG:
/************************************************************
SHUTDOWN_MSG: Shutting down NameNode at vmuser-VirtualBox/127.0.1.1
************************************************************/
adoop@vmuser-VirtualBox:/usr/local/hadoop$ ./sbin/start-dfs.sh
Starting namenodes on [localhost]
The authenticity of host 'localhost (127.0.0.1)' can't be established.
ECDSA key fingerprint is SHA256:Y2lX2D5tkCgVFTgJ6inqzO0v5o07tgEhD2sgea8w24M.
Are you sure you want to continue connecting (yes/no)? yes
localhost: Warning: Permanently added 'localhost' (ECDSA) to the list of known hosts.
localhost: Error: JAVA_HOME is not set and could not be found.
localhost: Error: JAVA_HOME is not set and could not be found.
hadoop@vmuser-VirtualBox:/usr/local/hadoop$ vim etc/hadoop/hadoop-env.sh +25
etc/hadoop/hadoop-env.sh:25:export JAVA_HOME=${JAVA_HOME}
etc/hadoop/hadoop-env.sh:25:export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
hadoop@vmuser-VirtualBox:/usr/local/hadoop$ ./sbin/start-dfs.sh
Starting namenodes on [localhost]
localhost: starting namenode, logging to /usr/local/hadoop/logs/hadoop-hadoop-namenode-vmuser-VirtualBox.out
localhost: starting datanode, logging to /usr/local/hadoop/logs/hadoop-hadoop-datanode-vmuser-VirtualBox.out
Starting secondary namenodes [0.0.0.0]
The authenticity of host '0.0.0.0 (0.0.0.0)' can't be established.
ECDSA key fingerprint is SHA256:Y2lX2D5tkCgVFTgJ6inqzO0v5o07tgEhD2sgea8w24M.
Are you sure you want to continue connecting (yes/no)? yes
0.0.0.0: Warning: Permanently added '0.0.0.0' (ECDSA) to the list of known hosts.
0.0.0.0: starting secondarynamenode, logging to /usr/local/hadoop/logs/hadoop-hadoop-secondarynamenode-vmuser-VirtualBox.out
hadoop@vmuser-VirtualBox:/usr/local/hadoop$ jps
8835 NameNode
9365 Jps
9241 SecondaryNameNode
9019 DataNode
access localhost:50070
hadoop@vmuser-VirtualBox:/usr/local/hadoop$ ./bin/hdfs dfs -mkdir -p /user/hadoop
hadoop@vmuser-VirtualBox:/usr/local/hadoop$ ./bin/hdfs dfs -put ./etc/hadoop/capacity-scheduler.xml input
put: Cannot create file/user/hadoop/input/capacity-scheduler.xml._COPYING_. Name node is in safe mode.
hadoop@vmuser-VirtualBox:/usr/local/hadoop$ jps
15058 DataNode
15574 Jps
14126 NameNode
14542 SecondaryNameNode
hadoop@vmuser-VirtualBox:/usr/local/hadoop$ ./bin/hdfs dfs -put ./etc/hadoop/capacity-scheduler.xml input
put: Cannot create file/user/hadoop/input/capacity-scheduler.xml._COPYING_. Name node is in safe mode.
hadoop@vmuser-VirtualBox:/usr/local/hadoop$ bin/hadoop dfsadmin -safemode leave
DEPRECATED: Use of this script to execute hdfs command is deprecated.
Instead use the hdfs command for it.
Safe mode is OFF
hadoop@vmuser-VirtualBox:/usr/local/hadoop$ ./bin/hdfs dfs -put ./etc/hadoop/capacity-scheduler.xml input
hadoop hadoop install (1)的更多相关文章
- hadoop redis install (4)
reference: http://dblab.xmu.edu.cn/blog/131/ https://github.com/dmajkic/redis https://blog.csdn ...
- hadoop kafka install (6)
reference: http://kafka.apache.org/quickstart http://dblab.xmu.edu.cn/blog/1096-2/ hadoop@iZuf68496 ...
- hadoop hive install (5)
reference : http://dblab.xmu.edu.cn/blog/install-hive/ http://dblab.xmu.edu.cn/blog/hive-in-practice ...
- hadoop mongodb install(3)
reference:http://dblab.xmu.edu.cn/blog/868-2/ root@iZuf68496ttdogcxs22w6sZ:~# mv mongodb-linux-x86_6 ...
- [Hadoop]Hadoop章2 HDFS原理及读写过程
HDFS(Hadoop Distributed File System )Hadoop分布式文件系统. HDFS有很多特点: ① 保存多个副本,且提供容错机制,副本丢失或宕机自动恢复.默认存3份. ② ...
- Hadoop hadoop 机架感知配置
机架感知脚本 使用python3编写机架感知脚本,报存到topology.py,给予执行权限 import sys import os DEFAULT_RACK="/default-rack ...
- hadoop mysql install (5)
reference : http://dblab.xmu.edu.cn/blog/install-mysql/ http://wiki.ubuntu.org.cn/MySQL #install mys ...
- hadoop hbase install (2)
reference: http://dblab.xmu.edu.cn/blog/install-hbase/ reference: http://dblab.xmu.edu.cn/blog/2139- ...
- hadoop kafka install multi-borker (7)
multi-borker function like cluster technology First we make a config file for each of the brokers (o ...
随机推荐
- python常见模块属性与方法
sys模块的变量 变量 描述 sys.path 模块搜索路径 path[0] 是当前脚本程序的路径名,否则为 '' sys.modules 已加载模块的字典 sys.version 版本信息字符串 s ...
- 平衡树之伸展树(Splay Tree)题目整理
目录 前言 练习1 BZOJ 3224 普通平衡树 练习2 BZOJ 3223 文艺平衡树 练习3 BZOJ 1588 [HNOI2002]营业额统计 练习4 BZOJ 1208 [HNOI2004] ...
- HDU 1358 Period(KMP+最小循环节)题解
思路: 这里只要注意一点,就是失配值和前后缀匹配值的区别,不懂的可以看看这里,这题因为对子串也要判定,所以用前后缀匹配值,其他的按照最小循环节做 代码: #include<iostream> ...
- 【附2】hystrix详述(2)- 配置
一.hystrix在生产中的建议 1.保持timeout的默认值(1000ms),除非需要修改(其实通常会修改) 2.保持threadpool的的线程数为10个,除非需要更多 3.依赖标准的报警和监控 ...
- undefined!=false之解 及==比较的规则
JS中有一个基本概念就是: JavaScript中undefined==null 但undefined!==null undefined与null转换成布尔值都是false 如果按照常规想法,比如下面 ...
- JObject 用法 、JProperty 用法、JArray 用法 Linq 转 Json
1.使用LINQ to JSON前,需要引用Newtonsoft.Json的dll和using Newtonsoft.Json.Linq的命名空间.LINQ to JSON主要使用到JObject, ...
- python 获取IP
第一种 import commandscmd = "ifconfig br0 | grep 'inet addr' | sed 's/^.*addr://g' |sed 's/ Bcast: ...
- Bata冲刺 第一天
一.冲刺第一天完成任务情况及贡献小时数: 姓名 今日已完成任务 时间(h) 马仲山 代码调整 2 马婧(12) 整理需求文档 2 马婧(13) 整理设计文档 2 马世芳 编写测试文档 2 张俊逸 ...
- Codeforces 918C - The Monster
918C - The Monster 思路1: 右键在新窗口打开图片 代码: #include<bits/stdc++.h> using namespace std; #define ll ...
- Codeforces 525A - Vitaliy and Pie
525A - Vitaliy and Pie 思路:贪心+hashing. 代码: #include<bits/stdc++.h> using namespace std; string ...