apache storm 的安装
原文: http://storm.praveendeshmane.co.in/storm/storm-0-10-0-installation-on-ubuntu-14-04.jsp
------------------------------------------
Apache Storm is a distributed stream processing computation framework written predominantly in the Clojure programming language. Originally created by Nathan Marz and team at BackType, the project was open sourced after being acquired by Twitter. It uses custom created "spouts" and "bolts" to define information sources and manipulations to allow batch, distributed processing of streaming data. The initial release was on 17 September 2011.
A Storm application is designed as a "topology" in the shape of a directed acyclic graph (DAG) with spouts and bolts acting as the graph vertices. Edges on the graph are named streams and direct data from one node to another. Together, the topology acts as a data transformation pipeline. At a superficial level the general topology structure is similar to a MapReduce job, with the main difference being that data is processed in real time as opposed to in individual batches.
Pre Requirements
1) A machine with Ubuntu 14.04 LTS operating system.
2) Apcahe ZooKeeper pre installed (How to install ZooKeeper on Ubuntu 14.04)
3) Apache Storm 0.10.0 software (Download Here)
Storm Installation Steps
Step 1 - Installing java 7. Open a terminal (CTRL + ALT + T) and type the following sudo command.
$ sudo apt-get install openjdk-7-jdk
Check it installed properly or not.
$ java -version
Step 2 - Edit $HOME/.bashrc file by adding the java path.
$ sudo gedit $HOME/.bashrc
$HOME/.bashrc file
export JAVA_HOME=/usr/lib/jvm/java-7-openjdk-amd64
Step 3 - Reload your changed $HOME/.bashrc settings
$ source $HOME/.bashrc
Step 4 - Creating storm directory. Open a terminal (CTRL + ALT + T) and type the following $ sudo command.
$ sudo mkdir /usr/local/storm
Step 5 - Change the ownership and permissions of the directory /usr/local/storm. Here 'hduser' is an Ubuntu username.
$ sudo chown -R hduser /usr/local/storm
$ sudo chmod -R 755 /usr/local/storm
Step 6 - Change the directory to /home/hduser/Desktop , In my case the downloaded apache-storm-0.10.0.tar.gz file is in /home/hduser/Desktop folder. For you it might be in /downloads folder check it.
$ cd /home/hduser/Desktop/
Step 7 - Untar the apache-storm-0.10.0.tar.gz file.
$ tar xzf apache-storm-0.10.0.tar.gz
Step 8 - Move the contents of apache-storm-0.10.0 folder to /usr/local/storm
$ mv apache-storm-0.10.0/* /usr/local/storm
Step 9 - Change the directory to /usr/local/storm
$ cd /usr/local/storm
Step 10 - Create /data, /data/nimbus and /data/supervisor directories.
$ sudo mkdir /usr/local/storm/data
$ sudo mkdir /usr/local/storm/data/nimbus
$ sudo mkdir /usr/local/storm/data/supervisor
Step 11 - Change the ownership and permissions of the directory /data, /data/nimbus and /data/supervisor directories.. Here 'hduser' is an Ubuntu username.
$ sudo chown -R hduser /usr/local/storm/data
$ sudo chown -R hduser /usr/local/storm/data/nimbus
$ sudo chown -R hduser /usr/local/storm/data/supervisor
$ sudo chmod -R 755 /usr/local/storm/data
$ sudo chmod -R 755 /usr/local/storm/data/nimbus
$ sudo chmod -R 755 /usr/local/storm/data/supervisor
Step 12 - Change the directory to /usr/local/storm/conf
$ cd /usr/local/storm/conf
Step 13 - Edit storm.yaml file.
$ gedit storm.yaml
Step 14 - Add these below lines to storm.yaml file. Save and close.
storm.zookeeper.servers:
- "localhost"
storm.local.dir: "/usr/local/storm/data"
nimbus.host: "localhost"
nimbus.thrift.port: 49627
storm.zookeeper.port: 2181
supervisor.slots.ports:
- 6700
- 6701
- 6702
- 6703
Step 15 - Start ZooKeeper. Open a new terminal (CTRL + ALT + T) and start zookeeper.
$ /usr/local/zookeeper/bin/zkServer.sh start
Step 16 - Open a new terminal (CTRL + ALT + T). Change the directory to /usr/local/storm
$ cd /usr/local/storm
Step 17 - Start nimbus
$ ./bin/storm nimbus
Step 18 - Open a new terminal (CTRL + ALT + T). Change the directory to /usr/local/storm
$ cd /usr/local/storm
Step 19 - Start supervisor
$ ./bin/storm supervisor
Step 20 - Open a new terminal (CTRL + ALT + T). Change the directory to /usr/local/storm
$ cd /usr/local/storm
Step 21 - Start web UI
$ ./bin/storm ui
Step 22 - Check the web UI here. Open a browser and type the following URL.
http://localhost:8080
Please share this blog post and follow me for latest updates on
apache storm 的安装的更多相关文章
- ubuntu14.04LTS 下storm单机版安装配置
1.下载storm 的安装文件 http://www.apache.org/dyn/closer.cgi/incubator/storm/apache-storm-0.9.2-incubating/a ...
- Apache Storm 1.1.0 中文文档 | ApacheCN
前言 Apache Storm 是一个免费的,开源的,分布式的实时计算系统. 官方文档: http://storm.apache.org 中文文档: http://storm.apachecn.org ...
- Apache Storm
作者:jiangzz 电话:15652034180 微信:jiangzz_wx 微信公众账号:jiangzz_wy 背景介绍 流计算:将大规模流动数据在不断变化的运动过程中实现数据的实时分析,捕捉到可 ...
- Apache Storm Installation
安装的过程参照此处的过程介绍(https://www.tutorialspoint.com/apache_storm/apache_storm_installation.htm) 安装的过程要安装3个 ...
- 在Docker Swarm上部署Apache Storm:第2部分
[编者按]本文来自 Baqend Tech Blog,描述了如何在 Docker Swarm,而不是在虚拟机上部署和调配Apache Storm集群.文章系国内 ITOM 管理平台 OneAPM 编译 ...
- 在Docker Swarm上部署Apache Storm:第1部分
[编者按]本文来自 Baqend Tech Blog,描述了如何在 Docker Swarm,而不是在虚拟机上部署和调配Apache Storm集群.文章系国内 ITOM 管理平台 OneAPM 编译 ...
- Apache Storm使用
Apache Storm 是 Apache 基金会的开源的分布式实时计算系统.与 Hadoop 的批处理相类似,Storm 可以对大量的数据流进行可靠的实时处理,这一过程也称为“流式处理”,是分布式大 ...
- 分布式流处理框架 Apache Storm —— 编程模型详解
一.简介 二.IComponent接口 三.Spout 3.1 ISpout接口 3.2 BaseRichSpout抽象类 四.Bolt 4.1 IBolt 接口 4. ...
- apache storm基本原理及使用总结
什么是Apache Storm Apache Storm是一个分布式实时大数据处理系统.Storm设计用于在容错和水平可扩展方法中处理大量数据.它是一个流数据框架,具有最高的摄取率.虽然Storm是无 ...
随机推荐
- poi 和jxl导出excel(2)
controller: /** * 导出报表 * @return */ @RequestMapping(value = "/export") @ResponseBody publi ...
- RAP接口文档的安装
本机环境 系统:CentOS 6.7 64 位 MySQL 5.6 JDK 1.8 Tomcat 8 Redis 3.0.7 Rap 0.14.1 Rap 说明 官网:https://github.c ...
- [转]Oracle Client 11g安裝經驗
本文转自:http://www.dotblogs.com.tw/shadow/archive/2011/11/08/54759.aspx 開發環境:本機(Win XP)的ASP.net 4 WebSi ...
- android fragment轻松监听返回键/Fragment中的popupwindow响应返回键隐藏
现在的开发我们基本上都是一个主activity中放多个fragment,点击返回按钮的时候,直接退出主activity,但是我们在fragment中经常会弹出例如popupWindow这样的布局,用户 ...
- 努比亚(nubia) Z18 mini NX611J 解锁BootLoader 并刷入recovery ROOT
努比亚(nubia) Z18 mini NX611J解锁BootLoader 并刷入recovery ROOT 工具下载链接:https://pan.baidu.com/s/1toU-mTR9FNE ...
- Android studio 中R.menu的创建
对于Android开发中的menu没有声明的情况: 首先,将鼠标定位到红色的menu上面, 然后,Alt+enter组合键,建立文件menu, 然后将以下代码复制进去: <item androi ...
- jQuery——尺寸位置
获取宽:$(".box").width() 设置宽:$(".box").width(200) 获取高:$(".box").height() ...
- 史上巨坑: vim的"set foldmethod=syntax"设置竟然是导致ctrl+p(ctrl+n)补全在文件稍大时光标位于中间位置补全效率变慢的元凶!
最近我的vim又让我闹心了. 问题出现在supertab的补全速度上, 有时候按下tab键半天才弹出补全列表, 即便是弹出了列表在列表上下移动也变得的相当缓慢, 这让我的很是蛋疼. 在完全无法接受这个 ...
- #1003 Max Sum
http://acm.hdu.edu.cn/showproblem.php?pid=1003 给你一串数列,让你求出其中 一段连续的子数列 并且 该子数列所有数字之和最大,输出该子数列的和.起点与终点 ...
- spring+spring MVC+mybatis 框架搭建
1.新建一个javaWeb工程Test,创建时记得勾选web.xml文件. 2.导入需要的jar包,Mybatis所有的jar,spring所有的jar,mysql驱动包. 这里mybatis和spr ...