apache storm 的安装
原文: http://storm.praveendeshmane.co.in/storm/storm-0-10-0-installation-on-ubuntu-14-04.jsp
------------------------------------------
Apache Storm is a distributed stream processing computation framework written predominantly in the Clojure programming language. Originally created by Nathan Marz and team at BackType, the project was open sourced after being acquired by Twitter. It uses custom created "spouts" and "bolts" to define information sources and manipulations to allow batch, distributed processing of streaming data. The initial release was on 17 September 2011.
A Storm application is designed as a "topology" in the shape of a directed acyclic graph (DAG) with spouts and bolts acting as the graph vertices. Edges on the graph are named streams and direct data from one node to another. Together, the topology acts as a data transformation pipeline. At a superficial level the general topology structure is similar to a MapReduce job, with the main difference being that data is processed in real time as opposed to in individual batches.
Pre Requirements
1) A machine with Ubuntu 14.04 LTS operating system.
2) Apcahe ZooKeeper pre installed (How to install ZooKeeper on Ubuntu 14.04)
3) Apache Storm 0.10.0 software (Download Here)
Storm Installation Steps
Step 1 - Installing java 7. Open a terminal (CTRL + ALT + T) and type the following sudo command.
$ sudo apt-get install openjdk-7-jdk
Check it installed properly or not.
$ java -version
Step 2 - Edit $HOME/.bashrc file by adding the java path.
$ sudo gedit $HOME/.bashrc
$HOME/.bashrc file
export JAVA_HOME=/usr/lib/jvm/java-7-openjdk-amd64
Step 3 - Reload your changed $HOME/.bashrc settings
$ source $HOME/.bashrc
Step 4 - Creating storm directory. Open a terminal (CTRL + ALT + T) and type the following $ sudo command.
$ sudo mkdir /usr/local/storm
Step 5 - Change the ownership and permissions of the directory /usr/local/storm. Here 'hduser' is an Ubuntu username.
$ sudo chown -R hduser /usr/local/storm
$ sudo chmod -R 755 /usr/local/storm
Step 6 - Change the directory to /home/hduser/Desktop , In my case the downloaded apache-storm-0.10.0.tar.gz file is in /home/hduser/Desktop folder. For you it might be in /downloads folder check it.
$ cd /home/hduser/Desktop/
Step 7 - Untar the apache-storm-0.10.0.tar.gz file.
$ tar xzf apache-storm-0.10.0.tar.gz
Step 8 - Move the contents of apache-storm-0.10.0 folder to /usr/local/storm
$ mv apache-storm-0.10.0/* /usr/local/storm
Step 9 - Change the directory to /usr/local/storm
$ cd /usr/local/storm
Step 10 - Create /data, /data/nimbus and /data/supervisor directories.
$ sudo mkdir /usr/local/storm/data
$ sudo mkdir /usr/local/storm/data/nimbus
$ sudo mkdir /usr/local/storm/data/supervisor
Step 11 - Change the ownership and permissions of the directory /data, /data/nimbus and /data/supervisor directories.. Here 'hduser' is an Ubuntu username.
$ sudo chown -R hduser /usr/local/storm/data
$ sudo chown -R hduser /usr/local/storm/data/nimbus
$ sudo chown -R hduser /usr/local/storm/data/supervisor
$ sudo chmod -R 755 /usr/local/storm/data
$ sudo chmod -R 755 /usr/local/storm/data/nimbus
$ sudo chmod -R 755 /usr/local/storm/data/supervisor
Step 12 - Change the directory to /usr/local/storm/conf
$ cd /usr/local/storm/conf
Step 13 - Edit storm.yaml file.
$ gedit storm.yaml
Step 14 - Add these below lines to storm.yaml file. Save and close.
storm.zookeeper.servers:
- "localhost"
storm.local.dir: "/usr/local/storm/data"
nimbus.host: "localhost"
nimbus.thrift.port: 49627
storm.zookeeper.port: 2181
supervisor.slots.ports:
- 6700
- 6701
- 6702
- 6703
Step 15 - Start ZooKeeper. Open a new terminal (CTRL + ALT + T) and start zookeeper.
$ /usr/local/zookeeper/bin/zkServer.sh start
Step 16 - Open a new terminal (CTRL + ALT + T). Change the directory to /usr/local/storm
$ cd /usr/local/storm
Step 17 - Start nimbus
$ ./bin/storm nimbus
Step 18 - Open a new terminal (CTRL + ALT + T). Change the directory to /usr/local/storm
$ cd /usr/local/storm
Step 19 - Start supervisor
$ ./bin/storm supervisor
Step 20 - Open a new terminal (CTRL + ALT + T). Change the directory to /usr/local/storm
$ cd /usr/local/storm
Step 21 - Start web UI
$ ./bin/storm ui
Step 22 - Check the web UI here. Open a browser and type the following URL.
http://localhost:8080
Please share this blog post and follow me for latest updates on
apache storm 的安装的更多相关文章
- ubuntu14.04LTS 下storm单机版安装配置
1.下载storm 的安装文件 http://www.apache.org/dyn/closer.cgi/incubator/storm/apache-storm-0.9.2-incubating/a ...
- Apache Storm 1.1.0 中文文档 | ApacheCN
前言 Apache Storm 是一个免费的,开源的,分布式的实时计算系统. 官方文档: http://storm.apache.org 中文文档: http://storm.apachecn.org ...
- Apache Storm
作者:jiangzz 电话:15652034180 微信:jiangzz_wx 微信公众账号:jiangzz_wy 背景介绍 流计算:将大规模流动数据在不断变化的运动过程中实现数据的实时分析,捕捉到可 ...
- Apache Storm Installation
安装的过程参照此处的过程介绍(https://www.tutorialspoint.com/apache_storm/apache_storm_installation.htm) 安装的过程要安装3个 ...
- 在Docker Swarm上部署Apache Storm:第2部分
[编者按]本文来自 Baqend Tech Blog,描述了如何在 Docker Swarm,而不是在虚拟机上部署和调配Apache Storm集群.文章系国内 ITOM 管理平台 OneAPM 编译 ...
- 在Docker Swarm上部署Apache Storm:第1部分
[编者按]本文来自 Baqend Tech Blog,描述了如何在 Docker Swarm,而不是在虚拟机上部署和调配Apache Storm集群.文章系国内 ITOM 管理平台 OneAPM 编译 ...
- Apache Storm使用
Apache Storm 是 Apache 基金会的开源的分布式实时计算系统.与 Hadoop 的批处理相类似,Storm 可以对大量的数据流进行可靠的实时处理,这一过程也称为“流式处理”,是分布式大 ...
- 分布式流处理框架 Apache Storm —— 编程模型详解
一.简介 二.IComponent接口 三.Spout 3.1 ISpout接口 3.2 BaseRichSpout抽象类 四.Bolt 4.1 IBolt 接口 4. ...
- apache storm基本原理及使用总结
什么是Apache Storm Apache Storm是一个分布式实时大数据处理系统.Storm设计用于在容错和水平可扩展方法中处理大量数据.它是一个流数据框架,具有最高的摄取率.虽然Storm是无 ...
随机推荐
- HTML--使用下拉列表框,节省空间
下拉列表在网页中也常会用到,它可以有效的节省网页空间.既可以单选.又可以多选.如下代码: 讲解: 1.value: 2.selected="selected": 设置selecte ...
- Elasticsearch之CURL命令的bulk批量操作
大家,也可去看看我下面的博客 Elasticsearch之批量操作bulk 官网上,是举例了新建一个requests文件. [hadoop@master elasticsearch-]$ pwd /h ...
- js的toFixed解惑
js中的toFixed,C#中的Math.round都是按照银行家算法的定义来算的,这里只拿js作参考,各个浏览器的计算方式并不一样,先看一张图,对比参数很容易就发现了其中的不同之处: 前三个Chro ...
- sublime text3配置插件
之前一直习惯用记事本写代码,懒得用IDE,虽然知道用 IDE效率高一些,不过觉得还是用记事本纯手写代码,比较容易记忆.直到昨天写代码遇到了点问题,截图给师兄看,师兄就问我是不是用记事本写代码,为什么不 ...
- SAS进阶《深入解析SAS》之Base SAS基础、读取外部数据到SAS数据集
SAS进阶<深入解析SAS>之Base SAS基础.读取外部数据到SAS数据集 前言:在学习完<SAS编程与商业案例>后,虽然能够接手公司的基本工作,但是为了更深入的SAS学习 ...
- Hibernate中使用子查询
子查询: 子查询是SQL语句中非常重要的功能特性,它可以在SQL语句中利用另外一条SQL语句的查询结果,在Hibernate中HQL查询同样对子查询功能提供了支持. 如下面代码所示: List ...
- POJ_3013_最短路
Big Christmas Tree Time Limit: 3000MS Memory Limit: 131072K Total Submissions: 23630 Accepted: 5 ...
- 人工智能时代,你为什么一定要学Python?
Python岗位年薪至少在10-20w之间,而且除了北.上.广.深外,杭州和合肥等二.三线城市的待遇正在与一线城市持平.未来,无论你身处何地,都能享受到人工智能.Python“带来的“市场红利”. P ...
- Centos7搭建ansible运维自动化工具
1)设置主机名和hosts文件 2)配置阿里云repo源 Wget -O /etc/yum.repos.d/aliyun.repo https://mirrors.aliyun.com/repo/Ce ...
- 使用Oracle函数在创建表的时候自动加入生成的流水号 生成格式是:前缀+年月日+00000
CREATE OR REPLACE FUNCTION fn_no_make(v_type VARCHAR2, v_number_col VARCHAR2, v_table_name VARCHAR2) ...