install Hadoop

CentOS:

https://www.vultr.com/docs/how-to-install-hadoop-in-stand-alone-mode-on-centos-7

https://www.linode.com/docs/databases/hadoop/how-to-install-and-set-up-hadoop-cluster/

http://www.michael-noll.com/tutorials/running-hadoop-on-ubuntu-linux-multi-node-cluster/

check nodes status

$HADOOP_HOME/bin/hadoop dfsadmin -report

https://ambari.apache.org/1.2.3/installing-hadoop-using-ambari/content/reference_chap2_1.html

　　Also can access name node web page with port 50070

install Spark

on CentOS:

　　https://aodba.com/how-to-install-apache-spark-in-centos-standalone/

https://bigdata-etl.com/how-to-install-apache-spark-standalone-in-centos/

https://www.tutorialspoint.com/apache_spark/apache_spark_installation.htm

https://gist.github.com/darcyliu/d47edccb923b0f03280a4cf8b66227c1

on Ubuntu:

install JAVA

install Scala

install Spark

big data env setup的更多相关文章

  1. Devlopment Env Setup install ubuntu16.04

    http://blog.csdn.net/ljheee/article/details/52966048 1.add chinese language support settings -> i ...

  2. Openwrt build env setup(9)

    reference : https://openwrt.org/docs/guide-developer/quickstart-build-images Install dependence pack ...

  3. 使用Active Database Duplication创建跨平台Data Guard设置 (Windows/Linux) (Doc ID 881421.1)

    Using Active Database Duplication to Create Cross Platform Data Guard Setup (Windows/Linux) (Doc ID ...

  4. sass-loader使用data引入公用文件或全局变量报错

    报错信息: ValidationError: Invalid options object. Sass Loader has been initialised using an options obj ...

  5. EntityFramework 7 开发纪录

    博文目录: 暂时开发模式 Code First 具体体现 DbContext 配置 Entity 映射关联配置 Migration 问题纪录(已解决) 之前的一篇博文:EF7 Code First O ...

  6. 引擎设计跟踪(九.14.2g) 将GNUMake集成到Visual Studio

    最近在做纹理压缩工具, 以及数据包的生成. shader编译已经在vs工程里面了, 使用custom build tool, build命令是调用BladeShaderComplier, 并且每个文件 ...

  7. 参照openRTSP写的一个RTSP client 加了一些注解

    #include "liveMedia.hh"  #include "BasicUsageEnvironment.hh"  #include "Gro ...

  8. Docker教程：dokcer machine的概念和安装

    http://blog.csdn.net/pipisorry/article/details/50920982 Docker machine介绍 做为Docker容器集群管理三剑客之一的Docker ...

  9. Tensorflow object detection API 搭建属于自己的物体识别模型

    一.下载Tensorflow object detection API工程源码 网址:https://github.com/tensorflow/models,可通过Git下载,打开Git Bash, ...

随机推荐

  1. linux 文件查找 find命令详解

    一,从索引库查找文件:locate 索引库:操作系统会周期性的遍历根文件系统,然后生成索引库 手动更新索引库:updatedb 语法:locate [OPTION]... PATTERN... 只匹配 ...

  2. QT程序在发布的时候应注意的地方

    ---恢复内容开始--- 我们用QT开发好的应用程序,如果要发布到其他计算机上运行怎么办呢?我们在用VC编程时,单独运行编译好的可执行文件时,经常会发现提示缺少动态库.用QT编程也不例外,在一定程度上 ...

  3. 别再误解MySQL和「幻读」了

    The so-called phantom problem occurs within a transaction when the same query produces different set ...

  4. sevlet

    https://blog.csdn.net/Mithrandir_One/article/details/52900425 大家现在做的比较多的基本上就是 web 应用.所以一定要把 sevlet 及 ...

  5. tar命令--数据归档（二）

    tar -cf all.tar *.jpg 这条命令是将所有.jpg的文件打成一个名为all.tar的包.-c是表示产生新的包,-f指定包的文件名. tar -rf all.tar *.gif 这条命 ...

  6. hadoop下生成echarts关系图

    数据 O700 O2833 O700 O331 O700 O3425 O700 O350 O700 O3516 O700 O3826 读取文件类 public class FileReadFromHd ...

  7. POJ-2762-Going from u to v or from v to u（强连通， 拓扑排序）

    链接: https://vjudge.net/problem/POJ-2762 题意: In order to make their sons brave, Jiajia and Wind take ...

  8. python 操作符**与*的用法

  9. 【leetcode】1213.Intersection of Three Sorted Arrays

    题目如下: Given three integer arrays arr1, arr2 and arr3 sorted in strictly increasing order, return a s ...

  10. eclipse在线安装ermaster插件

    eclipse在线安装ermaster插件: https://www.jianshu.com/p/449fbcd9141a ERMaster的安装和使用 https://www.cnblogs.com ...