Hadoop生态圈-CentOs7.5单机部署ClickHouse
Hadoop生态圈-CentOs7.5单机部署ClickHouse
作者：尹正杰
版权声明：原创作品，谢绝转载！否则将追究法律责任。
到了新的公司，认识了新的同事，生产环境也得你去适应新的集群环境，我新入职的公司的大数据开发同时并不看好hbase，而是对clickhouse青睐有加，听他们口头把ClickHouse吹的那么的出神入化，听的我是有种迫不及待想要对它一探究竟。
最新的一手资料请参考官网：https://clickhouse.yandex/
一.ClickHouse的独特功能
1>.真正的面向列的DBMS
因为有些系统可以单独存储不同列的值，但由于它们针对其他场景进行了优化，因此无法有效地处理分析查询。例如HBase，BigTable，Cassandra和HyperTable。在这些系统中，您将获得大约每秒十万行的吞吐量，但不会达到每秒数亿行。
值得注意的是，ClickHouse是一个数据库管理系统，而不是一个数据库。ClickHouse允许在运行时创建表和数据库，加载数据和运行查询，而无需重新配置和重新启动服务器。
2>.适用于在线查询
低延迟意味着可以在没有延迟的情况下处理查询，而无需在加载用户界面页面的同一时间提前准备答案。换句话说，在线。
3>.数据复制和数据完整性支持
使用异步多主机复制。写入任何可用副本后，数据将分发到后台的所有剩余副本。系统在不同的副本上维护相同的数据。在大多数故障后自动执行恢复，在复杂情况下 - 半自动执行。
4>.支持近似计算
ClickHouse提供了各种交易性能准确度的方法：
4.1>.聚合函数，用于近似计算不同值，中位数和分位数的数量。
4.2>.基于数据的一部分（样本）运行查询并获得近似结果。在这种情况下，从磁盘中检索的数据比例较少。
4.2>.为有限数量的随机密钥运行聚合，而不是为所有密钥运行聚合。在数据中密钥分发的某些条件下，这在使用较少资源的同时提供了相当准确的结果。
5>.SQL支持
ClickHouse支持基于SQL的声明性查询语言，在许多情况下与SQL标准相同。支持的查询包括GROUP BY，ORDER BY，FROM，IN和JOIN子句中的子查询以及标量子查询。不支持从属子查询和窗口函数。
以上说明并非我个人对ClickHouse的夸张说法，而是摘自官方文档的说明，可能官网对其软件有一定的夸张手法在里面，官网文档请参考：https://clickhouse.yandex/docs/en/
二.部署ClickHouse
推荐的安装方法参考：https://github.com/Altinity/clickhouse-rpm-install。文档写的头头是道，说下载一个安装shell脚本，结果你按照它的方法做会跑出一系列的坑，我在这里就不吐槽了，喜欢被虐的小伙伴可以去试试，我生产环境用的操作系统基本上都是CentOs的操作系统，我参考了网上多位网友的部署方法，找到了一种最简单有效的部署方法，我这里一句把坑给大家踩过了，只要我的按照步骤来部署就能成功！
1>.操作平台介绍
[root@yinzhengjie ~]# ip a | grep global | awk '{print $2}' | awk -F '/' '{print $1}'
10.1.2.105
[root@yinzhengjie ~]#
[root@yinzhengjie ~]# hostname
yinzhengjie
[root@yinzhengjie ~]#
[root@yinzhengjie ~]# cat /proc/cpuinfo| grep "processor"| wc -l
[root@yinzhengjie ~]#
[root@yinzhengjie ~]# cat /proc/cpuinfo| grep "cpu cores"| uniq
cpu cores :
[root@yinzhengjie ~]#
[root@yinzhengjie ~]# cat /proc/cpuinfo| grep "physical id"| sort| uniq| wc -l
[root@yinzhengjie ~]#
[root@yinzhengjie ~]# cat /proc/cpuinfo | grep name | cut -f2 -d: | uniq -c
Intel(R) Xeon(R) CPU E5- v4 @ .10GHz
[root@yinzhengjie ~]#
[root@yinzhengjie ~]# free -m
total used free shared buff/cache available
Mem:
Swap:
[root@yinzhengjie ~]#
[root@yinzhengjie ~]# free -g
total used free shared buff/cache available
Mem:
Swap:
[root@yinzhengjie ~]#
[root@yinzhengjie ~]#
[root@yinzhengjie ~]# uname -r
3.10.-.el7.x86_64
[root@yinzhengjie ~]#
[root@yinzhengjie ~]# uname -m
x86_64
[root@yinzhengjie ~]#
[root@yinzhengjie ~]# cat /etc/redhat-release
CentOS Linux release 7.5. (Core)
You have new mail in /var/spool/mail/root
[root@yinzhengjie ~]#
2>.检查是否支持SSE4.2
[root@yinzhengjie ~]# grep -q sse4_2 /proc/cpuinfo && echo "SSE 4.2 supported" || echo "SSE 4.2 not supported"
SSE 4.2 supported
[root@yinzhengjie ~]#
3>.下载依赖包
我已经将上面的rpm包下载好了，以及需要的依赖包都下载好啦～大家可以直接下载来用即可。百度云链接:https://pan.baidu.com/s/16gPGxOnHKfRZj2ytOil7qQ 密码:rs4u
[root@yinzhengjie download]# ll
total
-rw-r--r-- root root Oct : yinzhengjie-clickhouse.tar.gz
[root@yinzhengjie download]#
[root@yinzhengjie download]#
[root@yinzhengjie download]# tar zxf yinzhengjie-clickhouse.tar.gz -C ./
[root@yinzhengjie download]#
[root@yinzhengjie download]# ll
total
drwxr-xr-x root root Oct : clickhouse
-rw-r--r-- root root Oct : yinzhengjie-clickhouse.tar.gz
[root@yinzhengjie download]#
[root@yinzhengjie download]# cd clickhouse/
[root@yinzhengjie clickhouse]#
[root@yinzhengjie clickhouse]# ll
total
-rw-r--r-- root root Oct : clickhouse-client-1.1.-.el7.x86_64.rpm
-rw-r--r-- root root Oct : clickhouse-compressor-1.1.-.el7.x86_64.rpm
-rw-r--r-- root root Oct : clickhouse-debuginfo-1.1.-.el7.x86_64.rpm
-rw-r--r-- root root Oct : clickhouse-server-1.1.-.el7.x86_64.rpm
-rw-r--r-- root root Oct : clickhouse-server-common-1.1.-.el7.x86_64.rpm
-rw-r--r-- root root Oct : libtool-ltdl-2.4.-.el7_2.x86_64.rpm
-rw-r--r-- root root Oct : unixODBC-2.3.-.el7.x86_64.rpm
[root@yinzhengjie clickhouse]#
4>.安装ClickHouse的依赖及其rpm包
[root@yinzhengjie clickhouse]# ll
total
-rw-r--r-- root root Oct : clickhouse-client-1.1.-.el7.x86_64.rpm
-rw-r--r-- root root Oct : clickhouse-compressor-1.1.-.el7.x86_64.rpm
-rw-r--r-- root root Oct : clickhouse-debuginfo-1.1.-.el7.x86_64.rpm
-rw-r--r-- root root Oct : clickhouse-server-1.1.-.el7.x86_64.rpm
-rw-r--r-- root root Oct : clickhouse-server-common-1.1.-.el7.x86_64.rpm
-rw-r--r-- root root Oct : libtool-ltdl-2.4.-.el7_2.x86_64.rpm
-rw-r--r-- root root Oct : unixODBC-2.3.-.el7.x86_64.rpm
[root@yinzhengjie clickhouse]#
[root@yinzhengjie clickhouse]#
You have new mail in /var/spool/mail/root
[root@yinzhengjie clickhouse]#
[root@yinzhengjie clickhouse]# yum -y localinstall *.rpm
Loaded plugins: fastestmirror
Examining clickhouse-client-1.1.-.el7.x86_64.rpm: clickhouse-client-1.1.-.el7.x86_64
Marking clickhouse-client-1.1.-.el7.x86_64.rpm to be installed
Examining clickhouse-compressor-1.1.-.el7.x86_64.rpm: clickhouse-compressor-1.1.-.el7.x86_64
Marking clickhouse-compressor-1.1.-.el7.x86_64.rpm to be installed
Examining clickhouse-debuginfo-1.1.-.el7.x86_64.rpm: clickhouse-debuginfo-1.1.-.el7.x86_64
Marking clickhouse-debuginfo-1.1.-.el7.x86_64.rpm to be installed
Examining clickhouse-server-1.1.-.el7.x86_64.rpm: clickhouse-server-1.1.-.el7.x86_64
Marking clickhouse-server-1.1.-.el7.x86_64.rpm to be installed
Examining clickhouse-server-common-1.1.-.el7.x86_64.rpm: clickhouse-server-common-1.1.-.el7.x86_64
Marking clickhouse-server-common-1.1.-.el7.x86_64.rpm to be installed
Examining libtool-ltdl-2.4.-.el7_2.x86_64.rpm: libtool-ltdl-2.4.-.el7_2.x86_64
Marking libtool-ltdl-2.4.-.el7_2.x86_64.rpm to be installed
Examining unixODBC-2.3.-.el7.x86_64.rpm: unixODBC-2.3.-.el7.x86_64
Marking unixODBC-2.3.-.el7.x86_64.rpm to be installed
Resolving Dependencies
--> Running transaction check
---> Package clickhouse-client.x86_64 :1.1.-.el7 will be installed
---> Package clickhouse-compressor.x86_64 :1.1.-.el7 will be installed
--> Processing Dependency: libicudata.so.()(64bit) for package: clickhouse-compressor-1.1.-.el7.x86_64
Determining fastest mirrors
10gen | 2.5 kB ::
base | 3.6 kB ::
centosplus | 3.4 kB ::
epel | 3.2 kB ::
extras | 3.4 kB ::
updates | 3.4 kB ::
(/): epel/x86_64/updateinfo | kB ::
(/): epel/x86_64/primary | 3.6 MB ::
(/): updates//x86_64/primary_db | 6.0 MB ::
epel /
--> Processing Dependency: libicui18n.so.()(64bit) for package: clickhouse-compressor-1.1.-.el7.x86_64
--> Processing Dependency: libicuuc.so.()(64bit) for package: clickhouse-compressor-1.1.-.el7.x86_64
---> Package clickhouse-debuginfo.x86_64 :1.1.-.el7 will be installed
---> Package clickhouse-server.x86_64 :1.1.-.el7 will be installed
---> Package clickhouse-server-common.x86_64 :1.1.-.el7 will be installed
---> Package libtool-ltdl.x86_64 :2.4.-.el7_2 will be installed
---> Package unixODBC.x86_64 :2.3.-.el7 will be installed
--> Running transaction check
---> Package libicu.x86_64 :50.1.-.el7 will be installed
--> Finished Dependency Resolution Dependencies Resolved ===================================================================================================================================================================================================================
Package Arch Version Repository Size
===================================================================================================================================================================================================================
Installing:
clickhouse-client x86_64 1.1.-.el7 /clickhouse-client-1.1.-.el7.x86_64
clickhouse-compressor x86_64 1.1.-.el7 /clickhouse-compressor-1.1.-.el7.x86_64 2.9 M
clickhouse-debuginfo x86_64 1.1.-.el7 /clickhouse-debuginfo-1.1.-.el7.x86_64 M
clickhouse-server x86_64 1.1.-.el7 /clickhouse-server-1.1.-.el7.x86_64 M
clickhouse-server-common x86_64 1.1.-.el7 /clickhouse-server-common-1.1.-.el7.x86_64 k
libtool-ltdl x86_64 2.4.-.el7_2 /libtool-ltdl-2.4.-.el7_2.x86_64 k
unixODBC x86_64 2.3.-.el7 /unixODBC-2.3.-.el7.x86_64 1.2 M
Installing for dependencies:
libicu x86_64 50.1.-.el7 base 6.9 M Transaction Summary
===================================================================================================================================================================================================================
Install Packages (+ Dependent package) Total size: M
Total download size: 6.9 M
Installed size: M
Downloading packages:
libicu-50.1.-.el7.x86_64.rpm | 6.9 MB ::
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
Installing : libtool-ltdl-2.4.-.el7_2.x86_64 /
Installing : unixODBC-2.3.-.el7.x86_64 /
Installing : clickhouse-server-common-1.1.-.el7.x86_64 /
Installing : libicu-50.1.-.el7.x86_64 /
Installing : clickhouse-server-1.1.-.el7.x86_64 /
Installing : clickhouse-client-1.1.-.el7.x86_64 /
Installing : clickhouse-compressor-1.1.-.el7.x86_64 /
Installing : clickhouse-debuginfo-1.1.-.el7.x86_64 /
Verifying : clickhouse-debuginfo-1.1.-.el7.x86_64 /
Verifying : libtool-ltdl-2.4.-.el7_2.x86_64 /
Verifying : unixODBC-2.3.-.el7.x86_64 /
Verifying : libicu-50.1.-.el7.x86_64 /
Verifying : clickhouse-server-1.1.-.el7.x86_64 /
Verifying : clickhouse-client-1.1.-.el7.x86_64 /
Verifying : clickhouse-compressor-1.1.-.el7.x86_64 /
Verifying : clickhouse-server-common-1.1.-.el7.x86_64 / Installed:
clickhouse-client.x86_64 :1.1.-.el7 clickhouse-compressor.x86_64 :1.1.-.el7 clickhouse-debuginfo.x86_64 :1.1.-.el7 clickhouse-server.x86_64 :1.1.-.el7
clickhouse-server-common.x86_64 :1.1.-.el7 libtool-ltdl.x86_64 :2.4.-.el7_2 unixODBC.x86_64 :2.3.-.el7 Dependency Installed:
libicu.x86_64 :50.1.-.el7 Complete!
[root@yinzhengjie clickhouse]#
[root@yinzhengjie clickhouse]# yum -y localinstall *.rpm #直接通过yum安装即可
三.配置clickhouse
1>.备份clickhouse-server配置文件config.xml
[root@yinzhengjie ~]# cp /etc/clickhouse-server/config.xml /etc/clickhouse-server/config.xml.`date +%F`
[root@yinzhengjie ~]#
2>.修改clickhouse-server配置文件config.xml（指定clickhouse-server的主机地址）
[root@yinzhengjie ~]# sed -i 's#<listen_host>127.0.0.1</listen_host>#<listen_host>10.1.2.105</listen_host>#' /etc/clickhouse-server/config.xml
[root@yinzhengjie ~]#
3>.修改clickhouse-server配置文件config.xml（修改默认的9000端口，我们自定义该服务的端口为9999）
[root@yinzhengjie ~]# sed -i 's#<tcp_port>9000</tcp_port>#<tcp_port>9999</tcp_port>#' /etc/clickhouse-server/config.xml
[root@yinzhengjie ~]#
4>.备份clickhouse-server配置文件users.xml
[root@yinzhengjie ~]# cp /etc/clickhouse-server/users.xml /etc/clickhouse-server/users.xml.`date +%F`
[root@yinzhengjie ~]#
5>.修改clickhouse-server配置文件users.xml（修改/etc/clickhouse-server/users.xml配置文件的第83行，改地址改为你当前部署的服务器IP地址，不推荐使用主机名！下图是我修改后的值）
四.clickhouse管理方式
1>.启动clickhouse
[root@yinzhengjie ~]# clickhouse-server --config-file=/etc/clickhouse-server/config.xml
Include not found: clickhouse_remote_servers
Include not found: clickhouse_compression
2018.10. ::20.997594 [ ] <Warning> Application: Logging to console
2018.10. ::21.000228 [ ] <Information> : Starting daemon with revision
2018.10. ::21.000337 [ ] <Information> Application: starting up
2018.10. ::21.001299 [ ] <Debug> Application: Set max number of file descriptors to (was ).
2018.10. ::21.001317 [ ] <Debug> Application: Initializing DateLUT.
2018.10. ::21.001326 [ ] <Trace> Application: Initialized DateLUT with time zone `Asia/Shanghai'.
2018.10. ::21.048483 [ ] <Debug> Application: Configuration parameter 'interserver_http_host' doesn't exist or exists and empty. Will use 'yinzhengjie' as replica host.
2018.10. ::21.048676 [ ] <Debug> ConfigReloader: Loading config `/etc/clickhouse-server/users.xml'
2018.10. ::21.053081 [ ] <Warning> ConfigProcessor: Include not found: networks
2018.10. ::21.053114 [ ] <Warning> ConfigProcessor: Include not found: networks
2018.10. ::21.054194 [ ] <Information> Application: Loading metadata.
2018.10. ::21.055243 [ ] <Information> DatabaseOrdinary (default): Total tables.
2018.10. ::21.055919 [ ] <Debug> Application: Loaded metadata.
2018.10. ::21.056089 [ ] <Information> DatabaseOrdinary (system): Total tables.
2018.10. ::21.057059 [ ] <Information> Application: Listening http://[::1]:8123
2018.10. ::21.057138 [ ] <Information> Application: Listening tcp: [::]:
2018.10. ::21.057198 [ ] <Information> Application: Listening interserver: [::]:
2018.10. ::21.057289 [ ] <Information> Application: Listening http://10.1.2.105:8123
2018.10. ::21.057420 [ ] <Information> Application: Listening tcp: 10.1.2.105:
2018.10. ::21.057486 [ ] <Information> Application: Listening interserver: 10.1.2.105:
2018.10. ::21.057937 [ ] <Information> Application: Ready for connections.
2018.10. ::23.048780 [ ] <Debug> ConfigReloader: Loading config `/etc/clickhouse-server/config.xml'
2018.10. ::23.053196 [ ] <Warning> ConfigProcessor: Include not found: clickhouse_remote_servers
2018.10. ::23.053332 [ ] <Warning> ConfigProcessor: Include not found: clickhouse_compression
[root@yinzhengjie ~]# clickhouse-server --config-file=/etc/clickhouse-server/config.xml
2>.查看clickhouse服务是否启动
[root@yinzhengjie ~]# netstat -untalp | grep
tcp 10.1.2.105: 0.0.0.0:* LISTEN /clickhouse-se
tcp6 ::: :::* LISTEN /clickhouse-se
[root@yinzhengjie ~]#
3>.使用客户端链接测试
[root@yinzhengjie ~]# clickhouse-client --host=10.1.2.105 --port=
ClickHouse client version 1.1..
Connecting to 10.1.2.105:.
Connected to ClickHouse server version 1.1.. :)
:) select now(); SELECT now() ┌───────────────now()─┐
│ -- :: │
└─────────────────────┘ rows in set. Elapsed: 0.067 sec. :)
:) show tables; SHOW TABLES Ok. rows in set. Elapsed: 0.003 sec. :)
:)
[root@yinzhengjie ~]# clickhouse-client --host=10.1.2.105 --port=9999
4>.关闭clickhouse服务
[root@yinzhengjie ~]# ps -aux|grep clickhouse-server | grep -v grep
root 0.1 0.3 pts/ Sl+ : : clickhouse-server --config-file=/etc/clickhouse-server/config.xml
[root@yinzhengjie ~]#
[root@yinzhengjie ~]# kill - `ps -aux|grep clickhouse-server | grep -v grep | awk '{print $2}'`
[root@yinzhengjie ~]#
You have new mail in /var/spool/mail/root
[root@yinzhengjie ~]#
[root@yinzhengjie ~]# ps -aux|grep clickhouse-server | grep -v grep
[root@yinzhengjie ~]#
[root@yinzhengjie ~]#
[root@yinzhengjie ~]# kill -9 `ps -aux|grep clickhouse-server | grep -v grep | awk '{print $2}'`
5>.为clickhouse-server添加守护进程
[root@yinzhengjie ~]# nohup clickhouse-server --config-file=/etc/clickhouse-server/config.xml >/var/log/yinzhengjie-clickhouse.log >& & #会在"/var/log"目录下生成一个名称为“yinzhengjie-clickhouse.log”的日志文件
[]
[root@yinzhengjie ~]#
[root@yinzhengjie ~]# tail -10f /var/log/yinzhengjie-clickhouse.log
2018.10. ::53.981429 [ ] <Information> Application: Listening http://[::1]:8123
2018.10. ::53.981479 [ ] <Information> Application: Listening tcp: [::]:
2018.10. ::53.981516 [ ] <Information> Application: Listening interserver: [::]:
2018.10. ::53.981554 [ ] <Information> Application: Listening http://10.1.2.105:8123
2018.10. ::53.981590 [ ] <Information> Application: Listening tcp: 10.1.2.105:
2018.10. ::53.981624 [ ] <Information> Application: Listening interserver: 10.1.2.105:
2018.10. ::53.981796 [ ] <Information> Application: Ready for connections.
2018.10. ::55.973241 [ ] <Debug> ConfigReloader: Loading config `/etc/clickhouse-server/config.xml'
2018.10. ::55.977391 [ ] <Warning> ConfigProcessor: Include not found: clickhouse_remote_servers
2018.10. ::55.977455 [ ] <Warning> ConfigProcessor: Include not found: clickhouse_compression
2018.10. ::09.185410 [ ] <Trace> TCPConnectionFactory: TCP Request. Address: 10.1.2.105:
2018.10. ::09.185660 [ ] <Debug> TCPHandler: Connected ClickHouse client version 1.1., user: default.
2018.10. ::18.439436 [ ] <Debug> executeQuery: (from 10.1.2.105:) select now()
2018.10. ::18.440437 [ ] <Trace> InterpreterSelectQuery: FetchColumns -> Complete
2018.10. ::18.440867 [ ] <Debug> executeQuery: Query pipeline:
Expression
Expression
One 2018.10. ::18.441657 [ ] <Information> executeQuery: Read rows, 1.00 B in 0.002 sec., rows/sec., 510.97 B/sec.
2018.10. ::18.441764 [ ] <Debug> MemoryTracker: Peak memory usage (for query): 1.00 MiB.
2018.10. ::18.441804 [ ] <Debug> MemoryTracker: Peak memory usage (for user): 1.00 MiB.
2018.10. ::18.441849 [ ] <Debug> MemoryTracker: Peak memory usage (total): 1.00 MiB.
2018.10. ::18.441882 [ ] <Information> TCPHandler: Processed in 0.003 sec.
好啦～分享就到这里了，时间过的咋就这么快呢？一不小心就晚上九点多了，关于ClickHouse集群的部署等我后期研究好了也会分享给大家的，我得赶紧走了，不然赶不上地铁啦！晚安，兄弟们！
Hadoop生态圈-CentOs7.5单机部署ClickHouse的更多相关文章
- Hadoop生态圈-zookeeper完全分布式部署
Hadoop生态圈-zookeeper完全分布式部署 作者:尹正杰 版权声明:原创作品,谢绝转载!否则将追究法律责任. 本篇博客部署是建立在Hadoop高可用基础之上的,关于Hadoop高可用部署请参 ...
- Hadoop生态圈-phoenix完全分布式部署以及常用命令介绍
Hadoop生态圈-phoenix完全分布式部署 作者:尹正杰 版权声明:原创作品,谢绝转载!否则将追究法律责任. phoenix只是一个插件,我们可以用hive给hbase套上一个JDBC壳,但是你 ...
- Hadoop2-认识Hadoop大数据处理架构-单机部署
一.Hadoop原理介绍 1.请参考原理篇:Hadoop1-认识Hadoop大数据处理架构 二.centos7单机部署hadoop 前期准备 1.创建用户 [root@web3 ~]# useradd ...
- CentOS7下单机部署RabbltMQ环境的操作记录
一.RabbitMQ简单介绍在日常工作环境中,你是否遇到过两个(多个)系统间需要通过定时任务来同步某些数据?你是否在为异构系统的不同进程间相互调用.通讯的问题而苦恼.挣扎?如果是,那么恭喜你,消息服务 ...
- Centos7下单机部署Solr7.3
本章重点介绍CentOS7 下部署Solr7 ,添加核心Core配置,Dataimport导入,中文分词的相关操作. 一.准备工作 演示环境是在虚拟机下安装的CentOS7.java JDK8 ...
- Hadoop安装和基本单机部署
下载安装 # 下载 $ cd /usr/local $ wget http://mirrors.hust.edu.cn/apache/hadoop/common/hadoop-2.9.2/hadoo ...
- Hadoop生态圈-离线方式部署Cloudera Manager5.15.1
Hadoop生态圈-离线方式部署Cloudera Manager5.15.1 作者:尹正杰 版权声明:原创作品,谢绝转载!否则将追究法律责任. 到目前位置,Cloudera Manager和CDH最新 ...
- Clickhouse单机部署以及从mysql增量同步数据
背景: 随着数据量的上升,OLAP一直是被讨论的话题,虽然druid,kylin能够解决OLAP问题,但是druid,kylin也是需要和hadoop全家桶一起用的,异常的笨重,再说我也搞不定,那只能 ...
- Hadoop系列之（一）：Hadoop单机部署
1. Hadoop介绍 Hadoop是一个能够对海量数据进行分布式处理的系统架构. Hadoop框架的核心是:HDFS和MapReduce. HDFS分布式文件系统为海量的数据提供了存储, MapRe ...
随机推荐
- Linux内核设计第十七章笔记
第十七章 设备与模块 关于设备驱动和设备管理,四种内核成分 设备类型:在所有unix系统中为了统一普通设备的操作所采用的分类 模块:Linux内核中用于按需加载和卸载目标代码的机制 内核对象:内核数据 ...
- PHP预防跨站脚本（XSS）攻击且不影响html代码显示效果
什么是XSS 跨站脚本攻击(Cross Site Scripting),为不和层叠样式表(Cascading Style Sheets, CSS)的缩写混淆,故将跨站脚本攻击缩写为XSS.恶意攻击者往 ...
- logback基本入门
1. logback的定义 Logback是由log4j创始人设计的另一个开源日志组件,官方网站: http://logback.qos.ch.它当前分为下面下个模块: logback-core:其它 ...
- Mesos的用途
http://mesos.apache.org/ https://mesosphere.com/why-mesos/ http://www.cnblogs.com/ee900222/p/docker_ ...
- NF5280M4 安装 Win2016 的方法
1. 前提条件, 硬盘大于2T, 2. 必须使用最新版本的 Win2016 首先 win2016的可用序列号 • Windows Server 数据中心 CB7KF-BWN84-R7R2Y-793K2 ...
- node.js依赖express解析post请求四种数据格式（）
分别是这四种: www-form-urlencoded, form-data, application/json, text/xml www-form-urlencoded 这是http的post请求 ...
- 小项目分析之C++ 实现模拟银行排队
一.问题定义与分析 问题定义 •要解决的问题——银行一天之内的: 1.总客户数 2.客户总逗留时间 3.客户平均逗留时间 问题分析 •新来的人找个短的队伍,站在队尾开始排队 •排在队头的人可以办理 ...
- General Test Scenarios
1 all mandatory fields should be validated and indicated by askterisk(*) symbol2 validation error me ...
- nginx-匹配规则
location 指令的作用是根据用户请求的URI来执行不同的应用. locationn使用的语法为 location [=|~|~*|^~] uri { .... } location 语法说明表 ...
- BZOJ1785[USACO 2010 Jan Gold 3.Cow Telephones]——贪心
题目描述 奶牛们建立了电话网络,这个网络可看作为是一棵无根树连接n(1 n 100,000)个节点,节点编号为1 .. n.每个节点可能是(电话交换机,或者电话机).每条电话线连接两个节点.第i条电话 ...