Hadoop生态圈-CentOs7.5单机部署ClickHouse
Hadoop生态圈-CentOs7.5单机部署ClickHouse
作者：尹正杰
版权声明：原创作品，谢绝转载！否则将追究法律责任。
到了新的公司，认识了新的同事，生产环境也得你去适应新的集群环境，我新入职的公司的大数据开发同时并不看好hbase，而是对clickhouse青睐有加，听他们口头把ClickHouse吹的那么的出神入化，听的我是有种迫不及待想要对它一探究竟。
最新的一手资料请参考官网：https://clickhouse.yandex/
一.ClickHouse的独特功能
1>.真正的面向列的DBMS
因为有些系统可以单独存储不同列的值，但由于它们针对其他场景进行了优化，因此无法有效地处理分析查询。例如HBase，BigTable，Cassandra和HyperTable。在这些系统中，您将获得大约每秒十万行的吞吐量，但不会达到每秒数亿行。
值得注意的是，ClickHouse是一个数据库管理系统，而不是一个数据库。ClickHouse允许在运行时创建表和数据库，加载数据和运行查询，而无需重新配置和重新启动服务器。
2>.适用于在线查询
低延迟意味着可以在没有延迟的情况下处理查询，而无需在加载用户界面页面的同一时间提前准备答案。换句话说，在线。
3>.数据复制和数据完整性支持
使用异步多主机复制。写入任何可用副本后，数据将分发到后台的所有剩余副本。系统在不同的副本上维护相同的数据。在大多数故障后自动执行恢复，在复杂情况下 - 半自动执行。
4>.支持近似计算
ClickHouse提供了各种交易性能准确度的方法：
4.1>.聚合函数，用于近似计算不同值，中位数和分位数的数量。
4.2>.基于数据的一部分（样本）运行查询并获得近似结果。在这种情况下，从磁盘中检索的数据比例较少。
4.2>.为有限数量的随机密钥运行聚合，而不是为所有密钥运行聚合。在数据中密钥分发的某些条件下，这在使用较少资源的同时提供了相当准确的结果。
5>.SQL支持
ClickHouse支持基于SQL的声明性查询语言，在许多情况下与SQL标准相同。支持的查询包括GROUP BY，ORDER BY，FROM，IN和JOIN子句中的子查询以及标量子查询。不支持从属子查询和窗口函数。
以上说明并非我个人对ClickHouse的夸张说法，而是摘自官方文档的说明，可能官网对其软件有一定的夸张手法在里面，官网文档请参考：https://clickhouse.yandex/docs/en/
二.部署ClickHouse
推荐的安装方法参考：https://github.com/Altinity/clickhouse-rpm-install。文档写的头头是道，说下载一个安装shell脚本，结果你按照它的方法做会跑出一系列的坑，我在这里就不吐槽了，喜欢被虐的小伙伴可以去试试，我生产环境用的操作系统基本上都是CentOs的操作系统，我参考了网上多位网友的部署方法，找到了一种最简单有效的部署方法，我这里一句把坑给大家踩过了，只要我的按照步骤来部署就能成功！
1>.操作平台介绍
[root@yinzhengjie ~]# ip a | grep global | awk '{print $2}' | awk -F '/' '{print $1}'
10.1.2.105
[root@yinzhengjie ~]#
[root@yinzhengjie ~]# hostname
yinzhengjie
[root@yinzhengjie ~]#
[root@yinzhengjie ~]# cat /proc/cpuinfo| grep "processor"| wc -l
[root@yinzhengjie ~]#
[root@yinzhengjie ~]# cat /proc/cpuinfo| grep "cpu cores"| uniq
cpu cores :
[root@yinzhengjie ~]#
[root@yinzhengjie ~]# cat /proc/cpuinfo| grep "physical id"| sort| uniq| wc -l
[root@yinzhengjie ~]#
[root@yinzhengjie ~]# cat /proc/cpuinfo | grep name | cut -f2 -d: | uniq -c
Intel(R) Xeon(R) CPU E5- v4 @ .10GHz
[root@yinzhengjie ~]#
[root@yinzhengjie ~]# free -m
total used free shared buff/cache available
Mem:
Swap:
[root@yinzhengjie ~]#
[root@yinzhengjie ~]# free -g
total used free shared buff/cache available
Mem:
Swap:
[root@yinzhengjie ~]#
[root@yinzhengjie ~]#
[root@yinzhengjie ~]# uname -r
3.10.-.el7.x86_64
[root@yinzhengjie ~]#
[root@yinzhengjie ~]# uname -m
x86_64
[root@yinzhengjie ~]#
[root@yinzhengjie ~]# cat /etc/redhat-release
CentOS Linux release 7.5. (Core)
You have new mail in /var/spool/mail/root
[root@yinzhengjie ~]#
2>.检查是否支持SSE4.2
[root@yinzhengjie ~]# grep -q sse4_2 /proc/cpuinfo && echo "SSE 4.2 supported" || echo "SSE 4.2 not supported"
SSE 4.2 supported
[root@yinzhengjie ~]#
3>.下载依赖包
我已经将上面的rpm包下载好了，以及需要的依赖包都下载好啦～大家可以直接下载来用即可。百度云链接:https://pan.baidu.com/s/16gPGxOnHKfRZj2ytOil7qQ 密码:rs4u
[root@yinzhengjie download]# ll
total
-rw-r--r-- root root Oct : yinzhengjie-clickhouse.tar.gz
[root@yinzhengjie download]#
[root@yinzhengjie download]#
[root@yinzhengjie download]# tar zxf yinzhengjie-clickhouse.tar.gz -C ./
[root@yinzhengjie download]#
[root@yinzhengjie download]# ll
total
drwxr-xr-x root root Oct : clickhouse
-rw-r--r-- root root Oct : yinzhengjie-clickhouse.tar.gz
[root@yinzhengjie download]#
[root@yinzhengjie download]# cd clickhouse/
[root@yinzhengjie clickhouse]#
[root@yinzhengjie clickhouse]# ll
total
-rw-r--r-- root root Oct : clickhouse-client-1.1.-.el7.x86_64.rpm
-rw-r--r-- root root Oct : clickhouse-compressor-1.1.-.el7.x86_64.rpm
-rw-r--r-- root root Oct : clickhouse-debuginfo-1.1.-.el7.x86_64.rpm
-rw-r--r-- root root Oct : clickhouse-server-1.1.-.el7.x86_64.rpm
-rw-r--r-- root root Oct : clickhouse-server-common-1.1.-.el7.x86_64.rpm
-rw-r--r-- root root Oct : libtool-ltdl-2.4.-.el7_2.x86_64.rpm
-rw-r--r-- root root Oct : unixODBC-2.3.-.el7.x86_64.rpm
[root@yinzhengjie clickhouse]#
4>.安装ClickHouse的依赖及其rpm包
[root@yinzhengjie clickhouse]# ll
total
-rw-r--r-- root root Oct : clickhouse-client-1.1.-.el7.x86_64.rpm
-rw-r--r-- root root Oct : clickhouse-compressor-1.1.-.el7.x86_64.rpm
-rw-r--r-- root root Oct : clickhouse-debuginfo-1.1.-.el7.x86_64.rpm
-rw-r--r-- root root Oct : clickhouse-server-1.1.-.el7.x86_64.rpm
-rw-r--r-- root root Oct : clickhouse-server-common-1.1.-.el7.x86_64.rpm
-rw-r--r-- root root Oct : libtool-ltdl-2.4.-.el7_2.x86_64.rpm
-rw-r--r-- root root Oct : unixODBC-2.3.-.el7.x86_64.rpm
[root@yinzhengjie clickhouse]#
[root@yinzhengjie clickhouse]#
You have new mail in /var/spool/mail/root
[root@yinzhengjie clickhouse]#
[root@yinzhengjie clickhouse]# yum -y localinstall *.rpm
Loaded plugins: fastestmirror
Examining clickhouse-client-1.1.-.el7.x86_64.rpm: clickhouse-client-1.1.-.el7.x86_64
Marking clickhouse-client-1.1.-.el7.x86_64.rpm to be installed
Examining clickhouse-compressor-1.1.-.el7.x86_64.rpm: clickhouse-compressor-1.1.-.el7.x86_64
Marking clickhouse-compressor-1.1.-.el7.x86_64.rpm to be installed
Examining clickhouse-debuginfo-1.1.-.el7.x86_64.rpm: clickhouse-debuginfo-1.1.-.el7.x86_64
Marking clickhouse-debuginfo-1.1.-.el7.x86_64.rpm to be installed
Examining clickhouse-server-1.1.-.el7.x86_64.rpm: clickhouse-server-1.1.-.el7.x86_64
Marking clickhouse-server-1.1.-.el7.x86_64.rpm to be installed
Examining clickhouse-server-common-1.1.-.el7.x86_64.rpm: clickhouse-server-common-1.1.-.el7.x86_64
Marking clickhouse-server-common-1.1.-.el7.x86_64.rpm to be installed
Examining libtool-ltdl-2.4.-.el7_2.x86_64.rpm: libtool-ltdl-2.4.-.el7_2.x86_64
Marking libtool-ltdl-2.4.-.el7_2.x86_64.rpm to be installed
Examining unixODBC-2.3.-.el7.x86_64.rpm: unixODBC-2.3.-.el7.x86_64
Marking unixODBC-2.3.-.el7.x86_64.rpm to be installed
Resolving Dependencies
--> Running transaction check
---> Package clickhouse-client.x86_64 :1.1.-.el7 will be installed
---> Package clickhouse-compressor.x86_64 :1.1.-.el7 will be installed
--> Processing Dependency: libicudata.so.()(64bit) for package: clickhouse-compressor-1.1.-.el7.x86_64
Determining fastest mirrors
10gen | 2.5 kB ::
base | 3.6 kB ::
centosplus | 3.4 kB ::
epel | 3.2 kB ::
extras | 3.4 kB ::
updates | 3.4 kB ::
(/): epel/x86_64/updateinfo | kB ::
(/): epel/x86_64/primary | 3.6 MB ::
(/): updates//x86_64/primary_db | 6.0 MB ::
epel /
--> Processing Dependency: libicui18n.so.()(64bit) for package: clickhouse-compressor-1.1.-.el7.x86_64
--> Processing Dependency: libicuuc.so.()(64bit) for package: clickhouse-compressor-1.1.-.el7.x86_64
---> Package clickhouse-debuginfo.x86_64 :1.1.-.el7 will be installed
---> Package clickhouse-server.x86_64 :1.1.-.el7 will be installed
---> Package clickhouse-server-common.x86_64 :1.1.-.el7 will be installed
---> Package libtool-ltdl.x86_64 :2.4.-.el7_2 will be installed
---> Package unixODBC.x86_64 :2.3.-.el7 will be installed
--> Running transaction check
---> Package libicu.x86_64 :50.1.-.el7 will be installed
--> Finished Dependency Resolution Dependencies Resolved ===================================================================================================================================================================================================================
Package Arch Version Repository Size
===================================================================================================================================================================================================================
Installing:
clickhouse-client x86_64 1.1.-.el7 /clickhouse-client-1.1.-.el7.x86_64
clickhouse-compressor x86_64 1.1.-.el7 /clickhouse-compressor-1.1.-.el7.x86_64 2.9 M
clickhouse-debuginfo x86_64 1.1.-.el7 /clickhouse-debuginfo-1.1.-.el7.x86_64 M
clickhouse-server x86_64 1.1.-.el7 /clickhouse-server-1.1.-.el7.x86_64 M
clickhouse-server-common x86_64 1.1.-.el7 /clickhouse-server-common-1.1.-.el7.x86_64 k
libtool-ltdl x86_64 2.4.-.el7_2 /libtool-ltdl-2.4.-.el7_2.x86_64 k
unixODBC x86_64 2.3.-.el7 /unixODBC-2.3.-.el7.x86_64 1.2 M
Installing for dependencies:
libicu x86_64 50.1.-.el7 base 6.9 M Transaction Summary
===================================================================================================================================================================================================================
Install Packages (+ Dependent package) Total size: M
Total download size: 6.9 M
Installed size: M
Downloading packages:
libicu-50.1.-.el7.x86_64.rpm | 6.9 MB ::
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
Installing : libtool-ltdl-2.4.-.el7_2.x86_64 /
Installing : unixODBC-2.3.-.el7.x86_64 /
Installing : clickhouse-server-common-1.1.-.el7.x86_64 /
Installing : libicu-50.1.-.el7.x86_64 /
Installing : clickhouse-server-1.1.-.el7.x86_64 /
Installing : clickhouse-client-1.1.-.el7.x86_64 /
Installing : clickhouse-compressor-1.1.-.el7.x86_64 /
Installing : clickhouse-debuginfo-1.1.-.el7.x86_64 /
Verifying : clickhouse-debuginfo-1.1.-.el7.x86_64 /
Verifying : libtool-ltdl-2.4.-.el7_2.x86_64 /
Verifying : unixODBC-2.3.-.el7.x86_64 /
Verifying : libicu-50.1.-.el7.x86_64 /
Verifying : clickhouse-server-1.1.-.el7.x86_64 /
Verifying : clickhouse-client-1.1.-.el7.x86_64 /
Verifying : clickhouse-compressor-1.1.-.el7.x86_64 /
Verifying : clickhouse-server-common-1.1.-.el7.x86_64 / Installed:
clickhouse-client.x86_64 :1.1.-.el7 clickhouse-compressor.x86_64 :1.1.-.el7 clickhouse-debuginfo.x86_64 :1.1.-.el7 clickhouse-server.x86_64 :1.1.-.el7
clickhouse-server-common.x86_64 :1.1.-.el7 libtool-ltdl.x86_64 :2.4.-.el7_2 unixODBC.x86_64 :2.3.-.el7 Dependency Installed:
libicu.x86_64 :50.1.-.el7 Complete!
[root@yinzhengjie clickhouse]#
[root@yinzhengjie clickhouse]# yum -y localinstall *.rpm #直接通过yum安装即可
三.配置clickhouse
1>.备份clickhouse-server配置文件config.xml
[root@yinzhengjie ~]# cp /etc/clickhouse-server/config.xml /etc/clickhouse-server/config.xml.`date +%F`
[root@yinzhengjie ~]#
2>.修改clickhouse-server配置文件config.xml（指定clickhouse-server的主机地址）
[root@yinzhengjie ~]# sed -i 's#<listen_host>127.0.0.1</listen_host>#<listen_host>10.1.2.105</listen_host>#' /etc/clickhouse-server/config.xml
[root@yinzhengjie ~]#
3>.修改clickhouse-server配置文件config.xml（修改默认的9000端口，我们自定义该服务的端口为9999）
[root@yinzhengjie ~]# sed -i 's#<tcp_port>9000</tcp_port>#<tcp_port>9999</tcp_port>#' /etc/clickhouse-server/config.xml
[root@yinzhengjie ~]#
4>.备份clickhouse-server配置文件users.xml
[root@yinzhengjie ~]# cp /etc/clickhouse-server/users.xml /etc/clickhouse-server/users.xml.`date +%F`
[root@yinzhengjie ~]#
5>.修改clickhouse-server配置文件users.xml（修改/etc/clickhouse-server/users.xml配置文件的第83行，改地址改为你当前部署的服务器IP地址，不推荐使用主机名！下图是我修改后的值）
四.clickhouse管理方式
1>.启动clickhouse
[root@yinzhengjie ~]# clickhouse-server --config-file=/etc/clickhouse-server/config.xml
Include not found: clickhouse_remote_servers
Include not found: clickhouse_compression
2018.10. ::20.997594 [ ] <Warning> Application: Logging to console
2018.10. ::21.000228 [ ] <Information> : Starting daemon with revision
2018.10. ::21.000337 [ ] <Information> Application: starting up
2018.10. ::21.001299 [ ] <Debug> Application: Set max number of file descriptors to (was ).
2018.10. ::21.001317 [ ] <Debug> Application: Initializing DateLUT.
2018.10. ::21.001326 [ ] <Trace> Application: Initialized DateLUT with time zone `Asia/Shanghai'.
2018.10. ::21.048483 [ ] <Debug> Application: Configuration parameter 'interserver_http_host' doesn't exist or exists and empty. Will use 'yinzhengjie' as replica host.
2018.10. ::21.048676 [ ] <Debug> ConfigReloader: Loading config `/etc/clickhouse-server/users.xml'
2018.10. ::21.053081 [ ] <Warning> ConfigProcessor: Include not found: networks
2018.10. ::21.053114 [ ] <Warning> ConfigProcessor: Include not found: networks
2018.10. ::21.054194 [ ] <Information> Application: Loading metadata.
2018.10. ::21.055243 [ ] <Information> DatabaseOrdinary (default): Total tables.
2018.10. ::21.055919 [ ] <Debug> Application: Loaded metadata.
2018.10. ::21.056089 [ ] <Information> DatabaseOrdinary (system): Total tables.
2018.10. ::21.057059 [ ] <Information> Application: Listening http://[::1]:8123
2018.10. ::21.057138 [ ] <Information> Application: Listening tcp: [::]:
2018.10. ::21.057198 [ ] <Information> Application: Listening interserver: [::]:
2018.10. ::21.057289 [ ] <Information> Application: Listening http://10.1.2.105:8123
2018.10. ::21.057420 [ ] <Information> Application: Listening tcp: 10.1.2.105:
2018.10. ::21.057486 [ ] <Information> Application: Listening interserver: 10.1.2.105:
2018.10. ::21.057937 [ ] <Information> Application: Ready for connections.
2018.10. ::23.048780 [ ] <Debug> ConfigReloader: Loading config `/etc/clickhouse-server/config.xml'
2018.10. ::23.053196 [ ] <Warning> ConfigProcessor: Include not found: clickhouse_remote_servers
2018.10. ::23.053332 [ ] <Warning> ConfigProcessor: Include not found: clickhouse_compression
[root@yinzhengjie ~]# clickhouse-server --config-file=/etc/clickhouse-server/config.xml
2>.查看clickhouse服务是否启动
[root@yinzhengjie ~]# netstat -untalp | grep
tcp 10.1.2.105: 0.0.0.0:* LISTEN /clickhouse-se
tcp6 ::: :::* LISTEN /clickhouse-se
[root@yinzhengjie ~]#
3>.使用客户端链接测试
[root@yinzhengjie ~]# clickhouse-client --host=10.1.2.105 --port=
ClickHouse client version 1.1..
Connecting to 10.1.2.105:.
Connected to ClickHouse server version 1.1.. :)
:) select now(); SELECT now() ┌───────────────now()─┐
│ -- :: │
└─────────────────────┘ rows in set. Elapsed: 0.067 sec. :)
:) show tables; SHOW TABLES Ok. rows in set. Elapsed: 0.003 sec. :)
:)
[root@yinzhengjie ~]# clickhouse-client --host=10.1.2.105 --port=9999
4>.关闭clickhouse服务
[root@yinzhengjie ~]# ps -aux|grep clickhouse-server | grep -v grep
root 0.1 0.3 pts/ Sl+ : : clickhouse-server --config-file=/etc/clickhouse-server/config.xml
[root@yinzhengjie ~]#
[root@yinzhengjie ~]# kill - `ps -aux|grep clickhouse-server | grep -v grep | awk '{print $2}'`
[root@yinzhengjie ~]#
You have new mail in /var/spool/mail/root
[root@yinzhengjie ~]#
[root@yinzhengjie ~]# ps -aux|grep clickhouse-server | grep -v grep
[root@yinzhengjie ~]#
[root@yinzhengjie ~]#
[root@yinzhengjie ~]# kill -9 `ps -aux|grep clickhouse-server | grep -v grep | awk '{print $2}'`
5>.为clickhouse-server添加守护进程
[root@yinzhengjie ~]# nohup clickhouse-server --config-file=/etc/clickhouse-server/config.xml >/var/log/yinzhengjie-clickhouse.log >& & #会在"/var/log"目录下生成一个名称为“yinzhengjie-clickhouse.log”的日志文件
[]
[root@yinzhengjie ~]#
[root@yinzhengjie ~]# tail -10f /var/log/yinzhengjie-clickhouse.log
2018.10. ::53.981429 [ ] <Information> Application: Listening http://[::1]:8123
2018.10. ::53.981479 [ ] <Information> Application: Listening tcp: [::]:
2018.10. ::53.981516 [ ] <Information> Application: Listening interserver: [::]:
2018.10. ::53.981554 [ ] <Information> Application: Listening http://10.1.2.105:8123
2018.10. ::53.981590 [ ] <Information> Application: Listening tcp: 10.1.2.105:
2018.10. ::53.981624 [ ] <Information> Application: Listening interserver: 10.1.2.105:
2018.10. ::53.981796 [ ] <Information> Application: Ready for connections.
2018.10. ::55.973241 [ ] <Debug> ConfigReloader: Loading config `/etc/clickhouse-server/config.xml'
2018.10. ::55.977391 [ ] <Warning> ConfigProcessor: Include not found: clickhouse_remote_servers
2018.10. ::55.977455 [ ] <Warning> ConfigProcessor: Include not found: clickhouse_compression
2018.10. ::09.185410 [ ] <Trace> TCPConnectionFactory: TCP Request. Address: 10.1.2.105:
2018.10. ::09.185660 [ ] <Debug> TCPHandler: Connected ClickHouse client version 1.1., user: default.
2018.10. ::18.439436 [ ] <Debug> executeQuery: (from 10.1.2.105:) select now()
2018.10. ::18.440437 [ ] <Trace> InterpreterSelectQuery: FetchColumns -> Complete
2018.10. ::18.440867 [ ] <Debug> executeQuery: Query pipeline:
Expression
Expression
One 2018.10. ::18.441657 [ ] <Information> executeQuery: Read rows, 1.00 B in 0.002 sec., rows/sec., 510.97 B/sec.
2018.10. ::18.441764 [ ] <Debug> MemoryTracker: Peak memory usage (for query): 1.00 MiB.
2018.10. ::18.441804 [ ] <Debug> MemoryTracker: Peak memory usage (for user): 1.00 MiB.
2018.10. ::18.441849 [ ] <Debug> MemoryTracker: Peak memory usage (total): 1.00 MiB.
2018.10. ::18.441882 [ ] <Information> TCPHandler: Processed in 0.003 sec.
好啦～分享就到这里了，时间过的咋就这么快呢？一不小心就晚上九点多了，关于ClickHouse集群的部署等我后期研究好了也会分享给大家的，我得赶紧走了，不然赶不上地铁啦！晚安，兄弟们！
Hadoop生态圈-CentOs7.5单机部署ClickHouse的更多相关文章
- Hadoop生态圈-zookeeper完全分布式部署
Hadoop生态圈-zookeeper完全分布式部署 作者:尹正杰 版权声明:原创作品,谢绝转载!否则将追究法律责任. 本篇博客部署是建立在Hadoop高可用基础之上的,关于Hadoop高可用部署请参 ...
- Hadoop生态圈-phoenix完全分布式部署以及常用命令介绍
Hadoop生态圈-phoenix完全分布式部署 作者:尹正杰 版权声明:原创作品,谢绝转载!否则将追究法律责任. phoenix只是一个插件,我们可以用hive给hbase套上一个JDBC壳,但是你 ...
- Hadoop2-认识Hadoop大数据处理架构-单机部署
一.Hadoop原理介绍 1.请参考原理篇:Hadoop1-认识Hadoop大数据处理架构 二.centos7单机部署hadoop 前期准备 1.创建用户 [root@web3 ~]# useradd ...
- CentOS7下单机部署RabbltMQ环境的操作记录
一.RabbitMQ简单介绍在日常工作环境中,你是否遇到过两个(多个)系统间需要通过定时任务来同步某些数据?你是否在为异构系统的不同进程间相互调用.通讯的问题而苦恼.挣扎?如果是,那么恭喜你,消息服务 ...
- Centos7下单机部署Solr7.3
本章重点介绍CentOS7 下部署Solr7 ,添加核心Core配置,Dataimport导入,中文分词的相关操作. 一.准备工作 演示环境是在虚拟机下安装的CentOS7.java JDK8 ...
- Hadoop安装和基本单机部署
下载安装 # 下载 $ cd /usr/local $ wget http://mirrors.hust.edu.cn/apache/hadoop/common/hadoop-2.9.2/hadoo ...
- Hadoop生态圈-离线方式部署Cloudera Manager5.15.1
Hadoop生态圈-离线方式部署Cloudera Manager5.15.1 作者:尹正杰 版权声明:原创作品,谢绝转载!否则将追究法律责任. 到目前位置,Cloudera Manager和CDH最新 ...
- Clickhouse单机部署以及从mysql增量同步数据
背景: 随着数据量的上升,OLAP一直是被讨论的话题,虽然druid,kylin能够解决OLAP问题,但是druid,kylin也是需要和hadoop全家桶一起用的,异常的笨重,再说我也搞不定,那只能 ...
- Hadoop系列之（一）：Hadoop单机部署
1. Hadoop介绍 Hadoop是一个能够对海量数据进行分布式处理的系统架构. Hadoop框架的核心是:HDFS和MapReduce. HDFS分布式文件系统为海量的数据提供了存储, MapRe ...
随机推荐
- 第一次Sprint
项目刚开始做的话,离客户的需求应该,蛮远的. 用的是eclipse加安卓模拟器在弄. 目前主要弄APP的界面和一些主要的功能算法,各个功能板块的位置划分的内容. Github团队地址是:https:/ ...
- Python模块笔记
__name__属性 一个模块被另一个程序第一次引入时,其主程序将运行.如果我们想在模块被引入时,模块中的某一程序块不执行,我们可以用__name__属性来使该程序块仅在该模块自身运行时执行. #!/ ...
- 关于python项目路径导入自己写的库出错的一点思考
其实也是在写自己项目的时候遇到的,以前也遇到了但是一直采取的是回避的策略,这次总算弄清楚所以总结一下. 这个项目的顶级目录是medivac,他本身是一个python模块. 熟悉flask的人都知道,在 ...
- Mac 电脑 开发环境Idea环境搭建
1.应该把F1-->F9的快捷键禁用,不然调试的时候,需要走断点. ==>用手抓一下键盘==>到了桌面,然后==>点击系统偏好设置==>键盘==>键盘==>将 ...
- Java微信二次开发（八）
高级接口,先做了两个(获取用户信息和获取关注者列表) 第一步:找到包com.wtz.vo,新建类UserInfo.java package com.wtz.vo; /** * @author wang ...
- gitlab搭建与配置说明
1. 概述 Gitlab分为社区版和企业版,此次安装的是社区版(gitlab-ce). 2. 准备 本次使用系统为Ubuntu16.04. 3. 安装 添加GitLab仓库,并安装到服务器上(将git ...
- 一个死循环导致的栈溢出实例：StackOverFlowError
有一个功能,要用复选框组做成单选框效果,如果有三个复选框 CheckBox ,并且保证每次只能选中一个.刚开始添加了以下的值改变后的监听方法 addValueChangeListener ,却导致了栈 ...
- day13 生成器 三元运算 列表解析
本质上来说生成器迭代器都是一种数据类型,如果你直接打印生成器是无法得出值的,会得到一串内存地址,即一个对象想要得到生成器的值必须要用for或者next,list等来获取 生成器生成器就是一个可迭代对象 ...
- MT【209】打破对称
设正数$a,b,c$满足$ab+bc+ca=47$,求$(a^2+5)(b^2+5)(c^2+5)$的最小值_____ 解:$(a^2+5)(b^2+5)(c^2+5)=(a^2+5)(5(b+c)^ ...
- BZOJ5312 冒险（势能线段树）
BZOJ题目传送门 表示蒟蒻并不能一眼看出来这是个势能线段树. 不过仔细想想也并非难以理解,感性理解一下,在一个区间里又与又或,那么本来不相同的位也会渐渐相同,线段树每个叶子节点最多修改\(\log ...