Kafka的安装和部署及测试
1、简介
大数据分析处理平台包括数据的接入,数据的存储,数据的处理,以及后面的展示或者应用。今天我们连说一下数据的接入,数据的接入目前比较普遍的是采用kafka将前面的数据通过消息的方式,以数据流的形式将数据接入到大数据存储平台。
下面主要介绍一下kafka的安装和部署的过程。由于公司采用的是cloudera公司的产品,所以直接通过cloudera manager来安装部署kafka。只不过在安装的过程中,注意几个问题即可:
(1)存放kafka的log文件的位置
默认是/var/log/kafka/
(2)存放kafka的data的位置
默认是/var/local/kafka/data
这些路径需要手动创建,并注意他们的权限的问题。
2、kafka组建简介,并用控制台的方式来模拟一下
(1)kafka主要有:producer、broker、topic、consumer。在安装好kafka之后,首先需要创建一个topic,实际上这个topic就是对应的将消息进行分类,也可以理解成message queue。创建topic的命令是:
kafka-topics --create --zookeeper hadoop3:2181 --replication-factor 1 --partitions 1 --topic hello
在这里hadoop3表示的是部署zookeeper的主机名(由于我部署的zookeeper只有一台机器,所以对应的就是hadoop3,这个根据具体的情况会不同),hello是对应的topic名称,--replication-factor参数指定对应的消息的副本数
(2)创建好了topic之后,接下来就是生产者往这个topic中进行生产:
kafka-console-producer --broker-list hadoop2:9092 --topic hello
在这里hadoop2表示的是部署broker的主机名,hello对应的就是topic名称。
(3)第三步就是消费对应的消息了:
kafka-console-consumer --zookeeper hadoop3:2181 --topic hello --from-beginning
在这里hadoop3表示的是对应部署zookeeper的主机名,hello表示的就是topic的名称。
(4)查看topic的一些信息:
创建多副本的topic:kafka-topics --create --zookeeper hadoop3:2181 --replication-factor 3 --partitions 1 --topic multi
查看多副本的分布情况:kafka-topics --describe --zookeeper hadoop3:2181 --topic multi
aaarticlea/png;base64,iVBORw0KGgoAAAANSUhEUgAAA88AAAAsCAIAAADU0R0lAAANYklEQVR4nO2d247dthWG+Tx6kKSFgAJxMvB4pkjr+qgEbZO0afoCehA9x7xC06BA644zGU8Qw8nEM3FRBDV8uOnF3triYXGRIsVNSvq/u70lkYtci0u/KEoSX50//8XRA6FSNd3JhrYWIIC6XVvX1e02ZLqmym3LcllfXIElg3iWQW+UALwAEvH3sx/evXk3txWLYqc7N9oztzlgISCuwJJAPMugN0oAXgAJ+fLxs3cObue2AgAAAAAAgCXyt9On73zwm9xWAAAAAAAAsES+PH36LtQ2AAAAAAAAKfjq62e/xEoSAAAAAAAAUvCPby7rwzu5rQAAAAAAAGCJ/PPi6lfH+hsA09F0S3tD3PJalI6YvkI/g9LIFZMYCyAFiCsAEvLouxfvffjR3qorbDwPL/yBCtwDUNvTsHkfPl4KmxCvzAC1XTi770bgbW4bhi9pUKGNuAIgIafP/vv+7T8IoQ/EVb11smo6OstUTbeg5hMenrB1MX2VrZ/l96u29d4+bBDc3q29XdOs/RMMVLqavkesmSEdOcaCLfdP06NZRnfVdJKcrNu5nMhSZqSq6VaeNQDIyNnl/w7ufCqEkhOHIb4suWljJWpbCP1DWVXTTXYSmp3arppOlhPb01zZansHPngmhKhbOXpTKKqVqG1BKrGpYqyILFq35c/bps5I6ngBAOyVsx9f3rz3mRCealu78lb/bWt5jkRNEu47s/L0StsYCXo3q9cfva2ra4akpO6z20G1gKifOqcqX5Wi2jTBKpQMGGfQoen8Xca+uW097NnvE9NX8f1Mx+Tg/aFdSgF1a5SnKw5XycpORtTR9TrbK+9ljyyobUGoB20c274M55GvtljUdkQ240ZZtrEgeLXNZwauvT7RTtssb6FyjraPMwfPQG1PkJFs/qVuXkh7uOJK3d7XVhGFtzWRmXx9BMCiOX/+6vD+n4XwUNvElbcxYIdTmrZ5KMN29tLLcqntnUknbd1v7tOybDN7Pmat8pqVyTD7FQOjtlVUZ0tHN8PqPuXYmL4KPpaPyarpuq5VdPBuX5dcdZdsjyuuXr/2Qm37wKptTajoXojKV/xWj2w2GGyMsgxjof9n84crtIxDXe1lWsTb3P9nzTnCV8nNYQ1FfEZic45wz22TcaUUVTXdSdcpe9WtmlXNyymobQCEEOL86tXh/c+1P8lxT1x5y/spg85ajG08Bw5EKYlarxCgtmU0nzAZnmr79kRKNjeH2nbEpHHOUSby2RrdJfNq21av5fDRQG0LXT0YYoRQG4MbIvIVv3VcQjAjYf9jYdhumX1mbXa3194ih839HzFCbTclHnj83ojPSHzOEWFq2/BHpT1LOYObBgAUwZOr17ce+KhtiwLf/UltNwev5TwRqhx85uOhtmX0W7tKx+u3Gym1bfVUBrXtiknG+44aI0p2bnXX7gHUttCD2S4eqXiPyFf8VqdnHKNs/2Oh387MbTM2uyPR2iKXzZ7le6Dc8CyTpBlpt/tYtU1ZRWQzyygEAAw8uXp968FftD+htm2HW3aZT4Kx97U+N2KZ216I2nYEHdT2HJDUgz6157NWJ4fado+y7Gp7pM2zUNtiBnOwKTPSbvcUalvbv/BuBiAXT65f33roo7apP6dbSRKYUEerbTrlQm17nvULU9uumGTPQKNPblzJalxBbe8HVT1o85eOcZlyJcmI1VlTriQJHwuc2S4L3dmbW0nC2mzbJ4ji1fa0GWlvK0l8CgEAjFHbRlpV05f+1BFdiGUo6qe+SnlZqlS++ZSk4z0qkpGV/iy1yyo9s9QtsaRxZsnFmtEVb+66yvdoEddXwcfyMek4A5FRJz8PZC+Zjysftc231/lcEdS2INSD4qRaf+ROcWlUvuK3MtnMY5TlGQuMaHba7MreTIsco2z7FxfplpGi3evQb30USkRGSqW2tYvYulWfkjTChrqocWYzANbAxfUbWW1r6/OIzGdb81u3J+obteTN+mph+nj7WkZBjFiprrYezp5Vv2HQ3nJr5G0eVqk7jWtRcWhrWTnnDi+T2jaJfIOUXkFMXwUfa4lJLTyM39TB9GtwaHttceVXr7W96lbj/FSRfljjSUzuCT0pWF63p/g3Il9FZTN2lBH77GUsxGQGR3vZFjE2i3E5x3bVY6u1WEIykjvnEKGjXf8YqMuy5L+bjjwL2zsaahsAIcTF9ZsjY247BEy2AQDmAvIVAEHM624uAKVwcf3m6OEXExSEsxcAYC4gXwEQgGs5CgCAZpK5beVWFIYiAKBgkK8AGMEs1+QAUBjkGwABAAAAAAAAE/Dk+atbxrckAQAAAAAAABNwfvny8N6fclsBAAAAAADAEjn7/uebv/sktxUAAAAAAAAskcdP/3Pw29/ntgIAAAAAAIAlcvrt9Qe/xlP5qXB85XZxrK29PDG9gZ4EIC8YgwCAyfj3+Y83ju7ntmKxzCpfD+95gkachIWp7eG7cSt7a17/tfIy3LFxg+U9bLyP5O+fJraycLxyXYFj0EGu2JhhvVLteKfhLD04v8h5dPbDjcN74wxdBtRHsHEeEkJUtm+FVU03q84hPDyh/TG9MbeeFJvO7G22Rsg+0b2b1qIimrz7AnZDf5qH9VHdSn20+ZHc4Aio9Dy9KirCrZOQKzZi6q2aLku9ciFd1+1NbSuvLU8W04FWIXI86pULCYicf519/97h3VGHLARJ8QzflZuhDJqcxahtIfRPBsoDKZZ1qW3904tNl/80IYTY2zchy5JldKNZHxmHNPtTGaGony1McYVQllsnIVdsBNRL7DzeG+H11u3JSdtkGAeFRh0ix7fe0Mh59M3leue23Wpbvhodenb7b1vLEzBqx7vvVMpzN21jyK/dFZhfDFX93e6dbVoJux1U+4jyqVxAXZUrLZ5gFUoSiHTQG6jOnulm9w1q62HPfp+Y3ojvSTomB//Kt8HMAkbG1VC6Ed9laDXOEO6bkbz31YO7ptIGBVcyFznaPqFjhWz0SB8V40E7xkfCtcxk84JHft5i0T0R2dsZV6pNhBdmGhsT1DudZvKodzvzUpWktpnYWHBWWU/knF5cvX/8cKSZi8Cptqum0yW25NFNZA8pXtssV0Jnc72sOLW9M/ikrfuD+4Ept4g9e7E2e83IFnfVzqhtFdXZ0tHNsHpTOTamN4KP5WOyarqu2+WdytakgIxsdGMxWs1qiJYM6d7o0TZqOzfdSdcN7vApmYsckea8OMpHhpAtEVZtO7wQlZ/5rR7ZezBYC426lf4Y0rRe/AxjI7reQN0bWG+vzwpS2+7YWGZWWU/kPP7uxcGHH482dFlYup2al1RGAzEWtGLIcZVKkkrut14/rFxt80PWaN32REo2KIfadsSkofsm88b81Db1P9F9PYpHiGOlB+a8SuYiJ57I81MOhRGCevI2LjRZL0TkZ37ruBRH3CxNnCBzxUZEvbup2oBqw+odoqMste2IjWVmlfVEztmzn2/e/mOIrQuCzty8K6jt5iCyaKY049tntn6FaltB6Xj9QSxKbVs9lUFtu2LSz78hzE5tUw/Zaf63et9y3aUuyeJKtps1ETHnxWo+b9pQB6/5mBTnhYj8zG91etaRVRTLEzgiV2xMMkO5n5lRuZkFqW13bCwzq6wncs4vXx7e/Wy0ocsCatsoZkFq29LX+jywRWNBbZMlF6u2t0129TDnfafa9sjsGc6LPj6qyfUUpSLNbeu3bZxeyKS2fbKKsnVyd+SKjeB61a2jLQioV5+CMa6VE+N1lqRiY5lZZT2Rc3H1+ujB5+OsXBx+FznqX3ErSZIMmtFqmw7RVahtsy0zUNuumEynton7aWWIbZva5tvu8D6/ksSrVzOcF10+4leul4i6kkSbwXJ4IeVKEm6SzSOr+BsQQprYqNSH7Keq19xb643k9RY1t+3abY5ZBZEzcPHT2+Pmi1GHLA/7zRGt35mnJJkrJcfzENuqtGipg56SdLxlRWrCLhhHqW1p/7olLuxmo7ZVb9pGpkNtB/dG8LF8TPqo7YC46g+UHxMsxstWtW3OuEid5/C+IeyUpyTZki1mmVbHLCSwlW73kXmztZwLJiu1/tCV4jbeC1H5md/KZG+PuFKMoOY7ioyNpnOZFFavdpeeusqdvF6FYtS2T2zMMasgcga+ffH2uPnrqEOWhLbGjohu2z2Duj1R3zAlbyZuOBDHKxXrlo0cOZIlbT2ca6p+w6C95bbK2zxsVnca194MVKp3OecOL87bGk0sC7X6KKg3go+1xKQWAMZv7fCQjCzHzuiDJ6ciXKQ3zYiA4XDW+9qhXVPp2+0lj4mckV6gmqwVYfMRGVe5ByiH1v/Kv5bX7en+Dc3PUdmbjSvTgfTkGuHYMf1l9psQIiY2NoXbbhKG1mvUbVSQrl5th/0MBaqjbRlFs2muWQWRI3Px4u3xR+tV2+HMYGoIAABWCfLzhEQuzJ1dvWAqEDkyFz+9PVrx3HY4yOYAAFAmyM/TQayBX3S9YCoQOQrnV28OH6593fZYlJsYJdxVBwAAIIRAfgYAFMjXz18d3F/7O0kAAAAAAABIwqPLlzfurP192wAAAAAAAKTg/yfTF3GnvD22AAAAAElFTkSuQmCC" alt="" width="931" height="42" />
(5)测试一下容错机制
aaarticlea/png;base64,iVBORw0KGgoAAAANSUhEUgAAA0IAAABTCAIAAAA9XiCAAAAUgUlEQVR4nO2dy+7lRhHG/TyWUAghySOw8IIFIoQNEgskzOUx/CB+BZQrJ5cnYMECiWSYQUJigQSZzBuwOL50dVdVX213n/P9VjN/293VXdXl73Tb7u5Pf/lX/5NfdXUzTLfbNFxtBQAAAABATXz813//YPj11VYAAAAAAIBIPvvbf97+6W+utgIAAAAAAERy+/t/3/nZb6+2AgAAAAAARPLli+/e++B3V1sBAAAAAAAi+erl9+9/+IerrQAAAAAAAJF8/erN+7/849VWAAAAAACASL7+Zxsyrh/n2515vNoWABbG+TaP/dVWPALoSQCAy+NlhvIt+oqTcbtmquNrbf04b6aY35Bb7Cwk7NbCTo2YcS7cy8O0yV2hIf04z2PXdeMc0dYiXb3HlWFgWSeeTLsp5pJoV3B7conkOlLQOTQ9Flyazs97roq333Ntr2X9/Vr2BPXah6SyHLvdYNOtKt+iL1+9eY+TcWd33KotOOjXf/txNqNYuzDNihMbPky3eZ4PGJRaQ4bpNo9dN0yx9WZ2teW4giXrlT7KTbE4FwzzWOKDtHkeLGJbzM/9OBs36WGKUnL304VrFw0wjyMf2OYNfpjozd537WU8WMSGIYbiJb3x5as371Yu4+xDNLW3mCYWhul2m0ZZ3GTgkXHTcIGMW+TjASUrPGWKCQQyrkYeLGIbzs8btqDynEsjls/vbGDrdzrfny/jwSI2jLpk3Bev3rz7YZiMW2cTp2FfHqOn7fON3LSvdNT8O3fCHrXbmY6Mk2Y62WU8wah57J2Gp7RoWwwyK2fH7G0etTmqDPwyLiHajEvM/l4OKf1MjzLnJJes4Ysr+5woD/qn1k27p5H2Ni33vLSsRjvpLzc2fDaL7fXkDbUn79UYlin+O3EtkgktKTxiMmHXdepY8LVXOhqSkbSS/Zlfptn8TMtxTNvmxnzdwI/uB5FxFedYjm2Vf39gg/rRegxgtYCpn7vDenujwIIszxcv37zrfHBEEQHDdJ9AWo7vJ5rPr3XW1LLvaKdq2DVq11rd2bh9XXLRRgLUCNuocb7Ns9HwnBbd/bVb4jZ4TQ1nyjg2zMJvftRH4xTaz3vtYbNx0SUr6LO8WR7cindHClcWLWqr917PCWlZj3b3UQU3Ack26+1d/sTnDaNGOWJJzcE2HwdJQXbOKZoJzbGgt9fTG2o8h/Sk14M8reZn0gJZIvo6QUp6okIjRvKlVyTjuq6rM8eq1i4ZZU9jJCyd5rD158zGlZ9U/uLlmx9HyjjhVxOnTVcH6ke3KnkT7UVAR8ZRe7QuIrUwo4E+mpDRInLXY6rbj8fLuOUniHqR4sHlkCKq1EKXf+gJjPVmiIxLKznEZodMDxrFB+Yds5zzl4b0aOfuDNYtxfPOjKc9Ut7wFOJ6YR8vXptPwbGwWCa0x4LeXl9vaPEc1JNeD/K0mZ+36uJ+6FoouV2SYmR6MvLaq6gxx6rWGqt7+8+LZ5NxwfO8appgJtW8Mu5+gedCy3Z7PU/2VkduVnkt4o7TmUvz3xGD0uyFe1uYseDz4CLjYnPBvj7ClC32M624bMns9GLYz4M8D5LibYv9XUuSdcSI1turHdWjnVv1ZnpSsDkklLzniDJO8kKIzao9iT1pGc28X1siE7KzdEp7vb2R3ZOJ4qHF/Oy0IEXJMbHB20L/6tHt2t9TyR0LdeZYCfrQAmSc989lZRxJN2QtQe50e3FA9VZ3WppgRkZKTA7yFxmk0HeJiKet/5mmBQxsXcbllKzbXHmK0dN9KbwyLrZXDZsvk3EXP1jNzlCWyITsWNDb6+2N7J48QsZVmp8doufPhV+krC07dpNjbrYXUn+OFayFjAv4u+4vaSnBJTJ/kUkv68K4SfuMFoVNFy8mHTAovR5MiaS19+zGefrZqNjn4sSSA2x2KeJBacI/yqXlRzWDJ9pjTTBtDmlvuoyzrjMqO6PbZKRGl8qEbtP19vpX8uV4DunJ8jKuify8HgsPtT7kGU1Jxtkeb1rGVZRjWWvDZBzv/Npk3PdFZJzTr7Tx+tGO+X1P1wMCH6foOpKSyIU9fQllq9O40H2ENrVFg/XwZni/lUHx4P1Ijozr7A8YevrZPOuQknWbpbgq4UGhGxk5Sl6GIgVF/9RPwxPtzvyBaafPZq292xmpMs5KBUY5qs2HYs2gEuPzMyE/Fnzt9R+V4zmgJw+Yjas0P1s6jJdlA/+Kg7sCy/eb0JtkEERMmVxKfTnWY60u40wjt6CMknGikvFdm8wtWMYxD1BYN9SBLNoxMlxfRDRPcH+BGx1OgsS8wF11Ncucx/UEa4juy4v28eQWDdONvkotDFr1cAZeGRerHUiSje1n2lFrZ5coOQQ5rpI9yDSIuZ401+5J6boj8US7ZVmczWJ79byh9OQu4YyTmIwolHwgbne4olXpK/moZyy4VbuzWdJRb0aSr/Vnfn8/tZafybXClNhilFKnc703chSbvNdeSGU5VsIKwFVG9lbgGafNY9+ZxwKsEnsj6No0/vzy+3diZuOuQsnkNXL1DybZg8P2wkgL/Xgd6CDwSCCeTdAbNQAvlEKScQ1Jphq5LkA3vVubEG8MpBjwSCCeTdAbNQAvlOLzf7z+0S9sGQdyIHOnF79MB1KAB8EjgXg2QW/UALxQks//8fptyDgAAAAAgOb4DDIOAAAAAKBFPnsBGQcAAAAA0CCfvnj99ge/v9oKAAAAAAAQyacvXv8QMg4AAAAAoDlOlnGn7B95Ko/XouPI6Sv0M6iNq2ISYwEcAeKqVZ5bxu1vPUNenABkXBnu33XEJ5cOJCgzQMZVjvk1/qttqQK6E4QdRIirVtllHLfXx5MMgJxtbhuC8XDB1uX01WX9bG2bctb3KJPbu20BND77lzO5dHXq3sRHccVYkHJ/mR69ZHTTLTbve8aebUMKR2ak07YcBidDZJxny9iH5UlkXNfZH852N3JOpzkZZ215be/CfnDVme3FB9C7ZQ9r83/Fb9VPIuM69hZfKsaqyKKxm0hfwdEZiY4X8DhEyjh+09ct3sxfdTT6/IsUZIfc0Rn5zlbE2zbSzhbayzlkn+ndAqZ+Lll79jgusiB7AU5q3puuT7ivzZ2G/cz1nJy+yu9nPiZ37+/tsrYvd8qzb2W+kslJTtTx9Xrba54lRxZkXMfclqxxLH0pPiBfLQgyLiObaaPssrHQ6TJOzwxae0OiXd5WXcs51jneHNyAjCuQkST/ctOtxhm+uKLH19p6pvBpYDJTqI9AKhEyjvmt4ETCniutw3sZUlq0y/LJuM2kdat3Y7ybNquJXrUq6HfkBb/Xc1BkHIU627h63J+gINfm9FXytXpM9uM8zxMRWNu5Ph3kL1mOK63esPZCxoWgyjjrDmh7IStf6UcDstlusDPKLhgL61/uf/CFlnOpr71Ki3Sb17+JOacLlQgtLCfmZyQ153T+2Tg2rkhR/Tjf5pmcNUw0q7o6HTLucNhXHNiAYn4rmOcRb4rFSIGS6GFjdIrSEzLOxPKJkjq4ti8Zmm3uFTLOE5NOMiNTj2qN/pJ1GSfVK1weDWRcZ9+WnLsccxvb3ZCRr/SjcQnBjYTzx8J+XJgvU232t1dukcfm9Q85CmCbxEu8/jTyM5Kec7o0Gef4o7dehGhgmvPxCZZxgrTb/sgdd6NCSECpt6SQGUTIOBN7lYN0vD3zzsk40VMXyDhfTCre99SYUbL3qL/2ACDjOjuYZVXCxXtGvtKPej3jGWXnj4X1uDIbp9jsj0SxRT6bA8sPgCzR1MmhGWk7PVbGcVYx2UwYheAkIOMkq9jLhVPaiVy5r+1fc8Js3IPIOE/QQca1gHFbsicjQpatr5Bx/lF2uYyLtLkJGdc1MGt0ZEbaTj9CxlnnV97ND0nMoiq35l1oUTVxpEbLOH4sQ8YF3k4qk3G+mFRTW3TW1EqmcQUZdw70tmTNuHjG5ZGLqhEPKpRcVE0fC5rZPgv92VtbVFVtls5JonoZVzYjnbaoGlIIOJhwGeeMVzou7EeG+UIEH9s5tScf/THKd19x8LxdaxjZ22/Y+KyyQ3aYmMdGGotaMVUQb25dFXp1l9dXydfqMelJbWzUmQ/zyiXrcRUi4/T2eh8KhozrmNsScdJgPy9PXJqVr/SjSjYLGGXXjAVFjXlt9mVvpUWeUbb8SYt0YaRYs7P2ZG2lZGSko2Sc9etomOgrDk7YcGrZm81AJpaMs56BYIaU9FzVMN3oC/zmYfuJLP56+XmRjgkFo65p2NNyvx7YRZ3ZGvNYgFX0pLgWVYf1vJDm3P3d9aVJ7AvrdgU5fZV8rRCTVng4/+cu5l+O5u2V4iqsXrG99KiT+HrWD8+YHc2esJOC8HUP4t+MfJWVzdRRxpxzyljIyQye9qotUmzu4nKOJKelWqslJSP5cw4TOpawdqBPKJh/Hmf2Lix3NGTc0RTbjAvTAwCAVkC+AiCJttafngLIOADA04F8BUACvpVZcAFFZByZlYWPAQAVg3wFQARNLk8/E8Vm4wAAAAAAwJlAxgEAAAAANMmnL16/BRkHAAAAANAckHEAAAAAAE0CGQcAAAAA0CSQcYfybDvMPVt7dXJ6Az0JwLVgDII2gIw7lKYSwf5aOcRHER5Mxu1fbH+yj3SIu/hdwt0NwmcfdB+ZO48cbGXlBOW6Csegh6tio8F6jdrb/4TKJ08r47h9jZDgurwN7KuC8XBB+3N6o7We7OjWilXsI2x791iLqmjytqnRyH+7WPMR3ej0vt/s8Ranw6Xn8rfbKtxahJzYoNsPx8XGVTGZ1V6jkHmW9hIuD7frWZm6n1rGrW7ev+je4P21OA8j47rO/lg/3eY5j+eScfamB+NcyU/Yk3ZjqOt+zzc6ykfcFua1QTcMOEJ61uXWIiTEhnOJvd/9QfUyJ8d7I73eYbrdpjGlrZmUjzrIuE6TcfyWzstf6dba0u7WksPIrtyjc1+P3E64Xxd+NtusErYTqH1M+VyQsbsn83tj15UWmQy1Gkh/70vbgU/DfuZ6Tk5v5PckH5O7f821BHG/8Egv9XYarGYvK80QbbcG3fv04nnsrUGhlaxFjnVO6lhhGx3no/ZkXOfs5il5ISA/Lwg31Izs7Y0rahPjo+tjI2l0XxWTqfUuP+n7mmScEht6VoGM6yQZ14+zrd2Mzr936547rMNmJXyasMvKk3GbwbdpWC9ePW62SE2Lqs1Bc0jV/bpVZByFOtu4etyfkCHX5vRG8rV6TPbjPM9bKuylJiXcJJxurF/GWfmZ740V66B18jjf5nl3R0jJWuR0x9yqY3x0xc0rHlXGebyQlZ/1owHZezfYCo1hMv6wp2m7+CtjI3HX1KtiMrHeVTJWJOP8sSFmleeVcQZCJHAzKaSbmU62imEddpTWMSJSFKZPLuP0LOK0bsnQbIOukHGemHQERTFvtCfjuL8z3bdCPMJcazztHlSyFjn5ZNwytx/yx1hWFKonnF8wqhcy8rN+NC7FMcs7ByfITDmVLGuuismkevfoqEvGeWJDzCqQcZ2UEvTo4I673hFuxscETsj84hPKOIL1wAQ9yMk40VMXyDhfTIb5N4XmZJz0ApNxruh9QdDTpxO0kmWzClFk5qN+JUcHr/t0vOaFjPysH/V61pNViOUHJMuc2Ogz3s29KiYT6jWbWZGM88eG2HuQcR1kHFPMA8k4oa/tmSvh5g0Zx5ZcrYxbmuzrYc37XhkXcLO5QMZF+uj4eaFsjNk4e6LZ64WLZFxIViFHizshOTb4Zefj66VHoy1IqNf+be/8CDuYoLskFxuQcRphgr7kouohYRMt4/hR8xQyzm1LAzLOF5PHybiIxemTEWSc3naP9/VF1aBevUDGRfqoLRnXObM1Hi8cuaga8WCGb6SXT5hJsaE/ObqeoK57HhOTh9db1Wyc7zTIOA15htkKBeUVB+VXgedhxqUqK4CHpFccPO/eGk3YxkeUjDPOHybmR0wzMo56U0oWHhmX3BvJ1+oxGSLjEuJqvdB8xr8aL4syzp1doM+Oat53FAN5xUEtWTDLtTpnTU0qXfaRdZ/23rarYLCftSdu072QlZ/1o0r2DogrYgSnpS+JDd+c2Tj7TDomJo+ol1CNjAuJDcg4Hs8nROnsq93J9IV28zAza8tcLz8/0UUPZsOSadiTWL8e2EWd2VbzWIDN9KS49l6A9fyM5tz9Ox2L0cyjN6KPknoj+VohJq0AcP5vXZ5ykzBjJ/ri4khPR1m6VvKg6n3r0nns7eNyyTGRE+kFrslWEZqPSORcPTh9WP1P/ip83cP2b2p+zsrealy5DuQnks6NDba97mSduKyRWq9Tt1PBcfVaJ5wzHLiOljKKZZMnqzy5jEunnmUlAAAAJsjPBcl8bK65epsDMi4RpAkAAKgT5OdyMM8ZPnS97QEZlwCZHK1hgQkAAEDXdcjP4NmAjAMAAAAAaBLIOAAAAACAJoGMAwAAAABoEsg4AAAAAIAmgYwDAAAAAGgSyDgAAAAAgCYpLOOUzy6b3ynmXgLP+kZ8Rr1G7fhGDQAAAADaoZiM27YQGfnvLpobt9l7itEtRe+fbj6jXrOQeT5/bzUAAAAAgHTKL6ryn892d7BWNBO3YfCB9Q7T7TaNF2yRCwAAAACQDiPjMjbPXi535VBvyyd1r5RyMi6g3mW6roeMAwAAAEBTnCTjnD8qMi5RUCXWu0pGyDgAAAAAtMVJi6phMm55FSFtF7ykevfNdyHjAAAAANAWVcm4O+ZLCcfWe6+p3/4NGQcAAACAdvjk21NkXD/OVJmpz8YlPRyXUO+yfGwCJQcAAACARvjk2++ueFNVV3GMjFu+/abM0mXWi9k4AAAAALTFx66MO+IVh67rhmkr0fp+Wz/ON2up0ylinH0mxddLgIwDAAAAQFsUk3H7Vgk7VhHaPg1kfdORU8omDZn1WidAyQEAAACgFRgZVyH3jR3SZgcBAAAAAB6Sj7797q2f1y7j9s+CAAAAAACAOx9904CMAwAAAAAANh998z/IOAAAAACA5vg/Ooh7DC0XENsAAAAASUVORK5CYII=" alt="" />
虽然最初负责续写消息的leader down掉了,但之前的消息还是可以消费的:
> bin/kafka-console-consumer.sh --zookeeper hadoop3:2181 --from-beginning --topic mutil
...
my test message 1
my test message 2
Kafka的安装和部署及测试的更多相关文章
- 消息中间件kafka+zookeeper集群部署、测试与应用
业务系统中,通常会遇到这些场景:A系统向B系统主动推送一个处理请求:A系统向B系统发送一个业务处理请求,因为某些原因(断电.宕机..),B业务系统挂机了,A系统发起的请求处理失败:前端应用并发量过大, ...
- cobbler安装、部署、测试
cobbler功能介绍 官网:http://cobbler.github.io 安装 yum install -y httpd tftp dhcp cobbler cobbler-web pykick ...
- kafka安装和部署
阅读目录 一.环境配置 二.操作过程 Kafka介绍 安装及部署 回到顶部 一.环境配置 操作系统:Cent OS 7 Kafka版本:0.9.0.0 Kafka官网下载:请点击 JDK版本:1.7. ...
- Kafka安装及部署
安装及部署 一.环境配置 操作系统:Cent OS 7 Kafka版本:0.9.0.0 Kafka官网下载:请点击 JDK版本:1.7.0_51 SSH Secure Shell版本:XShell 5 ...
- kafka 的安装部署
Kafka 的简介: Kafka 是一款分布式消息发布和订阅系统,具有高性能.高吞吐量的特点而被广泛应用与大数据传输场景.它是由 LinkedIn 公司开发,使用 Scala 语言编写,之后成为 Ap ...
- Kafka Manager安装部署及使用
为了简化开发者和服务工程师维护Kafka集群的工作,yahoo构建了一个叫做Kafka管理器的基于Web工具,叫做 Kafka Manager.本文对其进行部署配置,并安装配置kafkatool对k ...
- LDAP-openldap服务部署和测试(YUM安装)
1. 概述2. 服务端部署过程2.1 软件包说明2.2 部署过程2.3 配置过程3. 测试4. 生成LDIF格式文件4.1 安装migrationtools工具4.2 用migrationtools生 ...
- 基于Ubuntu Server 16.04 LTS版本安装和部署Django之(五):测试项目
基于Ubuntu Server 16.04 LTS版本安装和部署Django之(一):安装Python3-pip和Django 基于Ubuntu Server 16.04 LTS版本安装和部署Djan ...
- 基于Ubuntu Server 16.04 LTS版本安装和部署Django之(三):设置上传文件夹权限(这里测试用完全共享)
基于Ubuntu Server 16.04 LTS版本安装和部署Django之(一):安装Python3-pip和Django 基于Ubuntu Server 16.04 LTS版本安装和部署Djan ...
随机推荐
- android 画虚线、实线,画圆角矩形,一半圆角
1.画虚线,实线: 建立dotted_line_gray.xml文件放在drawable文件夹下面. android:shape="line" 可以修改你想要的形状 <?xm ...
- python学习--字符串
python的字符串类型为str 定义字符串可以用 ‘abc' , "abc", '''abc''' 查看str的帮助 在python提示符里 help(str) python基于 ...
- SSIS excel2003文件导入列名显示为F1,F2 - FN
问题现象: 数据源2003版本的EXCEL文件,第一列是空白,数据是从第二列开始的.所以SSIS中用EXCEL数据源倒入这个EXCEL时,所有的列显示不出来,显示的是F1,F2 - FN. 解决方法 ...
- shell下>和>>的区别
> :如果文件不存在,同上,如果文件存在,先将文件清空,然后将数据填入此文件 >> :如果文件不存在,将创建新的文件,并将数据送至此文件:如果文件存在,则将数据添加在文件后面
- Swift函数的定义
//: Playground - noun: a place where people can play import Cocoa //基本的函数 //************************ ...
- jQuery 日常工作集中问题
1.siblings()方法:获得匹配元素中每个元素的同胞(除了自己外的其他同级元素) 导航栏选择效果 点击“华东地区”,其他同级元素灰色背景以及下拉菜单消失,当前元素有灰色背景以及下拉菜单.再次点击 ...
- Java注释@interface的用法【转】
Java用 @interface Annotation{ } 定义一个注解 @Annotation,一个注解是一个类.@Override,@Deprecated,@SuppressWarnings为 ...
- php 使用curl模拟登录discuz以及模拟发帖
<?php$discuz_url = 'http://127.0.0.1/discuz/';//论坛地址$login_url = $discuz_url .'logging.php?action ...
- win7远程连接 您的凭据不工作
1.查看远程连接有没有打开:计算机->属性->远程设置. 2.如果上面开启了还是不行,找到开始--- 运行-- 输入 gpedit.msc,打开注册表编辑器:然后依次找到菜单,计算机配置- ...
- 数组中的toString,toLocalString,valueOf方法有什么区别
1. 2.简单来说,tostring就是用字符串来代替对象.tolocalstring就是根据不同的语言环境吧对象转成字符串,实际上totolocalstring是有缺省参数的,如tolocalstr ...