通过phoenix创建hbase表失败,创建语句卡住,hbase-hmaster报错:exception=org.apache.hadoop.hbase.TableExistsException: SYNC_BUSINESS_INFO_BYDAY_EFFECT

问题描述:

　　前几天一个同事来说,通过phoenix创建表失败了,一直报表存在的错误,删除也报错,然后就针对这个问题找下解决方案.

问题分析:

1.通过phoenix创建表,一直卡住不动了.创建语句如下:

CREATE TABLE SYNC_BUSINESS_INFO_BYDAY_EFFECT(
ID VARCHAR,
DAY_ID VARCHAR,
PROV_ID VARCHAR,
PROV_NAME VARCHAR,
AREA_ID VARCHAR,
AREA_NAME VARCHAR,
CITY_CODE VARCHAR,
CITY_NAME VARCHAR,
STORE_ID VARCHAR,
STORE_NAME VARCHAR,
STORE_TYPE VARCHAR,
CHANNEL_ID VARCHAR,
CHANNEL_NAME VARCHAR,
CHANNEL_TYPE VARCHAR,
CHANNEL_TYPE_NAME VARCHAR,
STAFF_ID VARCHAR,
STAFF_NAME VARCHAR,
NET_TYPE VARCHAR,
NET_NAME VARCHAR,
TRADE_NUM VARCHAR,
SYNC_FILE_NAME VARCHAR,
SYNC_DATE VARCHAR CONSTRAINT pk PRIMARY KEY (ID)
);

备注:在phoenix中,一直卡住不动了,没有任何的报错.另外,这个表示以前创建过的,但是可能是创建没有成功,客户端就断掉了.

2.从两个路出发,首先看hbase中是否有这个表

hbase shell

备注:通过这个命令进入到hbase中,list命令,发现并没有这个表.

3.进入到zkCli.sh即zookeeper客户端中

ls /hbase/table
ls /hbase/table-lock

备注:发现是有这个表的记录的,所以通过以下命令进行删除:

rmr /hbase/table/SYNC_BUSINESS_INFO_BYDAY_EFFECT
rmr /hbase/table-lock/SYNC_BUSINESS_INFO_BYDAY_EFFECT

4.然后重启hbase,以为为解决这个问题,但是重新创建表还是卡住了.

5.查看hbase hmaster的日志,报下面的错误:

-- ::, INFO  [B.defaultRpcServer.handler=,queue=,port=] master.HMaster: Client=aiprd//10.191.5.227 create 'SYNC_BUSINESS_INFO_BYDAY_EFFECT', {TABLE_ATTRIBUTES => {coprocessor$1 => '|org.apache.phoenix.coprocessor.ScanRegionObserver|805306366|', coprocessor$2 => '|org.apache.phoenix.coprocessor.UngroupedAggregateRegionObserver|805306366|', coprocessor$3 => '|org.apache.phoenix.coprocessor.GroupedAggregateRegionObserver|805306366|', coprocessor$4 => '|org.apache.phoenix.coprocessor.ServerCachingEndpointImpl|805306366|', coprocessor$5 => '|org.apache.phoenix.hbase.index.Indexer|805306366|org.apache.hadoop.hbase.index.codec.class=org.apache.phoenix.index.PhoenixIndexCodec,index.builder=org.apache.phoenix.index.PhoenixIndexBuilder'}, {NAME => '0', BLOOMFILTER => 'ROW', VERSIONS => '1', IN_MEMORY => 'false', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'FAST_DIFF', TTL => 'FOREVER', COMPRESSION => 'NONE', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}
-- ::, INFO  [ProcedureExecutor-] procedure2.ProcedureExecutor: Rolledback procedure CreateTableProcedure (table=SYNC_BUSINESS_INFO_BYDAY_EFFECT) id= owner=aiprd state=ROLLEDBACK exec-time=251msec exception=org.apache.hadoop.hbase.TableExistsException: SYNC_BUSINESS_INFO_BYDAY_EFFECT

备注:一直报这个错误.

6.所以,就在网上找了一些方法

--1)比如,删/hbase目录重启hbase,zookpper,没有解决问题

--2)比如,关闭hbase,然后通过hbase clean --cleanZk删除hbase在zookeeper中的数据,没有解决问题

--3)比如,重启zookeeper,hbase,删除hbase在zookeeper中的数据,没有解决问题

备注:以上的方法,无论使用哪个,hmaster后台也一直报表存在的错误.

7.最后,没有办法继续找解决方法,有提到hbck命令,所以呢,就只能做尝试(这个命令的作用是检查一致性的),执行以下的命令:

[aiprd@host-10-191-5-225 hbase-1.2.6]$ bin/hbase hbck
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option PermSize=128m; support was removed in 8.0
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=128m; support was removed in 8.0
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/mnt/aiprd/app/hbase-1.2.6/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/common/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
2018-05-03 21:42:56,585 WARN  [main] util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
HBaseFsck command line options:
2018-05-03 21:42:57,137 INFO  [main] util.HBaseFsck: Launching hbck
2018-05-03 21:42:57,228 INFO  [main] zookeeper.RecoverableZooKeeper: Process identifier=hconnection-0x12299890 connecting to ZooKeeper ensemble=10.191.5.225:2181,10.191.5.226:2181,10.191.5.227:2181
2018-05-03 21:42:57,243 INFO  [main] zookeeper.ZooKeeper: Client environment:zookeeper.version=3.4.6-1569965, built on 02/20/2014 09:09 GMT
2018-05-03 21:42:57,243 INFO  [main] zookeeper.ZooKeeper: Client environment:host.name=host-10-191-5-225
2018-05-03 21:42:57,243 INFO  [main] zookeeper.ZooKeeper: Client environment:java.version=1.8.0_121
2018-05-03 21:42:57,243 INFO  [main] zookeeper.ZooKeeper: Client environment:java.vendor=Oracle Corporation
2018-05-03 21:42:57,243 INFO  [main] zookeeper.ZooKeeper: Client environment:java.home=/mnt/aiprd/app/java/jre
2018-05-03 21:42:57,243 INFO  [main] zookeeper.ZooKeeper: Client environment:java.class.path=/mnt/aiprd/app/hbase-1.2.6/bin/../conf:/mnt/aiprd/app/java/lib/tools.jar:/mnt/aiprd/app/hbase-1.2.6/bin/..:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/activation-1.1.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/aopalliance-1.0.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/apacheds-i18n-2.0.0-M15.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/apacheds-kerberos-codec-2.0.0-M15.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/api-asn1-api-1.0.0-M20.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/api-util-1.0.0-M20.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/asm-3.1.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/avro-1.7.4.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/commons-beanutils-1.7.0.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/commons-beanutils-core-1.8.0.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/commons-cli-1.2.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/commons-codec-1.9.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/commons-collections-3.2.2.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/commons-compress-1.4.1.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/commons-configuration-1.6.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/commons-daemon-1.0.13.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/commons-digester-1.8.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/commons-el-1.0.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/commons-httpclient-3.1.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/commons-io-2.4.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/commons-lang-2.6.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/commons-logging-1.2.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/commons-math-2.2.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/commons-math3-3.1.1.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/commons-net-3.1.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/disruptor-3.3.0.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/findbugs-annotations-1.3.9-1.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/guava-12.0.1.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/guice-3.0.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/guice-servlet-3.0.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/hadoop-annotations-2.5.1.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/hadoop-auth-2.5.1.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/hadoop-client-2.5.1.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/hadoop-common-2.5.1.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/hadoop-hdfs-2.5.1.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/hadoop-mapreduce-client-app-2.5.1.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/hadoop-mapreduce-client-common-2.5.1.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/hadoop-mapreduce-client-core-2.5.1.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/hadoop-mapreduce-client-jobclient-2.5.1.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/hadoop-mapreduce-client-shuffle-2.5.1.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/hadoop-yarn-api-2.5.1.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/hadoop-yarn-client-2.5.1.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/hadoop-yarn-common-2.5.1.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/hadoop-yarn-server-common-2.5.1.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/hbase-annotations-1.2.6.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/hbase-annotations-1.2.6-tests.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/hbase-client-1.2.6.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/hbase-common-1.2.6.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/hbase-common-1.2.6-tests.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/hbase-examples-1.2.6.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/hbase-external-blockcache-1.2.6.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/hbase-hadoop2-compat-1.2.6.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/hbase-hadoop-compat-1.2.6.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/hbase-it-1.2.6.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/hbase-it-1.2.6-tests.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/hbase-prefix-tree-1.2.6.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/hbase-procedure-1.2.6.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/hbase-protocol-1.2.6.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/hbase-resource-bundle-1.2.6.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/hbase-rest-1.2.6.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/hbase-server-1.2.6.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/hbase-server-1.2.6-tests.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/hbase-shell-1.2.6.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/hbase-thrift-1.2.6.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/htrace-core-3.1.0-incubating.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/httpclient-4.2.5.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/httpcore-4.4.1.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/jackson-core-asl-1.9.13.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/jackson-jaxrs-1.9.13.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/jackson-mapper-asl-1.9.13.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/jackson-xc-1.9.13.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/jamon-runtime-2.4.1.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/jasper-compiler-5.5.23.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/jasper-runtime-5.5.23.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/javax.inject-1.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/java-xmlbuilder-0.4.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/jaxb-api-2.2.2.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/jaxb-impl-2.2.3-1.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/jcodings-1.0.8.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/jersey-client-1.9.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/jersey-core-1.9.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/jersey-guice-1.9.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/jersey-json-1.9.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/jersey-server-1.9.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/jets3t-0.9.0.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/jettison-1.3.3.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/jetty-6.1.26.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/jetty-sslengine-6.1.26.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/jetty-util-6.1.26.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/joni-2.1.2.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/jruby-complete-1.6.8.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/jsch-0.1.42.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/jsp-2.1-6.1.14.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/jsp-api-2.1-6.1.14.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/junit-4.12.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/leveldbjni-all-1.8.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/libthrift-0.9.3.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/log4j-1.2.17.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/metrics-core-2.2.0.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/netty-all-4.0.23.Final.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/paranamer-2.3.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/phoenix-4.10.0-HBase-1.2-server.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/protobuf-java-2.5.0.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/servlet-api-2.5-6.1.14.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/servlet-api-2.5.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/slf4j-api-1.7.7.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/slf4j-log4j12-1.7.5.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/snappy-java-1.0.4.1.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/spymemcached-2.11.6.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/xmlenc-0.52.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/xz-1.0.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/zookeeper-3.4.6.jar:/mnt/aiprd/app/hadoop-2.6.0/etc/hadoop:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/common/lib/slf4j-log4j12-1.7.5.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/common/lib/jetty-6.1.26.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/common/lib/java-xmlbuilder-0.4.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/common/lib/hamcrest-core-1.3.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/common/lib/api-asn1-api-1.0.0-M20.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/common/lib/gson-2.2.4.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/common/lib/jaxb-impl-2.2.3-1.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/common/lib/apacheds-i18n-2.0.0-M15.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/common/lib/commons-logging-1.1.3.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/common/lib/htrace-core-3.0.4.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/common/lib/commons-math3-3.1.1.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/common/lib/commons-codec-1.4.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/common/lib/jsr305-1.3.9.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/common/lib/commons-configuration-1.6.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/common/lib/paranamer-2.3.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/common/lib/asm-3.2.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/common/lib/avro-1.7.4.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/common/lib/jersey-core-1.9.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/common/lib/commons-el-1.0.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/common/lib/jets3t-0.9.0.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/common/lib/jetty-util-6.1.26.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/common/lib/curator-recipes-2.6.0.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/common/lib/curator-framework-2.6.0.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/common/lib/commons-lang-2.6.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/common/lib/xz-1.0.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/common/lib/commons-beanutils-1.7.0.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/common/lib/commons-collections-3.2.1.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/common/lib/hadoop-annotations-2.6.0.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/common/lib/jersey-json-1.9.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/common/lib/curator-client-2.6.0.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/common/lib/jersey-server-1.9.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/common/lib/jsp-api-2.1.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/common/lib/httpcore-4.2.5.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/common/lib/jackson-xc-1.9.13.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/common/lib/log4j-1.2.17.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/common/lib/commons-io-2.4.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/common/lib/httpclient-4.2.5.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/common/lib/commons-cli-1.2.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/common/lib/guava-11.0.2.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/common/lib/jettison-1.1.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/common/lib/zookeeper-3.4.6.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/common/lib/jackson-core-asl-1.9.13.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/common/lib/activation-1.1.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/common/lib/commons-httpclient-3.1.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/common/lib/protobuf-java-2.5.0.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/common/lib/jackson-mapper-asl-1.9.13.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/common/lib/slf4j-api-1.7.5.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/common/lib/xmlenc-0.52.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/common/lib/commons-digester-1.8.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/common/lib/jsch-0.1.42.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/common/lib/jackson-jaxrs-1.9.13.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/common/lib/jasper-compiler-5.5.23.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/common/lib/commons-compress-1.4.1.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/common/lib/api-util-1.0.0-M20.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/common/lib/netty-3.6.2.Final.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/common/lib/commons-net-3.1.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/common/lib/jasper-runtime-5.5.23.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/common/lib/snappy-java-1.0.4.1.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/common/lib/servlet-api-2.5.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/common/lib/commons-beanutils-core-1.8.0.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/common/lib/apacheds-kerberos-codec-2.0.0-M15.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/common/lib/mockito-all-1.8.5.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/common/lib/junit-4.11.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/common/lib/stax-api-1.0-2.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/common/lib/hadoop-auth-2.6.0.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/common/lib/jaxb-api-2.2.2.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/common/hadoop-common-2.6.0-tests.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/common/hadoop-common-2.6.0.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/common/hadoop-nfs-2.6.0.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/hdfs:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/hdfs/lib/jetty-6.1.26.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/hdfs/lib/commons-daemon-1.0.13.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/hdfs/lib/xml-apis-1.3.04.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/hdfs/lib/commons-logging-1.1.3.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/hdfs/lib/htrace-core-3.0.4.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/hdfs/lib/xercesImpl-2.9.1.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/hdfs/lib/commons-codec-1.4.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/hdfs/lib/jsr305-1.3.9.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/hdfs/lib/asm-3.2.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/hdfs/lib/jersey-core-1.9.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/hdfs/lib/commons-el-1.0.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/hdfs/lib/jetty-util-6.1.26.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/hdfs/lib/commons-lang-2.6.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/hdfs/lib/jersey-server-1.9.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/hdfs/lib/jsp-api-2.1.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/hdfs/lib/log4j-1.2.17.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/hdfs/lib/commons-io-2.4.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/hdfs/lib/commons-cli-1.2.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/hdfs/lib/guava-11.0.2.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/hdfs/lib/jackson-core-asl-1.9.13.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/hdfs/lib/protobuf-java-2.5.0.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/hdfs/lib/jackson-mapper-asl-1.9.13.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/hdfs/lib/xmlenc-0.52.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/hdfs/lib/netty-3.6.2.Final.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/hdfs/lib/jasper-runtime-5.5.23.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/hdfs/lib/servlet-api-2.5.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/hdfs/hadoop-hdfs-2.6.0-tests.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/hdfs/hadoop-hdfs-nfs-2.6.0.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/hdfs/hadoop-hdfs-2.6.0.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/yarn/lib/jetty-6.1.26.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/yarn/lib/jaxb-impl-2.2.3-1.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/yarn/lib/commons-logging-1.1.3.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/yarn/lib/commons-codec-1.4.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/yarn/lib/jsr305-1.3.9.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/yarn/lib/jline-0.9.94.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/yarn/lib/asm-3.2.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/yarn/lib/javax.inject-1.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/yarn/lib/jersey-core-1.9.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/yarn/lib/jetty-util-6.1.26.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/yarn/lib/guice-3.0.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/yarn/lib/commons-lang-2.6.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/yarn/lib/jersey-guice-1.9.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/yarn/lib/xz-1.0.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/yarn/lib/commons-collections-3.2.1.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/yarn/lib/leveldbjni-all-1.8.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/yarn/lib/jersey-json-1.9.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/yarn/lib/jersey-server-1.9.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/yarn/lib/jackson-xc-1.9.13.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/yarn/lib/log4j-1.2.17.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/yarn/lib/commons-io-2.4.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/yarn/lib/commons-cli-1.2.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/yarn/lib/guava-11.0.2.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/yarn/lib/jettison-1.1.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/yarn/lib/zookeeper-3.4.6.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/yarn/lib/jackson-core-asl-1.9.13.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/yarn/lib/activation-1.1.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/yarn/lib/commons-httpclient-3.1.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/yarn/lib/protobuf-java-2.5.0.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/yarn/lib/jackson-mapper-asl-1.9.13.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/yarn/lib/jersey-client-1.9.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/yarn/lib/jackson-jaxrs-1.9.13.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/yarn/lib/aopalliance-1.0.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/yarn/lib/commons-compress-1.4.1.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/yarn/lib/netty-3.6.2.Final.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/yarn/lib/servlet-api-2.5.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/yarn/lib/guice-servlet-3.0.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/yarn/lib/stax-api-1.0-2.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/yarn/lib/jaxb-api-2.2.2.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/yarn/hadoop-yarn-api-2.6.0.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/yarn/hadoop-yarn-applications-distributedshell-2.6.0.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/yarn/hadoop-yarn-registry-2.6.0.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/yarn/hadoop-yarn-server-common-2.6.0.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/yarn/hadoop-yarn-server-resourcemanager-2.6.0.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/yarn/hadoop-yarn-server-web-proxy-2.6.0.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/yarn/hadoop-yarn-common-2.6.0.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/yarn/hadoop-yarn-server-applicationhistoryservice-2.6.0.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/yarn/hadoop-yarn-server-tests-2.6.0.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/yarn/hadoop-yarn-client-2.6.0.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/yarn/hadoop-yarn-applications-unmanaged-am-launcher-2.6.0.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/yarn/hadoop-yarn-server-nodemanager-2.6.0.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/mapreduce/lib/hamcrest-core-1.3.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/mapreduce/lib/paranamer-2.3.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/mapreduce/lib/asm-3.2.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/mapreduce/lib/avro-1.7.4.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/mapreduce/lib/javax.inject-1.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/mapreduce/lib/jersey-core-1.9.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/mapreduce/lib/guice-3.0.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/mapreduce/lib/jersey-guice-1.9.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/mapreduce/lib/xz-1.0.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/mapreduce/lib/hadoop-annotations-2.6.0.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/mapreduce/lib/leveldbjni-all-1.8.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/mapreduce/lib/jersey-server-1.9.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/mapreduce/lib/log4j-1.2.17.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/mapreduce/lib/commons-io-2.4.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/mapreduce/lib/jackson-core-asl-1.9.13.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/mapreduce/lib/protobuf-java-2.5.0.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/mapreduce/lib/jackson-mapper-asl-1.9.13.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/mapreduce/lib/aopalliance-1.0.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/mapreduce/lib/commons-compress-1.4.1.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/mapreduce/lib/netty-3.6.2.Final.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/mapreduce/lib/snappy-java-1.0.4.1.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/mapreduce/lib/guice-servlet-3.0.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/mapreduce/lib/junit-4.11.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/mapreduce/hadoop-mapreduce-client-common-2.6.0.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/mapreduce/hadoop-mapreduce-client-core-2.6.0.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-plugins-2.6.0.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.6.0.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-2.6.0.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.6.0-tests.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/mapreduce/hadoop-mapreduce-client-shuffle-2.6.0.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.6.0.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/mapreduce/hadoop-mapreduce-client-app-2.6.0.jar:/mnt/aiprd/app/hadoop-2.6.0/contrib/capacity-scheduler/*.jar
2018-05-03 21:42:57,245 INFO  [main] zookeeper.ZooKeeper: Client environment:java.library.path=/mnt/aiprd/app/hadoop-2.6.0/lib/native
2018-05-03 21:42:57,245 INFO  [main] zookeeper.ZooKeeper: Client environment:java.io.tmpdir=/tmp
2018-05-03 21:42:57,245 INFO  [main] zookeeper.ZooKeeper: Client environment:java.compiler=<NA>
2018-05-03 21:42:57,245 INFO  [main] zookeeper.ZooKeeper: Client environment:os.name=Linux
2018-05-03 21:42:57,245 INFO  [main] zookeeper.ZooKeeper: Client environment:os.arch=amd64
2018-05-03 21:42:57,245 INFO  [main] zookeeper.ZooKeeper: Client environment:os.version=2.6.32-431.el6.x86_64
2018-05-03 21:42:57,245 INFO  [main] zookeeper.ZooKeeper: Client environment:user.name=aiprd
2018-05-03 21:42:57,245 INFO  [main] zookeeper.ZooKeeper: Client environment:user.home=/mnt/aiprd
2018-05-03 21:42:57,245 INFO  [main] zookeeper.ZooKeeper: Client environment:user.dir=/mnt/aiprd/app/hbase-1.2.6
2018-05-03 21:42:57,246 INFO  [main] zookeeper.ZooKeeper: Initiating client connection, connectString=10.191.5.225:2181,10.191.5.226:2181,10.191.5.227:2181 sessionTimeout=90000 watcher=hconnection-0x122998900x0, quorum=10.191.5.225:2181,10.191.5.226:2181,10.191.5.227:2181, baseZNode=/hbase
2018-05-03 21:42:57,271 INFO  [main-SendThread(10.191.5.225:2181)] zookeeper.ClientCnxn: Opening socket connection to server 10.191.5.225/10.191.5.225:2181. Will not attempt to authenticate using SASL (unknown error)
2018-05-03 21:42:57,280 INFO  [main-SendThread(10.191.5.225:2181)] zookeeper.ClientCnxn: Socket connection established to 10.191.5.225/10.191.5.225:2181, initiating session
2018-05-03 21:42:57,292 INFO  [main-SendThread(10.191.5.225:2181)] zookeeper.ClientCnxn: Session establishment complete on server 10.191.5.225/10.191.5.225:2181, sessionid = 0x1632604ecbc0027, negotiated timeout = 40000
Version: 1.2.6
Number of live region servers: 3
Number of dead region servers: 0
Master: host-10-191-5-225,16000,1525354611207
Number of backup masters: 0
Average load: 26.0
Number of requests: 0
Number of regions: 78
Number of regions in transition: 7
2018-05-03 21:42:58,006 INFO  [main] util.HBaseFsck: Loading regionsinfo from the hbase:meta table

Number of empty REGIONINFO_QUALIFIER rows in hbase:meta: 0
2018-05-03 21:42:58,229 INFO  [main] util.HBaseFsck: getHTableDescriptors == tableNames => [IDX_TARGETCUSTDATA_SERIALNUMBER, SN_ORDERS, JDZMD_STOCK_DETAILS, SYN_SALE_PLUS_GOODS_DETAIL_SKU, SYSTEM.MUTEX, JDZMD_SALSER, SYN_SALE_PLUS_GOODS_DETAIL, JDZMD_AREAS, SYSTEM.SEQUENCE, OC_COMM_STOCK, PM_STORE_CHNL_RELATION_REPAIR, JDZMD_STOCK, SYSTEM.FUNCTION, SYN_SALE_PLUS_TRADE, SYNC_DEVELOP_INFO_BYDAY_EFFECT, TOTAL_COST_DETAIL, SYNC_BUSINESS_INFO_BYDAY_EFFECT01, DEPART_INCOME_COST_DETAIL, IDX_TARGETCUST_DATA_PROMO_ID, SYSTEM.STATS, PM_PRODUCT, SYNC01_BUSINESS_INFO_BYDAY_EFFECT, SYNC_TICKET_ACCOUNT_CHECK_INFO, SYN_SALE_PLUS_GOODS, SYNC_SALE_ORDER_INFO, SYNC_BUSINESS_INFO_BYYEAR_HUI_STORE, SYNC_BUSINESS_INFO_BYDAY_IDX_1, SYNC_BUSINESS_INFO_BYDAY_EFFECT02, ZMM_POPULATION, OC_COMM_BASE_INFO, EFFECT_INDEX, SYNC_DEVELOP_INFO_BYDAY, TOTAL_CURR_INCOME_DETAIL, JDZMD_ORDERS, SN_STORE, TOTAL_INCOME_DETAIL, SYN_SALE_PLUS_TRADE_DETAIL, JDZMD_BUSINESSES_AND_STORES, SYNC_COMM_STOCK_IMEI, TOTAL_LAST_INCOME_DETAIL, SYN_SALE_PLUS_STORE, RC_TOP_SALE_CATEGORY_20180103TMP, SYNC_BUSINESS_INFO_BYDAY_BAK1, SYN_SPARK_DEAL_LOG, TARGETCUST_DATA, PM_BRAND, SYNC_DEVELOP_INFO_BYDAY_BAK, RC_TOP_SALE_CATEGORY, SYNC_BUSINESS_INFO_BYDAY, SYNC_BUSINESS_INFO_BYDAY_HUI_STORE, SYNC_COMM_STOCK_BARCODE, SN_ORDERS_ITEM, SYNC_BUSINESS_INFO_BYDAY_EFFECT, SYSTEM.CATALOG, SN_STOCK, OC_ORDER_COMMODITY, hbase:namespace, ZMM2_POPULATION, PM_STORE, SYN_SALE_PLUS_EMPLOYEE, PROD_SALE, OC_ORDER_SALES, SN_EMPLOYEE_STORE, PM_STORE_CHNL_RELATION, PM_COMMODITY, PROD_SALE_TABLE]
2018-05-03 21:42:58,232 INFO  [main] zookeeper.RecoverableZooKeeper: Process identifier=hconnection-0x6a66a204 connecting to ZooKeeper ensemble=10.191.5.225:2181,10.191.5.226:2181,10.191.5.227:2181
2018-05-03 21:42:58,234 INFO  [main] zookeeper.ZooKeeper: Initiating client connection, connectString=10.191.5.225:2181,10.191.5.226:2181,10.191.5.227:2181 sessionTimeout=90000 watcher=hconnection-0x6a66a2040x0, quorum=10.191.5.225:2181,10.191.5.226:2181,10.191.5.227:2181, baseZNode=/hbase
2018-05-03 21:42:58,239 INFO  [main-SendThread(10.191.5.227:2181)] zookeeper.ClientCnxn: Opening socket connection to server 10.191.5.227/10.191.5.227:2181. Will not attempt to authenticate using SASL (unknown error)
2018-05-03 21:42:58,240 INFO  [main-SendThread(10.191.5.227:2181)] zookeeper.ClientCnxn: Socket connection established to 10.191.5.227/10.191.5.227:2181, initiating session
2018-05-03 21:42:58,251 INFO  [main-SendThread(10.191.5.227:2181)] zookeeper.ClientCnxn: Session establishment complete on server 10.191.5.227/10.191.5.227:2181, sessionid = 0x3632604f9540019, negotiated timeout = 40000
2018-05-03 21:42:58,285 INFO  [main] client.ConnectionManager$HConnectionImplementation: Closing master protocol: MasterService
2018-05-03 21:42:58,286 INFO  [main] client.ConnectionManager$HConnectionImplementation: Closing zookeeper sessionid=0x3632604f9540019
2018-05-03 21:42:58,291 INFO  [main] zookeeper.ZooKeeper: Session: 0x3632604f9540019 closed
2018-05-03 21:42:58,291 INFO  [main-EventThread] zookeeper.ClientCnxn: EventThread shut down
Number of Tables: 59
2018-05-03 21:42:58,301 INFO  [main] util.HBaseFsck: Loading region directories from HDFS
...
2018-05-03 21:42:59,690 INFO  [main] util.HBaseFsck: Loading region information from HDFS
2018-05-03 21:42:59,695 WARN  [hbasefsck-pool1-t23] util.HBaseFsck: No HDFS region dir found: { meta => SYNC_BUSINESS_INFO_BYDAY_EFFECT01,,1524566911288.55c734e322ed22f355d5d27b3bd5c060., hdfs => null, deployed => , replicaId => 0 } meta={ENCODED => 55c734e322ed22f355d5d27b3bd5c060, NAME => 'SYNC_BUSINESS_INFO_BYDAY_EFFECT01,,1524566911288.55c734e322ed22f355d5d27b3bd5c060.', STARTKEY => '', ENDKEY => ''}
2018-05-03 21:42:59,695 WARN  [hbasefsck-pool1-t8] util.HBaseFsck: No HDFS region dir found: { meta => SYNC01_BUSINESS_INFO_BYDAY_EFFECT,,1524576739370.691f074a362b80b6383d8894b659da88., hdfs => null, deployed => , replicaId => 0 } meta={ENCODED => 691f074a362b80b6383d8894b659da88, NAME => 'SYNC01_BUSINESS_INFO_BYDAY_EFFECT,,1524576739370.691f074a362b80b6383d8894b659da88.', STARTKEY => '', ENDKEY => ''}
2018-05-03 21:42:59,697 WARN  [hbasefsck-pool1-t32] util.HBaseFsck: No HDFS region dir found: { meta => SYNC_BUSINESS_INFO_BYDAY_EFFECT02,,1524579217478.7f505d85980d982e5ffb2c368cd948d1., hdfs => null, deployed => , replicaId => 0 } meta={ENCODED => 7f505d85980d982e5ffb2c368cd948d1, NAME => 'SYNC_BUSINESS_INFO_BYDAY_EFFECT02,,1524579217478.7f505d85980d982e5ffb2c368cd948d1.', STARTKEY => '', ENDKEY => ''}
2018-05-03 21:42:59,697 WARN  [hbasefsck-pool1-t14] util.HBaseFsck: No HDFS region dir found: { meta => ZMM_POPULATION,,1524562845934.851660046fc034c48aa58ea5a41808b9., hdfs => null, deployed => , replicaId => 0 } meta={ENCODED => 851660046fc034c48aa58ea5a41808b9, NAME => 'ZMM_POPULATION,,1524562845934.851660046fc034c48aa58ea5a41808b9.', STARTKEY => '', ENDKEY => ''}
.2018-05-03 21:42:59,904 WARN  [hbasefsck-pool1-t34] util.HBaseFsck: No HDFS region dir found: { meta => SYNC_BUSINESS_INFO_BYDAY_EFFECT,,1524575202010.cc93585a767dadcbfedff5e152c1e669., hdfs => null, deployed => , replicaId => 0 } meta={ENCODED => cc93585a767dadcbfedff5e152c1e669, NAME => 'SYNC_BUSINESS_INFO_BYDAY_EFFECT,,1524575202010.cc93585a767dadcbfedff5e152c1e669.', STARTKEY => '', ENDKEY => ''}
2018-05-03 21:42:59,905 WARN  [hbasefsck-pool1-t21] util.HBaseFsck: No HDFS region dir found: { meta => ZMM2_POPULATION,,1524576515935.de0beed9391b6d6ab68dcfd92697b662., hdfs => null, deployed => , replicaId => 0 } meta={ENCODED => de0beed9391b6d6ab68dcfd92697b662, NAME => 'ZMM2_POPULATION,,1524576515935.de0beed9391b6d6ab68dcfd92697b662.', STARTKEY => '', ENDKEY => ''}
2018-05-03 21:42:59,909 WARN  [hbasefsck-pool1-t25] util.HBaseFsck: No HDFS region dir found: { meta => PROD_SALE,,1524567998279.eeb0756bc3c35f22d510502624984ea8., hdfs => null, deployed => , replicaId => 0 } meta={ENCODED => eeb0756bc3c35f22d510502624984ea8, NAME => 'PROD_SALE,,1524567998279.eeb0756bc3c35f22d510502624984ea8.', STARTKEY => '', ENDKEY => ''}

2018-05-03 21:43:00,393 INFO  [main] zookeeper.RecoverableZooKeeper: Process identifier=hconnection-0x4e4efc1b connecting to ZooKeeper ensemble=10.191.5.225:2181,10.191.5.226:2181,10.191.5.227:2181
2018-05-03 21:43:00,393 INFO  [main] zookeeper.ZooKeeper: Initiating client connection, connectString=10.191.5.225:2181,10.191.5.226:2181,10.191.5.227:2181 sessionTimeout=90000 watcher=hconnection-0x4e4efc1b0x0, quorum=10.191.5.225:2181,10.191.5.226:2181,10.191.5.227:2181, baseZNode=/hbase
2018-05-03 21:43:00,395 INFO  [main-SendThread(10.191.5.226:2181)] zookeeper.ClientCnxn: Opening socket connection to server 10.191.5.226/10.191.5.226:2181. Will not attempt to authenticate using SASL (unknown error)
2018-05-03 21:43:00,396 INFO  [main-SendThread(10.191.5.226:2181)] zookeeper.ClientCnxn: Socket connection established to 10.191.5.226/10.191.5.226:2181, initiating session
2018-05-03 21:43:00,400 INFO  [main-SendThread(10.191.5.226:2181)] zookeeper.ClientCnxn: Session establishment complete on server 10.191.5.226/10.191.5.226:2181, sessionid = 0x2632604e6f7001e, negotiated timeout = 40000
2018-05-03 21:43:00,403 INFO  [main] zookeeper.RecoverableZooKeeper: Process identifier=hbase Fsck connecting to ZooKeeper ensemble=10.191.5.225:2181,10.191.5.226:2181,10.191.5.227:2181
2018-05-03 21:43:00,403 INFO  [main] zookeeper.ZooKeeper: Initiating client connection, connectString=10.191.5.225:2181,10.191.5.226:2181,10.191.5.227:2181 sessionTimeout=90000 watcher=hbase Fsck0x0, quorum=10.191.5.225:2181,10.191.5.226:2181,10.191.5.227:2181, baseZNode=/hbase
2018-05-03 21:43:00,405 INFO  [main-SendThread(10.191.5.225:2181)] zookeeper.ClientCnxn: Opening socket connection to server 10.191.5.225/10.191.5.225:2181. Will not attempt to authenticate using SASL (unknown error)
2018-05-03 21:43:00,406 INFO  [main-SendThread(10.191.5.225:2181)] zookeeper.ClientCnxn: Socket connection established to 10.191.5.225/10.191.5.225:2181, initiating session
2018-05-03 21:43:00,410 INFO  [main-SendThread(10.191.5.225:2181)] zookeeper.ClientCnxn: Session establishment complete on server 10.191.5.225/10.191.5.225:2181, sessionid = 0x1632604ecbc0028, negotiated timeout = 40000
2018-05-03 21:43:00,434 INFO  [main] zookeeper.ZooKeeper: Session: 0x1632604ecbc0028 closed
2018-05-03 21:43:00,434 INFO  [main-EventThread] zookeeper.ClientCnxn: EventThread shut down
2018-05-03 21:43:00,434 INFO  [main] client.ConnectionManager$HConnectionImplementation: Closing zookeeper sessionid=0x2632604e6f7001e
2018-05-03 21:43:00,438 INFO  [main] zookeeper.ZooKeeper: Session: 0x2632604e6f7001e closed
2018-05-03 21:43:00,439 INFO  [main-EventThread] zookeeper.ClientCnxn: EventThread shut down
2018-05-03 21:43:00,439 INFO  [main] util.HBaseFsck: Checking and fixing region consistency
ERROR: Region { meta => SYNC_BUSINESS_INFO_BYDAY_EFFECT01,,1524566911288.55c734e322ed22f355d5d27b3bd5c060., hdfs => null, deployed => , replicaId => 0 } found in META, but not in HDFS or deployed on any region server.
ERROR: Region { meta => ZMM_POPULATION,,1524562845934.851660046fc034c48aa58ea5a41808b9., hdfs => null, deployed => , replicaId => 0 } found in META, but not in HDFS or deployed on any region server.
ERROR: Region { meta => SYNC01_BUSINESS_INFO_BYDAY_EFFECT,,1524576739370.691f074a362b80b6383d8894b659da88., hdfs => null, deployed => , replicaId => 0 } found in META, but not in HDFS or deployed on any region server.
ERROR: Region { meta => SYNC_BUSINESS_INFO_BYDAY_EFFECT02,,1524579217478.7f505d85980d982e5ffb2c368cd948d1., hdfs => null, deployed => , replicaId => 0 } found in META, but not in HDFS or deployed on any region server.
ERROR: Region { meta => SYNC_BUSINESS_INFO_BYDAY_EFFECT,,1524575202010.cc93585a767dadcbfedff5e152c1e669., hdfs => null, deployed => , replicaId => 0 } found in META, but not in HDFS or deployed on any region server.
ERROR: Region { meta => ZMM2_POPULATION,,1524576515935.de0beed9391b6d6ab68dcfd92697b662., hdfs => null, deployed => , replicaId => 0 } found in META, but not in HDFS or deployed on any region server.
ERROR: Region { meta => PROD_SALE,,1524567998279.eeb0756bc3c35f22d510502624984ea8., hdfs => null, deployed => , replicaId => 0 } found in META, but not in HDFS or deployed on any region server.
2018-05-03 21:43:00,619 INFO  [main] util.HBaseFsck: Handling overlap merges in parallel. set hbasefsck.overlap.merge.parallel to false to run serially.
2018-05-03 21:43:00,634 WARN  [main] util.HBaseFsck: Naming new problem group: 20180124\x0020180125171008535733092133537437
ERROR: (region SYNC_BUSINESS_INFO_BYDAY_IDX_1,20180124\x0020180125171008535733092133537437,1522508011325.4a14624045253764c1056f646bbe6f78.) Multiple regions have the same startkey: 20180124\x0020180125171008535733092133537437
ERROR: (region SYNC_BUSINESS_INFO_BYDAY_IDX_1,20180124\x0020180125171008535733092133537437,1520317313775.7b26d838855e20824e961161d0faed68.) Multiple regions have the same startkey: 20180124\x0020180125171008535733092133537437
ERROR: (regions SYNC_BUSINESS_INFO_BYDAY_IDX_1,20180124\x0020180125171008535733092133537437,1520317313775.7b26d838855e20824e961161d0faed68. and SYNC_BUSINESS_INFO_BYDAY_IDX_1,20180226\x0020180227171513028226357214718886,1522508011325.c3cb4df74c322c55a482d8f9478a0324.) There is an overlap in the region chain.
2018-05-03 21:43:00,635 WARN  [main] util.HBaseFsck: reached end of problem group: null
ERROR: Found inconsistency in table SYNC_BUSINESS_INFO_BYDAY_IDX_1
2018-05-03 21:43:00,636 INFO  [main] util.HBaseFsck: Computing mapping of all store files
..
2018-05-03 21:43:00,902 INFO  [main] util.HBaseFsck: Validating mapping using HDFS state
2018-05-03 21:43:00,918 INFO  [main] zookeeper.RecoverableZooKeeper: Process identifier=hbase Fsck connecting to ZooKeeper ensemble=10.191.5.225:2181,10.191.5.226:2181,10.191.5.227:2181
2018-05-03 21:43:00,918 INFO  [main] zookeeper.ZooKeeper: Initiating client connection, connectString=10.191.5.225:2181,10.191.5.226:2181,10.191.5.227:2181 sessionTimeout=90000 watcher=hbase Fsck0x0, quorum=10.191.5.225:2181,10.191.5.226:2181,10.191.5.227:2181, baseZNode=/hbase
2018-05-03 21:43:00,919 INFO  [main-SendThread(10.191.5.225:2181)] zookeeper.ClientCnxn: Opening socket connection to server 10.191.5.225/10.191.5.225:2181. Will not attempt to authenticate using SASL (unknown error)
2018-05-03 21:43:00,920 INFO  [main-SendThread(10.191.5.225:2181)] zookeeper.ClientCnxn: Socket connection established to 10.191.5.225/10.191.5.225:2181, initiating session
2018-05-03 21:43:00,923 INFO  [main-SendThread(10.191.5.225:2181)] zookeeper.ClientCnxn: Session establishment complete on server 10.191.5.225/10.191.5.225:2181, sessionid = 0x1632604ecbc0029, negotiated timeout = 40000
2018-05-03 21:43:01,001 INFO  [main] zookeeper.ZooKeeper: Session: 0x1632604ecbc0029 closed
2018-05-03 21:43:01,002 INFO  [main-EventThread] zookeeper.ClientCnxn: EventThread shut down
2018-05-03 21:43:01,002 INFO  [main] zookeeper.RecoverableZooKeeper: Process identifier=hbase Fsck connecting to ZooKeeper ensemble=10.191.5.225:2181,10.191.5.226:2181,10.191.5.227:2181
2018-05-03 21:43:01,002 INFO  [main] zookeeper.ZooKeeper: Initiating client connection, connectString=10.191.5.225:2181,10.191.5.226:2181,10.191.5.227:2181 sessionTimeout=90000 watcher=hbase Fsck0x0, quorum=10.191.5.225:2181,10.191.5.226:2181,10.191.5.227:2181, baseZNode=/hbase
2018-05-03 21:43:01,005 INFO  [main-SendThread(10.191.5.227:2181)] zookeeper.ClientCnxn: Opening socket connection to server 10.191.5.227/10.191.5.227:2181. Will not attempt to authenticate using SASL (unknown error)
2018-05-03 21:43:01,006 INFO  [main-SendThread(10.191.5.227:2181)] zookeeper.ClientCnxn: Socket connection established to 10.191.5.227/10.191.5.227:2181, initiating session
2018-05-03 21:43:01,010 INFO  [main-SendThread(10.191.5.227:2181)] zookeeper.ClientCnxn: Session establishment complete on server 10.191.5.227/10.191.5.227:2181, sessionid = 0x3632604f954001a, negotiated timeout = 40000
2018-05-03 21:43:01,062 INFO  [main] zookeeper.ZooKeeper: Session: 0x3632604f954001a closed
Summary:
2018-05-03 21:43:01,062 INFO  [main-EventThread] zookeeper.ClientCnxn: EventThread shut down
Table IDX_TARGETCUSTDATA_SERIALNUMBER is okay.
    Number of regions: 1
    Deployed on:  host-10-191-5-225,16020,1525354612066
Table SYN_SALE_PLUS_GOODS is okay.
    Number of regions: 1
    Deployed on:  host-10-191-5-227,16020,1525354610418
Table SYN_SALE_PLUS_STORE is okay.
    Number of regions: 1
    Deployed on:  host-10-191-5-227,16020,1525354610418
Table SYN_SALE_PLUS_TRADE is okay.
    Number of regions: 1
    Deployed on:  host-10-191-5-225,16020,1525354612066
Table PM_PRODUCT is okay.
    Number of regions: 1
    Deployed on:  host-10-191-5-227,16020,1525354610418
Table PM_STORE_CHNL_RELATION_REPAIR is okay.
    Number of regions: 1
    Deployed on:  host-10-191-5-227,16020,1525354610418
Table JDZMD_BUSINESSES_AND_STORES is okay.
    Number of regions: 1
    Deployed on:  host-10-191-5-227,16020,1525354610418
Table OC_ORDER_COMMODITY is okay.
    Number of regions: 1
    Deployed on:  host-10-191-5-225,16020,1525354612066
Table SYNC_TICKET_ACCOUNT_CHECK_INFO is okay.
    Number of regions: 1
    Deployed on:  host-10-191-5-225,16020,1525354612066
Table TARGETCUST_DATA is okay.
    Number of regions: 3
    Deployed on:  host-10-191-5-226,16020,1525354610613
Table TOTAL_LAST_INCOME_DETAIL is okay.
    Number of regions: 1
    Deployed on:  host-10-191-5-227,16020,1525354610418
Table PM_STORE_CHNL_RELATION is okay.
    Number of regions: 1
    Deployed on:  host-10-191-5-225,16020,1525354612066
Table SYN_SALE_PLUS_GOODS_DETAIL_SKU is okay.
    Number of regions: 1
    Deployed on:  host-10-191-5-226,16020,1525354610613
Table EFFECT_INDEX is okay.
    Number of regions: 1
    Deployed on:  host-10-191-5-227,16020,1525354610418
Table TOTAL_INCOME_DETAIL is okay.
    Number of regions: 1
    Deployed on:  host-10-191-5-226,16020,1525354610613
Table JDZMD_STOCK_DETAILS is okay.
    Number of regions: 1
    Deployed on:  host-10-191-5-225,16020,1525354612066
Table hbase:meta is okay.
    Number of regions: 1
    Deployed on:  host-10-191-5-226,16020,1525354610613
Table DEPART_INCOME_COST_DETAIL is okay.
    Number of regions: 1
    Deployed on:  host-10-191-5-225,16020,1525354612066
Table SN_ORDERS_ITEM is okay.
    Number of regions: 1
    Deployed on:  host-10-191-5-226,16020,1525354610613
Table SN_ORDERS is okay.
    Number of regions: 1
    Deployed on:  host-10-191-5-226,16020,1525354610613
Table SYSTEM.CATALOG is okay.
    Number of regions: 1
    Deployed on:  host-10-191-5-225,16020,1525354612066
Table PROD_SALE_TABLE is okay.
    Number of regions: 1
    Deployed on:  host-10-191-5-225,16020,1525354612066
Table SYNC_BUSINESS_INFO_BYYEAR_HUI_STORE is okay.
    Number of regions: 1
    Deployed on:  host-10-191-5-225,16020,1525354612066
Table OC_COMM_BASE_INFO is okay.
    Number of regions: 1
    Deployed on:  host-10-191-5-227,16020,1525354610418
Table SYNC_BUSINESS_INFO_BYDAY_BAK1 is okay.
    Number of regions: 2
    Deployed on:  host-10-191-5-225,16020,1525354612066 host-10-191-5-227,16020,1525354610418
Table JDZMD_AREAS is okay.
    Number of regions: 1
    Deployed on:  host-10-191-5-227,16020,1525354610418
Table JDZMD_STOCK is okay.
    Number of regions: 1
    Deployed on:  host-10-191-5-226,16020,1525354610613
Table JDZMD_ORDERS is okay.
    Number of regions: 1
    Deployed on:  host-10-191-5-227,16020,1525354610418
Table PM_BRAND is okay.
    Number of regions: 1
    Deployed on:  host-10-191-5-226,16020,1525354610613
Table PM_STORE is okay.
    Number of regions: 1
    Deployed on:  host-10-191-5-226,16020,1525354610613
Table JDZMD_SALSER is okay.
    Number of regions: 1
    Deployed on:  host-10-191-5-227,16020,1525354610418
Table SYN_SALE_PLUS_EMPLOYEE is okay.
    Number of regions: 1
    Deployed on:  host-10-191-5-225,16020,1525354612066
Table SYNC_BUSINESS_INFO_BYDAY is okay.
    Number of regions: 5
    Deployed on:  host-10-191-5-225,16020,1525354612066 host-10-191-5-226,16020,1525354610613 host-10-191-5-227,16020,1525354610418
Table SYNC_DEVELOP_INFO_BYDAY_BAK is okay.
    Number of regions: 1
    Deployed on:  host-10-191-5-226,16020,1525354610613
Table SYNC_DEVELOP_INFO_BYDAY is okay.
    Number of regions: 4
    Deployed on:  host-10-191-5-225,16020,1525354612066 host-10-191-5-226,16020,1525354610613 host-10-191-5-227,16020,1525354610418
Table SYNC_COMM_STOCK_IMEI is okay.
    Number of regions: 2
    Deployed on:  host-10-191-5-226,16020,1525354610613
Table PM_COMMODITY is okay.
    Number of regions: 1
    Deployed on:  host-10-191-5-225,16020,1525354612066
Table SYN_SPARK_DEAL_LOG is okay.
    Number of regions: 1
    Deployed on:  host-10-191-5-227,16020,1525354610418
Table OC_ORDER_SALES is okay.
    Number of regions: 1
    Deployed on:  host-10-191-5-225,16020,1525354612066
Table hbase:namespace is okay.
    Number of regions: 1
    Deployed on:  host-10-191-5-227,16020,1525354610418
Table TOTAL_COST_DETAIL is okay.
    Number of regions: 2
    Deployed on:  host-10-191-5-226,16020,1525354610613 host-10-191-5-227,16020,1525354610418
Table SYN_SALE_PLUS_GOODS_DETAIL is okay.
    Number of regions: 1
    Deployed on:  host-10-191-5-225,16020,1525354612066
Table SYNC_SALE_ORDER_INFO is okay.
    Number of regions: 4
    Deployed on:  host-10-191-5-225,16020,1525354612066 host-10-191-5-226,16020,1525354610613
Table SYSTEM.SEQUENCE is okay.
    Number of regions: 1
    Deployed on:  host-10-191-5-225,16020,1525354612066
Table SYSTEM.FUNCTION is okay.
    Number of regions: 1
    Deployed on:  host-10-191-5-225,16020,1525354612066
Table SN_EMPLOYEE_STORE is okay.
    Number of regions: 1
    Deployed on:  host-10-191-5-227,16020,1525354610418
Table SYN_SALE_PLUS_TRADE_DETAIL is okay.
    Number of regions: 1
    Deployed on:  host-10-191-5-227,16020,1525354610418
Table SYNC_DEVELOP_INFO_BYDAY_EFFECT is okay.
    Number of regions: 1
    Deployed on:  host-10-191-5-225,16020,1525354612066
Table SYSTEM.MUTEX is okay.
    Number of regions: 1
    Deployed on:  host-10-191-5-227,16020,1525354610418
Table SYSTEM.STATS is okay.
    Number of regions: 1
    Deployed on:  host-10-191-5-227,16020,1525354610418
Table RC_TOP_SALE_CATEGORY_20180103TMP is okay.
    Number of regions: 1
    Deployed on:  host-10-191-5-227,16020,1525354610418
Table OC_COMM_STOCK is okay.
    Number of regions: 1
    Deployed on:  host-10-191-5-226,16020,1525354610613
Table IDX_TARGETCUST_DATA_PROMO_ID is okay.
    Number of regions: 1
    Deployed on:  host-10-191-5-227,16020,1525354610418
Table SYNC_COMM_STOCK_BARCODE is okay.
    Number of regions: 1
    Deployed on:  host-10-191-5-227,16020,1525354610418
Table SN_STOCK is okay.
    Number of regions: 1
    Deployed on:  host-10-191-5-226,16020,1525354610613
Table SN_STORE is okay.
    Number of regions: 1
    Deployed on:  host-10-191-5-227,16020,1525354610418
Table SYNC_BUSINESS_INFO_BYDAY_HUI_STORE is okay.
    Number of regions: 1
    Deployed on:  host-10-191-5-226,16020,1525354610613
Table SYNC_BUSINESS_INFO_BYDAY_IDX_1 is inconsistent.
    Number of regions: 4
    Deployed on:  host-10-191-5-225,16020,1525354612066 host-10-191-5-226,16020,1525354610613
Table RC_TOP_SALE_CATEGORY is okay.
    Number of regions: 1
    Deployed on:  host-10-191-5-225,16020,1525354612066
Table TOTAL_CURR_INCOME_DETAIL is okay.
    Number of regions: 1
    Deployed on:  host-10-191-5-226,16020,1525354610613
10 inconsistencies detected.
Status: INCONSISTENT
2018-05-03 21:43:01,075 INFO  [main] client.ConnectionManager$HConnectionImplementation: Closing master protocol: MasterService
2018-05-03 21:43:01,075 INFO  [main] client.ConnectionManager$HConnectionImplementation: Closing zookeeper sessionid=0x1632604ecbc0027
2018-05-03 21:43:01,078 INFO  [main] zookeeper.ZooKeeper: Session: 0x1632604ecbc0027 closed
2018-05-03 21:43:01,078 INFO  [main-EventThread] zookeeper.ClientCnxn: EventThread shut down

备注:通过以上的检查信息可以知道,就是说,这些表在zookeeper的META即元数据中存在,但是在hdfs中是不存在的,所以是不一致的,才会导致建表的时候一直报表存在的错误.

8.通过hbck修复元数据的选项-fixMeta修复该问题

[aiprd@host-10-191-5-225 hbase-1.2.6]$ bin/hbase hbck -fixMeta
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option PermSize=128m; support was removed in 8.0
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=128m; support was removed in 8.0
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/mnt/aiprd/app/hbase-1.2.6/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/common/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
2018-05-03 21:48:44,196 WARN  [main] util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
HBaseFsck command line options: -fixMeta
2018-05-03 21:48:44,876 INFO  [main] util.HBaseFsck: Launching hbck
2018-05-03 21:48:44,964 INFO  [main] zookeeper.RecoverableZooKeeper: Process identifier=hconnection-0x68dc098b connecting to ZooKeeper ensemble=10.191.5.225:2181,10.191.5.226:2181,10.191.5.227:2181
2018-05-03 21:48:44,970 INFO  [main] zookeeper.ZooKeeper: Client environment:zookeeper.version=3.4.6-1569965, built on 02/20/2014 09:09 GMT
2018-05-03 21:48:44,970 INFO  [main] zookeeper.ZooKeeper: Client environment:host.name=host-10-191-5-225
2018-05-03 21:48:44,970 INFO  [main] zookeeper.ZooKeeper: Client environment:java.version=1.8.0_121
2018-05-03 21:48:44,970 INFO  [main] zookeeper.ZooKeeper: Client environment:java.vendor=Oracle Corporation
2018-05-03 21:48:44,970 INFO  [main] zookeeper.ZooKeeper: Client environment:java.home=/mnt/aiprd/app/java/jre
2018-05-03 21:48:44,970 INFO  [main] zookeeper.ZooKeeper: Client environment:java.class.path=/mnt/aiprd/app/hbase-1.2.6/bin/../conf:/mnt/aiprd/app/java/lib/tools.jar:/mnt/aiprd/app/hbase-1.2.6/bin/..:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/activation-1.1.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/aopalliance-1.0.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/apacheds-i18n-2.0.0-M15.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/apacheds-kerberos-codec-2.0.0-M15.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/api-asn1-api-1.0.0-M20.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/api-util-1.0.0-M20.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/asm-3.1.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/avro-1.7.4.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/commons-beanutils-1.7.0.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/commons-beanutils-core-1.8.0.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/commons-cli-1.2.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/commons-codec-1.9.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/commons-collections-3.2.2.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/commons-compress-1.4.1.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/commons-configuration-1.6.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/commons-daemon-1.0.13.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/commons-digester-1.8.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/commons-el-1.0.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/commons-httpclient-3.1.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/commons-io-2.4.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/commons-lang-2.6.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/commons-logging-1.2.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/commons-math-2.2.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/commons-math3-3.1.1.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/commons-net-3.1.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/disruptor-3.3.0.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/findbugs-annotations-1.3.9-1.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/guava-12.0.1.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/guice-3.0.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/guice-servlet-3.0.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/hadoop-annotations-2.5.1.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/hadoop-auth-2.5.1.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/hadoop-client-2.5.1.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/hadoop-common-2.5.1.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/hadoop-hdfs-2.5.1.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/hadoop-mapreduce-client-app-2.5.1.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/hadoop-mapreduce-client-common-2.5.1.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/hadoop-mapreduce-client-core-2.5.1.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/hadoop-mapreduce-client-jobclient-2.5.1.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/hadoop-mapreduce-client-shuffle-2.5.1.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/hadoop-yarn-api-2.5.1.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/hadoop-yarn-client-2.5.1.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/hadoop-yarn-common-2.5.1.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/hadoop-yarn-server-common-2.5.1.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/hbase-annotations-1.2.6.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/hbase-annotations-1.2.6-tests.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/hbase-client-1.2.6.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/hbase-common-1.2.6.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/hbase-common-1.2.6-tests.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/hbase-examples-1.2.6.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/hbase-external-blockcache-1.2.6.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/hbase-hadoop2-compat-1.2.6.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/hbase-hadoop-compat-1.2.6.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/hbase-it-1.2.6.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/hbase-it-1.2.6-tests.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/hbase-prefix-tree-1.2.6.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/hbase-procedure-1.2.6.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/hbase-protocol-1.2.6.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/hbase-resource-bundle-1.2.6.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/hbase-rest-1.2.6.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/hbase-server-1.2.6.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/hbase-server-1.2.6-tests.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/hbase-shell-1.2.6.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/hbase-thrift-1.2.6.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/htrace-core-3.1.0-incubating.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/httpclient-4.2.5.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/httpcore-4.4.1.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/jackson-core-asl-1.9.13.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/jackson-jaxrs-1.9.13.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/jackson-mapper-asl-1.9.13.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/jackson-xc-1.9.13.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/jamon-runtime-2.4.1.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/jasper-compiler-5.5.23.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/jasper-runtime-5.5.23.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/javax.inject-1.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/java-xmlbuilder-0.4.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/jaxb-api-2.2.2.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/jaxb-impl-2.2.3-1.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/jcodings-1.0.8.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/jersey-client-1.9.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/jersey-core-1.9.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/jersey-guice-1.9.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/jersey-json-1.9.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/jersey-server-1.9.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/jets3t-0.9.0.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/jettison-1.3.3.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/jetty-6.1.26.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/jetty-sslengine-6.1.26.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/jetty-util-6.1.26.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/joni-2.1.2.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/jruby-complete-1.6.8.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/jsch-0.1.42.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/jsp-2.1-6.1.14.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/jsp-api-2.1-6.1.14.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/junit-4.12.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/leveldbjni-all-1.8.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/libthrift-0.9.3.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/log4j-1.2.17.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/metrics-core-2.2.0.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/netty-all-4.0.23.Final.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/paranamer-2.3.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/phoenix-4.10.0-HBase-1.2-server.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/protobuf-java-2.5.0.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/servlet-api-2.5-6.1.14.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/servlet-api-2.5.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/slf4j-api-1.7.7.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/slf4j-log4j12-1.7.5.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/snappy-java-1.0.4.1.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/spymemcached-2.11.6.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/xmlenc-0.52.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/xz-1.0.jar:/mnt/aiprd/app/hbase-1.2.6/bin/../lib/zookeeper-3.4.6.jar:/mnt/aiprd/app/hadoop-2.6.0/etc/hadoop:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/common/lib/slf4j-log4j12-1.7.5.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/common/lib/jetty-6.1.26.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/common/lib/java-xmlbuilder-0.4.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/common/lib/hamcrest-core-1.3.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/common/lib/api-asn1-api-1.0.0-M20.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/common/lib/gson-2.2.4.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/common/lib/jaxb-impl-2.2.3-1.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/common/lib/apacheds-i18n-2.0.0-M15.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/common/lib/commons-logging-1.1.3.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/common/lib/htrace-core-3.0.4.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/common/lib/commons-math3-3.1.1.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/common/lib/commons-codec-1.4.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/common/lib/jsr305-1.3.9.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/common/lib/commons-configuration-1.6.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/common/lib/paranamer-2.3.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/common/lib/asm-3.2.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/common/lib/avro-1.7.4.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/common/lib/jersey-core-1.9.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/common/lib/commons-el-1.0.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/common/lib/jets3t-0.9.0.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/common/lib/jetty-util-6.1.26.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/common/lib/curator-recipes-2.6.0.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/common/lib/curator-framework-2.6.0.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/common/lib/commons-lang-2.6.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/common/lib/xz-1.0.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/common/lib/commons-beanutils-1.7.0.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/common/lib/commons-collections-3.2.1.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/common/lib/hadoop-annotations-2.6.0.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/common/lib/jersey-json-1.9.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/common/lib/curator-client-2.6.0.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/common/lib/jersey-server-1.9.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/common/lib/jsp-api-2.1.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/common/lib/httpcore-4.2.5.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/common/lib/jackson-xc-1.9.13.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/common/lib/log4j-1.2.17.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/common/lib/commons-io-2.4.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/common/lib/httpclient-4.2.5.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/common/lib/commons-cli-1.2.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/common/lib/guava-11.0.2.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/common/lib/jettison-1.1.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/common/lib/zookeeper-3.4.6.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/common/lib/jackson-core-asl-1.9.13.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/common/lib/activation-1.1.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/common/lib/commons-httpclient-3.1.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/common/lib/protobuf-java-2.5.0.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/common/lib/jackson-mapper-asl-1.9.13.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/common/lib/slf4j-api-1.7.5.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/common/lib/xmlenc-0.52.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/common/lib/commons-digester-1.8.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/common/lib/jsch-0.1.42.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/common/lib/jackson-jaxrs-1.9.13.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/common/lib/jasper-compiler-5.5.23.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/common/lib/commons-compress-1.4.1.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/common/lib/api-util-1.0.0-M20.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/common/lib/netty-3.6.2.Final.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/common/lib/commons-net-3.1.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/common/lib/jasper-runtime-5.5.23.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/common/lib/snappy-java-1.0.4.1.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/common/lib/servlet-api-2.5.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/common/lib/commons-beanutils-core-1.8.0.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/common/lib/apacheds-kerberos-codec-2.0.0-M15.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/common/lib/mockito-all-1.8.5.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/common/lib/junit-4.11.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/common/lib/stax-api-1.0-2.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/common/lib/hadoop-auth-2.6.0.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/common/lib/jaxb-api-2.2.2.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/common/hadoop-common-2.6.0-tests.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/common/hadoop-common-2.6.0.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/common/hadoop-nfs-2.6.0.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/hdfs:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/hdfs/lib/jetty-6.1.26.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/hdfs/lib/commons-daemon-1.0.13.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/hdfs/lib/xml-apis-1.3.04.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/hdfs/lib/commons-logging-1.1.3.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/hdfs/lib/htrace-core-3.0.4.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/hdfs/lib/xercesImpl-2.9.1.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/hdfs/lib/commons-codec-1.4.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/hdfs/lib/jsr305-1.3.9.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/hdfs/lib/asm-3.2.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/hdfs/lib/jersey-core-1.9.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/hdfs/lib/commons-el-1.0.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/hdfs/lib/jetty-util-6.1.26.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/hdfs/lib/commons-lang-2.6.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/hdfs/lib/jersey-server-1.9.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/hdfs/lib/jsp-api-2.1.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/hdfs/lib/log4j-1.2.17.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/hdfs/lib/commons-io-2.4.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/hdfs/lib/commons-cli-1.2.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/hdfs/lib/guava-11.0.2.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/hdfs/lib/jackson-core-asl-1.9.13.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/hdfs/lib/protobuf-java-2.5.0.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/hdfs/lib/jackson-mapper-asl-1.9.13.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/hdfs/lib/xmlenc-0.52.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/hdfs/lib/netty-3.6.2.Final.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/hdfs/lib/jasper-runtime-5.5.23.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/hdfs/lib/servlet-api-2.5.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/hdfs/hadoop-hdfs-2.6.0-tests.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/hdfs/hadoop-hdfs-nfs-2.6.0.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/hdfs/hadoop-hdfs-2.6.0.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/yarn/lib/jetty-6.1.26.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/yarn/lib/jaxb-impl-2.2.3-1.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/yarn/lib/commons-logging-1.1.3.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/yarn/lib/commons-codec-1.4.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/yarn/lib/jsr305-1.3.9.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/yarn/lib/jline-0.9.94.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/yarn/lib/asm-3.2.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/yarn/lib/javax.inject-1.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/yarn/lib/jersey-core-1.9.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/yarn/lib/jetty-util-6.1.26.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/yarn/lib/guice-3.0.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/yarn/lib/commons-lang-2.6.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/yarn/lib/jersey-guice-1.9.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/yarn/lib/xz-1.0.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/yarn/lib/commons-collections-3.2.1.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/yarn/lib/leveldbjni-all-1.8.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/yarn/lib/jersey-json-1.9.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/yarn/lib/jersey-server-1.9.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/yarn/lib/jackson-xc-1.9.13.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/yarn/lib/log4j-1.2.17.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/yarn/lib/commons-io-2.4.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/yarn/lib/commons-cli-1.2.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/yarn/lib/guava-11.0.2.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/yarn/lib/jettison-1.1.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/yarn/lib/zookeeper-3.4.6.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/yarn/lib/jackson-core-asl-1.9.13.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/yarn/lib/activation-1.1.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/yarn/lib/commons-httpclient-3.1.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/yarn/lib/protobuf-java-2.5.0.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/yarn/lib/jackson-mapper-asl-1.9.13.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/yarn/lib/jersey-client-1.9.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/yarn/lib/jackson-jaxrs-1.9.13.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/yarn/lib/aopalliance-1.0.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/yarn/lib/commons-compress-1.4.1.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/yarn/lib/netty-3.6.2.Final.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/yarn/lib/servlet-api-2.5.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/yarn/lib/guice-servlet-3.0.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/yarn/lib/stax-api-1.0-2.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/yarn/lib/jaxb-api-2.2.2.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/yarn/hadoop-yarn-api-2.6.0.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/yarn/hadoop-yarn-applications-distributedshell-2.6.0.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/yarn/hadoop-yarn-registry-2.6.0.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/yarn/hadoop-yarn-server-common-2.6.0.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/yarn/hadoop-yarn-server-resourcemanager-2.6.0.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/yarn/hadoop-yarn-server-web-proxy-2.6.0.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/yarn/hadoop-yarn-common-2.6.0.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/yarn/hadoop-yarn-server-applicationhistoryservice-2.6.0.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/yarn/hadoop-yarn-server-tests-2.6.0.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/yarn/hadoop-yarn-client-2.6.0.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/yarn/hadoop-yarn-applications-unmanaged-am-launcher-2.6.0.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/yarn/hadoop-yarn-server-nodemanager-2.6.0.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/mapreduce/lib/hamcrest-core-1.3.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/mapreduce/lib/paranamer-2.3.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/mapreduce/lib/asm-3.2.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/mapreduce/lib/avro-1.7.4.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/mapreduce/lib/javax.inject-1.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/mapreduce/lib/jersey-core-1.9.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/mapreduce/lib/guice-3.0.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/mapreduce/lib/jersey-guice-1.9.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/mapreduce/lib/xz-1.0.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/mapreduce/lib/hadoop-annotations-2.6.0.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/mapreduce/lib/leveldbjni-all-1.8.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/mapreduce/lib/jersey-server-1.9.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/mapreduce/lib/log4j-1.2.17.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/mapreduce/lib/commons-io-2.4.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/mapreduce/lib/jackson-core-asl-1.9.13.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/mapreduce/lib/protobuf-java-2.5.0.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/mapreduce/lib/jackson-mapper-asl-1.9.13.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/mapreduce/lib/aopalliance-1.0.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/mapreduce/lib/commons-compress-1.4.1.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/mapreduce/lib/netty-3.6.2.Final.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/mapreduce/lib/snappy-java-1.0.4.1.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/mapreduce/lib/guice-servlet-3.0.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/mapreduce/lib/junit-4.11.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/mapreduce/hadoop-mapreduce-client-common-2.6.0.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/mapreduce/hadoop-mapreduce-client-core-2.6.0.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-plugins-2.6.0.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.6.0.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-2.6.0.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.6.0-tests.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/mapreduce/hadoop-mapreduce-client-shuffle-2.6.0.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.6.0.jar:/mnt/aiprd/app/hadoop-2.6.0/share/hadoop/mapreduce/hadoop-mapreduce-client-app-2.6.0.jar:/mnt/aiprd/app/hadoop-2.6.0/contrib/capacity-scheduler/*.jar
2018-05-03 21:48:44,973 INFO  [main] zookeeper.ZooKeeper: Client environment:java.library.path=/mnt/aiprd/app/hadoop-2.6.0/lib/native
2018-05-03 21:48:44,973 INFO  [main] zookeeper.ZooKeeper: Client environment:java.io.tmpdir=/tmp
2018-05-03 21:48:44,973 INFO  [main] zookeeper.ZooKeeper: Client environment:java.compiler=<NA>
2018-05-03 21:48:44,973 INFO  [main] zookeeper.ZooKeeper: Client environment:os.name=Linux
2018-05-03 21:48:44,973 INFO  [main] zookeeper.ZooKeeper: Client environment:os.arch=amd64
2018-05-03 21:48:44,973 INFO  [main] zookeeper.ZooKeeper: Client environment:os.version=2.6.32-431.el6.x86_64
2018-05-03 21:48:44,973 INFO  [main] zookeeper.ZooKeeper: Client environment:user.name=aiprd
2018-05-03 21:48:44,973 INFO  [main] zookeeper.ZooKeeper: Client environment:user.home=/mnt/aiprd
2018-05-03 21:48:44,973 INFO  [main] zookeeper.ZooKeeper: Client environment:user.dir=/mnt/aiprd/app/hbase-1.2.6
2018-05-03 21:48:44,974 INFO  [main] zookeeper.ZooKeeper: Initiating client connection, connectString=10.191.5.225:2181,10.191.5.226:2181,10.191.5.227:2181 sessionTimeout=90000 watcher=hconnection-0x68dc098b0x0, quorum=10.191.5.225:2181,10.191.5.226:2181,10.191.5.227:2181, baseZNode=/hbase
2018-05-03 21:48:44,994 INFO  [main-SendThread(10.191.5.227:2181)] zookeeper.ClientCnxn: Opening socket connection to server 10.191.5.227/10.191.5.227:2181. Will not attempt to authenticate using SASL (unknown error)
2018-05-03 21:48:44,996 INFO  [main-SendThread(10.191.5.227:2181)] zookeeper.ClientCnxn: Socket connection established to 10.191.5.227/10.191.5.227:2181, initiating session
2018-05-03 21:48:45,012 INFO  [main-SendThread(10.191.5.227:2181)] zookeeper.ClientCnxn: Session establishment complete on server 10.191.5.227/10.191.5.227:2181, sessionid = 0x3632604f954001b, negotiated timeout = 40000
Version: 1.2.6
Number of live region servers: 3
Number of dead region servers: 0
Master: host-10-191-5-225,16000,1525354611207
Number of backup masters: 0
Average load: 26.0
Number of requests: 0
Number of regions: 78
Number of regions in transition: 7
2018-05-03 21:48:45,783 INFO  [main] util.HBaseFsck: Loading regionsinfo from the hbase:meta table

Number of empty REGIONINFO_QUALIFIER rows in hbase:meta: 0
2018-05-03 21:48:45,963 INFO  [main] util.HBaseFsck: getHTableDescriptors == tableNames => [IDX_TARGETCUSTDATA_SERIALNUMBER, SN_ORDERS, JDZMD_STOCK_DETAILS, SYN_SALE_PLUS_GOODS_DETAIL_SKU, SYSTEM.MUTEX, JDZMD_SALSER, SYN_SALE_PLUS_GOODS_DETAIL, JDZMD_AREAS, SYSTEM.SEQUENCE, OC_COMM_STOCK, PM_STORE_CHNL_RELATION_REPAIR, JDZMD_STOCK, SYSTEM.FUNCTION, SYN_SALE_PLUS_TRADE, SYNC_DEVELOP_INFO_BYDAY_EFFECT, TOTAL_COST_DETAIL, SYNC_BUSINESS_INFO_BYDAY_EFFECT01, DEPART_INCOME_COST_DETAIL, IDX_TARGETCUST_DATA_PROMO_ID, SYSTEM.STATS, PM_PRODUCT, SYNC01_BUSINESS_INFO_BYDAY_EFFECT, SYNC_TICKET_ACCOUNT_CHECK_INFO, SYN_SALE_PLUS_GOODS, SYNC_SALE_ORDER_INFO, SYNC_BUSINESS_INFO_BYYEAR_HUI_STORE, SYNC_BUSINESS_INFO_BYDAY_IDX_1, SYNC_BUSINESS_INFO_BYDAY_EFFECT02, ZMM_POPULATION, OC_COMM_BASE_INFO, EFFECT_INDEX, SYNC_DEVELOP_INFO_BYDAY, TOTAL_CURR_INCOME_DETAIL, JDZMD_ORDERS, SN_STORE, TOTAL_INCOME_DETAIL, SYN_SALE_PLUS_TRADE_DETAIL, JDZMD_BUSINESSES_AND_STORES, SYNC_COMM_STOCK_IMEI, TOTAL_LAST_INCOME_DETAIL, SYN_SALE_PLUS_STORE, RC_TOP_SALE_CATEGORY_20180103TMP, SYNC_BUSINESS_INFO_BYDAY_BAK1, SYN_SPARK_DEAL_LOG, TARGETCUST_DATA, PM_BRAND, SYNC_DEVELOP_INFO_BYDAY_BAK, RC_TOP_SALE_CATEGORY, SYNC_BUSINESS_INFO_BYDAY, SYNC_BUSINESS_INFO_BYDAY_HUI_STORE, SYNC_COMM_STOCK_BARCODE, SN_ORDERS_ITEM, SYNC_BUSINESS_INFO_BYDAY_EFFECT, SYSTEM.CATALOG, SN_STOCK, OC_ORDER_COMMODITY, hbase:namespace, ZMM2_POPULATION, PM_STORE, SYN_SALE_PLUS_EMPLOYEE, PROD_SALE, OC_ORDER_SALES, SN_EMPLOYEE_STORE, PM_STORE_CHNL_RELATION, PM_COMMODITY, PROD_SALE_TABLE]
2018-05-03 21:48:45,966 INFO  [main] zookeeper.RecoverableZooKeeper: Process identifier=hconnection-0x6865c751 connecting to ZooKeeper ensemble=10.191.5.225:2181,10.191.5.226:2181,10.191.5.227:2181
2018-05-03 21:48:45,966 INFO  [main] zookeeper.ZooKeeper: Initiating client connection, connectString=10.191.5.225:2181,10.191.5.226:2181,10.191.5.227:2181 sessionTimeout=90000 watcher=hconnection-0x6865c7510x0, quorum=10.191.5.225:2181,10.191.5.226:2181,10.191.5.227:2181, baseZNode=/hbase
2018-05-03 21:48:45,971 INFO  [main-SendThread(10.191.5.225:2181)] zookeeper.ClientCnxn: Opening socket connection to server 10.191.5.225/10.191.5.225:2181. Will not attempt to authenticate using SASL (unknown error)
2018-05-03 21:48:45,971 INFO  [main-SendThread(10.191.5.225:2181)] zookeeper.ClientCnxn: Socket connection established to 10.191.5.225/10.191.5.225:2181, initiating session
2018-05-03 21:48:45,979 INFO  [main-SendThread(10.191.5.225:2181)] zookeeper.ClientCnxn: Session establishment complete on server 10.191.5.225/10.191.5.225:2181, sessionid = 0x1632604ecbc002a, negotiated timeout = 40000
2018-05-03 21:48:46,016 INFO  [main] client.ConnectionManager$HConnectionImplementation: Closing master protocol: MasterService
2018-05-03 21:48:46,016 INFO  [main] client.ConnectionManager$HConnectionImplementation: Closing zookeeper sessionid=0x1632604ecbc002a
2018-05-03 21:48:46,020 INFO  [main-EventThread] zookeeper.ClientCnxn: EventThread shut down
2018-05-03 21:48:46,021 INFO  [main] zookeeper.ZooKeeper: Session: 0x1632604ecbc002a closed
Number of Tables: 59
2018-05-03 21:48:46,032 INFO  [main] util.HBaseFsck: Loading region directories from HDFS
...
2018-05-03 21:48:46,508 INFO  [main] util.HBaseFsck: Loading region information from HDFS
2018-05-03 21:48:46,513 WARN  [hbasefsck-pool1-t49] util.HBaseFsck: No HDFS region dir found: { meta => SYNC_BUSINESS_INFO_BYDAY_EFFECT01,,1524566911288.55c734e322ed22f355d5d27b3bd5c060., hdfs => null, deployed => , replicaId => 0 } meta={ENCODED => 55c734e322ed22f355d5d27b3bd5c060, NAME => 'SYNC_BUSINESS_INFO_BYDAY_EFFECT01,,1524566911288.55c734e322ed22f355d5d27b3bd5c060.', STARTKEY => '', ENDKEY => ''}
2018-05-03 21:48:46,514 WARN  [hbasefsck-pool1-t20] util.HBaseFsck: No HDFS region dir found: { meta => SYNC01_BUSINESS_INFO_BYDAY_EFFECT,,1524576739370.691f074a362b80b6383d8894b659da88., hdfs => null, deployed => , replicaId => 0 } meta={ENCODED => 691f074a362b80b6383d8894b659da88, NAME => 'SYNC01_BUSINESS_INFO_BYDAY_EFFECT,,1524576739370.691f074a362b80b6383d8894b659da88.', STARTKEY => '', ENDKEY => ''}
2018-05-03 21:48:46,515 WARN  [hbasefsck-pool1-t25] util.HBaseFsck: No HDFS region dir found: { meta => SYNC_BUSINESS_INFO_BYDAY_EFFECT02,,1524579217478.7f505d85980d982e5ffb2c368cd948d1., hdfs => null, deployed => , replicaId => 0 } meta={ENCODED => 7f505d85980d982e5ffb2c368cd948d1, NAME => 'SYNC_BUSINESS_INFO_BYDAY_EFFECT02,,1524579217478.7f505d85980d982e5ffb2c368cd948d1.', STARTKEY => '', ENDKEY => ''}
2018-05-03 21:48:46,515 WARN  [hbasefsck-pool1-t28] util.HBaseFsck: No HDFS region dir found: { meta => ZMM_POPULATION,,1524562845934.851660046fc034c48aa58ea5a41808b9., hdfs => null, deployed => , replicaId => 0 } meta={ENCODED => 851660046fc034c48aa58ea5a41808b9, NAME => 'ZMM_POPULATION,,1524562845934.851660046fc034c48aa58ea5a41808b9.', STARTKEY => '', ENDKEY => ''}
.2018-05-03 21:48:46,637 WARN  [hbasefsck-pool1-t7] util.HBaseFsck: No HDFS region dir found: { meta => SYNC_BUSINESS_INFO_BYDAY_EFFECT,,1524575202010.cc93585a767dadcbfedff5e152c1e669., hdfs => null, deployed => , replicaId => 0 } meta={ENCODED => cc93585a767dadcbfedff5e152c1e669, NAME => 'SYNC_BUSINESS_INFO_BYDAY_EFFECT,,1524575202010.cc93585a767dadcbfedff5e152c1e669.', STARTKEY => '', ENDKEY => ''}
2018-05-03 21:48:46,641 WARN  [hbasefsck-pool1-t17] util.HBaseFsck: No HDFS region dir found: { meta => ZMM2_POPULATION,,1524576515935.de0beed9391b6d6ab68dcfd92697b662., hdfs => null, deployed => , replicaId => 0 } meta={ENCODED => de0beed9391b6d6ab68dcfd92697b662, NAME => 'ZMM2_POPULATION,,1524576515935.de0beed9391b6d6ab68dcfd92697b662.', STARTKEY => '', ENDKEY => ''}
2018-05-03 21:48:46,642 WARN  [hbasefsck-pool1-t20] util.HBaseFsck: No HDFS region dir found: { meta => PROD_SALE,,1524567998279.eeb0756bc3c35f22d510502624984ea8., hdfs => null, deployed => , replicaId => 0 } meta={ENCODED => eeb0756bc3c35f22d510502624984ea8, NAME => 'PROD_SALE,,1524567998279.eeb0756bc3c35f22d510502624984ea8.', STARTKEY => '', ENDKEY => ''}

2018-05-03 21:48:46,968 INFO  [main] zookeeper.RecoverableZooKeeper: Process identifier=hconnection-0x2755d705 connecting to ZooKeeper ensemble=10.191.5.225:2181,10.191.5.226:2181,10.191.5.227:2181
2018-05-03 21:48:46,968 INFO  [main] zookeeper.ZooKeeper: Initiating client connection, connectString=10.191.5.225:2181,10.191.5.226:2181,10.191.5.227:2181 sessionTimeout=90000 watcher=hconnection-0x2755d7050x0, quorum=10.191.5.225:2181,10.191.5.226:2181,10.191.5.227:2181, baseZNode=/hbase
2018-05-03 21:48:46,970 INFO  [main-SendThread(10.191.5.225:2181)] zookeeper.ClientCnxn: Opening socket connection to server 10.191.5.225/10.191.5.225:2181. Will not attempt to authenticate using SASL (unknown error)
2018-05-03 21:48:46,971 INFO  [main-SendThread(10.191.5.225:2181)] zookeeper.ClientCnxn: Socket connection established to 10.191.5.225/10.191.5.225:2181, initiating session
2018-05-03 21:48:46,975 INFO  [main-SendThread(10.191.5.225:2181)] zookeeper.ClientCnxn: Session establishment complete on server 10.191.5.225/10.191.5.225:2181, sessionid = 0x1632604ecbc002b, negotiated timeout = 40000
2018-05-03 21:48:46,979 INFO  [main] zookeeper.RecoverableZooKeeper: Process identifier=hbase Fsck connecting to ZooKeeper ensemble=10.191.5.225:2181,10.191.5.226:2181,10.191.5.227:2181
2018-05-03 21:48:46,979 INFO  [main] zookeeper.ZooKeeper: Initiating client connection, connectString=10.191.5.225:2181,10.191.5.226:2181,10.191.5.227:2181 sessionTimeout=90000 watcher=hbase Fsck0x0, quorum=10.191.5.225:2181,10.191.5.226:2181,10.191.5.227:2181, baseZNode=/hbase
2018-05-03 21:48:46,980 INFO  [main-SendThread(10.191.5.226:2181)] zookeeper.ClientCnxn: Opening socket connection to server 10.191.5.226/10.191.5.226:2181. Will not attempt to authenticate using SASL (unknown error)
2018-05-03 21:48:46,981 INFO  [main-SendThread(10.191.5.226:2181)] zookeeper.ClientCnxn: Socket connection established to 10.191.5.226/10.191.5.226:2181, initiating session
2018-05-03 21:48:46,988 INFO  [main-SendThread(10.191.5.226:2181)] zookeeper.ClientCnxn: Session establishment complete on server 10.191.5.226/10.191.5.226:2181, sessionid = 0x2632604e6f7001f, negotiated timeout = 40000
2018-05-03 21:48:47,035 INFO  [main-EventThread] zookeeper.ClientCnxn: EventThread shut down
2018-05-03 21:48:47,036 INFO  [main] zookeeper.ZooKeeper: Session: 0x2632604e6f7001f closed
2018-05-03 21:48:47,036 INFO  [main] client.ConnectionManager$HConnectionImplementation: Closing zookeeper sessionid=0x1632604ecbc002b
2018-05-03 21:48:47,039 INFO  [main-EventThread] zookeeper.ClientCnxn: EventThread shut down
2018-05-03 21:48:47,039 INFO  [main] zookeeper.ZooKeeper: Session: 0x1632604ecbc002b closed
2018-05-03 21:48:47,039 INFO  [main] util.HBaseFsck: Checking and fixing region consistency
ERROR: Region { meta => SYNC_BUSINESS_INFO_BYDAY_EFFECT01,,1524566911288.55c734e322ed22f355d5d27b3bd5c060., hdfs => null, deployed => , replicaId => 0 } found in META, but not in HDFS or deployed on any region server.
ERROR: Region { meta => ZMM_POPULATION,,1524562845934.851660046fc034c48aa58ea5a41808b9., hdfs => null, deployed => , replicaId => 0 } found in META, but not in HDFS or deployed on any region server.
ERROR: Region { meta => SYNC01_BUSINESS_INFO_BYDAY_EFFECT,,1524576739370.691f074a362b80b6383d8894b659da88., hdfs => null, deployed => , replicaId => 0 } found in META, but not in HDFS or deployed on any region server.
ERROR: Region { meta => SYNC_BUSINESS_INFO_BYDAY_EFFECT02,,1524579217478.7f505d85980d982e5ffb2c368cd948d1., hdfs => null, deployed => , replicaId => 0 } found in META, but not in HDFS or deployed on any region server.
ERROR: Region { meta => SYNC_BUSINESS_INFO_BYDAY_EFFECT,,1524575202010.cc93585a767dadcbfedff5e152c1e669., hdfs => null, deployed => , replicaId => 0 } found in META, but not in HDFS or deployed on any region server.
ERROR: Region { meta => ZMM2_POPULATION,,1524576515935.de0beed9391b6d6ab68dcfd92697b662., hdfs => null, deployed => , replicaId => 0 } found in META, but not in HDFS or deployed on any region server.
ERROR: Region { meta => PROD_SALE,,1524567998279.eeb0756bc3c35f22d510502624984ea8., hdfs => null, deployed => , replicaId => 0 } found in META, but not in HDFS or deployed on any region server.
2018-05-03 21:48:47,141 INFO  [hbasefsck-pool1-t26] util.HBaseFsck: Deleted SYNC_BUSINESS_INFO_BYDAY_EFFECT01,,1524566911288.55c734e322ed22f355d5d27b3bd5c060. from META
2018-05-03 21:48:47,142 INFO  [hbasefsck-pool1-t15] util.HBaseFsck: Deleted SYNC01_BUSINESS_INFO_BYDAY_EFFECT,,1524576739370.691f074a362b80b6383d8894b659da88. from META
2018-05-03 21:48:47,142 INFO  [hbasefsck-pool1-t19] util.HBaseFsck: Deleted ZMM_POPULATION,,1524562845934.851660046fc034c48aa58ea5a41808b9. from META
2018-05-03 21:48:47,142 INFO  [hbasefsck-pool1-t37] util.HBaseFsck: Deleted PROD_SALE,,1524567998279.eeb0756bc3c35f22d510502624984ea8. from META
2018-05-03 21:48:47,142 INFO  [hbasefsck-pool1-t35] util.HBaseFsck: Deleted SYNC_BUSINESS_INFO_BYDAY_EFFECT02,,1524579217478.7f505d85980d982e5ffb2c368cd948d1. from META
2018-05-03 21:48:47,142 INFO  [hbasefsck-pool1-t10] util.HBaseFsck: Deleted SYNC_BUSINESS_INFO_BYDAY_EFFECT,,1524575202010.cc93585a767dadcbfedff5e152c1e669. from META
2018-05-03 21:48:47,142 INFO  [hbasefsck-pool1-t4] util.HBaseFsck: Deleted ZMM2_POPULATION,,1524576515935.de0beed9391b6d6ab68dcfd92697b662. from META
2018-05-03 21:48:47,323 INFO  [main] util.HBaseFsck: Handling overlap merges in parallel. set hbasefsck.overlap.merge.parallel to false to run serially.
2018-05-03 21:48:47,341 WARN  [main] util.HBaseFsck: Naming new problem group: 20180124\x0020180125171008535733092133537437
ERROR: (region SYNC_BUSINESS_INFO_BYDAY_IDX_1,20180124\x0020180125171008535733092133537437,1522508011325.4a14624045253764c1056f646bbe6f78.) Multiple regions have the same startkey: 20180124\x0020180125171008535733092133537437
ERROR: (region SYNC_BUSINESS_INFO_BYDAY_IDX_1,20180124\x0020180125171008535733092133537437,1520317313775.7b26d838855e20824e961161d0faed68.) Multiple regions have the same startkey: 20180124\x0020180125171008535733092133537437
ERROR: (regions SYNC_BUSINESS_INFO_BYDAY_IDX_1,20180124\x0020180125171008535733092133537437,1520317313775.7b26d838855e20824e961161d0faed68. and SYNC_BUSINESS_INFO_BYDAY_IDX_1,20180226\x0020180227171513028226357214718886,1522508011325.c3cb4df74c322c55a482d8f9478a0324.) There is an overlap in the region chain.
2018-05-03 21:48:47,341 WARN  [main] util.HBaseFsck: reached end of problem group: null
ERROR: Found inconsistency in table SYNC_BUSINESS_INFO_BYDAY_IDX_1
2018-05-03 21:48:47,350 INFO  [main] util.HBaseFsck: Computing mapping of all store files
..
2018-05-03 21:48:47,558 INFO  [main] util.HBaseFsck: Validating mapping using HDFS state
2018-05-03 21:48:47,573 INFO  [main] zookeeper.RecoverableZooKeeper: Process identifier=hbase Fsck connecting to ZooKeeper ensemble=10.191.5.225:2181,10.191.5.226:2181,10.191.5.227:2181
2018-05-03 21:48:47,574 INFO  [main] zookeeper.ZooKeeper: Initiating client connection, connectString=10.191.5.225:2181,10.191.5.226:2181,10.191.5.227:2181 sessionTimeout=90000 watcher=hbase Fsck0x0, quorum=10.191.5.225:2181,10.191.5.226:2181,10.191.5.227:2181, baseZNode=/hbase
2018-05-03 21:48:47,574 INFO  [main-SendThread(10.191.5.226:2181)] zookeeper.ClientCnxn: Opening socket connection to server 10.191.5.226/10.191.5.226:2181. Will not attempt to authenticate using SASL (unknown error)
2018-05-03 21:48:47,575 INFO  [main-SendThread(10.191.5.226:2181)] zookeeper.ClientCnxn: Socket connection established to 10.191.5.226/10.191.5.226:2181, initiating session
2018-05-03 21:48:47,582 INFO  [main-SendThread(10.191.5.226:2181)] zookeeper.ClientCnxn: Session establishment complete on server 10.191.5.226/10.191.5.226:2181, sessionid = 0x2632604e6f70020, negotiated timeout = 40000
2018-05-03 21:48:47,593 INFO  [main] zookeeper.ZooKeeper: Session: 0x2632604e6f70020 closed
2018-05-03 21:48:47,593 INFO  [main-EventThread] zookeeper.ClientCnxn: EventThread shut down
2018-05-03 21:48:47,594 INFO  [main] zookeeper.RecoverableZooKeeper: Process identifier=hbase Fsck connecting to ZooKeeper ensemble=10.191.5.225:2181,10.191.5.226:2181,10.191.5.227:2181
2018-05-03 21:48:47,594 INFO  [main] zookeeper.ZooKeeper: Initiating client connection, connectString=10.191.5.225:2181,10.191.5.226:2181,10.191.5.227:2181 sessionTimeout=90000 watcher=hbase Fsck0x0, quorum=10.191.5.225:2181,10.191.5.226:2181,10.191.5.227:2181, baseZNode=/hbase
2018-05-03 21:48:47,595 INFO  [main-SendThread(10.191.5.226:2181)] zookeeper.ClientCnxn: Opening socket connection to server 10.191.5.226/10.191.5.226:2181. Will not attempt to authenticate using SASL (unknown error)
2018-05-03 21:48:47,596 INFO  [main-SendThread(10.191.5.226:2181)] zookeeper.ClientCnxn: Socket connection established to 10.191.5.226/10.191.5.226:2181, initiating session
2018-05-03 21:48:47,600 INFO  [main-SendThread(10.191.5.226:2181)] zookeeper.ClientCnxn: Session establishment complete on server 10.191.5.226/10.191.5.226:2181, sessionid = 0x2632604e6f70021, negotiated timeout = 40000
2018-05-03 21:48:47,643 INFO  [main] zookeeper.ZooKeeper: Session: 0x2632604e6f70021 closed
2018-05-03 21:48:47,643 INFO  [main-EventThread] zookeeper.ClientCnxn: EventThread shut down
2018-05-03 21:48:47,757 INFO  [main] util.HBaseFsck: Finishing hbck
Summary:
Table IDX_TARGETCUSTDATA_SERIALNUMBER is okay.
    Number of regions: 1
    Deployed on:  host-10-191-5-225,16020,1525354612066
Table SYN_SALE_PLUS_GOODS is okay.
    Number of regions: 1
    Deployed on:  host-10-191-5-227,16020,1525354610418
Table SYN_SALE_PLUS_STORE is okay.
    Number of regions: 1
    Deployed on:  host-10-191-5-227,16020,1525354610418
Table SYN_SALE_PLUS_TRADE is okay.
    Number of regions: 1
    Deployed on:  host-10-191-5-225,16020,1525354612066
Table PM_PRODUCT is okay.
    Number of regions: 1
    Deployed on:  host-10-191-5-227,16020,1525354610418
Table PM_STORE_CHNL_RELATION_REPAIR is okay.
    Number of regions: 1
    Deployed on:  host-10-191-5-227,16020,1525354610418
Table JDZMD_BUSINESSES_AND_STORES is okay.
    Number of regions: 1
    Deployed on:  host-10-191-5-227,16020,1525354610418
Table OC_ORDER_COMMODITY is okay.
    Number of regions: 1
    Deployed on:  host-10-191-5-225,16020,1525354612066
Table SYNC_TICKET_ACCOUNT_CHECK_INFO is okay.
    Number of regions: 1
    Deployed on:  host-10-191-5-225,16020,1525354612066
Table TARGETCUST_DATA is okay.
    Number of regions: 3
    Deployed on:  host-10-191-5-226,16020,1525354610613
Table TOTAL_LAST_INCOME_DETAIL is okay.
    Number of regions: 1
    Deployed on:  host-10-191-5-227,16020,1525354610418
Table PM_STORE_CHNL_RELATION is okay.
    Number of regions: 1
    Deployed on:  host-10-191-5-225,16020,1525354612066
Table SYN_SALE_PLUS_GOODS_DETAIL_SKU is okay.
    Number of regions: 1
    Deployed on:  host-10-191-5-226,16020,1525354610613
Table EFFECT_INDEX is okay.
    Number of regions: 1
    Deployed on:  host-10-191-5-227,16020,1525354610418
Table TOTAL_INCOME_DETAIL is okay.
    Number of regions: 1
    Deployed on:  host-10-191-5-226,16020,1525354610613
Table JDZMD_STOCK_DETAILS is okay.
    Number of regions: 1
    Deployed on:  host-10-191-5-225,16020,1525354612066
Table hbase:meta is okay.
    Number of regions: 1
    Deployed on:  host-10-191-5-226,16020,1525354610613
Table DEPART_INCOME_COST_DETAIL is okay.
    Number of regions: 1
    Deployed on:  host-10-191-5-225,16020,1525354612066
Table SN_ORDERS_ITEM is okay.
    Number of regions: 1
    Deployed on:  host-10-191-5-226,16020,1525354610613
Table SN_ORDERS is okay.
    Number of regions: 1
    Deployed on:  host-10-191-5-226,16020,1525354610613
Table SYSTEM.CATALOG is okay.
    Number of regions: 1
    Deployed on:  host-10-191-5-225,16020,1525354612066
Table PROD_SALE_TABLE is okay.
    Number of regions: 1
    Deployed on:  host-10-191-5-225,16020,1525354612066
Table SYNC_BUSINESS_INFO_BYYEAR_HUI_STORE is okay.
    Number of regions: 1
    Deployed on:  host-10-191-5-225,16020,1525354612066
Table OC_COMM_BASE_INFO is okay.
    Number of regions: 1
    Deployed on:  host-10-191-5-227,16020,1525354610418
Table SYNC_BUSINESS_INFO_BYDAY_BAK1 is okay.
    Number of regions: 2
    Deployed on:  host-10-191-5-225,16020,1525354612066 host-10-191-5-227,16020,1525354610418
Table JDZMD_AREAS is okay.
    Number of regions: 1
    Deployed on:  host-10-191-5-227,16020,1525354610418
Table JDZMD_STOCK is okay.
    Number of regions: 1
    Deployed on:  host-10-191-5-226,16020,1525354610613
Table JDZMD_ORDERS is okay.
    Number of regions: 1
    Deployed on:  host-10-191-5-227,16020,1525354610418
Table PM_BRAND is okay.
    Number of regions: 1
    Deployed on:  host-10-191-5-226,16020,1525354610613
Table PM_STORE is okay.
    Number of regions: 1
    Deployed on:  host-10-191-5-226,16020,1525354610613
Table JDZMD_SALSER is okay.
    Number of regions: 1
    Deployed on:  host-10-191-5-227,16020,1525354610418
Table SYN_SALE_PLUS_EMPLOYEE is okay.
    Number of regions: 1
    Deployed on:  host-10-191-5-225,16020,1525354612066
Table SYNC_BUSINESS_INFO_BYDAY is okay.
    Number of regions: 5
    Deployed on:  host-10-191-5-225,16020,1525354612066 host-10-191-5-226,16020,1525354610613 host-10-191-5-227,16020,1525354610418
Table SYNC_DEVELOP_INFO_BYDAY_BAK is okay.
    Number of regions: 1
    Deployed on:  host-10-191-5-226,16020,1525354610613
Table SYNC_DEVELOP_INFO_BYDAY is okay.
    Number of regions: 4
    Deployed on:  host-10-191-5-225,16020,1525354612066 host-10-191-5-226,16020,1525354610613 host-10-191-5-227,16020,1525354610418
Table SYNC_COMM_STOCK_IMEI is okay.
    Number of regions: 2
    Deployed on:  host-10-191-5-226,16020,1525354610613
Table PM_COMMODITY is okay.
    Number of regions: 1
    Deployed on:  host-10-191-5-225,16020,1525354612066
Table SYN_SPARK_DEAL_LOG is okay.
    Number of regions: 1
    Deployed on:  host-10-191-5-227,16020,1525354610418
Table OC_ORDER_SALES is okay.
    Number of regions: 1
    Deployed on:  host-10-191-5-225,16020,1525354612066
Table hbase:namespace is okay.
    Number of regions: 1
    Deployed on:  host-10-191-5-227,16020,1525354610418
Table TOTAL_COST_DETAIL is okay.
    Number of regions: 2
    Deployed on:  host-10-191-5-226,16020,1525354610613 host-10-191-5-227,16020,1525354610418
Table SYN_SALE_PLUS_GOODS_DETAIL is okay.
    Number of regions: 1
    Deployed on:  host-10-191-5-225,16020,1525354612066
Table SYNC_SALE_ORDER_INFO is okay.
    Number of regions: 4
    Deployed on:  host-10-191-5-225,16020,1525354612066 host-10-191-5-226,16020,1525354610613
Table SYSTEM.SEQUENCE is okay.
    Number of regions: 1
    Deployed on:  host-10-191-5-225,16020,1525354612066
Table SYSTEM.FUNCTION is okay.
    Number of regions: 1
    Deployed on:  host-10-191-5-225,16020,1525354612066
Table SN_EMPLOYEE_STORE is okay.
    Number of regions: 1
    Deployed on:  host-10-191-5-227,16020,1525354610418
Table SYN_SALE_PLUS_TRADE_DETAIL is okay.
    Number of regions: 1
    Deployed on:  host-10-191-5-227,16020,1525354610418
Table SYNC_DEVELOP_INFO_BYDAY_EFFECT is okay.
    Number of regions: 1
    Deployed on:  host-10-191-5-225,16020,1525354612066
Table SYSTEM.MUTEX is okay.
    Number of regions: 1
    Deployed on:  host-10-191-5-227,16020,1525354610418
Table SYSTEM.STATS is okay.
    Number of regions: 1
    Deployed on:  host-10-191-5-227,16020,1525354610418
Table RC_TOP_SALE_CATEGORY_20180103TMP is okay.
    Number of regions: 1
    Deployed on:  host-10-191-5-227,16020,1525354610418
Table OC_COMM_STOCK is okay.
    Number of regions: 1
    Deployed on:  host-10-191-5-226,16020,1525354610613
Table IDX_TARGETCUST_DATA_PROMO_ID is okay.
    Number of regions: 1
    Deployed on:  host-10-191-5-227,16020,1525354610418
Table SYNC_COMM_STOCK_BARCODE is okay.
    Number of regions: 1
    Deployed on:  host-10-191-5-227,16020,1525354610418
Table SN_STOCK is okay.
    Number of regions: 1
    Deployed on:  host-10-191-5-226,16020,1525354610613
Table SN_STORE is okay.
    Number of regions: 1
    Deployed on:  host-10-191-5-227,16020,1525354610418
Table SYNC_BUSINESS_INFO_BYDAY_HUI_STORE is okay.
    Number of regions: 1
    Deployed on:  host-10-191-5-226,16020,1525354610613
Table SYNC_BUSINESS_INFO_BYDAY_IDX_1 is inconsistent.
    Number of regions: 4
    Deployed on:  host-10-191-5-225,16020,1525354612066 host-10-191-5-226,16020,1525354610613
Table RC_TOP_SALE_CATEGORY is okay.
    Number of regions: 1
    Deployed on:  host-10-191-5-225,16020,1525354612066
Table TOTAL_CURR_INCOME_DETAIL is okay.
    Number of regions: 1
    Deployed on:  host-10-191-5-226,16020,1525354610613
10 inconsistencies detected.
Status: INCONSISTENT
2018-05-03 21:48:47,761 INFO  [main] client.ConnectionManager$HConnectionImplementation: Closing master protocol: MasterService
2018-05-03 21:48:47,762 INFO  [main] client.ConnectionManager$HConnectionImplementation: Closing zookeeper sessionid=0x3632604f954001b
2018-05-03 21:48:47,765 INFO  [main] zookeeper.ZooKeeper: Session: 0x3632604f954001b closed
2018-05-03 21:48:47,765 INFO  [main-EventThread] zookeeper.ClientCnxn: EventThread shut down

备注:通过以上的信息可以知道,已经将上面的表从META即元数据中删掉了.

9.再次在phoenix中执行表创建,没有问题,删除表也没有问题

小结:

　　使用hbase本身的工具可以解决很多的问题,在生产环境中要经常使用hbck检查不一致性的问题.

　　hbase hbck

　　hbase hbck -fixMeta 修复元数据的问题.

文档创建时间:2018年5月4日10:13:16