记一次phoenix在不加索引的情况调优，由6s以上时间变成不到1s
背景：
网约车预约单查询：
这里面恶心的地方是：
1个时间窗口要查询6种时间：推送订单时间（来自mongodb）、有效抢单时间（来自mongodb）、抢单成功时间（实时kafka）、取消订单时间（实时kafka）、到达目的地时间（实时kafka）、支付时间（实时kafka）
根据现实的业务情况，乘客即便打车了，也不是立马支付，很可能过些天才支付。所以不能做表关联，否则结果会不对（开心不）
因此需要做的就是，将一张表拆成6张表，然后做union操作
最后结果存入了Hbase，用phoenix做查询；
优化前，phoenix的查询操作：
SELECT
"TB1".driver_id,
"TB1".driver_name,
"TB1".mobile,
"TB1".alliance_name,
"TB1".register_city,
"TB1".driver_type,
SUM("TB1".pushOrderNum) AS "pushOrderNum" ,
SUM("TB1".effectiveSlogan) AS "effectiveSlogan",
SUM("TB1".grab_sus_order) AS "grab_sus_order",
SUM("TB1".cancel_order_num) AS "cancel_order_num",
SUM("TB1".cancel_passenger_num) AS "cancel_passenger_num",
SUM("TB1".cancel_driver_num) AS "cancel_driver_num",
SUM("TB1".cancel_service_num) AS "cancel_service_num",
SUM("TB1".destination_reached_num) AS "destination_reached_num",
SUM("TB1".pay_num) AS "pay_num",
SUM("TB1".order_cumulative_num) AS "order_cumulative_num",
SUM("TB1".order_pay_cumulative_num) AS "order_pay_cumulative_num"
from
(SELECT
"driver_id" ,
"driver_name" ,
"mobile" ,
"alliance_name" ,
"register_city" ,
"driver_type" ,
sum(to_number("pushOrderNum")) AS "pushOrderNum" ,
sum() AS "effectiveSlogan" ,
sum() AS "grab_sus_order" ,
sum() AS "cancel_order_num",
sum() AS "cancel_passenger_num",
sum() AS "cancel_driver_num",
sum() AS "cancel_service_num" ,
sum() AS "destination_reached_num" ,
sum() AS "pay_num" ,
sum() AS "order_cumulative_num",
sum() AS "order_pay_cumulative_num"
FROM
"_crontab_reservation"
WHERE
CASE WHEN "pushOrderTime" != '' THEN
TO_DATE("pushOrderTime",'yyyy-MM-dd') >= TO_DATE(#{start_time},'yyyy-MM-dd')
AND
TO_DATE("pushOrderTime",'yyyy-MM-dd') <= TO_DATE(#{end_time},'yyyy-MM-dd')
ELSE = END
<if test="driver_id != null and driver_id !=''">
and "driver_id"= #{driver_id}
</if>
<if test="driver_name != null and driver_name !=''">
and "driver_name" = #{driver_name}
</if>
<if test="mobile != null and mobile !=''">
and "mobile" = #{mobile}
</if>
<if test="alliance_name != null and alliance_name !=''">
and "driver_management_id" = #{alliance_name}
</if>
<if test="register_city != null and register_city !=''">
and substr("register_city" , , ) = substr(#{register_city} , , )
</if>
<if test="driver_type != null and driver_type !=''">
and "driver_type" = #{driver_type}
</if>
GROUP BY
"driver_id" ,
"driver_name" ,
"mobile" ,
"alliance_name" ,
"register_city" ,
"driver_type" union all SELECT
"driver_id" ,
"driver_name" ,
"mobile" ,
"alliance_name" ,
"register_city" ,
"driver_type" ,
sum() AS "pushOrderNum" ,
sum(to_number("effectiveSlogan")) AS "effectiveSlogan" ,
sum() AS "grab_sus_order" ,
sum() AS "cancel_order_num",
sum() AS "cancel_passenger_num",
sum() AS "cancel_driver_num",
sum() AS "cancel_service_num" ,
sum() AS "destination_reached_num" ,
sum() AS "pay_num" ,
sum() AS "order_cumulative_num",
sum() AS "order_pay_cumulative_num"
FROM
"_crontab_reservation"
WHERE
CASE WHEN "_effective_glab_time" != '' THEN
TO_DATE("_effective_glab_time",'yyyy-MM-dd') >= TO_DATE(#{start_time},'yyyy-MM-dd')
AND
TO_DATE("_effective_glab_time",'yyyy-MM-dd') <= TO_DATE(#{end_time},'yyyy-MM-dd')
ELSE = END
<if test="driver_id != null and driver_id !=''">
and "driver_id"= #{driver_id}
</if>
<if test="driver_name != null and driver_name !=''">
and "driver_name" = #{driver_name}
</if>
<if test="mobile != null and mobile !=''">
and "mobile" = #{mobile}
</if>
<if test="alliance_name != null and alliance_name !=''">
and "driver_management_id" = #{alliance_name}
</if>
<if test="register_city != null and register_city !=''">
and substr("register_city" , , ) = substr(#{register_city} , , )
</if>
<if test="driver_type != null and driver_type !=''">
and "driver_type" = #{driver_type}
</if>
GROUP BY
"driver_id" ,
"driver_name" ,
"mobile" ,
"alliance_name" ,
"register_city" ,
"driver_type" union all SELECT
"driver_id" ,
"driver_name" ,
"mobile" ,
"alliance_name" ,
"register_city" ,
"driver_type" ,
sum() AS "pushOrderNum" ,
sum() AS "effectiveSlogan" ,
sum(to_number("grab_sus_order")) AS "grab_sus_order" ,
sum() AS "cancel_order_num",
sum() AS "cancel_passenger_num",
sum() AS "cancel_driver_num",
sum() AS "cancel_service_num" ,
sum() AS "destination_reached_num" ,
sum() AS "pay_num" ,
sum() AS "order_cumulative_num",
sum() AS "order_pay_cumulative_num"
FROM
"_crontab_reservation"
WHERE
CASE WHEN "grab_sus_time" != '' THEN
TO_DATE("grab_sus_time" , 'yyyy-MM-dd') >= TO_DATE(#{start_time} , 'yyyy-MM-dd')
AND
TO_DATE("grab_sus_time" , 'yyyy-MM-dd') <= TO_DATE(#{end_time} , 'yyyy-MM-dd')
ELSE = END
<if test="driver_id != null and driver_id !=''">
and "driver_id"= #{driver_id}
</if>
<if test="driver_name != null and driver_name !=''">
and "driver_name" = #{driver_name}
</if>
<if test="mobile != null and mobile !=''">
and "mobile" = #{mobile}
</if>
<if test="alliance_name != null and alliance_name !=''">
and "driver_management_id" = #{alliance_name}
</if>
<if test="register_city != null and register_city !=''">
and substr("register_city" , , ) = substr(#{register_city} , , )
</if>
<if test="driver_type != null and driver_type !=''">
and "driver_type" = #{driver_type}
</if>
GROUP BY
"driver_id" ,
"driver_name" ,
"mobile" ,
"alliance_name" ,
"register_city" ,
"driver_type" union all SELECT
"driver_id" ,
"driver_name" ,
"mobile" ,
"alliance_name" ,
"register_city" ,
"driver_type" ,
sum() AS "pushOrderNum" ,
sum() AS "effectiveSlogan" ,
sum() AS "grab_sus_order" ,
sum(to_number("cancel_order_num")) AS "cancel_order_num",
sum(to_number("cancel_passenger_num")) AS "cancel_passenger_num",
sum(to_number("cancel_driver_num")) AS "cancel_driver_num",
sum(to_number("cancel_service_num")) AS "cancel_service_num" ,
sum() AS "destination_reached_num" ,
sum() AS "pay_num" ,
sum() AS "order_cumulative_num",
sum() AS "order_pay_cumulative_num"
FROM
"_crontab_reservation"
WHERE
CASE WHEN "cancel_time" != '' THEN
TO_DATE("cancel_time",'yyyy-MM-dd') >= TO_DATE(#{start_time},'yyyy-MM-dd')
AND
TO_DATE("cancel_time",'yyyy-MM-dd') <= TO_DATE(#{end_time},'yyyy-MM-dd')
ELSE = END
<if test="driver_id != null and driver_id !=''">
and "driver_id"= #{driver_id}
</if>
<if test="driver_name != null and driver_name !=''">
and "driver_name" = #{driver_name}
</if>
<if test="mobile != null and mobile !=''">
and "mobile" = #{mobile}
</if>
<if test="alliance_name != null and alliance_name !=''">
and "driver_management_id" = #{alliance_name}
</if>
<if test="register_city != null and register_city !=''">
and substr("register_city" , , ) = substr(#{register_city} , , )
</if>
<if test="driver_type != null and driver_type !=''">
and "driver_type" = #{driver_type}
</if>
GROUP BY
"driver_id" ,
"driver_name" ,
"mobile" ,
"alliance_name" ,
"register_city" ,
"driver_type" union all SELECT
"driver_id" ,
"driver_name" ,
"mobile" ,
"alliance_name" ,
"register_city" ,
"driver_type" ,
sum() AS "pushOrderNum" ,
sum() AS "effectiveSlogan" ,
sum() AS "grab_sus_order" ,
sum() AS "cancel_order_num",
sum() AS "cancel_passenger_num",
sum() AS "cancel_driver_num",
sum() AS "cancel_service_num" ,
sum(to_number("destination_reached_num")) AS "destination_reached_num" ,
sum() AS "pay_num" ,
sum() AS "order_cumulative_num",
sum() AS "order_pay_cumulative_num"
FROM
"_crontab_reservation"
WHERE
CASE WHEN "close_gps_time" != '' THEN
TO_DATE("close_gps_time",'yyyy-MM-dd') >= TO_DATE(#{start_time},'yyyy-MM-dd')
AND
TO_DATE("close_gps_time",'yyyy-MM-dd') <= TO_DATE(#{end_time},'yyyy-MM-dd')
ELSE = END
<if test="driver_id != null and driver_id !=''">
and "driver_id"= #{driver_id}
</if>
<if test="driver_name != null and driver_name !=''">
and "driver_name" = #{driver_name}
</if>
<if test="mobile != null and mobile !=''">
and "mobile" = #{mobile}
</if>
<if test="alliance_name != null and alliance_name !=''">
and "driver_management_id" = #{alliance_name}
</if>
<if test="register_city != null and register_city !=''">
and substr("register_city" , , ) = substr(#{register_city} , , )
</if>
<if test="driver_type != null and driver_type !=''">
and "driver_type" = #{driver_type}
</if>
GROUP BY
"driver_id" ,
"driver_name" ,
"mobile" ,
"alliance_name" ,
"register_city" ,
"driver_type" union all SELECT
"driver_id" ,
"driver_name" ,
"mobile" ,
"alliance_name" ,
"register_city" ,
"driver_type" ,
sum() AS "pushOrderNum" ,
sum() AS "effectiveSlogan" ,
sum() AS "grab_sus_order" ,
sum() AS "cancel_order_num",
sum() AS "cancel_passenger_num",
sum() AS "cancel_driver_num",
sum() AS "cancel_service_num" ,
sum() AS "destination_reached_num" ,
sum(to_number("pay_num")) AS "pay_num" ,
sum(to_number("order_cumulative_num")) AS "order_cumulative_num",
sum(to_number("order_pay_cumulative_num")) AS "order_pay_cumulative_num"
FROM
"_crontab_reservation"
WHERE
CASE WHEN "pay_time" != '' THEN
TO_DATE("pay_time",'yyyy-MM-dd') >= TO_DATE(#{start_time},'yyyy-MM-dd')
AND
TO_DATE("pay_time",'yyyy-MM-dd') <= TO_DATE(#{end_time},'yyyy-MM-dd')
ELSE = END
<if test="driver_id != null and driver_id !=''">
and "driver_id"= #{driver_id}
</if>
<if test="driver_name != null and driver_name !=''">
and "driver_name" = #{driver_name}
</if>
<if test="mobile != null and mobile !=''">
and "mobile" = #{mobile}
</if>
<if test="alliance_name != null and alliance_name !=''">
and "driver_management_id" = #{alliance_name}
</if>
<if test="register_city != null and register_city !=''">
and substr("register_city" , , ) = substr(#{register_city} , , )
</if>
<if test="driver_type != null and driver_type !=''">
and "driver_type" = #{driver_type}
</if>
GROUP BY
"driver_id" ,
"driver_name" ,
"mobile" ,
"alliance_name" ,
"register_city" ,
"driver_type"
) AS "TB1"
GROUP BY
"TB1".driver_id ,
"TB1".driver_name ,
"TB1".mobile ,
"TB1".alliance_name ,
"TB1".register_city ,
"TB1".driver_type
优化前的phoenix查询操作
问题：
随着时间推移，不到两个月，数据涨到大概200万左右。每一次查询能慢到怀疑人生。
问题排查：
phoenix的查询语句大概是
select a1 , a2 , sum(a3) , sum(a4) ..... from( select * from table where 条件（推送订单） union all select * from table where 条件（有效抢单） union all select * from table where 条件（抢单成功） union all select * from table where 条件（取消订单） union all select * from table where 条件（到达目的地） union all select * from table where 条件（支付） )
group by a1 , a2
请注意，200万条数据，phoenix不加索引，那么每一个表查询大概花费1秒左右，那么union了6次，就会花费6秒的时间，在加上最后的结果聚合，因此花费的时间超过了6s
并且每个时间字段格式是：yyyy-mm-dd HH:mm:ss
那么每一个司机每天都会有很多的订单，这样就导致数据边的很大。
优化
其实所谓的优化就是瘦表。想办法让phoenix在查询的时候，数据边的少一些，基于这个宗旨开始使用spark做定时任务的优化，然后在写入hbase
//基表
val _reservation = session
.read
.format(GlobalConfigUtils.customHbasePath)
.options(
Map(
GlobalConfigUtils.sparksql_table_schema -> GlobalConfigUtils._spark_reservation,
GlobalConfigUtils.hbase_table_name -> "_crontab_reservation",
GlobalConfigUtils.hbase_table_schema -> GlobalConfigUtils._hbase_reservation
)).load()
_reservation.createOrReplaceTempView("reservation")
//缓存表
session.sqlContext.cacheTable("reservation")
然后迭代出6张小表
//1）：推送订单表
val _push_order_tb: DataFrame = session.sql(Optimization_resevation._push_order_tb)
//2）：有效订单表
val _effective_tb: DataFrame = session.sql(Optimization_resevation._effective_tb)
//3）：抢单成功表
val _success_glab_order: DataFrame = session.sql(Optimization_resevation._success_glab_order)
//4）：取消订单表
val _cancel_order:DataFrame = session.sql(Optimization_resevation._cancel_order)
//5）：到达目的地表
val _destination_reached_order:DataFrame = session.sql(Optimization_resevation._destination_reached_order)
//6）：支付订单表
val _pay_order:DataFrame = session.sql(Optimization_resevation._pay_order)
sql语句就是对每个时间对应的表做时间瘦表，把yyyy-mm-dd HH:mm:ss变成yyyy-mm-dd
lazy val _push_order_tb =
"""
|select
|tb.driver_id ,
|tb.driver_name ,
|tb.mobile ,
|tb.alliance_name ,
|tb.driver_management_id ,
|tb.register_city ,
|tb.driver_type ,
|cast(tb.close_gps_time as string) as close_gps_time,
|cast(tb.pushOrderTime as string) as pushOrderTime ,
|cast(tb._effective_glab_time as string) as _effective_glab_time ,
|cast(tb.pay_time as string) as pay_time,
|cast(tb.cancel_time as string) as cancel_time,
|cast(tb.grab_sus_time as string) as grab_sus_time ,
|sum(tb.pushOrderNum) as pushOrderNum ,
|sum(0) AS effectiveSlogan ,
|sum(0) AS grab_sus_order ,
|sum(0) AS cancel_order_num,
|sum(0) AS cancel_passenger_num,
|sum(0) AS cancel_driver_num,
|sum(0) AS cancel_service_num ,
|sum(0) AS destination_reached_num ,
|sum(0) AS pay_num ,
|sum(0) AS order_cumulative_num,
|sum(0) AS order_pay_cumulative_num
|from
|(
|select
|driver_id ,
|driver_name ,
|mobile ,
|alliance_name ,
|driver_management_id ,
|register_city ,
|driver_type ,
|0 as close_gps_time ,
|date_format(pushOrderTime,'yyyy-MM-dd') as pushOrderTime ,
|0 as _effective_glab_time ,
|0 as pay_time ,
|0 as cancel_time ,
|0 as grab_sus_time ,
|pushOrderNum ,
|0 AS effectiveSlogan ,
|0 AS grab_sus_order ,
|0 AS cancel_order_num,
|0 AS cancel_passenger_num,
|0 AS cancel_driver_num,
|0 AS cancel_service_num ,
|0 AS destination_reached_num ,
|0 AS pay_num ,
|0 AS order_cumulative_num,
|0 AS order_pay_cumulative_num
|from
|reservation
|where
|pushOrderTime != '0'
|and
|cast(date_format(pushOrderTime , 'yyyyMMdd') as bigint) < cast(date_format(now() , 'yyyyMMdd') as bigint)
|) tb
|GROUP BY
|tb.driver_id ,
|tb.driver_name ,
|tb.mobile ,
|tb.alliance_name ,
|tb.driver_management_id ,
|tb.register_city ,
|tb.driver_type ,
|tb.close_gps_time ,
|tb.pushOrderTime ,
|tb._effective_glab_time ,
|tb.pay_time ,
|tb.cancel_time ,
|tb.grab_sus_time
""".stripMargin
然后数据写入hbase
dF.write
.mode(SaveMode.Append)
.options(Map(HBaseTableCatalog.tableCatalog -> catalog , HBaseTableCatalog.newTable -> "8"))
.format("org.apache.spark.sql.execution.datasources.hbase")
.save()
代码其实就是将一个表拆成6张表（对应每个不同的时间维度），然后将时间yyyy-mm-dd HH:mm:ss中的数据按照司机ID和yyyy-mm-dd做分组求和。这样落到hbase就是6张小表了。
然后phoenix在去分别查询6张小表，在聚合起来即可；
这样200万的数据经过优化后，在不加索引的前提下查询不到1s；
最后的phoenix查询语句就是分别查6个小表：
SELECT
"TB1".driver_id,
"TB1".driver_name,
"TB1".mobile,
"TB1".alliance_name,
"TB1".register_city,
"TB1".driver_type,
SUM("TB1".pushOrderNum) AS "pushOrderNum" ,
SUM("TB1".effectiveSlogan) AS "effectiveSlogan",
SUM("TB1".grab_sus_order) AS "grab_sus_order",
SUM("TB1".cancel_order_num) AS "cancel_order_num",
SUM("TB1".cancel_passenger_num) AS "cancel_passenger_num",
SUM("TB1".cancel_driver_num) AS "cancel_driver_num",
SUM("TB1".cancel_service_num) AS "cancel_service_num",
SUM("TB1".destination_reached_num) AS "destination_reached_num",
SUM("TB1".pay_num) AS "pay_num",
SUM("TB1".order_cumulative_num) AS "order_cumulative_num",
SUM("TB1".order_pay_cumulative_num) AS "order_pay_cumulative_num"
from
(SELECT
"driver_id" ,
"driver_name" ,
"mobile" ,
"alliance_name" ,
"register_city" ,
"driver_type" ,
sum(to_number("pushOrderNum")) AS "pushOrderNum" ,
sum(0) AS "effectiveSlogan" ,
sum(0) AS "grab_sus_order" ,
sum(0) AS "cancel_order_num",
sum(0) AS "cancel_passenger_num",
sum(0) AS "cancel_driver_num",
sum(0) AS "cancel_service_num" ,
sum(0) AS "destination_reached_num" ,
sum(0) AS "pay_num" ,
sum(0) AS "order_cumulative_num",
sum(0) AS "order_pay_cumulative_num"
FROM
"_push_order_tb"
WHERE
CASE WHEN "pushOrderTime" != '0' THEN
TO_DATE("pushOrderTime",'yyyy-MM-dd') >= TO_DATE(#{start_time},'yyyy-MM-dd')
AND
TO_DATE("pushOrderTime",'yyyy-MM-dd') <= TO_DATE(#{end_time},'yyyy-MM-dd')
ELSE 1=1 END
<if test="driver_id != null and driver_id !=''">
and "driver_id"= #{driver_id}
</if>
<if test="driver_name != null and driver_name !=''">
and "driver_name" = #{driver_name}
</if>
<if test="mobile != null and mobile !=''">
and "mobile" = #{mobile}
</if>
<if test="alliance_name != null and alliance_name !=''">
and "driver_management_id" = #{alliance_name}
</if>
<if test="register_city != null and register_city !=''">
and substr("register_city" , 0 , 4) = substr(#{register_city} , 0 , 4)
</if>
<if test="driver_type != null and driver_type !=''">
and "driver_type" = #{driver_type}
</if>
GROUP BY
"driver_id" ,
"driver_name" ,
"mobile" ,
"alliance_name" ,
"register_city" ,
"driver_type" union all SELECT
"driver_id" ,
"driver_name" ,
"mobile" ,
"alliance_name" ,
"register_city" ,
"driver_type" ,
sum(0) AS "pushOrderNum" ,
sum(to_number("effectiveSlogan")) AS "effectiveSlogan" ,
sum(0) AS "grab_sus_order" ,
sum(0) AS "cancel_order_num",
sum(0) AS "cancel_passenger_num",
sum(0) AS "cancel_driver_num",
sum(0) AS "cancel_service_num" ,
sum(0) AS "destination_reached_num" ,
sum(0) AS "pay_num" ,
sum(0) AS "order_cumulative_num",
sum(0) AS "order_pay_cumulative_num"
FROM
"_effective_tb"
WHERE
CASE WHEN "_effective_glab_time" != '0' THEN
TO_DATE("_effective_glab_time",'yyyy-MM-dd') >= TO_DATE(#{start_time},'yyyy-MM-dd')
AND
TO_DATE("_effective_glab_time",'yyyy-MM-dd') <= TO_DATE(#{end_time},'yyyy-MM-dd')
ELSE 1=1 END
<if test="driver_id != null and driver_id !=''">
and "driver_id"= #{driver_id}
</if>
<if test="driver_name != null and driver_name !=''">
and "driver_name" = #{driver_name}
</if>
<if test="mobile != null and mobile !=''">
and "mobile" = #{mobile}
</if>
<if test="alliance_name != null and alliance_name !=''">
and "driver_management_id" = #{alliance_name}
</if>
<if test="register_city != null and register_city !=''">
and substr("register_city" , 0 , 4) = substr(#{register_city} , 0 , 4)
</if>
<if test="driver_type != null and driver_type !=''">
and "driver_type" = #{driver_type}
</if>
GROUP BY
"driver_id" ,
"driver_name" ,
"mobile" ,
"alliance_name" ,
"register_city" ,
"driver_type" union all SELECT
"driver_id" ,
"driver_name" ,
"mobile" ,
"alliance_name" ,
"register_city" ,
"driver_type" ,
sum(0) AS "pushOrderNum" ,
sum(0) AS "effectiveSlogan" ,
sum(to_number("grab_sus_order")) AS "grab_sus_order" ,
sum(0) AS "cancel_order_num",
sum(0) AS "cancel_passenger_num",
sum(0) AS "cancel_driver_num",
sum(0) AS "cancel_service_num" ,
sum(0) AS "destination_reached_num" ,
sum(0) AS "pay_num" ,
sum(0) AS "order_cumulative_num",
sum(0) AS "order_pay_cumulative_num"
FROM
"_success_glab_order_tb"
WHERE
CASE WHEN "grab_sus_time" != '0' THEN
TO_DATE("grab_sus_time" , 'yyyy-MM-dd') >= TO_DATE(#{start_time} , 'yyyy-MM-dd')
AND
TO_DATE("grab_sus_time" , 'yyyy-MM-dd') <= TO_DATE(#{end_time} , 'yyyy-MM-dd')
ELSE 1=1 END
<if test="driver_id != null and driver_id !=''">
and "driver_id"= #{driver_id}
</if>
<if test="driver_name != null and driver_name !=''">
and "driver_name" = #{driver_name}
</if>
<if test="mobile != null and mobile !=''">
and "mobile" = #{mobile}
</if>
<if test="alliance_name != null and alliance_name !=''">
and "driver_management_id" = #{alliance_name}
</if>
<if test="register_city != null and register_city !=''">
and substr("register_city" , 0 , 4) = substr(#{register_city} , 0 , 4)
</if>
<if test="driver_type != null and driver_type !=''">
and "driver_type" = #{driver_type}
</if>
GROUP BY
"driver_id" ,
"driver_name" ,
"mobile" ,
"alliance_name" ,
"register_city" ,
"driver_type" union all SELECT
"driver_id" ,
"driver_name" ,
"mobile" ,
"alliance_name" ,
"register_city" ,
"driver_type" ,
sum(0) AS "pushOrderNum" ,
sum(0) AS "effectiveSlogan" ,
sum(0) AS "grab_sus_order" ,
sum(to_number("cancel_order_num")) AS "cancel_order_num",
sum(to_number("cancel_passenger_num")) AS "cancel_passenger_num",
sum(to_number("cancel_driver_num")) AS "cancel_driver_num",
sum(to_number("cancel_service_num")) AS "cancel_service_num" ,
sum(0) AS "destination_reached_num" ,
sum(0) AS "pay_num" ,
sum(0) AS "order_cumulative_num",
sum(0) AS "order_pay_cumulative_num"
FROM
"_cancel_order_tb"
WHERE
CASE WHEN "cancel_time" != '0' THEN
TO_DATE("cancel_time",'yyyy-MM-dd') >= TO_DATE(#{start_time},'yyyy-MM-dd')
AND
TO_DATE("cancel_time",'yyyy-MM-dd') <= TO_DATE(#{end_time},'yyyy-MM-dd')
ELSE 1=1 END
<if test="driver_id != null and driver_id !=''">
and "driver_id"= #{driver_id}
</if>
<if test="driver_name != null and driver_name !=''">
and "driver_name" = #{driver_name}
</if>
<if test="mobile != null and mobile !=''">
and "mobile" = #{mobile}
</if>
<if test="alliance_name != null and alliance_name !=''">
and "driver_management_id" = #{alliance_name}
</if>
<if test="register_city != null and register_city !=''">
and substr("register_city" , 0 , 4) = substr(#{register_city} , 0 , 4)
</if>
<if test="driver_type != null and driver_type !=''">
and "driver_type" = #{driver_type}
</if>
GROUP BY
"driver_id" ,
"driver_name" ,
"mobile" ,
"alliance_name" ,
"register_city" ,
"driver_type" union all SELECT
"driver_id" ,
"driver_name" ,
"mobile" ,
"alliance_name" ,
"register_city" ,
"driver_type" ,
sum(0) AS "pushOrderNum" ,
sum(0) AS "effectiveSlogan" ,
sum(0) AS "grab_sus_order" ,
sum(0) AS "cancel_order_num",
sum(0) AS "cancel_passenger_num",
sum(0) AS "cancel_driver_num",
sum(0) AS "cancel_service_num" ,
sum(to_number("destination_reached_num")) AS "destination_reached_num" ,
sum(0) AS "pay_num" ,
sum(0) AS "order_cumulative_num",
sum(0) AS "order_pay_cumulative_num"
FROM
"_destination_reached_order_tb"
WHERE
CASE WHEN "close_gps_time" != '0' THEN
TO_DATE("close_gps_time",'yyyy-MM-dd') >= TO_DATE(#{start_time},'yyyy-MM-dd')
AND
TO_DATE("close_gps_time",'yyyy-MM-dd') <= TO_DATE(#{end_time},'yyyy-MM-dd')
ELSE 1=1 END
<if test="driver_id != null and driver_id !=''">
and "driver_id"= #{driver_id}
</if>
<if test="driver_name != null and driver_name !=''">
and "driver_name" = #{driver_name}
</if>
<if test="mobile != null and mobile !=''">
and "mobile" = #{mobile}
</if>
<if test="alliance_name != null and alliance_name !=''">
and "driver_management_id" = #{alliance_name}
</if>
<if test="register_city != null and register_city !=''">
and substr("register_city" , 0 , 4) = substr(#{register_city} , 0 , 4)
</if>
<if test="driver_type != null and driver_type !=''">
and "driver_type" = #{driver_type}
</if>
GROUP BY
"driver_id" ,
"driver_name" ,
"mobile" ,
"alliance_name" ,
"register_city" ,
"driver_type" union all SELECT
"driver_id" ,
"driver_name" ,
"mobile" ,
"alliance_name" ,
"register_city" ,
"driver_type" ,
sum(0) AS "pushOrderNum" ,
sum(0) AS "effectiveSlogan" ,
sum(0) AS "grab_sus_order" ,
sum(0) AS "cancel_order_num",
sum(0) AS "cancel_passenger_num",
sum(0) AS "cancel_driver_num",
sum(0) AS "cancel_service_num" ,
sum(0) AS "destination_reached_num" ,
sum(to_number("pay_num")) AS "pay_num" ,
sum(to_number("order_cumulative_num")) AS "order_cumulative_num",
sum(to_number("order_pay_cumulative_num")) AS "order_pay_cumulative_num"
FROM
"_pay_order_tb"
WHERE
CASE WHEN "pay_time" != '0' THEN
TO_DATE("pay_time",'yyyy-MM-dd') >= TO_DATE(#{start_time},'yyyy-MM-dd')
AND
TO_DATE("pay_time",'yyyy-MM-dd') <= TO_DATE(#{end_time},'yyyy-MM-dd')
ELSE 1=1 END
<if test="driver_id != null and driver_id !=''">
and "driver_id"= #{driver_id}
</if>
<if test="driver_name != null and driver_name !=''">
and "driver_name" = #{driver_name}
</if>
<if test="mobile != null and mobile !=''">
and "mobile" = #{mobile}
</if>
<if test="alliance_name != null and alliance_name !=''">
and "driver_management_id" = #{alliance_name}
</if>
<if test="register_city != null and register_city !=''">
and substr("register_city" , 0 , 4) = substr(#{register_city} , 0 , 4)
</if>
<if test="driver_type != null and driver_type !=''">
and "driver_type" = #{driver_type}
</if>
GROUP BY
"driver_id" ,
"driver_name" ,
"mobile" ,
"alliance_name" ,
"register_city" ,
"driver_type"
) AS "TB1"
GROUP BY
"TB1".driver_id ,
"TB1".driver_name ,
"TB1".mobile ,
"TB1".alliance_name ,
"TB1".register_city ,
"TB1".driver_type
瘦表后的查询
记一次phoenix在不加索引的情况调优，由6s以上时间变成不到1s的更多相关文章
- MySQL索引和SQL调优手册
MySQL索引 MySQL支持诸多存储引擎,而各种存储引擎对索引的支持也各不相同,因此MySQL数据库支持多种索引类型,如BTree索引,哈希索引,全文索引等等.为了避免混乱,本文将只关注于BTree ...
- mysql 索引优化 性能调优 锁
1 检查mysql 是否安装 rpm -qa|grep -i mysql 2 ntsysv 查看和设置开机启动列表 3 mysql 在 centos 上默认 的数据目录是 /var/lib/mysql ...
- 记一次令人窒息的线上fullgc调优
今天第二篇采坑了... ... 现场因为处理太急促没有保留,而且是一旁协助,没有收集到所有信息实在是有些遗憾...只能靠记忆回想一些细节 情况是一台服务器一启动就开始full gc,短短1分钟可以有几 ...
- SqlServer之like、charindex、patindex 在有无索引的情况下分析
1.环境介绍 测试环境 SQL2005 测试数据 200W条 2.环境准备 2.1建表 CREATE TABLE [dbo].[Depratments]( [Dep_id] [int] ...
- 《高性能SQL调优精要与案例解析》一书谈主流关系库SQL调优（SQL TUNING或SQL优化）核心机制之——索引（index）
继<高性能SQL调优精要与案例解析>一书谈SQL调优(SQL TUNING或SQL优化),我们今天就谈谈各主流关系库中,占据SQL调优技术和工作半壁江山的.最重要的核心机制之一——索引(i ...
- Oracle外键不加索引会引起死锁问题
转载链接:http://www.jb51.net/article/50161.htm 这篇文章主要介绍了Oracle外键不加索引引起死锁的情况及解决,需要的朋友可以参考下 --创建一个表,此表作为子表 ...
- postgresql大数据查询加索引和不加索引耗时总结
1.创建测试表 CREATE TABLE big_data( id character varying(50) NOT NULL, name character varying(50), dat ...
- Phoenix表和索引分区优化方法
Phoenix表和索引分区,基本优化方法 优化方法 1. SALT_BUCKETS RowKey SALT_BUCKETS 分区 2. Pre-split RowKey分区 3. 分列族 4. 使用压 ...
- SQL Server调优系列基础篇（索引运算总结）
前言 上几篇文章我们介绍了如何查看查询计划.常用运算符的介绍.并行运算的方式,有兴趣的可以点击查看. 本篇将分析在SQL Server中,如何利用先有索引项进行查询性能优化,通过了解这些索引项的应用方 ...
随机推荐
- MongoDB的复合唯一索引
一 创建 JavaScript Shell db.room.ensureIndex({'floor':1,'num':1}) Spring Data @Data // lombok @Document ...
- Tag Helper1
Tag Helpers是服务器段的C#代码,在Razor文件里,参与到创建和渲染HTML元素的过程 和HTML Helpers类似 跟HTML的命名规范一致 内置了很多Tag Helpers也可以自定 ...
- luogu1313计算系数题解--二项式定理
题目链接 https://www.luogu.org/problemnew/show/P1313 分析 二项式定理 \((a+b)^n=\sum_{k=0}^{n}{C^k_n a^k b^{n-k} ...
- python简答
解释 GIL 全局解释器锁 def func(*args): for i in args: print(i) func(3,2,1,4,7) 在我们不知道该传递多少关键字参数时,使用**kwargs ...
- JS-实现横向手风琴
横向手风琴-- 鼠标悬浮某一张图片,图片显示,其他图片影藏. <div class="content"> <ul> <li class="c ...
- Jboss未授权访问部署木马 利用exp
查看系统名称 java -jar jboss_exploit_fat.jar -i http://www.any.com:8080/invoker/JMXInvokerServlet get jbos ...
- 7.移动端自动化测试-小知识 try...except...finally语句
异常Error 我们在写代码的时候,经常会遇见程序抛出Error无法执行的情况 一般情况下,在Python无法正常处理程序时就会发生一个异常.异常是Python对象,表示一个错误.当Python脚本发 ...
- MySQL8.0 caching_sha2_password报错问题
在bin目录下执行mysql -uroot -p123456 登录后执行: use mysql; select host, user, plugin from user; 打印: +--------- ...
- Windows下同时安装两个版本Jdk
在项目开发中遇到了jdk版本切换的问题,于是尝试在电脑中安装jdk1.6和jdk1.7,话不多说马上开始 1 准备好两个版本的jdk路径 2 设置两个JAVA_HOME 3 设置总的动态切换的JAVA ...
- 万能模拟器eve-ng介绍
作为一名网络工程师,通过模拟器来搭建实验环境是必不可少的,主流的网络模拟器有如下几款:Packet Tracer是思科公司出的一款模拟器,只能模拟思科自己的设备,由于此款软件是纯软件模拟,所以思科设备 ...