mysql 查询语句去重 通过sql去重
url 在表中具有唯一性；
由于定时任务的多进程/代码的多线程/定时任务的调度/脚本的执行时间，导致，破坏了唯一性；
SELECT COUNT(1) FROM test_error_tmp
UNION ALL
SELECT COUNT(DISTINCT url) FROM test_error_tmp;
COUNT(1)
40972
40910
差62
SELECT * FROM (
SELECT COUNT(1) AS c,url FROM test_error_tmp GROUP BY url
) AS tab
WHERE c>1;
;
c url
2 b2b.88152.com/show-2677485.html
2 b2b.88152.com/show-2989753.html
2 b2b.88152.com/show-2994375.html
2 b2b.88152.com/show-3071983.html
2 haikou.kuyiso.com/huizhan/29251188x.htm
2 info.123cha.com/show_1007121.html
2 info.b2b168.com/s168-48159518.html
2 info.b2b168.com/s168-58451934.html
2 info.b2b168.com/s168-60677934.html
2 lanzhou.kuyiso.com/canyin/1328.htm
2 site.leshou.com/s/39010445.html
2 site.leshou.com/s/40131383.html
2 site.leshou.com/s/45062793.html
2 site.leshou.com/s/45714581.html
2 site.leshou.com/s/46021681.html
2 site.leshou.com/s/49557679.html
2 site.leshou.com/s/50777239.html
2 site.leshou.com/s/52407278.html
2 wap.ailaba.org/sell/182965618167809.html
2 www.01ny.cn/xinxi/25324778.html
2 www.01ny.cn/xinxi/37941846.html
2 www.05188.com/fenlei/22739365.html
2 www.51sole.com/b2b/pd_26691448.htm
2 www.587766.com/gongqiu/show_2096363.html
2 www.afinance.cn/fenlei/25435008.html
2 www.afinance.cn/fenlei/37074884.html
2 www.afinance.cn/fenlei/38574413.html
2 www.ailaba.org/sell/179521059297846.html
2 www.ailaba.org/sell/183760178499696.html
2 www.bzcm.net/xinxi/29968353.html
2 www.cnhan.com/fenlei/34281043.html
2 www.cntrades.com/b2b/hnszbjh/sell/itemid-85394203.html
2 www.gtobal.com/?m=trade&c=detail&a=preview&id=4930157402&type=2
2 www.gtobal.com/?m=trade&c=detail&a=preview&id=5060488002&type=2
2 www.hizh.cn/xinxi/10229031.html
2 www.hizh.cn/xinxi/3524483.html
2 www.kuyiso.com/html/101/ershoushebei/18348452x.htm
2 www.kuyiso.com/html/21/zhiyepx/29045579x.htm
2 www.laiwunews.cn/xinxi/13358851.html
2 www.laiwunews.cn/xinxi/18715417.html
2 www.laiwunews.cn/xinxi/18774478.html
2 www.laiwunews.cn/xinxi/18917168.html
2 www.laiwunews.cn/xinxi/19054895.html
2 www.laiwunews.cn/xinxi/20974600.html
2 www.laiwunews.cn/xinxi/20975830.html
2 www.laiwunews.cn/xinxi/21115975.html
2 www.laiwunews.cn/xinxi/21361687.html
2 www.laiwunews.cn/xinxi/21416448.html
2 www.laiwunews.cn/xinxi/21512940.html
2 www.laiwunews.cn/xinxi/25450210.html
2 www.laiwunews.cn/xinxi/27248072.html
2 www.laiwunews.cn/xinxi/27340981.html
2 www.qhnews.com/bianmin/30874042.html
2 www.qhnews.com/bianmin/32388347.html
2 www.qhnews.com/bianmin/32550280.html
2 www.qixiangwang.cn/fenlei/20773619.html
2 www.qiyegu.com/sell/201412/13/deatil_10657611.html
2 www.sooshong.com/chxchx/offerdetail-150433811437054.html
2 www.waaku.com/infoview11788636.html
2 www.yuqinge.com/xinxi/4683460.html
2 www.zznews.cn/xinxi/21212448.html
2 yt.qu114.com/zhuanli/37012081.html
恰好共计62条
现在通过sql删除62条，注意不是62*2条
删除每个url出现晚的数据行
先选出主键id，再执行删除
SELECT id,url FROM test_error_tmp WHERE url IN
(
SELECT url FROM
(
SELECT COUNT(1) AS c,url FROM test_error_tmp GROUP BY url
) AS tab
WHERE c>1
)
;
id url
2204 b2b.88152.com/show-2677485.html
8647 b2b.88152.com/show-2677485.html
2293 b2b.88152.com/show-2989753.html
8426 b2b.88152.com/show-2989753.html
2318 b2b.88152.com/show-2994375.html
8654 b2b.88152.com/show-2994375.html
2220 b2b.88152.com/show-3071983.html
8662 b2b.88152.com/show-3071983.html
1264 haikou.kuyiso.com/huizhan/29251188x.htm
8477 haikou.kuyiso.com/huizhan/29251188x.htm
1935 info.123cha.com/show_1007121.html
8603 info.123cha.com/show_1007121.html
1887 info.b2b168.com/s168-48159518.html
9577 info.b2b168.com/s168-48159518.html
3000 info.b2b168.com/s168-58451934.html
8474 info.b2b168.com/s168-58451934.html
9428 info.b2b168.com/s168-60677934.html
9431 info.b2b168.com/s168-60677934.html
2210 lanzhou.kuyiso.com/canyin/1328.htm
8674 lanzhou.kuyiso.com/canyin/1328.htm
1909 site.leshou.com/s/39010445.html
9076 site.leshou.com/s/39010445.html
1867 site.leshou.com/s/40131383.html
9089 site.leshou.com/s/40131383.html
1885 site.leshou.com/s/45062793.html
9086 site.leshou.com/s/45062793.html
1859 site.leshou.com/s/45714581.html
9090 site.leshou.com/s/45714581.html
1889 site.leshou.com/s/46021681.html
9084 site.leshou.com/s/46021681.html
1900 site.leshou.com/s/49557679.html
9081 site.leshou.com/s/49557679.html
1873 site.leshou.com/s/50777239.html
9088 site.leshou.com/s/50777239.html
1904 site.leshou.com/s/52407278.html
9079 site.leshou.com/s/52407278.html
2992 wap.ailaba.org/sell/182965618167809.html
8475 wap.ailaba.org/sell/182965618167809.html
1610 www.01ny.cn/xinxi/25324778.html
9539 www.01ny.cn/xinxi/25324778.html
9536 www.01ny.cn/xinxi/37941846.html
9543 www.01ny.cn/xinxi/37941846.html
2282 www.05188.com/fenlei/22739365.html
9288 www.05188.com/fenlei/22739365.html
1913 www.51sole.com/b2b/pd_26691448.htm
8449 www.51sole.com/b2b/pd_26691448.htm
1230 www.587766.com/gongqiu/show_2096363.html
8480 www.587766.com/gongqiu/show_2096363.html
2275 www.afinance.cn/fenlei/25435008.html
9304 www.afinance.cn/fenlei/25435008.html
10328 www.afinance.cn/fenlei/37074884.html
10329 www.afinance.cn/fenlei/37074884.html
1917 www.afinance.cn/fenlei/38574413.html
8440 www.afinance.cn/fenlei/38574413.html
1924 www.ailaba.org/sell/179521059297846.html
8421 www.ailaba.org/sell/179521059297846.html
7276 www.ailaba.org/sell/183760178499696.html
9114 www.ailaba.org/sell/183760178499696.html
1222 www.bzcm.net/xinxi/29968353.html
8481 www.bzcm.net/xinxi/29968353.html
2954 www.cnhan.com/fenlei/34281043.html
8476 www.cnhan.com/fenlei/34281043.html
1276 www.cntrades.com/b2b/hnszbjh/sell/itemid-85394203.html
8428 www.cntrades.com/b2b/hnszbjh/sell/itemid-85394203.html
2262 www.gtobal.com/?m=trade&c=detail&a=preview&id=4930157402&type=2
9414 www.gtobal.com/?m=trade&c=detail&a=preview&id=4930157402&type=2
9450 www.gtobal.com/?m=trade&c=detail&a=preview&id=5060488002&type=2
9452 www.gtobal.com/?m=trade&c=detail&a=preview&id=5060488002&type=2
2271 www.hizh.cn/xinxi/10229031.html
9311 www.hizh.cn/xinxi/10229031.html
2043 www.hizh.cn/xinxi/3524483.html
9360 www.hizh.cn/xinxi/3524483.html
9566 www.kuyiso.com/html/101/ershoushebei/18348452x.htm
9583 www.kuyiso.com/html/101/ershoushebei/18348452x.htm
9515 www.kuyiso.com/html/21/zhiyepx/29045579x.htm
9522 www.kuyiso.com/html/21/zhiyepx/29045579x.htm
1293 www.laiwunews.cn/xinxi/13358851.html
8416 www.laiwunews.cn/xinxi/13358851.html
2213 www.laiwunews.cn/xinxi/18715417.html
8670 www.laiwunews.cn/xinxi/18715417.html
3181 www.laiwunews.cn/xinxi/18774478.html
8299 www.laiwunews.cn/xinxi/18774478.html
2259 www.laiwunews.cn/xinxi/18917168.html
9417 www.laiwunews.cn/xinxi/18917168.html
2280 www.laiwunews.cn/xinxi/19054895.html
9293 www.laiwunews.cn/xinxi/19054895.html
1282 www.laiwunews.cn/xinxi/20974600.html
8427 www.laiwunews.cn/xinxi/20974600.html
2217 www.laiwunews.cn/xinxi/20975830.html
8664 www.laiwunews.cn/xinxi/20975830.html
2256 www.laiwunews.cn/xinxi/21115975.html
9422 www.laiwunews.cn/xinxi/21115975.html
2211 www.laiwunews.cn/xinxi/21361687.html
8672 www.laiwunews.cn/xinxi/21361687.html
2214 www.laiwunews.cn/xinxi/21416448.html
8669 www.laiwunews.cn/xinxi/21416448.html
2272 www.laiwunews.cn/xinxi/21512940.html
9403 www.laiwunews.cn/xinxi/21512940.html
2267 www.laiwunews.cn/xinxi/25450210.html
9407 www.laiwunews.cn/xinxi/25450210.html
2290 www.laiwunews.cn/xinxi/27248072.html
9387 www.laiwunews.cn/xinxi/27248072.html
2288 www.laiwunews.cn/xinxi/27340981.html
9390 www.laiwunews.cn/xinxi/27340981.html
3825 www.qhnews.com/bianmin/30874042.html
8303 www.qhnews.com/bianmin/30874042.html
9559 www.qhnews.com/bianmin/32388347.html
9572 www.qhnews.com/bianmin/32388347.html
2541 www.qhnews.com/bianmin/32550280.html
8442 www.qhnews.com/bianmin/32550280.html
2564 www.qixiangwang.cn/fenlei/20773619.html
8288 www.qixiangwang.cn/fenlei/20773619.html
2946 www.qiyegu.com/sell/201412/13/deatil_10657611.html
8478 www.qiyegu.com/sell/201412/13/deatil_10657611.html
1992 www.sooshong.com/chxchx/offerdetail-150433811437054.html
8614 www.sooshong.com/chxchx/offerdetail-150433811437054.html
1299 www.waaku.com/infoview11788636.html
8414 www.waaku.com/infoview11788636.html
2297 www.yuqinge.com/xinxi/4683460.html
9257 www.yuqinge.com/xinxi/4683460.html
2215 www.zznews.cn/xinxi/21212448.html
8666 www.zznews.cn/xinxi/21212448.html
9425 yt.qu114.com/zhuanli/37012081.html
9426 yt.qu114.com/zhuanli/37012081.html
2204 b2b.88152.com/show-2677485.html
8647 b2b.88152.com/show-2677485.html
2293 b2b.88152.com/show-2989753.html
8426 b2b.88152.com/show-2989753.html
2318 b2b.88152.com/show-2994375.html
8654 b2b.88152.com/show-2994375.html
2220 b2b.88152.com/show-3071983.html
8662 b2b.88152.com/show-3071983.html
1264 haikou.kuyiso.com/huizhan/29251188x.htm
8477 haikou.kuyiso.com/huizhan/29251188x.htm
1935 info.123cha.com/show_1007121.html
8603 info.123cha.com/show_1007121.html
1887 info.b2b168.com/s168-48159518.html
9577 info.b2b168.com/s168-48159518.html
3000 info.b2b168.com/s168-58451934.html
8474 info.b2b168.com/s168-58451934.html
9428 info.b2b168.com/s168-60677934.html
9431 info.b2b168.com/s168-60677934.html
2210 lanzhou.kuyiso.com/canyin/1328.htm
8674 lanzhou.kuyiso.com/canyin/1328.htm
1909 site.leshou.com/s/39010445.html
9076 site.leshou.com/s/39010445.html
1867 site.leshou.com/s/40131383.html
9089 site.leshou.com/s/40131383.html
1885 site.leshou.com/s/45062793.html
9086 site.leshou.com/s/45062793.html
1859 site.leshou.com/s/45714581.html
9090 site.leshou.com/s/45714581.html
1889 site.leshou.com/s/46021681.html
9084 site.leshou.com/s/46021681.html
1900 site.leshou.com/s/49557679.html
9081 site.leshou.com/s/49557679.html
1873 site.leshou.com/s/50777239.html
9088 site.leshou.com/s/50777239.html
1904 site.leshou.com/s/52407278.html
9079 site.leshou.com/s/52407278.html
2992 wap.ailaba.org/sell/182965618167809.html
8475 wap.ailaba.org/sell/182965618167809.html
1610 www.01ny.cn/xinxi/25324778.html
9539 www.01ny.cn/xinxi/25324778.html
9536 www.01ny.cn/xinxi/37941846.html
9543 www.01ny.cn/xinxi/37941846.html
2282 www.05188.com/fenlei/22739365.html
9288 www.05188.com/fenlei/22739365.html
1913 www.51sole.com/b2b/pd_26691448.htm
8449 www.51sole.com/b2b/pd_26691448.htm
1230 www.587766.com/gongqiu/show_2096363.html
8480 www.587766.com/gongqiu/show_2096363.html
2275 www.afinance.cn/fenlei/25435008.html
9304 www.afinance.cn/fenlei/25435008.html
10328 www.afinance.cn/fenlei/37074884.html
10329 www.afinance.cn/fenlei/37074884.html
1917 www.afinance.cn/fenlei/38574413.html
8440 www.afinance.cn/fenlei/38574413.html
1924 www.ailaba.org/sell/179521059297846.html
8421 www.ailaba.org/sell/179521059297846.html
7276 www.ailaba.org/sell/183760178499696.html
9114 www.ailaba.org/sell/183760178499696.html
1222 www.bzcm.net/xinxi/29968353.html
8481 www.bzcm.net/xinxi/29968353.html
2954 www.cnhan.com/fenlei/34281043.html
8476 www.cnhan.com/fenlei/34281043.html
1276 www.cntrades.com/b2b/hnszbjh/sell/itemid-85394203.html
8428 www.cntrades.com/b2b/hnszbjh/sell/itemid-85394203.html
2262 www.gtobal.com/?m=trade&c=detail&a=preview&id=4930157402&type=2
9414 www.gtobal.com/?m=trade&c=detail&a=preview&id=4930157402&type=2
9450 www.gtobal.com/?m=trade&c=detail&a=preview&id=5060488002&type=2
9452 www.gtobal.com/?m=trade&c=detail&a=preview&id=5060488002&type=2
2271 www.hizh.cn/xinxi/10229031.html
9311 www.hizh.cn/xinxi/10229031.html
2043 www.hizh.cn/xinxi/3524483.html
9360 www.hizh.cn/xinxi/3524483.html
9566 www.kuyiso.com/html/101/ershoushebei/18348452x.htm
9583 www.kuyiso.com/html/101/ershoushebei/18348452x.htm
9515 www.kuyiso.com/html/21/zhiyepx/29045579x.htm
9522 www.kuyiso.com/html/21/zhiyepx/29045579x.htm
1293 www.laiwunews.cn/xinxi/13358851.html
8416 www.laiwunews.cn/xinxi/13358851.html
2213 www.laiwunews.cn/xinxi/18715417.html
8670 www.laiwunews.cn/xinxi/18715417.html
3181 www.laiwunews.cn/xinxi/18774478.html
8299 www.laiwunews.cn/xinxi/18774478.html
2259 www.laiwunews.cn/xinxi/18917168.html
9417 www.laiwunews.cn/xinxi/18917168.html
2280 www.laiwunews.cn/xinxi/19054895.html
9293 www.laiwunews.cn/xinxi/19054895.html
1282 www.laiwunews.cn/xinxi/20974600.html
8427 www.laiwunews.cn/xinxi/20974600.html
2217 www.laiwunews.cn/xinxi/20975830.html
8664 www.laiwunews.cn/xinxi/20975830.html
2256 www.laiwunews.cn/xinxi/21115975.html
9422 www.laiwunews.cn/xinxi/21115975.html
2211 www.laiwunews.cn/xinxi/21361687.html
8672 www.laiwunews.cn/xinxi/21361687.html
2214 www.laiwunews.cn/xinxi/21416448.html
8669 www.laiwunews.cn/xinxi/21416448.html
2272 www.laiwunews.cn/xinxi/21512940.html
9403 www.laiwunews.cn/xinxi/21512940.html
2267 www.laiwunews.cn/xinxi/25450210.html
9407 www.laiwunews.cn/xinxi/25450210.html
2290 www.laiwunews.cn/xinxi/27248072.html
9387 www.laiwunews.cn/xinxi/27248072.html
2288 www.laiwunews.cn/xinxi/27340981.html
9390 www.laiwunews.cn/xinxi/27340981.html
3825 www.qhnews.com/bianmin/30874042.html
8303 www.qhnews.com/bianmin/30874042.html
9559 www.qhnews.com/bianmin/32388347.html
9572 www.qhnews.com/bianmin/32388347.html
2541 www.qhnews.com/bianmin/32550280.html
8442 www.qhnews.com/bianmin/32550280.html
2564 www.qixiangwang.cn/fenlei/20773619.html
8288 www.qixiangwang.cn/fenlei/20773619.html
2946 www.qiyegu.com/sell/201412/13/deatil_10657611.html
8478 www.qiyegu.com/sell/201412/13/deatil_10657611.html
1992 www.sooshong.com/chxchx/offerdetail-150433811437054.html
8614 www.sooshong.com/chxchx/offerdetail-150433811437054.html
1299 www.waaku.com/infoview11788636.html
8414 www.waaku.com/infoview11788636.html
2297 www.yuqinge.com/xinxi/4683460.html
9257 www.yuqinge.com/xinxi/4683460.html
2215 www.zznews.cn/xinxi/21212448.html
8666 www.zznews.cn/xinxi/21212448.html
9425 yt.qu114.com/zhuanli/37012081.html
9426 yt.qu114.com/zhuanli/37012081.html
SELECT MIN(id),url FROM test_error_tmp WHERE url IN
(
SELECT url FROM
(
SELECT COUNT(1) AS c,url FROM test_error_tmp GROUP BY url
) AS tab
WHERE c>1
) GROUP BY url
;
MIN(id) url
2204 b2b.88152.com/show-2677485.html
2293 b2b.88152.com/show-2989753.html
2318 b2b.88152.com/show-2994375.html
2220 b2b.88152.com/show-3071983.html
1264 haikou.kuyiso.com/huizhan/29251188x.htm
1935 info.123cha.com/show_1007121.html
1887 info.b2b168.com/s168-48159518.html
3000 info.b2b168.com/s168-58451934.html
9428 info.b2b168.com/s168-60677934.html
2210 lanzhou.kuyiso.com/canyin/1328.htm
1909 site.leshou.com/s/39010445.html
1867 site.leshou.com/s/40131383.html
1885 site.leshou.com/s/45062793.html
1859 site.leshou.com/s/45714581.html
1889 site.leshou.com/s/46021681.html
1900 site.leshou.com/s/49557679.html
1873 site.leshou.com/s/50777239.html
1904 site.leshou.com/s/52407278.html
2992 wap.ailaba.org/sell/182965618167809.html
1610 www.01ny.cn/xinxi/25324778.html
9536 www.01ny.cn/xinxi/37941846.html
2282 www.05188.com/fenlei/22739365.html
1913 www.51sole.com/b2b/pd_26691448.htm
1230 www.587766.com/gongqiu/show_2096363.html
2275 www.afinance.cn/fenlei/25435008.html
10328 www.afinance.cn/fenlei/37074884.html
1917 www.afinance.cn/fenlei/38574413.html
1924 www.ailaba.org/sell/179521059297846.html
7276 www.ailaba.org/sell/183760178499696.html
1222 www.bzcm.net/xinxi/29968353.html
2954 www.cnhan.com/fenlei/34281043.html
1276 www.cntrades.com/b2b/hnszbjh/sell/itemid-85394203.html
2262 www.gtobal.com/?m=trade&c=detail&a=preview&id=4930157402&type=2
9450 www.gtobal.com/?m=trade&c=detail&a=preview&id=5060488002&type=2
2271 www.hizh.cn/xinxi/10229031.html
2043 www.hizh.cn/xinxi/3524483.html
9566 www.kuyiso.com/html/101/ershoushebei/18348452x.htm
9515 www.kuyiso.com/html/21/zhiyepx/29045579x.htm
1293 www.laiwunews.cn/xinxi/13358851.html
2213 www.laiwunews.cn/xinxi/18715417.html
3181 www.laiwunews.cn/xinxi/18774478.html
2259 www.laiwunews.cn/xinxi/18917168.html
2280 www.laiwunews.cn/xinxi/19054895.html
1282 www.laiwunews.cn/xinxi/20974600.html
2217 www.laiwunews.cn/xinxi/20975830.html
2256 www.laiwunews.cn/xinxi/21115975.html
2211 www.laiwunews.cn/xinxi/21361687.html
2214 www.laiwunews.cn/xinxi/21416448.html
2272 www.laiwunews.cn/xinxi/21512940.html
2267 www.laiwunews.cn/xinxi/25450210.html
2290 www.laiwunews.cn/xinxi/27248072.html
2288 www.laiwunews.cn/xinxi/27340981.html
3825 www.qhnews.com/bianmin/30874042.html
9559 www.qhnews.com/bianmin/32388347.html
2541 www.qhnews.com/bianmin/32550280.html
2564 www.qixiangwang.cn/fenlei/20773619.html
2946 www.qiyegu.com/sell/201412/13/deatil_10657611.html
1992 www.sooshong.com/chxchx/offerdetail-150433811437054.html
1299 www.waaku.com/infoview11788636.html
2297 www.yuqinge.com/xinxi/4683460.html
2215 www.zznews.cn/xinxi/21212448.html
9425 yt.qu114.com/zhuanli/37012081.html
SELECT id FROM test_error_tmp
WHERE url IN
(
SELECT url FROM
(
SELECT COUNT(1) AS c,url FROM test_error_tmp GROUP BY url
) AS tab
WHERE c>1
)
AND id NOT IN (
SELECT MIN(id) FROM test_error_tmp WHERE url IN
(
SELECT url FROM
(
SELECT COUNT(1) AS c,url FROM test_error_tmp GROUP BY url
) AS tab
WHERE c>1
) GROUP BY url
)
;
id
8647
8426
8654
8662
8477
8603
9577
8474
9431
8674
9076
9089
9086
9090
9084
9081
9088
9079
8475
9539
9543
9288
8449
8480
9304
10329
8440
8421
9114
8481
8476
8428
9414
9452
9311
9360
9583
9522
8416
8670
8299
9417
9293
8427
8664
9422
8672
8669
9403
9407
9387
9390
8303
9572
8442
8288
8478
8614
8414
9257
8666
9426
[SQL]DELETE FROM test_error_tmp WHERE id IN (
SELECT id FROM test_error_tmp
WHERE url IN
(
SELECT url FROM
(
SELECT COUNT(1) AS c,url FROM test_error_tmp GROUP BY url
) AS tab
WHERE c>1
)
AND id NOT IN (
SELECT MIN(id) FROM test_error_tmp WHERE url IN
(
SELECT url FROM
(
SELECT COUNT(1) AS c,url FROM test_error_tmp GROUP BY url
) AS tab
WHERE c>1
) GROUP BY url
)
);
[Err] 1093 - You can't specify target table 'test_error_tmp' for update in FROM clause
SELECT * FROM
(
SELECT id FROM test_error_tmp
WHERE url IN
(
SELECT url FROM
(
SELECT COUNT(1) AS c,url FROM test_error_tmp GROUP BY url
) AS tab
WHERE c>1
)
AND id NOT IN (
SELECT MIN(id) FROM test_error_tmp WHERE url IN
(
SELECT url FROM
(
SELECT COUNT(1) AS c,url FROM test_error_tmp GROUP BY url
) AS tab
WHERE c>1
) GROUP BY url
)
) AS tab
id
8647
8426
8654
8662
8477
8603
9577
8474
9431
8674
9076
9089
9086
9090
9084
9081
9088
9079
8475
9539
9543
9288
8449
8480
9304
10329
8440
8421
9114
8481
8476
8428
9414
9452
9311
9360
9583
9522
8416
8670
8299
9417
9293
8427
8664
9422
8672
8669
9403
9407
9387
9390
8303
9572
8442
8288
8478
8614
8414
9257
8666
9426
[SQL]DELETE FROM test_error_tmp WHERE id IN
(
SELECT * FROM
(
SELECT id FROM test_error_tmp
WHERE url IN
(
SELECT url FROM
(
SELECT COUNT(1) AS c,url FROM test_error_tmp GROUP BY url
) AS tab
WHERE c>1
)
AND id NOT IN (
SELECT MIN(id) FROM test_error_tmp WHERE url IN
(
SELECT url FROM
(
SELECT COUNT(1) AS c,url FROM test_error_tmp GROUP BY url
) AS tab
WHERE c>1
) GROUP BY url
)
) AS tab
);
受影响的行: 62
时间: 0.403s
校验
SELECT COUNT(1) FROM test_error_tmp
UNION ALL
SELECT COUNT(DISTINCT url) FROM test_error_tmp;
COUNT(1)
40946
40946
1:1
完成。
该表在读写中，进行的删除。
SELECT COUNT(1) FROM test_error
UNION ALL
SELECT COUNT(DISTINCT url) FROM test_error;
DELETE FROM test_error WHERE id IN
(
SELECT * FROM
(
SELECT id FROM test_error
WHERE url IN
(
SELECT url FROM
(
SELECT COUNT(1) AS c,url FROM test_error GROUP BY url
) AS tab
WHERE c>1
)
AND id NOT IN (
SELECT MIN(id) FROM test_error WHERE url IN
(
SELECT url FROM
(
SELECT COUNT(1) AS c,url FROM test_error GROUP BY url
) AS tab
WHERE c>1
) GROUP BY url
)
) AS tab
) AND payoff_status=0;
mysql 查询语句去重 通过sql去重的更多相关文章
- MySQL查询语句执行过程及性能优化（JOIN/ORDER BY)-图
http://blog.csdn.net/iefreer/article/details/12622097 MySQL查询语句执行过程及性能优化-查询过程及优化方法(JOIN/ORDER BY) 标签 ...
- MYSQL查询语句大全集锦
MYSQL查询语句大全集锦 1:使用SHOW语句找出在服务器上当前存在什么数据库: mysql> SHOW DATABASES; 2:2.创建一个数据库MYSQLDATA mysql> C ...
- MySQL查询语句执行过程及性能优化-查询过程及优化方法（JOIN/ORDER BY）
在上一篇文章MySQL查询语句执行过程及性能优化-基本概念和EXPLAIN语句简介中介绍了EXPLAIN语句,并举了一个慢查询例子:
- [转]MySQL查询语句执行过程详解
Mysql查询语句执行原理 数据库查询语句如何执行?语法分析:首先进行语法分析,对使用sql表示的查询进行语法分析,生成查询语法分析树.语义检查:检查sql中所涉及的对象以及是否在数据库中存在,用户是 ...
- php面试专题---MYSQL查询语句优化
php面试专题---MYSQL查询语句优化 一.总结 一句话总结: mysql的性能优化包罗甚广: 索引优化,查询优化,查询缓存,服务器设置优化,操作系统和硬件优化,应用层面优化(web服务器,缓存) ...
- Mysql查询语句执行过程
Mysql查询语句执行过程 Mysql分为server层和存储引擎两部分,或许可以再加一层连接层 连接层(器) Mysql使用的是典型的C/S架构.连接器通过典型的TCP握手完成连接. 需要注 ...
- 在magento里把查询语句转换称sql语句
在magento里把查询语句转换称sql语句 $order->getSelectsql(true); order为对象.
- mysql查询语句，通过limit来限制查询的行数。
mysql查询语句,通过limit来限制查询的行数. 例如: select name from usertb where age > 20 limit 0, 1; //限制从第一条开始,显示1条 ...
- MySQL查询语句执行过程及性能优化-基本概念和EXPLAIN语句简介
网站或服务的性能关键点很大程度在于数据库的设计(假设你选择了合适的语言开发框架)以及如何查询数据上. 我们知道MySQL的性能优化方法,一般有建立索引.规避复杂联合查询.设置冗余字段.建立中间表.查询 ...
- mysql查询语句集
1. mysql 查询出某字段的值不为空的语句 1.不为空 select * from table where id <> ""; select * from tabl ...
随机推荐
- 单点登录-CAS
使用插件 phpCAS ,插件链接:https://github.com/apereo/phpCAS 1. CAS 单点登录流程及插件的API的简介 图片链接来源于wiki:https://wiki. ...
- 转载：shell脚本之前的基础知识
转载地址:http://www.92csz.com/study/linux/12.htm 第十二章 学习 shell脚本之前的基础知识 日常的linux系统管理工作中必不可少的就是shell脚本,如果 ...
- 2016 Multi-University Training Contest 3 solutions BY 绍兴一中
1001 Sqrt Bo 由于有\(5\)次的这个限制,所以尝试寻找分界点. 很容易发现是\(2^{32}\),所以我们先比较输入的数字是否比这个大,然后再暴力开根. 复杂度是\(O(\log\log ...
- rabbitmq management Login Failed
默认用户guest 只允许localhost登录. so... 我们自己建立用户 1. 用户管理 用户管理包括增加用户,删除用户,查看用户列表,修改用户密码. 相应的命令 (1) 新增一个用户 rab ...
- poj2689素数问题
打算重新刷一下数论题,忘了很多了,水平也很差,此题入手就不顺了,刷了一个早上,只是一个简单 的素数应用罢了.题意:找出区间长度不超过10^6的最近的素数和最远的素数(相邻的), 算法:数在int范围内 ...
- Codeforces 659D Bicycle Race【计算几何】
题目链接: http://codeforces.com/contest/659/problem/D 题意: 若干条直线围城的湖,给定直线的端点,判断多少个转点会有危险?(危险是指直走的的话会掉进水里) ...
- Django的form，model自定制
一.Form组件原理: django框架提供了一个form类,来处理web开发中的表单相关事项.众所周知,form最常做的是对用户输入的内容进行验证,为此django的forms类提供了全面的内容验证 ...
- 学习日常笔记<day17>jdbc基础
1.jdbc入门 1.1.jdbc定义 使用java代码发送sql语句的技术就是jdbc技术 1.2.使用jdbc发送sql前提 需要登录数据库服务器(数据库的IP地址,端口,数据库用户名,密码) / ...
- java学习——关于搜索异常处理的总结
根据网上的资料可以知道,异常处理是为了检测到程序运行中发生的非正常情况的检测而设立的一种机制,异常的英文单词是exception,字面翻译就是“意外.例外”的意思,也就是非正常情况.关于平常我们经常遇 ...
- Myeclipse配置jad
下载地址:http://pan.baidu.com/s/1bnpMEuF 1.下载jad158g.win.zip 下载后解压.解压缩后将jad.exe拷贝到自定义的文件夹内:我这里用的是D:/jad/ ...