Mysql占用CPU过高如何优化，如何解决

2017-02-28 15:13 331人阅读 评论(0) 收藏 举报

 

MySQL占用CPU过高如何优化

一次生产DB服务器的 超负荷运行问题解决：

1.查看生产DB服务器top列表，

执行 top 命令

查看Cpu(s) 参数一直处于 98% 状态 ，load average达到了 5  (4核服务器)

可见DB已经超负荷运行了

2.使用root用户登录mysql

执行 show full processlist  查看慢查询，反复执行，发现一直有3个select 查询语句存在，为了缓解DB服务器压力，直接使用kill命令杀掉

kill  慢查询的id

服务器压力缓解，恢复正常。

使用这条SQL语句去项目中找到对应功能，经查是没用的功能，直接隐藏这个功能，问题解决。

Mysql占用CPU过高如何优化？

MySQL处在高负载环境下，磁盘IO读写过多，肯定会占用很多资源，必然CP会U占用过高。
占用CPU过高，可以做如下考虑：
1.打开慢查询日志，查询是否是某个SQL语句占用过多资源，如果是的话，可以对SQL语句进行优化，比如优化 insert 语句、优化 group by 语句、优化 order by 语句、优化 join 语句等等；
2.考虑索引问题；
3.定期分析表，使用optimize table；
4.优化数据库对象；
5.考虑是否是锁问题；
6.调整一些MySQL Server参数，比如key_buffer_size、table_cache、innodb_buffer_pool_size、innodb_log_file_size等等；
7.如果数据量过大，可以考虑使用MySQL集群或者搭建高可用环境。

mysql show processlist + show status + kill Id命令详解

1. SHOW PROCESSLIST
显示哪些线程正在运行。您也可以使用mysqladmin processlist语句得到此信息。如果您有SUPER权限，您可以看到所有线程。否则，您只能看到您自己的线程（也就是，与您正在使用的MySQL 账户相关的线程）。请参见13.5.5.3节，“KILL语法”。如果您不使用FULL关键词，则只显示每个查询的前100个字符。

本语句报告TCP/IP连接的主机名称（采用host_name:client_port格式），以方便地判定哪个客户端正在做什么。

如果您得到“too many connections”错误信息，并且想要了解正在发生的情况，本语句是非常有用的。MySQL保留一个额外的连接，让拥有SUPER权限的 账户使用，以确保管理员能够随时连接和检查系统（假设您没有把此权限给予所有的用户）。

这个命令中最关键的就是state列，mysql列出的状态主要有以下几种：

Checking table
　正在检查数据表（这是自动的）。
Closing tables
　正在将表中修改的数据刷新到磁盘中，同时正在关闭已经用完的表。这是一个很快的操作，如果不是这样的话，就应该确认磁盘空间是否已经满了或者磁盘是否正处于重负中。
Connect Out
　复制从服务器正在连接主服务器。
Copying to tmp table on disk
　由于临时结果集大于tmp_table_size，正在将临时表从内存存储转为磁盘存储以此节省内存。
Creating tmp table
　正在创建临时表以存放部分查询结果。
deleting from main table
　服务器正在执行多表删除中的第一部分，刚删除第一个表。
deleting from reference tables
　服务器正在执行多表删除中的第二部分，正在删除其他表的记录。
Flushing tables
　正在执行FLUSH TABLES，等待其他线程关闭数据表。
Killed
　发送了一个kill请求给某线程，那么这个线程将会检查kill标志位，同时会放弃下一个kill请求。MySQL会在每次的主循环中检查 kill标志位，不过有些情况下该线程可能会过一小段才能死掉。如果该线程程被其他线程锁住了，那么kill请求会在锁释放时马上生效。
Locked
　被其他查询锁住了。
Sending data
　正在处理SELECT查询的记录，同时正在把结果发送给客户端。
Sorting for group
　正在为GROUP BY做排序。
　Sorting for order
　正在为ORDER BY做排序。
Opening tables
　这个过程应该会很快，除非受到其他因素的干扰。例如，在执ALTER TABLE或LOCK TABLE语句行完以前，数据表无法被其他线程打开。正尝试打开一个表。
Removing duplicates
　正在执行一个SELECT DISTINCT方式的查询，但是MySQL无法在前一个阶段优化掉那些重复的记录。因此，MySQL需要再次去掉重复的记录，然后再把结果发送给客户端。
Reopen table
　获得了对一个表的锁，但是必须在表结构修改之后才能获得这个锁。已经释放锁，关闭数据表，正尝试重新打开数据表。
Repair by sorting
　修复指令正在排序以创建索引。
Repair with keycache
　修复指令正在利用索引缓存一个一个地创建新索引。它会比Repair by sorting慢些。
Searching rows for update
　正在讲符合条件的记录找出来以备更新。它必须在UPDATE要修改相关的记录之前就完成了。
Sleeping
　正在等待客户端发送新请求.
System lock
　正在等待取得一个外部的系统锁。如果当前没有运行多个mysqld服务器同时请求同一个表，那么可以通过增加--skip-external-locking参数来禁止外部系统锁。
Upgrading lock
　INSERT DELAYED正在尝试取得一个锁表以插入新记录。
Updating
　正在搜索匹配的记录，并且修改它们。
User Lock
　正在等待GET_LOCK()。
Waiting for tables
　该线程得到通知，数据表结构已经被修改了，需要重新打开数据表以取得新的结构。然后，为了能的重新打开数据表，必须等到所有其他线程关闭这个 表。以下几种情况下会产生这个通知：FLUSH TABLES tbl_name, ALTER TABLE, RENAME TABLE, REPAIR TABLE, ANALYZE TABLE,或OPTIMIZE TABLE。
waiting for handler insert
　INSERT DELAYED已经处理完了所有待处理的插入操作，正在等待新的请求。
　大部分状态对应很快的操作，只要有一个线程保持同一个状态好几秒钟，那么可能是有问题发生了，需要检查一下。
　还有其他的状态没在上面中列出来，不过它们大部分只是在查看服务器是否有存在错误是才用得着。

mysql 查看当前连接数

命令： show processlist;
如果是root帐号，你能看到所有用户的当前连接。如果是其它普通帐号，只能看到自己占用的连接。
show processlist;只列出前100条，如果想全列出请使用show full processlist;
mysql> show processlist;

2. show status

Aborted_clients 由于客户没有正确关闭连接已经死掉，已经放弃的连接数量。
Aborted_connects 尝试已经失败的MySQL服务器的连接的次数。
Connections 试图连接MySQL服务器的次数。
Created_tmp_tables 当执行语句时，已经被创造了的隐含临时表的数量。
Delayed_insert_threads 正在使用的延迟插入处理器线程的数量。
Delayed_writes 用INSERT DELAYED写入的行数。
Delayed_errors 用INSERT DELAYED写入的发生某些错误(可能重复键值)的行数。
Flush_commands 执行FLUSH命令的次数。
Handler_delete 请求从一张表中删除行的次数。
Handler_read_first 请求读入表中第一行的次数。
Handler_read_key 请求数字基于键读行。
Handler_read_next 请求读入基于一个键的一行的次数。
Handler_read_rnd 请求读入基于一个固定位置的一行的次数。
Handler_update 请求更新表中一行的次数。
Handler_write 请求向表中插入一行的次数。
Key_blocks_used 用于关键字缓存的块的数量。
Key_read_requests 请求从缓存读入一个键值的次数。
Key_reads 从磁盘物理读入一个键值的次数。
Key_write_requests 请求将一个关键字块写入缓存次数。
Key_writes 将一个键值块物理写入磁盘的次数。
Max_used_connections 同时使用的连接的最大数目。
Not_flushed_key_blocks 在键缓存中已经改变但是还没被清空到磁盘上的键块。
Not_flushed_delayed_rows 在INSERT DELAY队列中等待写入的行的数量。
Open_tables 打开表的数量。
Open_files 打开文件的数量。
Open_streams 打开流的数量(主要用于日志记载）
Opened_tables 已经打开的表的数量。
Questions 发往服务器的查询的数量。
Slow_queries 要花超过long_query_time时间的查询数量。
Threads_connected 当前打开的连接的数量。
Threads_running 不在睡眠的线程数量。
Uptime 服务器工作了多少秒。

After create

This occurs when the thread creates a table (including internal temporary tables), at the end of the function that creates the table. This state is used even if the table could not be created due to some error.

Analyzing

The thread is calculating a MyISAM table key distributions (for example, for ANALYZE TABLE).

checking permissions

The thread is checking whether the server has the required privileges to execute the statement.

Checking table

The thread is performing a table check operation.

cleaning up

The thread has processed one command and is preparing to free memory and reset certain state variables.

closing tables

The thread is flushing the changed table data to disk and closing the used tables. This should be a fast operation. If not, you should verify that you do not have a full disk and that the disk is not in very heavy use.

converting HEAP to MyISAM

The thread is converting an internal temporary table from a MEMORY table to an on-disk MyISAM table.

copy to tmp table

The thread is processing an ALTER TABLE statement. This state occurs after the table with the new structure has been created but before rows are copied into it.

Copying to group table

If a statement has different ORDER BY and GROUP BY criteria, the rows are sorted by group and copied to a temporary table.

Copying to tmp table

The server is copying to a temporary table in memory.

Copying to tmp table on disk

The server is copying to a temporary table on disk. The temporary result set was larger than tmp_table_size and the thread is changing the temporary table from in-memory to disk-based format to save memory.

Creating index

The thread is processing ALTER TABLE ... ENABLE KEYS for a MyISAM table.

Creating sort index

The thread is processing a SELECT that is resolved using an internal temporary table.

creating table

The thread is creating a table. This includes creation of temporary tables.

Creating tmp table

The thread is creating a temporary table in memory or on disk. If the table is created in memory but later is converted to an on-disk table, the state during that operation will be Copying to tmp table on disk.

deleting from main table

The server is executing the first part of a multiple-table delete. It is deleting only from the first table, and saving columns and offsets to be used for deleting from the other (reference) tables.

deleting from reference tables

The server is executing the second part of a multiple-table delete and deleting the matched rows from the other tables.

discard_or_import_tablespace

The thread is processing an ALTER TABLE ... DISCARD TABLESPACE or ALTER TABLE ... IMPORT TABLESPACE statement.

end

This occurs at the end but before the cleanup of ALTER TABLE, CREATE VIEW, DELETE, INSERT, SELECT, or UPDATE statements.

executing

The thread has begun executing a statement.

Execution of init_command

The thread is executing statements in the value of the init_command system variable.

freeing items

The thread has executed a command. This state is usually followed by cleaning up.

Flushing tables

The thread is executing FLUSH TABLES and is waiting for all threads to close their tables.

FULLTEXT initialization

The server is preparing to perform a natural-language full-text search.

init

This occurs before the initialization of ALTER TABLE, DELETE, INSERT, SELECT, or UPDATE statements.

Killed

Someone has sent a KILL statement to the thread and it should abort next time it checks the kill flag. The flag is checked in each major loop in MySQL, but in some cases it might still take a short time for the thread to die. If the thread is locked by some other thread, the kill takes effect as soon as the other thread releases its lock.

Locked

The query is locked by another query.

logging slow query

The thread is writing a statement to the slow-query log.

NULL

This state is used for the SHOW PROCESSLIST state.

login

The initial state for a connection thread until the client has been authenticated successfully.

Opening tables, Opening table

The thread is trying to open a table. This is should be very fast procedure, unless something prevents opening. For example, an ALTER TABLE or a LOCK TABLE statement can prevent opening a table until the statement is finished.

preparing

This state occurs during query optimization.

Purging old relay logs

The thread is removing unneeded relay log files.

query end

This state occurs after processing a query but before the freeing items state.

Reading from net

The server is reading a packet from the network.

Removing duplicates

The query was using SELECT DISTINCT in such a way that MySQL could not optimize away the distinct operation at an early stage. Because of this, MySQL requires an extra stage to remove all duplicated rows before sending the result to the client.

removing tmp table

The thread is removing an internal temporary table after processing a SELECT statement. This state is not used if no temporary table was created.

rename

The thread is renaming a table.

rename result table

The thread is processing an ALTER TABLE statement, has created the new table, and is renaming it to replace the original table.

Reopen tables

The thread got a lock for the table, but noticed after getting the lock that the underlying table structure changed. It has freed the lock, closed the table, and is trying to reopen it.

Repair by sorting

The repair code is using a sort to create indexes.

Repair done

The thread has completed a multi-threaded repair for a MyISAM table.

Repair with keycache

The repair code is using creating keys one by one through the key cache. This is much slower than Repair by sorting.

Rolling back

The thread is rolling back a transaction.

Saving state

For MyISAM table operations such as repair or analysis, the thread is saving the new table state to the .MYI file header. State includes information such as number of rows, the AUTO_INCREMENT counter, and key distributions.

Searching rows for update

The thread is doing a first phase to find all matching rows before updating them. This has to be done if the UPDATE is changing the index that is used to find the involved rows.

Sending data

The thread is processing rows for a SELECT statement and also is sending data to the client.

setup

The thread is beginning an ALTER TABLE operation.

Sorting for group

The thread is doing a sort to satisfy a GROUP BY.

Sorting for order

The thread is doing a sort to satisfy a ORDER BY.

Sorting index

The thread is sorting index pages for more efficient access during a MyISAM table optimization operation.

Sorting result

For a SELECT statement, this is similar to Creating sort index, but for nontemporary tables.

statistics

The server is calculating statistics to develop a query execution plan.

System lock

The thread is going to request or is waiting for an internal or external system lock for the table. If this state is being caused by requests for external locks and you are not using multiple mysqld servers that are accessing the same tables, you can disable external system locks with the --skip-external-locking option. However, external locking is disabled by default, so it is likely that this option will have no effect. For SHOW PROFILE, this state means the thread is requesting the lock (not waiting for it).

Table lock

The next thread state after System lock. The thread has acquired an external lock and is going to request an internal table lock.

Updating

The thread is searching for rows to update and is updating them.

updating main table

The server is executing the first part of a multiple-table update. It is updating only the first table, and saving columns and offsets to be used for updating the other (reference) tables.

updating reference tables

The server is executing the second part of a multiple-table update and updating the matched rows from the other tables.

User lock

The thread is going to request or is waiting for an advisory lock requested with a GET_LOCK() call. For SHOW PROFILE, this state means the thread is requesting the lock (not waiting for it).

Waiting for tables, Waiting for table

The thread got a notification that the underlying structure for a table has changed and it needs to reopen the table to get the new structure. However, to reopen the table, it must wait until all other threads have closed the table in question.

This notification takes place if another thread has used FLUSH TABLES or one of the following statements on the table in question: FLUSH TABLES tbl_name, ALTER TABLE, RENAME TABLE, REPAIR TABLE, ANALYZE TABLE, or OPTIMIZE TABLE.

Waiting on cond

A generic state in which the thread is waiting for a condition to become true. No specific state information is available.

Writing to net

The server is writing a packet to the network.
3. kill mysql 执行线程
(1) show processlist;

引用

-----------+
| 152 | root | 192.168.0.100:1808 | atomic_data | Query   | 5600 | copy to tmp table | alter table `atomic_data`.`basic_point_beijing_p1`
   change `valid` `type` int(2) NULL ,
   chang |
| 155 | root | 192.168.0.100:1944 | atomic_data | Query   | 4702 | Locked            | alter table `atomic_data`.`basic_point_beijing_p1`
   change `valid` `type` int(2) NULL ,
   chang |
| 158 | root | localhost          | atomic_data | Query   |    0 | NULL              | show processlist                                                                                     |

(2) kill 152;

引用

Query OK, 0 rows affected (0.00 sec)

(3) show preocesslist;

引用

+-----+------+--------------------+-------------+---------+------+-------------------+------------------------------------------------------------------------------------------------------+
| Id  | User | Host               | db          | Command | Time | State             | Info                                                                                                 |
+-----+------+--------------------+-------------+---------+------+-------------------+------------------------------------------------------------------------------------------------------+
| 155 | root | 192.168.0.100:1944 | atomic_data | Query   | 4840 | copy to tmp table | alter table `atomic_data`.`basic_point_beijing_p1`
   change `valid` `type` int(2) NULL ,
   chang |
| 158 | root | localhost          | atomic_data | Query   |    0 | NULL              | show processlist

批量 kill mysql 中运行时间长的sql

以下内容来自mysql手册：

13.5.5.3. KILL语法
KILL [CONNECTION | QUERY] thread_id
每个与mysqld的连接都在一个独立的线程里运行，您可以使用SHOW PROCESSLIST语句查看哪些线程正在运行，并使用KILL thread_id语句终止一个线程。

KILL允许自选的CONNECTION或QUERY修改符：

· KILL CONNECTION与不含修改符的KILL一样：它会终止与给定的thread_id有关的连接。

· KILL QUERY会终止连接当前正在执行的语句，但是会保持连接的原状。

如果您拥有PROCESS权限，则您可以查看所有线程。如果您拥有SUPER权限，您可以终止所有线程和语句。否则，您只能查看和终止您自己的线程和语句。

您也可以使用mysqladmin processlist和mysqladmin kill命令来检查和终止线程。

注释：您不能同时使用KILL和Embedded MySQL Server库，因为内植的服务器只运行主机应用程序的线程。它不能创建任何自身的连接线程。

当您进行一个KILL时，对线程设置一个特有的终止标记。在多数情况下，线程终止可能要花一些时间，这是因为终止标记只会在在特定的间隔被检查：

· 在SELECT, ORDER BY和GROUP BY循环中，在读取一组行后检查标记。如果设置了终止标记，则该语句被放弃。

· 在ALTER TABLE过程中，在每组行从原来的表中被读取前，检查终止标记。如果设置了终止标记，则语句被放弃，临时表被删除。

· 在UPDATE或DELETE运行期间，在每个组读取之后以及每个已更行或已删除的行之后，检查终止标记。如果终止标记被设置，则该语句被放弃。注意，如果您正在使用事务，则变更不会被 回滚。

· GET_LOCK()会放弃和返回NULL。

· INSERT DELAYED线程会快速地刷新（插入）它在存储器中的所有的行，然后终止。

· 如果线程在表锁定管理程序中（状态：锁定），则表锁定被快速地放弃。

· 如果在写入调用中，线程正在等待空闲的磁盘空间，则写入被放弃，并伴随"disk full"错误消息。

· 警告：对MyISAM表终止一个REPAIR TABLE或OPTIMIZE TABLE操作会导致出现一个被损坏的没有用的表。对这样的表的任何读取或写入都会失败，直到您再次优化或修复它（不中断）。

1、通过information_schema.processlist表中的连接信息生成需要处理掉的MySQL连接的语句临时文件，然后执行临时文件中生成的指令

mysql> select concat('KILL ',id,';') from information_schema.processlist where user='root';
+------------------------+
| concat('KILL ',id,';')
+------------------------+
| KILL 3101;
| KILL 2946;
+------------------------+
2 rows in set (0.00 sec)

mysql>select concat('KILL ',id,';') from information_schema.processlist where user='root' into outfile '/tmp/a.txt';
Query OK, 2 rows affected (0.00 sec)

mysql>source /tmp/a.txt;
Query OK, 0 rows affected (0.00 sec)

2、

　　杀掉当前所有的MySQL连接

mysqladmin -uroot -p processlist|awk -F "|" '{print $2}'|xargs -n 1 mysqladmin -uroot -p kill

　　杀掉指定用户运行的连接，这里为Mike

mysqladmin -uroot -p processlist|awk -F "|" '{if($3 == "Mike")print $2}'|xargs -n 1 mysqladmin -uroot -p kill

　

3、通过SHEL脚本实现

#杀掉锁定的MySQL连接
for id in `mysqladmin processlist|grep -i locked|awk '{print $1}'`
do
   mysqladmin kill ${id}
done

4、通过Maatkit工具集中提供的mk-kill命令进行

#杀掉超过60秒的sql
mk-kill -busy-time 60 -kill
#如果你想先不杀，先看看有哪些sql运行超过60秒
mk-kill -busy-time 60 -print
#如果你想杀掉，同时输出杀掉了哪些进程
mk-kill -busy-time 60 -print –kill