mysql启动失败,一直crash,报错如下:

2019-03-14T11:15:12.937923Z 0 [Note] InnoDB: Uncompressed page, stored checksum in field1 1118110825, calculated checksums for field1: crc32 2889511119/2044155182, innodb 4005355497, none 3735928559, stored checksum in field2 1244127832, calculated checksums for field2: crc32 2889511119/2044155182, innodb 952099433, none 3735928559, page LSN 4 2819295388, low 4 bytes of LSN at page end 2486405135, page number (if stored to page already) 3971, space id (if created with >= MySQL-4.1.1 and stored already) 1840
InnoDB: Page may be an index page where index id is 6640
2019-03-14T11:15:12.937950Z 0 [Note] InnoDB: Index 6640 is `GEN_CLUST_INDEX` in table `zabbix`.`history_uint`
2019-03-14T11:15:12.937955Z 0 [Note] InnoDB: It is also possible that your operating system has corrupted its own file cache and rebooting your computer removes the error. If the corrupt page is an index page. You can also try to fix the corruption by dumping, dropping, and reimporting the corrupt table. You can use CHECK TABLE to scan your table for corruption. Please refer to http://dev.mysql.com/doc/refman/5.7/en/forcing-innodb-recovery.html for information about forcing recovery.

看起来是文件损坏了,我有备库,所以希望主库尽快恢复,然后再恢复数据,直接移走有问题的frm和ibd,再启动,报错:

2019-03-14T11:23:37.246589Z 0 [ERROR] InnoDB: Tablespace 1840 was not found at ./zabbix/history_uint.ibd.
2019-03-14T11:23:37.246594Z 0 [ERROR] InnoDB: Set innodb_force_recovery=1 to ignore this and to permanently lose all changes to the tablespace.
2019-03-14T11:23:37.247018Z 0 [ERROR] InnoDB: Cannot continue operation.

修改my.cnf

innodb_force_recovery=1

再重启正常,尝试重新新建表,但是各种报错:

mysql> CREATE TABLE `history` (
-> `itemid` bigint(20) unsigned NOT NULL,
-> `clock` int(11) NOT NULL DEFAULT '',
-> `value` double(16,4) NOT NULL DEFAULT '0.0000',
-> `ns` int(11) NOT NULL DEFAULT '',
-> KEY `history_1` (`itemid`,`clock`)
-> ) ENGINE=InnoDB DEFAULT CHARSET=utf8 COLLATE=utf8_bin;
ERROR 1813 (HY000): Tablespace '`zabbix`.`history`' exists.
mysql> alter table history discard tablespace;
ERROR 1146 (42S02): Table 'zabbix.history' doesn't exist
mysql> drop tablespace history;
ERROR 1529 (HY000): Failed to drop TABLESPACE history
mysql> drop table history;
ERROR 1051 (42S02): Unknown table 'zabbix.history'

完全没有办法操作,这时的解决方法是:

  • CREATE TABLE bad_table ENGINE=MyISAM ...
  • rm bad_table.ibd
  • DROP TABLE bad_table
  • CREATE TABLE bad_table ENGINE=INNODB ...

然后再恢复数据

也有可能报错时并没有指出具体出错的文件或者表,类似

2019-11-25T12:54:59.258844Z 0 [ERROR] InnoDB: Database page corruption on disk or a failed file read of page [page id: space=0, page number=593]. You may have to recover from a backup.

这时需要使用mysqlcheck命令来检查某个数据库

# mysqlcheck -uroot -proot $dbname

也可以检查所有的数据库

# echo 'show databases'|mysql -uroot -proot|xargs -i mysqlcheck -uroot -proot {}

输出结果为两列,第二列是status,正常为OK,其他值为异常,找到异常的表,删除或者将数据导出后再导入,则问题修复;

参考:
https://stackoverflow.com/questions/15694168/error-tablespace-for-table-xxx-exists-please-discard-the-tablespace-before-imp

【原创】大叔问题定位分享(32)mysql故障恢复的更多相关文章

  1. 【原创】大叔问题定位分享(28)openssh升级到7.4之后ssh跳转异常

    服务器集群之间忽然ssh跳转不通 # ssh 192.168.0.1The authenticity of host '192.168.0.1 (192.168.0.1)' can't be esta ...

  2. 【原创】大叔问题定位分享(13)HBase Region频繁下线

    问题现象:hive执行sql报错 select count(*) from test_hive_table; 报错 Error: java.io.IOException: org.apache.had ...

  3. 【原创】大叔问题定位分享(6)Dubbo monitor服务iowait高,负载高

    一 问题 Dubbo monitor所在服务器状态异常,iowait一直很高,load也一直很高,监控如下: iowait如图: load如图: 二 分析 通过iotop命令可以查看当前系统中磁盘io ...

  4. 【原创】大叔问题定位分享(5)Kafka客户端报错SocketException: Too many open files 打开的文件过多

    kafka0.8.1 一 问题 10月22号应用系统忽然报错: [2014/12/22 11:52:32.738]java.net.SocketException: 打开的文件过多 [2014/12/ ...

  5. 【原创】大叔问题定位分享(4)Kafka集群broker节点从zookeeper上消失

    kafka_2.8.0-0.8.1 一 现象 生产环境一组kafka集群经常发生问题,现象是kafka在zookeeper上的broker节点消失,此时kafka进程和端口都在,然后每个broker都 ...

  6. 【原创】大叔问题定位分享(3)Kafka集群broker进程逐个报错退出

    kafka0.8.1 一 问题现象 生产环境kafka服务器134.135.136分别在10月11号.10月13号挂掉: 134日志 [2014-10-13 16:45:41,902] FATAL [ ...

  7. 【原创】大叔问题定位分享(31)hive metastore报错

    hive metastore在建表时报错 [pool-5-thread-2]: MetaException(message:Got exception: java.net.ConnectExcepti ...

  8. 【原创】大叔问题定位分享(30)mesos agent启动失败:Failed to perform recovery: Incompatible agent info detected

    mesos agent启动失败,报错如下: Feb 15 22:03:18 server1.bj mesos-slave[1190]: E0215 22:03:18.622994 1192 slave ...

  9. 【原创】大叔问题定位分享(29)datanode启动报错:50020端口被占用

    集群中有一台datanode一直启动报错如下: java.net.BindException: Problem binding to [$server1:50020] java.net.BindExc ...

随机推荐

  1. 09、 在QQ音乐中查找七里香这首歌的精彩评论

       找到七里香这首歌的精彩评论      URL https://c.y.qq.com/base/fcgi-bin/fcg_global_comment_h5.fcg?g_tk=5381&l ...

  2. 自学python 7.

    内容:join , fromkeys , set , .add , .update , .pop , .remove , 交集(s1&s2) , 并集(s1|s2) , 深浅拷贝 , .cop ...

  3. Spark 整合ElasticSearch

    Spark 整合ElasticSearch 因为做资料搜索用到了ElasticSearch,最近又了解一下 Spark ML,先来演示一个Spark 读取/写入 ElasticSearch 简单示例. ...

  4. adduser Ubuntu添加sudo用户

    第一种方法: 添加sudo用户 当你安装Ubuntu的时候,它会自动添加第一个用户到sudo组,允许这个用户通过键入其自身帐户密码来获得超级用户(root)身份.然而,系统不会再自动添加其他的用户到s ...

  5. vue中element-ui树形控件自定义节点,注意一下

    在项目中用到菜单项编辑删除,在 element-ui自定义节点内容时, 有说明:使用render-content指定渲染函数, 得环境支持 JSX 语法,只要正确地配置了相关依赖,就可以正常运行. 一 ...

  6. 24.join算法/锁_1

    一. JOIN算法1.1. JOIN 语法 mysql> select * from t4; +---+------+ | a | b | +---+------+ | | 11 | | | 5 ...

  7. vivalidi 一款由Web技术诞生的Web浏览器

    vivalidi https://vivaldi.com/ A million ways to customize everything The world is a colorful place b ...

  8. Spring 快速开始 启动Spring

    [启动Spring必须配置] [web.xml部署描述符方式] 1.配置Servlet级别上下文 <servlet> <servlet-name>springDispatche ...

  9. web api HttpResponseMessage的简单使用

    using Lemon.Common; using Lemon.WeChat.Model; using Lemon.WeChat.Services; using Newtonsoft.Json; us ...

  10. Linux 常用命令(2)

    wc命令:用于对文件的行数.单词数和字符数进行统计. wc -w file.txt,显示文件中的单词数,wc -l file.txt,显示文件中的行数,wc -m file.txt,显示文件中的字符数 ...