ESXi6.5上的Ubuntu虚机在远程SSH时宕机

情况是这样的, 有一个ESXi6.5上跑的Ubuntu虚机, 版本是18.04.1, 今天升级成18.04.2后, 就发现远程连接SSH不对劲, 在本地登录和操作都好好的, 只要远程SSH一连接, 服务器立马奔溃. 开始以为是SSHD配置问题, 修改各种参数没效果, 而系统/var/log/syslog里面也没有任何相关的错误信息. 后来实在没办法, 重装了, 还是不行. 怀疑是这个虚机的文件坏了, 把虚机删了, 再新建虚机重装, 发现还是这个问题.

但是在重建虚机安装后, 连接SSH时在本地的命令行看到了一个很有价值的错误输出

[84978.843130] kernel BUG at drivers/net/vmxnet3/vmxnet3_drv.c:!
[84978.843167] invalid opcode:  [#] SMP
[84978.843191] Modules linked in: vmw_vsock_vmci_transport(E) vsock(E) xt_conntrack(E) iptable_mangle(E) iptable_nat(E) nf_conntrack_ipv4(E) nf_defrag_ipv4(E) nf_nat_ipv4(E) nf_nat(E) iptable_filter(E) ip_tables(E) xt_LOG(E) nf_conntrack(E) coretemp(E) hwmon(E) kvm_intel(E) kvm(E) irqbypass(E) mousedev(E) hid_generic(E) aesni_intel(E) vmw_balloon(E) aes_x86_64(E) glue_helper(E) lrw(E) gf128mul(E) ablk_helper(E) evdev(E) cryptd(E) psmouse(E) usbhid(E) hid(E) nfit(E) intel_agp(E) vmw_vmci(E) battery(E) i2c_piix4(E) intel_gtt(E) acpi_cpufreq(E) tpm_tis(E) tpm_tis_core(E) tpm(E) ac(E) button(E) sch_fq_codel(E) crc32c_intel(E) uhci_hcd(E) ehci_pci(E) ehci_hcd(E) usbcore(E) usb_common(E) autofs4(E)

在VMWare的官网上查到了对应的bug和解决方案 https://kb.vmware.com/s/article/2151480

This issue occurs due to a bug in vmxnet3 vNIC backend which is part of the vmkernel. This issue occurs if all the below are true:

1. Linux VM is running kernel >= 4.8
2. HW version of VM is >=13
3. ESXi version is 6.5

This is a known issue affecting VMware ESXi 6.5.
This issue is resolved in VMware ESXi 6.5 Update 1, available at VMware Downloads.
To work around this issue if you do not want to upgrade, use any one of these options.
1. Add the vmxnet3.rev.30 = FALSE parameter in the vmx file of virtual machine:
2. Power off the virtual machine.
3. Edit the vmx file and add the below parameter:
    vmxnet3.rev.30 = FALSE
4. Power on the virtual machine.

If you do not want to power off the virtual machine, disable the receive data ring for each vmxnet3 vNIC on the VM by running this command:
    ethtool -G ethX rx-mini 0
Note: Replace ethX with virtual machine interface name.

注: 对应的文件在 /vmfs/volumes/584f7xxx-7xx749b4-3461-x0... / 目录下, 找到对应的虚机文件目录, 在下面找到vmx文件, 在最后添加上面提到的参数