Huge Packet Drops (Tx drops) Observed on NetScaler

来源  https://support.citrix.com/article/CTX215843

Article | NetworkingPerformance | Created: 15 Dec 2014 | Modified: 11 Aug 2016

Applicable Products

  • NetScaler

Symptoms or Error

  • Huge packet drops was observed on NetScaler. The drops are TX drops (software drops).
  • Customer attempted to ping NSIP and could see that most of the packet gets dropped. All configured services goes down and comes up frequently. Users could see that application gets disconnected very frequently (Applications have been launched through NetScaler Gateway).
  • We can see from newnslog that "nic_err_congestionlimit_pkts_dropped" and "nic_err_tx_dropped" counters" are incrementing:
    /var/nslog]$ nsconmsg101 -K newnslog -g nic_err -s disptime=1 -d current | grep congest |more
    6195 0 6159352 48887 6944 nic_err_congestionlimit_pkts_dropped interface(0/1) Wed Dec 3 17:17:38 2014
    6201 0 6208229 48877 6962 nic_err_congestionlimit_pkts_dropped interface(0/1) Wed Dec 3 17:17:45 2014
    6402 0 6257108 48879 6943 nic_err_congestionlimit_pkts_dropped interface(0/1) Wed Dec 3 17:25:32 2014
    6462 0 6305991 48883 6953 nic_err_congestionlimit_pkts_dropped interface(0/1) Wed Dec 3 17:27:45 2014
    6468 0 6354868 48877 6932 nic_err_congestionlimit_pkts_dropped interface(0/1) Wed Dec 3 17:27:52 2014
    6805 0 6403744 48876 6942 nic_err_congestionlimit_pkts_dropped interface(0/1) Wed Dec 3 17:40:55 2014
    /var/nslog]$ nsconmsg101 -K newnslog -g nic_err_tx_dropped -s time=04Dec2014:17:00 -s disptime=1 -d current | more reltime:mili second between two records Thu Dec 4 17:06:08 2014
    Index rtime totalcount-val delta rate/sec symbol-name&device-no&time
    0 2697427 8897070 48877 6942 nic_err_tx_dropped interface(0/1) Thu Dec 4 17:06:08 2014
    1 7040 8945947 48877 6942 nic_err_tx_dropped interface(0/1) Thu Dec 4 17:06:15 2014
    2 7170 8994878 48931 6824 nic_err_tx_dropped interface(0/1) Thu Dec 4 17:06:22 2014
    3 78151 9092633 97755 13885 nic_err_tx_dropped interface(0/1) Thu Dec 4 17:07:40 2014

Solution

Ensure that you are not using Management port (interface 0/1) for production traffic.

The best practice is to use management card only for Management traffic and the Network card for the Production traffic.

Management interfaces uses software RSS instead of hardware RSS, so this can impact performance.

Enabled the interface 1/1 and then disabled the interface 0/1.


Problem Cause

Management port (interface 0/1) used for production/data traffic.


see https://discussions.citrix.com/topic/390985-rx-drops-in-vpx-inf-11/  ---  RX Drops in VPX inf 1/1

Shell Command : nsconmsg101 -K newnslog -g nic_err -d statswt0

NetScaler VPX Loses Network Connectivity on VMware ESXi 5.1.0 2191751, VMware ESXi 5.5 2143827 and also on VMware ESXi 6.0

来源 https://support.citrix.com/article/CTX200278

Article | ConnectivityNetworkingThird PartyFixed Issues | 163 found this helpful | Created: 05 Nov 2014 | Modified: 29 Sep 2016

Applicable Products

  • NetScaler

Symptoms or Error

The following are the issues encountered when NetScaler VPX is run on VMware ESXi 5.1.0 2191751, VMware ESXi 5.5 2143827 and also on VMware ESXi 6.0

  • Network disconnectivity.

  • NetScaler GUI access is affected.

  • When running show interfaces command from NetScaler CLI you might notice that one of the interfaces has stalls and errors.


Solution

To fix this issue upgrade to NetScaler 10.5 build 55.8. This firmware can be download from Citrix Downloads.

For more information, see Fixed Issues section from NetScaler 10.5 Build 55.8 Release Notes.

The following are the workarounds for this issue:

Workaround 1

Install NetScaler VPX appliance on a previously known good VMware version and build.

Workaround 2

  1. SSH and log on to Citrix NetScaler VPX appliance as nsroot.

  2. Type shell.

  3. Change directory (cd) to /flash/boot.

  4. Create file /flash/boot/loader.conf.local (if not present) with same permissions as/flash/boot/loader.conf. Add the following line and reboot:
    hw.em.txd=512
    Note: To create the file, use command touch loader.conf.local.

vi Commands

The following are the vi commands to edit the document:

  1. From NetScaler shell type:
    vi <filename>

  2. Move the cursor to the last character of text in the file, type "a" and click Enter.

  3. Type the line:
    hw.em.txd=512

  4. Press the ESC key and then ":" key. The cursor will move to the bottom of the page, then type wq!.


Problem Cause

NetScaler VPX network connectivity issue on VMware ESXi 5.1.0 build 2191751 and VMware ESXi 5.5 build 2143827 is caused by "tx_ring_length" mismatch, which causes TX stalls.


Additional Resources

For more information refer to VMware Knowledge Base - 2092809.


Netscaler NIC Counters

来源 https://support.citrix.com/article/CTX125102

Article | Other | Created: 14 Jun 2017 | Modified: 14 Jun 2017

Applicable Products

  • NetScaler

Information

This article provides a correlation of the newnslog NIC counters, their SNMP counterpart, and a brief description.

Newnslog NIC Counter

The table below lists the 59 different newnslog NIC counters, a simple description of the counter, and the matching SNMP object name.

Newnslog Counter Name

SNMP Name

Description

nic_tot_rx_bytes

TotRxBytes

Number of bytes received by this interface

nic_rx_avg_bandwidth

RxAvgBandwidthUsage

The average rx bandwidth on interface in bits per second

nic_tot_rx_packets

TotRxPkts

Number of packets received by this interface

nic_rx_avg_packet_rate

RxAvgPacketRate

The average rate of incoming packets on the interface since the system start

nic_tot_rx_mbits

TotRxMbits

Number of megabits received by this interface

nic_tot_tx_bytes

TotTxBytes

Number of bytes transmitted by this interface

nic_tx_avg_bandwidth

TxAvgBandwidthUsage

The average tx bandwidth on interface in bits per second

nic_tot_tx_packets

TotTxPkts

Number of packets transmitted by this interface

nic_tx_avg_packet_rate

TxAvgPacketRate

The average rate of outgoing packets on the interface since the system start

nic_tot_tx_mbits

TotTxMbits

Number of megabits transmitted by this interface

nic_tot_multicast_pkts

nicTotMulticastPkts

Number of multicast packets received by this interface

nic_tot_netscaler_pkts

TotNetScalerPkts

Number of NetScaler packets received by this interface

nic_tot_rx_lacpdus

RxLacpdu

Number of LACPDUs received by this interface

nic_tot_tx_lacpdus

TxLacpdu

Number of LACPDUs transmitted by this interface

nic_err_rx

ErrPktRx

Number of erroneous packets received by this interface

nic_err_tx

ErrPktTx

Number of errors encountered during transmitting by this interface

nic_cur_link_state

CurLinkState

Current link state

nic_cur_link_uptime

CurLinkUptime

Current link uptime

nic_cur_link_downtime

CurLinkDowntime

Current link downtime

nic_err_dropped_pkts

ErrDroppedRxPkts

Number of received packets dropped by this interface

nic_err_tx_dropped

ErrDroppedTxPkts

Number of packets dropped in transmission by this interface

nic_tot_bdg_mac_moved

TotMacMoved

Number of times MAC moved between ports

nic_err_bdg_muted

ErrNicMuted

Number of times NIC become bridging-muted because of loop detection

nic_cur_txqlen

CurTxQLen

Number of packets queued for transmission

nic_err_link_hangs

ErrLinkHangs

Number of NIC hangs detected

nic_err_duplex_mismatch

ErrDuplexMismatch

Number of duplex mismatches detected

nic_err_rx_nonsb

ErrTxNoNSB

Number of times there was not enough memory for processing of the received data

nic_err_rx_crc

RxCRCErrors

Number of received packets with CRC errors (alignment or FCS)

nic_err_rx_long_frame

RxFrameErrors

Number of too long packets received on the interface

nic_err_rx_alignment

RxAlignmentErrors

The number of alignment errors received on the interface

nic_err_ifInDiscards

ErrIfInDiscards

Number of inbound error-free packets discarded internally by NIC

nic_err_ifOutDiscards

nicErrIfOutDiscards

Number of outbound error-free packets discarded

nic_err_link_reinits

LinkReinits

Number of link re-initializations (attempts to establish the link)

nic_err_link_sts_stalls

nicStsStalls

Number of Status stalls detected

nic_err_link_tx_stalls

nicTxStalls

Number of Transmit stalls detected

nic_err_link_rx_stalls

nicRxStalls

Number of Receive stalls detected

nic_err_link_errdisables

nicErrDisables

Number of times link has been error-disabled

nic_cur_Ptxqlen

HpTxQLen

Number of high-priority packets queued for transmission

nic_cur_LPtxqlen

LpTxQLen

Number of low-priority packets queued for transmit

nic_err_tx_nonucpkt

ErrTxNonUcPkt

Number of output non-unicast packets

nic_err_tx_collisions

TxCollisions

Number of collisions in transmission (half-duplex only)

nic_err_tx_excess_collisions

TxExcessCollisions

Number of excess collisions in transmission (half-duplex only)

nic_err_tx_late_collisions

TxLateCollisions

Number of late collisions in transmission (half-duplex only)

nic_err_tx_multi_collision_frame

TxMultiCollisionErrors

Number of multi collision during transmission (half-duplex only)

nic_err_tx_carrier

TxCarrierError

Number of carrier errors during transmission

nic_err_rx_nobufs

ErrRxNoBuffs

Number of times there was no memory for Rx replacement buffers

nic_err_tx_overflow

ErrTxOverflow

Number of times there was an overflow of the Tx ring (Informational)

nic_err_congested_pkts_dropped

ErrCongestedPktsDrops

Number of output packets dropped because of congestion

nic_err_congestionlimit_pkts_dropped

ErrCongestionLimitPktDrops

Number of output packets dropped during the congestion time

nic_err_rx_FCS

ErrRxFCS

Number of packets received with Frame Check Sequence (FCS) errors

nic_err_rx_fifo

ErrRxFIFO

NIC receive FIFO memory overruns

nic_tot_outXoffSent

TotXoffSent

Number of XOFF pause frames sent

nic_tot_outXonSent

TotXonSent

Number of XON pause frames sent

nic_tot_xoffPauseFramesReceived

TotRxXoffPause

Number of XOFF pause frames received

nic_tot_xoffStateEntered

TotXoffStateEntered

Number of times the XOFF state was entered

nic_tot_xonPauseFramesReceived

TotRxXonPause

Number of XON pause frames received

nic_err_tx_deferred

ErrTxDeferred

Number of times a packet transmission was deferred (Informational - half-duplex only)

nic_err_tx_fifo

ErrTxFIFO

NIC transmission FIFO under-runs

nic_err_tx_heartbeat

ErrTxHeartBeat

No 10Mb link heartbeats (Informational - 10Mb half-duplex only)

====================== End

Huge Packet Drops (Tx drops) Observed on NetScaler的更多相关文章

  1. HDU 5336——XYZ and Drops——————【广搜BFS】

    XYZ and Drops Time Limit: 3000/1500 MS (Java/Others)    Memory Limit: 65536/65536 K (Java/Others)Tot ...

  2. SNMP OID Reference - NetScaler 10

    SNMP OID Reference - NetScaler 10 https://docs.citrix.com/content/dam/docs/en-us/netscaler/10/downlo ...

  3. Cisco IOS Debug Command Reference I through L

    debug iapp through debug ip ftp debug iapp : to begin debugging of IAPP operations(in privileged EXE ...

  4. python爬乌云dorps文章

    有时候翻看drops 无赖drops没有一个全部文章列表,所以就有了这个想法把所有文件标题链接都爬出来这样又直观又好找感兴趣的文章 #coding=utf-8 import re import url ...

  5. bootstrap daterangepicker 添加阴历及节假日

    所有的新增都用 'myAdd start'和'myAdd end'标注,所有的修改都用'myChange start'和'myChange end'标注. 借用了 1900-2100区间内的公历.农历 ...

  6. Processing 编程学习指南 (丹尼尔·希夫曼 著)

    https://processing.org/reference/ 第1章 像素 (已看) 第2章 Processing (已看) 第3章 交互 (已看) 第4章 变量 (已看) 第5章 条件语句 ( ...

  7. DPDK QoS_meter 源码阅读

    main.c /* SPDX-License-Identifier: BSD-3-Clause * Copyright(c) 2010-2016 Intel Corporation */ #inclu ...

  8. xssbypass小记

    简单整理下bypass的一些点 标签外 如果是标签之外 又有htmlspecialchars函数的点 就别想了 在标签外同时能xss但是有长度限制 如果是储存型可以利用多个点 然后构造<scri ...

  9. I.MX6 AW-NB177NF wifi HAL 调试修改

    /************************************************************************* * I.MX6 AW-NB177NF wifi H ...

随机推荐

  1. Table 'xxx.hibernate_sequence' doesn't exist

    Caused by: com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: Table 'xxx.hibernate_sequence' ...

  2. JS 省市联动 ajax

    省市联动 //省市联动     $('.locationCode').change(function () {         var val = $(this).val();         if ...

  3. react+laravel与服务端渲染的几点思考

    一.前后端完全分离 1.用React.js做MVC中的V,剩下的交给Laravel 2.Laravel用来做API接口开发. 3.好处:实现了前后端开发的分离,从而加快前后端开发效率.另外若是多端的如 ...

  4. [Ubuntu]18安装百度网盘

     1.下载客户端 下载地址: 选择linux版本,我选择的是deb格式,下载就可以了. 2.安装 进入下载目录,点击右键,选择在终端打开. 之后输入 以下代码愉快的安装就好了 注意:dpkg后面跟的文 ...

  5. List自定义对象的排序,根据对象的某一列进行排序

    在工作中,经常需要对List对象集合进行排序操作,下面总结下搞个通用排序对象,原理是使用JAVA的 Comparator    接口实现排序   不多说直接上“干货” 1.存在实体类: @Data @ ...

  6. mysql 松散索引与紧凑索引扫描(引入数据结构)

    这一篇文章本来应该是放在 mysql 高性能日记中的,并且其优化程度并不高,但考虑到其特殊性和原理(索引结构也在这里稍微讲一下) 一,mysql 索引结构 (B.B+树) 要问到 mysql 的索引用 ...

  7. 拒绝LOW ---青鸟影院购票系统

    1.首先我们需要了解这个软件的功能: 1).影院每天更新放映列表,系统支持实时查看,包括电影放映场次的时间: 2).影院提供三类影票:普通票,学生票和赠票: 3).允许用户查看某场次座位的售出情况: ...

  8. 【Zookeeper】本地模式安装

    安装步骤 上传gz包 通过rz命令,将zookeeper-3.4.10.tar.gz安装包上传到/opt/soft/文件夹下. [root@bigdata111 soft]# rz [root@big ...

  9. 关于Java新手开发配置各种环境可能会遇到的的坑

    一.软件的安装 虽然国内的软件都支持中文目录安装,部分国外软件也支持,但是作为一名合格的程序开发者,必须做到以下几点 Windows下开发软件的安装目录和环境变量中永远不要包含中文字符,包括汉字[]. ...

  10. Java 基础篇之IO

    IO Java IO 流使用了一种装饰器设计模式,它将 IO 流分成底层节点流和上层处理流,其中节点流用于和物理节点直接关联,程序可以把不同物理节点流包装成统一的处理流,从而允许程序使用统一的输入.输 ...