Cisco Catalyst 6000 Series Switches

Document ID: 12027

Contents

Introduction
Prerequisites
      Requirements
      Components Used
      Conventions
Physical Layer Troubleshooting
      Using the LEDs to Troubleshoot
      Check the Cable and Both Sides of the Connection
      Ethernet Copper and Fiber Cables
      Gigabit Ethernet Troubleshooting
      Connected vs Notconnected
Most Common Port and Interface Troubleshooting Commands for CatOS and Cisco IOS
Understanding Specific Port and Interface Counter Output for CatOS and Cisco IOS
      Show Port for CatOS and Show Interfaces for Cisco IOS
      Show Mac for CatOS and Show Interfaces Counters for Cisco IOS
      Show Counters for CatOS and Show Counters Interface for Cisco IOS
      Show Controller Ethernet-Controller for Cisco IOS
      Show Top for CatOS
Common System Error Messages
      Error Messages on WS-X6348 Modules
      %PAGP-5-PORTTO / FROMSTP and %ETHC-5-PORTTO / FROMSTP
      %SPANTREE-3-PORTDEL_FAILNOTFOUND
      %SYS-4-PORT_GBICBADEEPROM: / %SYS-4-PORT_GBICNOTSUPP
      %AMDP2_FE-3-UNDERFLO
      %INTR_MGR-DFC1-3-INTR: Queueing Engine (Blackwater) [1]: FIC Fabric-A Received Unexpected Control Code
      Command Rejected: [Interface] not a Switching Port
Common Port and Interface Problems
      Port or Interface Status is Disable or Shutdown
      Port or Interface Status is errDisable
      Port or Interface Status is Inactive
      Uplink Port or Interface Status is Inactive
      Deferred Counter on the Catalyst Switch Interface Starts Incrementing
      Intermittent Failure to set timer [value] from vlan [vlan no]
      Trunking Mode Mismatch
      Jumbos, Giants, and Baby Giants
      Cannot Ping End Device
      Using Set Port Host or Switchport Host to Fix Startup Delays
      Speed/Duplex, Autonegotiation, or NIC Issues
      Spanning Tree Loops
      UDLD: One-Way Link
      Deferred Frames (Out-Lost or Out-Discard)
      Software Problems
      Hardware Problems
      Input Errors on a Layer 3 Interface Connected to a Layer 2 Switchport
      Rapidly Incrementing Rx-No-Pkt-Buff Counter and Input Errors
      Trunking between a Switch and a Router
      Connectivity Issues due to Oversubscription
      Subinterfaces in SPA Modules
      Troubleshooting rxTotalDrops
NetPro Discussion Forums - Featured Conversations
Related Information

Introduction

This document is intended to help determine why a port or interface experiences problems. This document applies to Catalyst switches that run CatOS Software on the Supervisor or Cisco IOS^® System Software on the Supervisor.

Prerequisites

Requirements

There are no specific requirements for this document.

Components Used

This document is not restricted to specific software and hardware versions.

Conventions

Refer to Cisco Technical Tips Conventions for more information on document conventions.

Physical Layer Troubleshooting

Using the LEDs to Troubleshoot

If you have physical access to the switch, it can save time to look at the port LEDs which give you the link status or can indicate an error condition (if red or orange). The table describes the LED status indicators for Ethernet modules or fixed-configuration switches:

Ensure that both sides have a link. A single broken wire or one shutdown port can cause the problem where one side has a link light, but the other side does not.

A link light does not guarantee that the cable is fully functional. The cable can have encountered physical stress that causes it to be functional at a marginal level. Normally you can identify this situation if the port has many packet errors, or the port constantly flaps (loses and regains link).

Check the Cable and Both Sides of the Connection

If the link light for the port does not come on, you can consider these possibilities:

Ethernet Copper and Fiber Cables

Make sure you have the correct cable for the type of connection you are making. Category 3 copper cable can be used for 10 Mbps unshielded twisted pair (UTP) connections, but must never be used for 10/100 or 10/100/1000Mbps UTP connections. Always use either Category 5, Category 5e, or Category 6 UTP for 10/100 or 10/100/1000Mbps connections.

Warning: Category 5e and Category 6 cables can store high levels of static electricity because of the dielectric properties of the materials used in their construction. Always ground the cables (especially in new cable runs) to a suitable and safe earth ground before you connect them to the module.

For fiber, make sure you have the correct cable for the distances involved and the type of fiber ports that are used. The two options are singlemode fiber (SMF) or multimode fiber (MMF). Make sure the ports on the devices that are connected together are both SMF, or both are MMF ports.

Note: For fiber connections, make sure the transmit lead of one port is connected to the receive lead of the other port. Connections for transmit-to-transmit and receive-to-receive do not work.

Ethernet and Fast Ethernet Maximum Transmission Distances

For more details on the different types of cables/connectors, cabling requirements, optical requirements (distance, type, patch cables, etc.), how to connect the different cables, and which cables are used by most Cisco switches and modules, refer to Catalyst Switch Cable Guide.

Gigabit Ethernet Troubleshooting

If you have device A connected to device B over a Gigabit link, and the link does not come up, perform this procedure.

Step-by-Step Procedure

1. Verify device A and B use the same GBIC, short wavelength (SX), long wavelength (LX), long haul (LH), extended wavelength (ZX), or copper UTP (TX). Both devices must use the same type of GBIC to establish link. An SX GBIC needs to connect with an SX GBIC. An SX GBIC does not link with an LX GBIC. Refer to Mode-Conditioning Patch Cord Installation Note for more information.

2. Verify distance and cable used per GBIC as defined in this table.

   1000BASE-T and 1000BASE-X Port Cabling Specifications

   GBIC	Wavelength (nm)	Copper/Fiber Type	Core Size1 (Microns)	Modal Bandwidth (MHz/km)	Cable Distance2
   WS-G5483 1000Base - T (copper)		Category 5 UTP Category 5e UTP Category 6 UTP			328 ft (100 m)
   WS-G5484 1000BASE-SX3	850	MMF	62.5 62.5 50.0 50.0	160 200 400 500	722 ft (220 m) 902 ft (275 m) 1640 ft (500 m) 1804 ft (550 m)
   WS-G5486 1000BASE-LX/LH	1310	MMF4 SMF	62.5 50.0 50.0 8.3/9/10	500 400 500 -	1804 ft (550 m) 1804 ft (550 m) 1804 ft (550 m) 6.2 miles (10 km)
   WS-G5487 1000BASE-ZX5	1550	MMF SMF6	8.3/9/10 8.3/9/10		43.5 miles (70 km)7 62.1 miles (100 km)

   1. The numbers given for multimode fiber-optic cable refer to the core diameter. For single-mode fiber-optic cable, 8.3 microns refers to the core diameter. The 9-micron and 10-micron values refer to the mode-field diameter (MFD), which is the diameter of the light-carrying portion of the fiber. This area consists of the fiber core plus a small portion of the surrounding cladding. The MFD is a function of the core diameter, the wavelength of the laser, and the refractive index difference between the core and the cladding.

   2. Distances are based on fiber loss. Multiple splices and substandard fiber-optic cable reduce cabling distances.

   3. Use with MMF only.

   4. When you use an LX/LH GBIC with 62.5-micron diameter MMF, you must install a mode-conditioning patch cord (CAB-GELX-625 or equivalent) between the GBIC and the MMF cable on both the transmit and receive ends of the link. The mode-conditioning patch cord is required for link distances less than 328 feet (100 m) or greater than 984 feet (300 m). The mode-conditioning patch cord prevents overdriving the receiver for short lengths of MMF and reduces differential mode delay for long lengths of MMF. Refer to Mode-Conditioning Patch Cord Installation Note for more information.

   5. Use with SMF only.

   6. Dispersion-shifted single-mode fiber-optic cable.

   7. The minimum link distance for ZX GBICs is 6.2 miles (10 km) with an 8-dB attenuator installed at each end of the link. Without attenuators, the minimum link distance is 24.9 miles (40 km).

3. If either device has multiple Gigabit ports, connect the ports to each other. This tests each device and verifies that the Gigabit interface functions correctly. For example, you have a switch that has two Gigabit ports. Wire Gigabit port one to Gigabit port two. Does the link come up? If so, the port is good. STP blocks on the port and prevents any loops (port one receive (RX) goes to port two transmit (TX), and port one TX goes to port two RX).

4. If single connection or Step 3 fails with SC connectors, loop the port back to itself (port one RX goes to port one TX). Does the port come up? If not, contact the TAC, as this can be a faulty port.

5. If steps 3 and 4 are successful, but a connection between device A and B cannot be established, loop ports with the cable that adjoins the two devices. Verify that there is not a faulty cable.

6. Verify that each device supports 802.3z specification for Gigabit auto-negotiation. Gigabit Ethernet has an auto-negotiation procedure that is more extensive than the one used for 10/100 Ethernet (Gigabit auto-negotiation spec: IEEE Std 802.3z-1998). When you enable link negotiation, the system auto-negotiates flow control, duplex mode, and remote fault information. You must either enable or disable link negotiation on both ends of the link. Both ends of the link must be set to the same value or the link cannot connect. Problems have been seen when you connect to devices manufactured before the IEEE 802.3z standard was ratified. If either device does not support Gigabit auto-negotiation, disable the Gigabit auto-negotiation, and it forces the link up. It takes 300msec for the card firmware to notify the software that a 10/100/1000BASE-TX link/port is down. The 300msec default debounce timer comes from the firmware polling timer to the linecards, which occurs every 300 msec. If this link is run in 1G (1000BASE-TX) mode, Gigabit sync, which occurs every 10msec, must be able to detect the link down faster. There is a difference in the link failure detection times when you run GigabitEthenet on copper versus GigabitEthernet over Fibre. This difference in detection time is based on the IEEE standards.

   Warning: Disabling auto-negotiation hides link drops or physical layer problems. Disabling auto-negotiation is only required if end-devices such as older Gigabit NICs are used which cannot support IEEE 802.3z. Do not disable auto-negotiation between switches unless absolutely required to do so, as physical layer problems can go undetected, which results in STP loops. The alternative is to contact the vendor for software/hardware upgrade for IEEE 802.3z Gigabit auto-negotiation support.

In order to troubleshoot the error message: %SYS-4-PORT_GBICBADEEPROM: / %SYS-4-PORT_GBICNOTSUPP, refer to Common CatOS Error Messages on Catalyst 6000/6500 Series Switches.

For GigabitEthernet system requirements as well as Gigabit Interface Converters (GBICs), Coarse Wavelength Division Multiplexing (CWDM), and Small Form-Factor Pluggable (SFP) system requirements, refer to these:

• System Requirements to Implement Gigabit Ethernet on Catalyst Switches

• Catalyst GigaStack Gigabit Interface Converter Switch Compatibility Matrix

• Cisco Gigabit Ethernet Transceiver Modules Compatibility Matrix

• GBIC, SFP, and CWDM Documentation

For general configuration and troubleshooting information, refer to Configuring and Troubleshooting Ethernet 10/100/1000 MB Half/Full Duplex Auto-Negotiation.

Connected vs Notconnected

Most Cisco switches default to having a port in the notconnect state. This means it is currently not connected to anything, but it will connect if it has a good connection to another operational device. If you connect a good cable to two switch ports in the notconnect state, the link light must become green for both ports, and the port status must indicate connected. This means that the port is up as far as Layer 1 (L1) is concerned.

For CatOS, you can use the show port command to verify whether the port has a connected or notconnect status, or whether it is another state that would cause connectivity to fail, like disabled or errdisable.

Switch> (enable) sh port status 3/1
  Port  Name                 Status     Vlan       Duplex Speed Type
  ----- -------------------- ---------- ---------- ------ ----- ------------
   3/1                       disabled   1            auto  auto 10/100BaseTX
!--- The show port status {mod/port} command show the port is disabled.
!--- Use the set port enable {mod/port}command to try and re-enable it.

For Cisco IOS, you can use the show interfaces command to verify whether the interface is "up, line protocol is up (connected)". The first "up" refers to the physical layer status of the interface. The "line protocol up" message shows the data link layer status of the interface and says that the interface can send and receive keepalives.

Router#show interfaces fastEthernet 6/1
FastEthernet6/1 is down, line protocol is down (notconnect)
!--- The interface is down and line protocol is down.
!--- Reasons: In this case,
!--- 1) A cable is not properly connected or not connected at all to this port.
!--- 2) The connected cable is faulty.
!--- 3) Other end of the cable is not connected to an active port or device.

!--- Note: For gigabit connections, GBICs need to be matched on each 
!--- side of the connection. 
!--- There are different types of GBICs, depending on the cable and 
!--- distances involved: short wavelength (SX), 
!--- long-wavelength/long-haul (LX/LH) and extended distance (ZX). 
!--- An SX GBIC needs to connect with an SX GBIC; 
!--- an SX GBIC does not link with an LX GBIC. Also, some gigabit 
!--- connections require conditioning cables, 
!--- depending on the lengths involved.

Router#show interfaces fastEthernet 6/1
FastEthernet6/1 is up, line protocol is down (notconnect)
!--- The interface is up (or not in a shutdown state), but line protocol down.
!--- Reason: In this case, the device on the other side of the wire is a 
!--- CatOS switch with its port disabled.

Router#sh interfaces fas 6/1 status
Port   Name    Status       Vlan    Duplex   Speed  Type
Fa6/1          notconnect    1       auto     auto   10/100BaseTX
!--- The show interfaces card-type [slot/port] status command is the equivalent
!--- of show port status for CatOS.

If show port shows connected or show interfaces shows up/ line protocol up (connected) but you see errors incrementing in the output of either command, refer to the Understanding Specfic Port and Interface Counter Output for CatOS or Cisco IOS or Common Port and Interface Problems sections of this document for troubleshooting advice.

Most Common Port and Interface Troubleshooting Commands for CatOS and Cisco IOS

This table shows the most common commands used for troubleshooting port or interface problems on switches that run CatOS Software on the Supervisor or Cisco IOS System Software on the Supervisor.

Note: Choose a command in the left hand column to go to documentation for that command. The right hand column gives a brief description of what the command does and lists any exceptions to it's use per platform.

These commands are supported by the Output Interpreter tool for CatOS and can be used to assist in troubleshooting switch port or problems: show version, show module, show port, show counters, or show mac.

If you have the output of the supported commands from your Cisco device, you can use to display potential issues and fixes. In order to use Output Interpreter, you must be a registered user, be logged in, and have JavaScript enabled.

Understanding Specific Port and Interface Counter Output for CatOS and Cisco IOS

Most switches have some way to track the packets and errors that occur on a port or interface. The common commands used to find this type of information are described in the Most Common Port and Interface Troubleshooting Commands for CatOS and Cisco IOS section of this document.

Note: There can be differences in the implementation of the counters across various platforms and releases. Although the values of the counters are largely accurate, they are not very precise by design. In order to pull the exact statistics of the traffic, it is suggested that you use a sniffer to monitor the necessary ingress and egress interfaces.

Excessive errors for certain counters usually indicate a problem. When you operate at half-duplex setting, some data link errors incrementing in Frame Check Sequence (FCS), alignment, runts, and collision counters are normal. Generally, a one percent ratio of errors to total traffic is acceptable for half-duplex connections. If the ratio of errors to input packets is greater than two or three percent, performance degradation can be noticed.

In half-duplex environments, it is possible for both the switch and the connected device to sense the wire and transmit at exactly the same time and result in a collision. Collisions can cause runts, FCS, and alignment errors due to the frame not being completely copied to the wire, which results in fragmented frames.

When you operate at full-duplex, errors in FCS, Cyclic Redundancy Checks (CRC), alignment, and runt counters must be minimal. If the link operates at full-duplex, the collision counter is not active. If the FCS, CRC, alignment, or runt counters increment, check for a duplex mismatch. Duplex mismatch is a situation where the switch operates at full-duplex and the connected device operates at half-duplex, or vice versa. The results of a duplex mismatch are extremely slow performance, intermittent connectivity, and loss of connection. Other possible causes of data link errors at full-duplex are bad cables, faulty switch ports, or NIC software/hardware issues. See the Common Port and Interface Problems section of this document for more information.

Show Port for CatOS and Show Interfaces for Cisco IOS

The show port {mod/port} command is used when that run CatOS on the Supervisor. An alternative to this command is the show port counters {mod/port} which only displays the port error counters. Refer to Table 1 for explanations of the error counter output.

Switch> (enable) sh port counters 3/1
  
  Port  Align-Err  FCS-Err    Xmit-Err   Rcv-Err    UnderSize
 ----- ---------- ---------- ---------- ---------- ---------
  3/1           0          0          0          0         0
 
  Port  Single-Col Multi-Coll Late-Coll  Excess-Col Carri-Sen Runts     Giants
 ----- ---------- ---------- ---------- ---------- --------- --------- ---------
  3/1           0          0          0          0         0         0         0

The show interfaces card-type {slot/port} command is the equivalent command for Cisco IOS on the Supervisor. An alternative to this command (for Catalyst 6000, 4000, 3550, 2970 2950/2955, and 3750 series switches) is the show interfaces card-type {slot/port} counters errors command which only displays the interface error counters.

Note: For 2900/3500XL Series switches use the show interfaces card-type {slot/port} command with the show controllers Ethernet-controller command.

Router#sh interfaces fastEthernet 6/1
FastEthernet6/1 is up, line protocol is up (connected)
   Hardware is C6k 100Mb 802.3, address is 0009.11f3.8848 (bia 0009.11f3.8848)
   MTU 1500 bytes, BW 100000 Kbit, DLY 100 usec,
      reliability 255/255, txload 1/255, rxload 1/255
   Encapsulation ARPA, loopback not set
   Full-duplex, 100Mb/s
   input flow-control is off, output flow-control is off
   ARP type: ARPA, ARP Timeout 04:00:00
   Last input 00:00:14, output 00:00:36, output hang never
   Last clearing of "show interface" counters never
   Input queue: 0/2000/0/0 (size/max/drops/flushes); Total output drops: 0
   Queueing strategy: fifo
   Output queue :0/40 (size/max)
   5 minute input rate 0 bits/sec, 0 packets/sec
   5 minute output rate 0 bits/sec, 0 packets/sec

The show interfaces command output up to this point is explained here (in order) :

• up, line protocol is up (connected) - The first "up" refers to the physical layer status of the interface. The "line protocol up" message shows the data link layer status of the interface and says that the interface can send and receive keepalives.

• MTU - The Maximum Transmission Unit (MTU) is 1500 bytes for Ethernet by default (for the max data portion of the frame).

• Full-duplex, 100Mb/s - Full-duplex and 100Mbps is the current speed and duplex setting of the interface. This does not tell you whether autoneg was used to achieve this. Use the show interfaces fas 6/1 status command to display this:

  Router#sh interfaces fas 6/1 status
  	
  	Port    Name               Status       Vlan       Duplex  Speed Type
  	Fa6/1                      connected    1          a-full  a-100 10/100BaseTX
  	!--- Autonegotiation was used to achieve full-duplex and 100Mbps.
  

• Last input, output - The number of hours, minutes, and seconds since the last packet was successfully received or transmitted by the interface. This is usefulto know when a dead interface failed.

• Last clearing of "show interface" counters - The last time the clear counters command was issued since the last time the switch was rebooted. The clear counters command is used to reset interface statistics.

  Note: Variables that can affect routing (for example, load and reliability) are not cleared when the counters are cleared.

• Input queue - The number of packets in the input queue. Size/max/drops = the current number of frames in the queue / the max number of frames the queue can hold before it must start dropping frames / the actual number of frames dropped because the max queue size was exceeded. Flushes is used to count Selective Packet Discard (SPD) drops on the the Catalyst 6000 Series that run Cisco IOS. (The flushes counter can be used but never increments on the Catalyst 4000 Series that run Cisco IOS.) SPD is a mechanism that quickly drops low priority packets when the CPU is overloaded in order to save some processing capacity for high priority packets. The flushes counter in the show interface command output increments as part of selective packet discard (SPD), which implements a selective packet drop policy on the IP process queue of the router. Therefore, it applies to only process switched traffic.

  The purpose of SPD is to ensure that important control packets, such as routing updates and keepalives, are not dropped when the IP input queue is full. When the size of the IP input queue is between the minimum and maximum thresholds, normal IP packets are dropped based on a certain drop probability. These random drops are called SPD flushes.

• Total output drops - The number of packets dropped because the output queue is full. A common cause of this might be traffic from a high bandwidth link being switched to a lower bandwidth link or traffic from multiple inbound links being switched to a single outbound link. For example, if a large amount of bursty traffic comes in on a gigabit interface and is switched out to a 100Mbps interface, this might cause output drops to increment on the 100Mbps interface. This is because the output queue on that interface is overwhelmed by the excess traffic due to the speed mismatch between the inbound and outbound bandwidths.

• Output queue - The number of packets in the output queue. Size/max means the current number of frames in the queue/the max number of frames the queue can hold before it is full and must start dropping frames.

• 5 minute input/output rate - The average input and output rate seen by the interface in the last five minutes. In order to get a more accurate reading by specifying a shorter period of time (to better detect traffic bursts for example), issue the load-interval <seconds> interface command.

The rest of the show interfaces command displays error counter output which is similar or equivalent to CatOS error counter output. See Table 1 for explanations of the error counter output.

!--- ...show interfaces command output continues.
     1117058 packets input, 78283238 bytes, 0 no buffer
      Received 1117035 broadcasts, 0 runts, 0 giants, 0 throttles
      0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored
      0 watchdog, 0 multicast, 0 pause input
      0 input packets with dribble condition detected
      285811 packets output, 27449284 bytes, 0 underruns
      0 output errors, 0 collisions, 2 interface resets
      0 babbles, 0 late collision, 0 deferred
      0 lost carrier, 0 no carrier
      0 output buffer failures, 0 output buffers swapped out

The show interfaces card-type {slot/port} counters errors command is the equivalent Cisco IOS command to show port counters for CatOS. See Table 1 for explanations of the error counter output.

Router#sh interfaces fastEthernet 6/1 counters errors
  
 Port        Align-Err    FCS-Err   Xmit-Err    Rcv-Err UnderSize OutDiscards
 Fa6/1               0          0          0          0         0           0
 
 Port      Single-Col Multi-Col  Late-Col Excess-Col Carri-Sen     Runts    Giants
 Fa6/1              0         0         0          0         0         0         0

Table 1:

CatOS error counter output for show port or show port counters for the Catalyst 6000, 5000 and 4000 Series. Cisco IOS error counter output for show interfaces or show interfaces card-type x/y counters errors for the Catalyst 6000 and 4000 Series.

Show Mac for CatOS and Show Interfaces Counters for Cisco IOS

The show mac {mod/port}command is useful when that run CatOS on the supervisor to monitor inbound and outbound traffic on the port as displayed by the receive (Rcv) and transmit (Xmit) counters for unicast, multicast, and broadcast traffic. This output is from a Catalyst 6000 that run CatOS:

Console> (enable) sh mac 3/1
  
  Port     Rcv-Unicast          Rcv-Multicast        Rcv-Broadcast
  -------- -------------------- -------------------- --------------------
   3/1                      177               256272                 3694
  
  Port     Xmit-Unicast         Xmit-Multicast       Xmit-Broadcast
  -------- -------------------- -------------------- --------------------
   3/1                       30               680377                  153
  
  Port     Rcv-Octet            Xmit-Octet
  -------- -------------------- --------------------
   3/1                 22303565             48381168
  
  MAC      Dely-Exced MTU-Exced  In-Discard Out-Discard
  -------- ---------- ---------- ---------- -----------
   3/1              0          0     233043          17
  
  Port  Last-Time-Cleared
  ----- --------------------------
   3/1  Sun Jun 1 2003, 12:22:47

This command also has these error counters: Dely-Exced, MTU-Exced, In-Discard and Out-Discard.

• Dely-Exced - The number of frames discarded by this port due to an excessive transmit delay through the switch. This counter must never go up unless the port is under very high utilization.

• MTU Exceed - This is an indication that one of the devices on that port or segment transmits more than the allowed frame size (1518 bytes for non-jumbo Ethernet).

• In-Discard - The result of inbound valid frames that were discarded because the frame did not need to be switched. This can be normal if a hub is connected to a port and two devices on that hub exchange data. The switch port still sees the data but does not have to switch it (since the CAM table shows the MAC address of both devices associated with the same port), and so it is discarded. This counter can also increment on a port configured as a trunk if that trunk blocks for some VLANs, or on a port that is the only member of a VLAN.

• Out-Discard - The number of outbound packets chosen to be discarded even though no packet errors have been detected. One possible reason to discard such a packet can be to free up buffer space.

Catalyst 4000 and 5000 series switches that run CatOS have two additional error counters in the show mac command. These are the In-Lost and Out-Lost counters:

MAC      Dely-Exced MTU-Exced  In-Discard Lrn-Discrd In-Lost    Out-Lost
-------- ---------- ---------- ---------- ---------- ---------- ---------- 
5/1              0          0          0          0          0          0

• In-Lost - On the Catalyst 4000, this counter is the sum of all error packets received on the port. The In-Lost counter on the Catalyst 5000, on the other hand, tracks the sum of all receive buffer failures.

• Out-Lost - On the Catalyst 4000 and 5000, these are outbound frames that were lost before they were forwarded (due to insufficient buffer space). This is commonly caused by oversubscribing the port.

The show interfaces card-type {slot/port} counters command is used when you run Cisco IOS on the Supervisor.

Note: There are no equivalent counters to the CatOS show mac error counters: Dely-Exced, MTU-Exced and In-Discard in this command There is, however, an Out-Discard counter in the Cisco IOS show interfaces counters errors command which is explained in Table 1.

Router#sh interfaces fas 6/1 counters
  
  Port            InOctets   InUcastPkts   InMcastPkts   InBcastPkts
  Fa6/1           47856076            23        673028           149
  
  Port           OutOctets  OutUcastPkts  OutMcastPkts  OutBcastPkts
  Fa6/1           22103793            17        255877          3280
  Router#
  !--- Cisco IOS counters used to monitor inbound and outbound unicast, multicast
  !--- and broadcast packets on the interface.

Show Counters for CatOS and Show Counters Interface for Cisco IOS

The show counters [mod/port] command offers even more detailed statistics for ports and interfaces. This command is available for CatOS and the equivalent show counters interface card-type {slot/port} command was introduced in Cisco IOS software version 12.1(13)E for the Catalyst 6000 series only. These commands display the 32-bit and 64-bit error counters per port or interface. Refer to the CatOS command documentation for show counters for more information.

Note: Counter stats for Catalyst 6000 series switches that run Cisco IOS are in Hex.

Console> (enable) sh counters 3/1
  64 bit counters
  0  rxHCTotalPkts                      =               260555
  1  txHCTotalPkts                      =               687411
  2  rxHCUnicastPkts                    =                  177
  3  txHCUnicastPkts                    =                   30
  4  rxHCMulticastPkts                  =               256684
  5  txHCMulticastPkts                  =               687228
  6  rxHCBroadcastPkts                  =                 3694
  7  txHCBroadcastPkts                  =                  153
  8  rxHCOctets                         =             22386167
  9  txHCOctets                         =             48850817
  10 rxTxHCPkts64Octets                 =               228929
  11 rxTxHCPkts65to127Octets            =               701493
  12 rxTxHCPkts128to255Octets           =                  285
  13 rxTxHCPkts256to511Octets           =                17090
  14 rxTxHCpkts512to1023Octets          =                  168
  15 rxTxHCpkts1024to1518Octets         =                    1
  16 txHCTrunkFrames                    =               395217
  17 rxHCTrunkFrames                    =               236459
  18 rxHCDropEvents                     =                    0
  32 bit counters
  0  rxCRCAlignErrors                   =          0
  1  rxUndersizedPkts                   =          0
  2  rxOversizedPkts                    =          0
  3  rxFragmentPkts                     =          0
  4  rxJabbers                          =          0
  5  txCollisions                       =          0
  6  ifInErrors                         =          0
  7  ifOutErrors                        =          0
  8  ifInDiscards                       =     233043
  9  ifInUnknownProtos                  =          2
  10 ifOutDiscards                      =         17
  !--- Output suppressed.

Show Controller Ethernet-Controller for Cisco IOS

For Catalyst 3750, 3550, 2970, 2950/2955, 2940, and 2900/3500XL switches use the command show controller ethernet-controller to display traffic counter and error counter output that is similar to the sh port, sh interface, sh mac and show counters output for Catalyst 6000, 5000 and 4000 series switches.

3550-1#sh controller ethernet-controller fastEthernet 0/1
  !--- Output from a Catalyst 3550.
  
    Transmit FastEthernet0/1           Receive
          0 Bytes                        0 Bytes
          0 Unicast frames               0 Unicast frames
          0 Multicast frames             0 Multicast frames
          0 Broadcast frames             0 Broadcast frames
          0 Discarded frames             0 No dest, unicast
          0 Too old frames               0 No dest, multicast
          0 Deferred frames              0 No dest, broadcast
          0  1 collision frames
          0  2 collision frames          0 FCS errors
          0  3 collision frames          0 Oversize frames
          0  4 collision frames          0 Undersize frames
          0  5 collision frames          0 Collision fragments
          0  6 collision frames
          0  7 collision frames          0 Minimum size frames
          0  8 collision frames          0 65 to 127 byte frames
          0  9 collision frames          0 128 to 255 byte frames
          0 10 collision frames          0 256 to 511 byte frames
          0 11 collision frames          0 512 to 1023 byte frames
          0 12 collision frames          0 1024 to 1518 byte frames
          0 13 collision frames
          0 14 collision frames          0 Flooded frames
          0 15 collision frames          0 Overrun frames
          0 Excessive collisions         0 VLAN filtered frames
          0 Late collisions              0 Source routed frames
          0 Good (1 coll) frames         0 Valid oversize frames
          0 Good(>1 coll) frames         0 Pause frames
          0 Pause frames                 0 Symbol error frames
          0 VLAN discard frames          0 Invalid frames, too large
          0 Excess defer frames          0 Valid frames, too large
          0 Too large frames             0 Invalid frames, too small
          0 64 byte frames               0 Valid frames, too small
          0 127 byte frames
          0 255 byte frames
          0 511 byte frames
          0 1023 byte frames
          0 1518 byte frames
 
 3550-1#
 !--- See table for additional counter output for 2900/3500XL Series switches.

Show Top for CatOS

The command show top allows you to collect and analyze data for each physical port on a switch. The command displays this data for each physical port:

• Port utilization (Uti %)

• Number of in and out bytes (Bytes)

• Number of in and out packets (Pkts)

• Number of in and out broadcast packets (Bcst)

• Number of in and out multicast packets (Mcst)

• Number of in errors (Error)

• Number of buffer-overflow errors (Overflow)

 Console> (enable) sh top
Start Time:     Mar 28 2007 06:58:41
End Time:       Mar 28 2007 06:59:11
PortType:       all
Metric:         util
Port  Band- Uti Bytes                Pkts       Bcst       Mcst       Error Over
      width  %  (Tx + Rx)            (Tx + Rx)  (Tx + Rx)  (Tx + Rx)  (Rx)  flow
----- ----- --- -------------------- ---------- ---------- ---------- ----- ----
 3/11  a-10   0               334187       1561         22       1536     0    0
 3/12 a-100   0               333608       1557         22       1532     0    0
 3/25 a-100   0               333622       1555         22       1533     0    0
 6/2   1000   0                    0          0          0          0     0    0
 6/1   1000   0                    0          0          0          0     0    0
 4/8   1000   0                    0          0          0          0     0    0
 4/7   1000   0                    0          0          0          0     0    0
 4/6   1000   0                    0          0          0          0     0    0
 4/5   1000   0                    0          0          0          0     0    0
 4/4   1000   0                    0          0          0          0     0    0
 4/3   1000   0                    0          0          0          0     0    0
 4/2   1000   0                    0          0          0          0     0    0
 4/1   1000   0                    0          0          0          0     0    0
 3/48  auto   0                    0          0          0          0     0    0
 3/47  auto   0                    0          0          0          0     0    0
 3/46  auto   0                    0          0          0          0     0    0

Note: When you calculate port utilization, the command bundles the Tx and Rx lines into the same counter and also looks at the full-duplex bandwidth when it calculates the percentage of utilization. For example, a Gigabit Ethernet port is 2000 Mbps full duplex.

In Errors is the sum of all error packets received on that port.

Buffer Overflow means that the port receivesg more traffic than it can store in its buffer. This can be caused a bursty traffic, as well an overrun of buffers. The suggested action is to decrease the transmission of the source device.

Also refer to 'In-Lost' and 'Out-Lost' counters from the show mac command.

Common System Error Messages

Cisco IOS sometimes has a different format for system messages. You can examine CatOS system messages and Cisco IOS system messages for a comparison. You can refer to the Messages and Recovery Procedures Guide for the release of software you run. For example, you can look at Messages and Recovery Procedures for CatOS software Version 7.6 and compare them to the Messages and Recovery Procedures for Cisco IOS 12.1 E Releases.

Error Messages on WS-X6348 Modules

Look at these error messages:

• Coil Pinnacle Header Checksum

• Coil Mdtif State Machine Error

• Coil Mdtif Packet CRC Error

• Coil Pb Rx Underflow Error

• Coil Pb Rx Parity Error

You can see syslog messages with one of those errors listed:

%SYS-5-SYS_LCPERR5:Module 9: Coil Pinnacle Header Checksum Error - Port #37

If you see this type of message or you notice that groups of 10/100 ports that fail on WS-X6348 modules, refer to these documents for further troubleshooting advice based on the operating system you use:

• Troubleshooting WS-X6348 Module Port Connectivity for Catalyst 6000 Using CatOS

• Troubleshooting WS-X6348 Module Port Connectivity on a Catalyst 6500/6000 that run Cisco IOS System Software

%PAGP-5-PORTTO / FROMSTP and %ETHC-5-PORTTO / FROMSTP

For CatOS, use the show logging buffer command to view stored log messages. For Cisco IOS, use the show logging command.

Console> (enable) sh logging buffer
  2003 Jun 02 20:12:43 %PAGP-5-PORTTOSTP:Port 3/2 joined bridge port 3/2
  2003 Jun 02 20:59:56 %PAGP-5-PORTFROMSTP:Port 3/1 left bridge port 3/1
  !--- This is the command to view the logging buffer on switches that run CatOS.

This message can cause concern for customers but for the most part it is informational in nature.

%PAGP-5-PORTTO / FROMSTP and %ETHC-5-PORTTO / FROMSTP 

The Port Aggregation Protocol (PAgP) negotiates Etherchannel links between switches. Whenever a device joins or leaves a bridge port, an informational message is displayed on the console. In most cases this message is completely normal, but if you see these messages on ports that should not flap for any reason, you must investigate further.

In CatOS software version 7.x and later, "PAGP-5" was changed to "ETHC-5" to make the message more understandable.

This message is specific to the Catalyst 4000, 5000, and 6000