OpManager showing to many Interface Discard Alarms

OpManager showing to many Interface Discard Alarms

Description

OpManager application is raising a lot of alarms for High Discard rates on mdl0 interface. Alarms on the SRP Lab in Chch, this might make it easier for Dev to capture and analyse the problem. Similar to SRP I see that the discards have increase as well. They were only about 2% but now we see much higher rates
Discards rate for Interface 'mdl0-mdl0' is 11.961%, threshold value for this monitor is 1%.RRU-1-700WirelessUnAssignedTrouble18 minutes ago
Discards rate for Interface 'mdl0-mdl0' is 38.475%, threshold value for this monitor is 2%.RRU-3-700WirelessUnAssignedTrouble18 minutes ago
Discards rate for Interface 'mdl0-mdl0' is 11.861%, threshold value for this monitor is 2%.RRU-1-700WirelessUnAssignedTrouble18 minutes ago
Discards rate for Interface 'mdl0-mdl0' is 11.861%, threshold value for this monitor is 2%.RRU-1-700WirelessUnAssignedTrouble18 minutes ago

Problem

There are many packets types and virtually infinite number of configurations in which a large number of IP packets can be dropped. Being shocked by a large number because it is LARGE isn't an issue if you can explain its origins.

Some protocols (like PVST) can be transmitted and dropped all the time (by either Tx or Rx interface, again, depends on configuration). This issue applies to other packets, imagine that tomorrow someone reconfigures the local switch and add a feature which would generate a new packet stream across the Air interfaces, those packets from the protocol may be unregistered with radio config and hence it might drop the data, adding to the counter. This is usually done unintentionally and no one will appreciate the side-effect. 

Diagnostics

Logs show the mdl0 Rx-Drop count was high on RRU-1-700, but then it was noted that the firewall configuration on Tornado-bases (1+1) was disabled. 

After enabling firewall on 1+1 pair to prevent the PVST packets from going over air (mdl0) link now which increase the Rx-drop count on mdl0 of RRU-T. After the firewall rule applied, I see the Rx-drop count is stable now on RRU-1-700. Let's monitor the same. 
Yakir Matusovsky the double-tagged packets are too heavy for the ghost to deal with :-)

We've previously done the exercise of;
  1. Capturing traffic using TCPDump
  2. Checking which packets are sent and analys whcich could potentially be dropped, Requires an understanding of what data the MMX Radio will process vs Drop.
  3. Seeing how the dropped counters can ignore or filter them out before counted.

Solution

In the latest case there was a need to add firewall rules to drop PVST packets on BRU-Ts. the configuration can be added as below:

In Routed mode:
  1. configure security firewall rule routed drop lan wan -proto pvst
In Bridged mode:
  1. configure security firewall rule bridged drop bridge bridge -proto pvst