Raslog message TO-1006 and HA Out Of Sync issue observed on Gen7 Brocade switch due to TSB 2023-289-A
Applies to
- Brocade Gen 7 directors (X7-4 & X7-8) and G7** switch
- FOS v9.1.x
Issue
- HA Out Of Sync issue observed on X7 director switch post upgrading the FOS to version 9.1.1b.
- CP failover happened with error code
RAS-1004
.
2023/05/05-23:43:50 (IST), [SULB-1005], 692023, SLOT 1 | CHASSIS, INFO, switch, Current Active CP is preparing to failover.
2023/05/05-23:43:50 (IST), [RAS-1007], 692024, SLOT 1 | CHASSIS, INFO, switch, System is about to reload.
2023/05/05-23:43:54 (IST), [FSSM-1003], 692025, SLOT 1 | CHASSIS, WARNING, switch, HA State out of sync.
2023/05/05-23:41:04 (IST), [HAM-1004], 692026, SLOT 1 | CHASSIS, INFO, switch, Processor rebooted - HaFailover.
-
Gen 7 directors and switches that have encountered an oversubscription management event will observe the following Traffic Optimizer RASlog in
errdump
as shown below:2023/06/16-08:57:56 (+08),[TO-1006], 1011618/1002267, FID 128, INFO, Switch_100, Flows destined to xxx device have been moved to PG_OVER_SUBSCRIPTION_4G_16G PG
Note: Gen 7 products running on FOS v9.0.x are not at risk to these failure conditions.
- Additional symptoms that could appear due to these identified issues could be as follows:
1) Large counts of CRC errors on a link may be observed that are not fixed with optic / cable replacement.
2) Frames may be discarded, credit on a link can be lost
3) Ports may be faulted, ASIC may halt and be faulted.
4) A director may observe an unexpected HA fail-over or even a cold restart of the director
5) Switches may observe a cold restart