Sunday, June 15, 2008

OSPF - overload protection

This lab again uses the topology of another OSPF post:

R2 ---> R5 in area 25
R5 ---> R4 in area 0
R4 ---> R3 in area 345

Area 25 is an NSSA.
R2 is an ASBR redistruting connected and static routes into OSPF:

R2#show run | section router ospf
router ospf 1
log-adjacency-changes
area 25 nssa
redistribute maximum-prefix 4 50 warning-only
redistribute connected subnets tag 100
redistribute static metric-type 1 subnets tag 200
network 172.12.25.0 0.0.0.255 area 25

R4 has the following ospf database:

R4# show ip ospf database

OSPF Router with ID (4.4.4.4) (Process ID 1)

Router Link States (Area 0)

Link ID ADV Router Age Seq# Checksum Link count
4.4.4.4 4.4.4.4 70 0x80000003 0x009BEB 3
5.5.5.5 5.5.5.5 112 0x8000000D 0x0093E2 3
6.6.6.6 6.6.6.6 71 0x8000000F 0x0050F6 4

Summary Net Link States (Area 0)

Link ID ADV Router Age Seq# Checksum
3.3.3.3 4.4.4.4 124 0x80000001 0x009A84
4.4.4.4 4.4.4.4 124 0x80000001 0x0062B9
172.12.25.0 5.5.5.5 982 0x80000006 0x000210
172.12.34.0 4.4.4.4 124 0x80000001 0x004E03

Router Link States (Area 345)

Link ID ADV Router Age Seq# Checksum Link count
3.3.3.3 3.3.3.3 140 0x80000014 0x009F90 2
4.4.4.4 4.4.4.4 121 0x80000002 0x00F23E 2

Net Link States (Area 345)

Link ID ADV Router Age Seq# Checksum
172.12.34.3 3.3.3.3 140 0x80000001 0x0089AA

Summary ASB Link States (Area 345)

Link ID ADV Router Age Seq# Checksum
5.5.5.5 4.4.4.4 106 0x80000001 0x009E39

Type-5 AS External Link States

Link ID ADV Router Age Seq# Checksum Tag
1.1.1.1 5.5.5.5 459 0x80000002 0x00BB83 200
2.2.2.2 5.5.5.5 459 0x80000002 0x00021D 100
2.2.2.3 5.5.5.5 459 0x80000002 0x00F726 100
2.2.2.4 5.5.5.5 459 0x80000002 0x00ED2F 100
2.2.2.5 5.5.5.5 459 0x80000002 0x00E338 100
5.5.5.5 5.5.5.5 985 0x80000006 0x006013 0
172.12.123.0 5.5.5.5 459 0x80000002 0x00BA38 100
192.168.254.0 5.5.5.5 459 0x80000002 0x004397 200

There are 19 LSAs here, 6 of them generated by this router (router-id = 4.4.4.4) for a total of 13 non-self-generated LSAs. We can limit the number of these such LSA received by the OSPF process as follows:

R4(config)#router ospf 1
R4(config-router)#max-lsa ?
<1-4294967294> Maximum number of non self-generated LSAs this process can receive

R4(config-router)#max-lsa 14 ?
<1-100> Threshold value (%) at which to generate a warning msg

R4(config-router)#max-lsa 14 50 ?
ignore-count maximum number of times adjacencies can be suppressed
ignore-time time during which all adjacencies are suppressed

reset-time time after which ignore-count is reset to zero

warning-only Only give warning message when limit is exceeded


R4(config-router)#max-lsa 14 50 ignore-count 1 ignore-time ?
<1-17895> number of minutes during which all adjacencies are suppressed

R4(config-router)#max-lsa 14 50 ignore-count 1 ignore-time 1 reset-time ?
<2-35791> number of minutes after which ignore-count is reset to zero

R4(config-router)#max-lsa 14 50 ignore-count 1 ignore-time 1 reset-time 2

After the command is entered, the adjacencies are reset and th follwoing log message appears when the adjacencies come back up.

R4(config-router)#
Mar 1 03:07:58.209: %OSPF-4-OSPF_MAX_LSA_THR: Threshold for maximum number of n
on self-generated LSA has been reached "ospf 1" - 7 LSAs

So we have passed the 50% threshold and are near the limit, let's add a new LSA buy creating a new loopback being advertised on a distant router:

R2(config)#int lo 4
R2(config-if)#ip address 2.2.2.6 255.255.255.255

Nothing happened on R4, so let's add another network to OSPF:

R2(config)#int lo 5
R2(config-if)#ip address 2.2.2.7 255.255.255.255

Now we see something:

Mar 1 03:10:59.144: %OSPF-4-OSPF_MAX_LSA: Maximum number of non self-generated
LSA has been exceeded "ospf 1" - 15 LSAs

One minute later the adjacencies are dropped:

Mar 1 03:11:59.264: %OSPF-5-ADJCHG: Process 1, Nbr 6.6.6.6 on Serial1/0 from FULL to DOWN, Neighbor Down: Interface down or detached
Mar 1 03:11:59.272: %OSPF-5-ADJCHG: Process 1, Nbr 5.5.5.5 on Serial1/0 from FULL to DOWN, Neighbor Down: Interface down or detached
Mar 1 03:11:59.320: %OSPF-5-ADJCHG: Process 1, Nbr 3.3.3.3 on FastEthernet0/1 from FULL to DOWN, Neighbor Down: Interface down or detached

R4#show ip ospf neighbor

R4#

After 1 minute the adjacencies come back up:

Mar 1 03:13:01.403: %OSPF-4-OSPF_MAX_LSA_THR: Threshold for maximum number of non self-generated LSA has been reached "ospf 1" - 7 LSAs
Mar 1 03:13:01.407: %OSPF-5-ADJCHG: Process 1, Nbr 3.3.3.3 on FastEthernet0/1 from LOADING to FULL, Loading Done
Mar 1 03:13:16.335: %OSPF-5-ADJCHG: Process 1, Nbr 5.5.5.5 on Serial1/0 from LOADING to FULL, Loading Done
Mar 1 03:13:27.123: %OSPF-5-ADJCHG: Process 1, Nbr 6.6.6.6 on Serial1/0 from LOADING to FULL, Loading Done
R4#
R4#show ip ospf neighbor

Neighbor ID Pri State Dead Time Address Interface
6.6.6.6 0 FULL/ - 00:01:44 172.12.45.6 Serial1/0
5.5.5.5 0 FULL/ - 00:01:44 172.12.45.5 Serial1/0
3.3.3.3 1 FULL/BDR 00:00:35 172.12.34.3 FastEthernet0/1

-An Anti-summary-

Let's review exactly what happens with these timers with a new example On R4:

R4(config)#router ospf 1
R4(config-router)#max-lsa 15 50 ignore-count 1 ignore-time 2 reset-time 5

R2(config-if)#int lo 6
R2(config-if)#ip address 2.2.2.8 255.255.255.255

R4#
Mar 1 03:24:54.040: %OSPF-4-OSPF_MAX_LSA: Maximum number of non self-generated
LSA has been exceeded "ospf 1" - 16 LSAs

Neighbors are still up:

R4#show ip ospf neighbor

Neighbor ID Pri State Dead Time Address Interface
6.6.6.6 0 FULL/ - 00:01:38 172.12.45.6 Serial1/0
5.5.5.5 0 FULL/ - 00:01:57 172.12.45.5 Serial1/0
3.3.3.3 1 FULL/DR 00:00:32 172.12.34.3 FastEthernet0/

Neighbors finally come down after 1 minute:

Mar 1 03:25:54.160: %OSPF-5-ADJCHG: Process 1, Nbr 6.6.6.6 on Serial1/0 from FULL to DOWN, Neighbor Down: Interface down or detached
Mar 1 03:25:54.168: %OSPF-5-ADJCHG: Process 1, Nbr 5.5.5.5 on Serial1/0 from FULL to DOWN, Neighbor Down: Interface down or detached
Mar 1 03:25:54.212: %OSPF-5-ADJCHG: Process 1, Nbr 3.3.3.3 on FastEthernet0/1 from FULL to DOWN, Neighbor Down: Interface down or detached

After 2 minutes, the adjacencies should come back up, you can view th time remaining as follows:

R4#show ip ospf | section Maximum number of non self-generated
Maximum number of non self-generated LSA allowed 15
Threshold for warning message 50%
Ignore-time 2 minutes, reset-time 5 minutes
Ignore-count allowed 1, current ignore-count 1
Ignoring all neighbors due to max-lsa limit, time remaining: 00:01:06

R4#show ip ospf | section Maximum number of non self-generated
Maximum number of non self-generated LSA allowed 15
Threshold for warning message 50%
Ignore-time 2 minutes, reset-time 5 minutes
Ignore-count allowed 1, current ignore-count 1
Ignoring all neighbors due to max-lsa limit, time remaining: 00:00:25

Adjacencies come back up after two minutes and the warning threshold is reached:

Mar 1 03:27:57.127: %OSPF-4-OSPF_MAX_LSA_THR: Threshold for maximum number of non self-generated LSA has been reached "ospf 1" - 7 LSAs
Mar 1 03:27:57.127: %OSPF-5-ADJCHG: Process 1, Nbr 6.6.6.6 on Serial1/0 from LOADING to FULL, Loading Done
Mar 1 03:28:01.367: %OSPF-5-ADJCHG: Process 1, Nbr 3.3.3.3 on FastEthernet0/1 from LOADING to FULL, Loading Done
Mar 1 03:28:24.531: %OSPF-5-ADJCHG: Process 1, Nbr 5.5.5.5 on Serial1/0 from LOADING to FULL, Loading Done

R4#show ip ospf neighbor

Neighbor ID Pri State Dead Time Address Interface
5.5.5.5 0 FULL/ - 00:01:59 172.12.45.5 Serial1/0
6.6.6.6 0 FULL/ - 00:01:59 172.12.45.6 Serial1/0
3.3.3.3 1 FULL/BDR 00:00:33 172.12.34.3 FastEthernet0/1

Let's look at the next timer in effect:

R4#show ip ospf | section Maximum number of non self-generated
Maximum number of non self-generated LSA allowed 15
Threshold for warning message 50%
Ignore-time 2 minutes, reset-time 5 minutes
Ignore-count allowed 1, current ignore-count 1 - time remaining: 00:03:12

Notice that we have already ignored our neighbors once - this was the ignore-count in the configuration. We cannot ignore our neighbors again until the ignore count is reset at 5 minutes. While we waiting for the timer to count down, R4 has normal adjacencies with all it's neighbors and a full LSDB.

Roughly 5 minutes later and nothing has happened again:

R4#show clock
03:35:09.169 UTC Fri Mar 1 2002

R4#show ip ospf neighbor

Neighbor ID Pri State Dead Time Address Interface
5.5.5.5 0 FULL/ - 00:01:34 172.12.45.5 Serial1/0
6.6.6.6 0 FULL/ - 00:01:45 172.12.45.6 Serial1/0
3.3.3.3 1 FULL/BDR 00:00:39 172.12.34.3 FastEthernet0/1

R4#show ip ospf | section Maximum number of non self-generated
Maximum number of non self-generated LSA allowed 15
Threshold for warning message 50%
Ignore-time 2 minutes, reset-time 5 minutes
Ignore-count allowed 1, current ignore-count 0

hmmm....what gives?

Well it turns out I lost an LSA somewhere a long the way so I still had 15 after adjacencies came back up! Very strange...I quickly created a new one on R2 and we'll try to catch up to where we were...

I'm back waiting for the 5 minute timer to expire. In the meantime adjacencies are up and max-lsa limit has been reached again.

Mar 1 03:41:31.451: %OSPF-4-OSPF_MAX_LSA: Maximum number of non self-generated
LSA has been exceeded "ospf 1" - 16 LSAs

R4#show ip ospf | section Maximum number of non self-generated
Maximum number of non self-generated LSA allowed 15
Threshold for warning message 50%
Ignore-time 2 minutes, reset-time 5 minutes
Ignore-count allowed 1, current ignore-count 1 - time remaining: 00:04:47

R4#show ip ospf neighbor

Neighbor ID Pri State Dead Time Address Interface
5.5.5.5 0 FULL/ - 00:01:46 172.12.45.5 Serial1/0
6.6.6.6 0 FULL/ - 00:01:57 172.12.45.6 Serial1/0
3.3.3.3 1 FULL/BDR 00:00:31 172.12.34.3 FastEthernet0/1

But wait! Adjacencies come down about a minute later:

R4#
Mar 1 03:42:31.575: %OSPF-5-ADJCHG: Process 1, Nbr 5.5.5.5 on Serial1/0 from FULL to DOWN, Neighbor Down: Interface down or detached
Mar 1 03:42:31.579: %OSPF-5-ADJCHG: Process 1, Nbr 6.6.6.6 on Serial1/0 from FULL to DOWN, Neighbor Down: Interface down or detached
Mar 1 03:42:31.627: %OSPF-5-ADJCHG: Process 1, Nbr 3.3.3.3 on FastEthernet0/1 from FULL to DOWN, Neighbor Down: Interface down or detached

Notice the last line:

R4#show ip ospf | section Maximum number of non self-generated
Maximum number of non self-generated LSA allowed 15
Threshold for warning message 50%
Ignore-time 2 minutes, reset-time 5 minutes
Ignore-count allowed 1, current ignore-count 2
Permanently ignoring all neighbors due to max-lsa limit

R4#show ip ospf neighbor

R4#

The configured ignore-count is 1 so upon reaching the max-lsa limit the second time, all neighbors are now ignored until ospf is cleared manually. Well this was a pretty helter skelter lab. I hope it gives you some idea of what overload protection does, and maybe gives you some ideas for testing it yourself.

3 comments:

  1. A follow up to this post is in need. I read more about the ignore-count option and this what I found from cisco doc:

    "Each time the OSPF process gets into an ignore state a counter is incremented. If this counter exceeds the number of minutes configured by the ignore-count keyword, the OSPF process stays permanently in the same ignore state and manual intervention is required to get the OSPF process out of the ignore state. "

    This is what happened after we reached the max-lsa limit the 2nd time. Our ignore-count was 1, so the 2nd time caused the router to permanently ignore all neighbors.

    More can be found by googling "OSPF Link-State Database Overload Protection" and reading the first link.

    ReplyDelete
  2. Agreed...
    http://www.cisco.com/en/US/partner/docs/ios/iproute/configuration/guide/irp_ospf_lk_state_db_ps6441_TSD_Products_Configuration_Guide_Chapter.html

    ReplyDelete
  3. Thanks for the link. I got a lot of my ideas from the DocCD. That is one of the best ways to practice!

    ReplyDelete

Note: Only a member of this blog may post a comment.