If you decided to read the content of this post, you probably know something (or a lot) about Border Gateway Protocol (BGP) and its benefits. Or I might be wrong, and you have heard about this cool protocol numerous times, but just did not figure out how it works. In either case, feel free to proceed with reading.
The content of this article does not require a thorough understanding of BGP, but still, it is assumed you have a prior knowledge of routing protocols, and that you are familiar with the idea of interior/exterior routing protocols and neighbor relationships.
As you probably remember from the CCNA level, autonomous systems (AS in further text) are routing domains with their own independent routing policies and unique routing protocols. They usually fall under the administration of a single network engineer or organization.
The role of BGP is to route information between autonomous systems (mainly ISP networks). It is therefore also called an interdomain routing protocol.
From BGP’s perspective, the whole Internet is an entity of ASs, each of which is identified by a unique AS number. Routers that run a BGP process are referred to as BGP speakers. Routers that form neighbor relationship between each other in order to exchange routing information are referred to as BGP peers. When BGP peering is established between two routers within the same AS, we refer to internal BGP relationship, IBGP. When BGP peering is established between routers in different ASs, we refer to external BGP, EBGP.
The above figure represents the idea of IBGP and EBGP from the perspective of an Internet Service Provider (ISP). An ISP is running IBGP within its own autonomous system. Establishing an EBGP peering with another ISP enables the exchange of routes, that is, communication between ISPs.
To avoid routing loops, BGP specifies that routes learned via IBGP should not be advertised to other IBGP peers (BGP split-horizon rule). In order for all BGP routers internal to the AS learn all the external routes (from all external sources), they need to establish an internal BGP peering with all other internal BGP routers, that is, to form a full IBGP mesh (a network of interconnected IBGP peers). This solution is feasible if you have a small number of IBGP routers in the AS. With only 10 routers, you would need to maintain 45 BGP sessions; with 15 routers even more, 105 BGP sessions. Logically, as the number of routers increases, so does the number of BGP sessions required to establish a full mesh of IBGP peers. So, the solution is not so scalable, can get pretty complex with a large number of IBGP peers, and has the potential to affect the performance of the overall network. Namely, a large number of IBGP sessions might consume a significant amount of bandwidth.
The formula to calculate a number of BGP sessions, considering the number of routers is as follows:
IBGP sessions = n(n – 1)/2
In the figure above, every BGP router in the AS established 9 BGP sessions with all other BGP routers inside that AS, which means 45 IBGP sessions.
The second, more scalable option is to consider the implementation of route reflectors (RRs). Routers configured as RRs are allowed to propagate routes learned via IBGP to other IBGP peers in the AS, thereby violating a typical behavior of IBGP routers.
BGP Route Reflectors
Let us assume that the routers are already running BGP in your network. Moreover, you are dealing with a large network, consisting of hundreds of routers, which constitute a large internal BGP mesh. Tracking hundreds of BGP sessions in such a network seems quite an administratively demanding task, don’t you agree?
Instead of establishing an IBGP peering with each and every router of the internal BGP mesh, routers establish IBGP sessions with route reflectors only. RRs represent a focal point for IBGP sessions, and form a cluster together with its clients, that is, other routers within an internal BGP mesh.
Due to the fact that RRs propagate IBGP routes to other IBGP peers, a full mesh of IBGP peers, that is, peering between the clients is no longer needed. This saves a large number of IBGP sessions that should otherwise be maintained, and therefore reduces BGP routing traffic. You should ensure that the RRs are fully meshed with IBGP so that routes learned are propagated throughout the autonomous system. It is important to mention that RRs send only the best route specified by its BGP process, which further reduces the memory usage on route reflector clients compared to full IBGP mesh.
RRs will replicate (reflect) routes from IBGP router to all other IBGP routers, thereby violating a typical behavior of IBGP router. Which router will be a RR depends on your decision. It is recommended that you choose routers that have a good performance since RRs may increase processing overhead. Also, they should be configured cautiously, otherwise they might introduce loop in routing and route instability within your AS. A misconfigured route reflector can result in routes sent from one cluster to return to that same cluster.
It is strongly recommended that you implement a backup RR for redundancy in case the primary fails, thereby increasing the reliability of your network. With two RRs in one cluster, each should be configured with a cluster ID, which will allow them to recognize updates and discard routes from each other.
BGP Route Reflector Issues
Be aware that if you do not follow the recommendation of having at least one backup RR, a single RR failure will cause clients to be isolated from the rest of the network. In other words, having a single point of failure means inability to propagate routes throughout the autonomous system. Therefore, make sure you implement at least two route reflectors and that you establish an IBGP peering session between them.
As already said, the implementation of RRs poses a risk of creating a loop in routing within the AS. With poor configuration, there is a possibility that the route sent from one cluster returns to that same cluster. To maintain loop prevention, RRs use an optional ORIGINATOR_ID attribute. Namely, upon reflecting an IBGP route from one IBGP peer to another, the RR appends the ORIGINATOR_ID attribute, together with the cluster ID, to the route. ORIGINATOR_ID carries the router ID of the originator of the route. If a BGP router receives a route with the ORIGINATOR_ID set to its own router ID, it will ignore that route.
Configuring a Route Reflector
We will shortly go through a basic configuration of route reflector and its clients. In the figure below, route reflector clients established an IBGP peering via route reflector.
We should first configure a RR to peer with both Client A and Client B:
RR(config)# router bgp 200
RR(config-router)# neighbor 192.168.1.5 remote-as 200
RR(config-router)# neighbor 172.16.3.5 remote-as 200
We should also configure neighboring routers to establish a peering with RR:
Client_A(config)# router bgp 200
Client_A (config-router)# neighbor 192.168.1.6 remote-as 200
Client_B (config)# router bgp 200
Client_B (config-router)# neighbor 172.16.3.6 remote-as 200
Make sure to explicitly tell that the specified neighbors are route reflector clients:
RR(config)# router bgp 200
RR(config-router)# neighbor 192.168.1.5 route-reflector-client
RR(config-router)# neighbor 172.16.3.5 route-reflector-client
Before implementing RRs, you should consider the following:
- which routers will be the reflectors and which will be the clients
- will you divide a single AS into multiple clusters, each having a single or more RRs, or
- the whole AS will be a single cluster with one or several RRs
It all depends on the size of your network/AS, and your administrative demands and preferences. When choosing a route reflector, it is also recommended to consider your physical topology as an indicator of which router would be the best candidate. A physical redundancy may very well have an impact on the logical route reflector redundancy. If the RR becomes physically isolated from some part of the AS, it might not be able to reflect the routes to IBGP routers residing in another part of that same AS.
Bear in mind that implementing at least two RR in the autonomous system means increased redundancy and the avoidance of a single point of failure.
Keep calm and love BGP, till next time!
Can you add some detail into configuring dual RR. Does the RR need to peer IBGP with the other RR ? Do they peer as RR client or just plain IBGP ?
simple and straight to the point
18266 109682 An fascinating discussion is worth comment. I believe which you need to write a lot more on this subject, it may well not be a taboo topic but typically folks are not enough to speak on such topics. Towards the next. Cheers 428959
i am very new to BGP and having hell lot of doubts regarding BGP.
Thanks for post.
Thanks very useful!
I hope one day to manage a BGP network, so far only OSPF experiences.
Hi Ingrid !!
Nice article 🙂
I’m just gonna be a little ‘picky’ here ….
(hope you’ll take some constructive feedback in high spirit 🙂
– “In order to learn all the routes WITHIN the AS, routers should establish an internal BGP peering with each other, that is, form a full IBGP mesh”.
I would re-write this to:
– “In order for all BGP routers internal to the AS to learn all the external routes (from all external sources), they need to establish an internal BGP peering with all other internal BGP routers, that is; to form a full IBGP mesh”.
Your sentence could lead people to think that BGP knows about routes WITHIN the AS !
(Which they typically don’t !)
BGP is essentially only used to learn the external routes (from outside the AS).
To learn the internal routes you’ll always – unless you’re a masochist – use an IGP as IS-IS or OSPF. (As you’ll probably be very aware of 🙂
And many fun aspects of the manual nature of BGP and the synchronization between EGP and IGP then becomes challenging and tricky issues in vendor certifications and IRL … LOL …
Basically it’s very puzzling that we still rely on a mostly manual protocol as BGP, and even enhance it’s capabilities with MP-BGP, etc. Instead of redesigning a dynamic protocol from the ground up. But I guess the World would be without BGP Guru’s and the whole ‘hero-mysticism’ surrounding these, if we did that 🙂
(Even Mckeown at Stanford and the SDN/NFV movement hasn’t dared to question the ‘hero-myth’ …
and we would also have to disrupt the daily life of the Internet 🙂
Hi Mikael, thank you for the feedback. I agree with a proposed change, thanks 🙂
Pingback: BGP Network Design - BGP Confederation - Router Freak