BGP community-based peering policy - Part 2 - Customer services
Scaling BGP with communities - Part 2. The customer services design and implementation.
The BGP community attribute is an excellent tool for controlling and scaling BGP peering. Most big ISPs use the BGP communities for their internal policies and customer traffic engineering.
In Part 1 of the series, we discussed simple concepts on how the whole thing works and allowed to the customer to influence how our ISP will send him the traffic. This is ok for the simplest scenarios. In this Part 2, I will extend the basics and the internal working of such network design with some real-life customer services.
Prefix classifications - the implementation
Now that we know the scenario, we need to classify and tag all the prefixes received with communities for all the peers and assign a policy. We will also adjust the statically routed customers and our own prefixes. For the sake of the example, let's assume we have one upstream, IXP, and one peer. Our AS is 65432.
The community list and function description
|AS65432:1100||AS65432 aggregate sub-prefixes|
|AS65432:1500||Statically routed customer PI prefixes|
|AS65432:2000||Customers who get transit|
|AS65432:2100||Customers who get IXP|
|AS65432:2200||Customers who get customer routes|
|AS65432:3000||Routes learned from the IXP|
The community configuration will be as follows:
ip community-list 10 permit 65432:1000 ip community-list 11 permit 65432:1100 ip community-list 15 permit 65432:1500 ip community-list 20 permit 65432:2000 ip community-list 21 permit 65432:2100 ip community-list 22 permit 65432:2200 ip community-list 30 permit 65432:3000 ip community-list 40 permit 65432:4000 ip community-list 50 permit 65432:5000
The idea is to tag all the inbound prefixes with a community describing the service and filter them on the outbound so that the customer gets the prefixes from the partner network and vice versa.
Internally originated prefixes
Our AS65432 has some prefixes of its own; it also originates some prefixes, that are not our own - for example, prefixes of a customer, that is running his own Provider Independent prefix. Such configuration can look similar to this:
router bgp 65432 network 192.0.2.0 mask 255.255.255.0 route-map as65432-prefixes redistribute static route-map static-to-as65432 ! ip route 192.0.2.0 255.255.255.0 Null0 ! ip prefix-list as65432-block permit 192.0.2.0/24 le 32 ! route-map as65432-prefixes permit 10 set community 65432:1000 ! route-map static-to-as65432 permit 10 match ip address prefix-list as65432-block set community 65432:1100 ! route-map static-to-as65432 permit 20 set community 65432:1500
If we break down the configuration:
- we are originating the prefix
- in case we introduce any static route, it will be processed by a route-map
static-to-as65432and the decision will be made if it belongs to us via prefix-list
as65432-blockand receives community 65432:1100 (AS 65432 sub-prefixes) or not and receives 65432:1500 (statically routed PI).
Please bear in mind, that there are already some opinionated configuration details - for example - I don't have a problem introducing redistribution of static routes, because it is my design decision, I am aware of all the consequences and will act accordingly. Also, pointing an aggregate to Null to get the prefix has some alternatives. My approach is to originate my own prefixes everywhere, I will have the more specific networks from the IGP. I am sending traffic that doesn't have a route from my IGP to Null (I don't want to forward traffic to a destination that doesn't exist).
Our ISP offers its customers the following services:
- Full Transit (all routes)
- Full Transit (default plus AS65432)
- Upstream only (routes from the upstream providers)
- IXP only (routes from the Internet Exchange Point(s))
- BGP customers only (only routes of our directly connected customers)
For all the services, we will be adding our aggregates, our statically routed PI customers, and directly connected customers, as we don't want to utilize our upstream transit provider for this traffic when we can push it through the customer link.
|Service||AS65432 aggregates||AS65432 specifics||PI||Full transit customers||IXP||Upstream||Default|
|Transit (default + AS65432)||X||-||-||-||-||-||X|
|BGP customers only||X||-||X||X||-||-||-|
From a scalability point of view, the easiest way to implement such a solution is to use peer-groups or peer-templates. I will use peer-groups.
Services templates implementation
- Full Transit (all routes) service definition.
The customer is getting all the routes, from all sources (plus the default route)
router bgp 65432 neighbor transit-full peer-group neighbor transit-full route-map transit-full-out out neighbor transit-full route-map transit-full-in in ! route-map transit-full-out permit 10 match ip community 10 15 20 21 22 30 40 50 ! route-map transit-full-in permit 10 set community 65432:2000 65432:2100 65432:2200
- Full Transit (default route plus AS65432 aggregates) service definition.
The customer is getting a default route and our aggregates. The alternative here would be to do the
default-originatefor the peer-group with some route-map, validating the functionality of the upstream connectivity.
router bgp 65432 neighbor transit-default peer-group neighbor transit-default route-map transit-default-out out neighbor transit-default route-map transit-default-in in ! route-map transit-default-out permit 10 match ip community 10 50 ! route-map transit-default-in permit 10 set community 65432:2000 65432:2100 65432:2200
- Transit-only (Just the upstream provider prefixes) service definition.
The customer is getting just our upstream and our aggregates.
router bgp 65432 neighbor transit-only peer-group neighbor transit-only route-map transit-only-out out neighbor transit-only route-map transit-only-in in ! route-map transit-only-out permit 10 match ip community 10 15 20 22 50 ! route-map transit-only-in permit 10 set community 65432:2000
- IXP-only (Just the IXP prefixes) service definition.
The customer is getting just IXP prefixes and our aggregates.
router bgp 65432 neighbor ixp-only peer-group neighbor ixp-only route-map ixp-only-out out neighbor ixp-only route-map ixp-only-in in ! route-map ixp-only-out permit 10 match ip community 10 15 20 30 ! route-map ixp-only-in permit 10 set community 65432:2100
- BGP-only (Just the customer prefixes) service definition.
The customer is getting just other BGP customer prefixes, our aggregates, and PI prefixes.
router bgp 65432 neighbor bgp-only peer-group neighbor bgp-only route-map bgp-only-out out neighbor bgp-only route-map bgp-only-in in ! route-map bgp-only-out permit 10 match ip community 10 15 20 22 ! route-map bgp-only-in permit 10 set community 65432:2200
It is crucial to achieve symmetry, i.e., ensure that if you advertise customers' prefixes to a peer, they should get prefixes from that peer. If there is a mistake, you will face asymmetric routing issues.
Customer service activation
So let's assume we have one customer for each service. How will the service activation look after we are done with binging up the connectivity?
router bgp 65432 neighbor a.a.a.a remote-as 200 neighbor a.a.a.a peer-group transit-full neighbor a.a.a.a prefix-list pfx-as200-in neighbor b.b.b.b remote-as 300 neighbor b.b.b.b peer-group transit-default neighbor b.b.b.b prefix-list pfx-as300-in neighbor c.c.c.c remote-as 400 neighbor c.c.c.c peer-group transit-only neighbor c.c.c.c prefix-list pfx-as400-in neighbor d.d.d.d remote-as 500 neighbor d.d.d.d peer-group ixp-only neighbor d.d.d.d prefix-list pfx-as500-in neighbor e.e.e.e remote-as 600 neighbor e.e.e.e peer-group bgp-only neighbor e.e.e.e prefix-list pfx-as600-in
Customers are placed into the correct peer-group according to the service they are buying. As an additional security measure, we apply the prefix-list filtering on the inbound to the customers.
Note: Originating the default route
There is an everlasting debate about whether and how to originate the default route and how to check it is functional. Dummy generating of the default route to the customers can cause serious issues in case your upstream connectivity fails, as the route gets generated no matter what, and the traffic will get blackholed. It is wise to implement some sort of test that the upstream is alive. This topic is outside the scope of this article, you can read more about this topic in a great article by Ivan Pepelnjak here.
In this Part 2 of the series, we established a connectivity service offering and how to implement it with a Cisco router. In the next part, we will look at the economy of the ISP and how it translates to our community design.