BGP community-based peering policy - Part 1 - Principles

Introduction

The BGP community attribute is an excellent tool for controlling and scaling BGP peering. Most big ISPs use the BGP communities for their internal policies and customer traffic engineering.

I would like to go through some basic design options and principles in this article.

The use of BGP community attribute for multi-homed scenarios is described in RFC1998. This document is quite aged but still contains valuable information we can build on. The RFC1998 has been extended for use with BGP Large Communities with RFC8092 and RFC8195. However, BGP Large communities are outside of the scope of this post. Implementation should be quite similar, though.

Simple scenario - dual-homing

RFC1998 uses an example of MCI, how to do load-sharing and backup with multiple upstream AS links. The BGP communities, in this case, determine the local preference in the upstream network. This, in turn, gives control to the customer, who can then use a special community himself, without having to call his ISP technical support. Which is great for both parties :)

As a basis, the RFC1998 uses an example of MCI network, for example:

If this is implemented on the ISP PE device, the customer can control how the ISP will handle his traffic. As an example. The customer is dual-homed to AS100 and wants to use that particular path as a backup path, he would announce his prefix to that peer with a community of 100:90. The ISP would then, based on this received community, set the local preference for this prefix to be 90.

Sample configuration of the customer router (Cisco IOS, IOS-XE). Please note this is just a minimal example. The customer originates only his own prefixes (via the as-path ACL) and sets the community as described above.

router bgp 200
 address-family ipv4 unicast
  neighbor 192.0.2.1 remote-as 100
  neighbor 192.0.2.1 description ISP backup path
  neighbor 192.0.2.1 route-map as100-out out
  neighbor 192.0.2.1 send-community
  neighbor 192.0.2.1 activate
! 
ip as-path access-list 10 permit ^$
!
route-map as100-out permit 10
 match as-path 10
 set community 100:90

So now the customer is sending his prefix with a community of 100:90. How is the configuration in the ISP PE router?

router bgp 100
 address-family ipv4 unicast
  neighbor 192.0.2.2 remote-as 200
  neighbor 192.0.2.2 route-map customer-in in
  neighbor 192.0.2.2 activate
!
! Homed to another ISP
ip community-list 7 permit 100:70
! Homed to another ISP with same ASPATH length
ip community-list 8 permit 100:80
! Customer backup routes
ip community-list 9 permit 100:90
!
route-map customer-in permit 10
 match community 7
 set local-preference 70
!
route-map customer-in permit 20
 match community 8
 set local-preference 80
!
route-map customer-in permit 30
 match community 9
 set local-preference 90
!
route-map customer-in permit 40
 set local-preference 100

From the ISP perspective, this looks more complicated than the usual per-customer peering and configuration, but it scales better in the long run.

This was a simple example based on RFC1998. It is probably ok for very simple dual-homed situations. Typical ISP has to handle more and more complex situations. Therefore they need to have a more complex policy that will ease the operations and provide their customers with more policy control.

Typical ISP usage

A more complex scenario would allow more control for the customer. In this case, we would like to allow customers to change local preference and our as-prepending towards our upstream providers. There can also be a situation when the customer is under attack. In that case, he can force us to blackhole all the traffic coming to his network with Remote-Triggered Black Hole (RTBH) functionality.

The actual example implementation of this is going to look like this (in the example, we have one customer and one upstream provider.

router bgp 100
 address-family ipv4 unicast
  neighbor 192.168.0.2 remote-as 200
  neighbor 192.168.0.2 route-map customer-in in
  neighbor 192.168.0.2 activate
  neighbor 10.0.0.2 remote-as 300
  neighbor 10.0.0.2 route-map upstream-out out
  neighbor 10.0.0.2 activate
!
ip community-list 1 permit 100:1
ip community-list 2 permit 100:2
ip community-list 3 permit 100:3
ip community-list 80 permit 100:80
ip community-list 120 permit 100:120
ip community-list 666 permit 100:666
!
!
ip route 192.0.2.1 255.255.255.255 Null0
!
!
route-map customer-in permit 10
 match community 80
 set local-preference 80
!
route-map customer-in permit 20
 match community 120
 set local-preference 120
!
route-map customer-in permit 30
 match community 666
 set ip next-hop 192.0.2.1
!
route-map customer-in permit 40
 set local-preference 100
!
!
route-map upstream-out permit 10
 match community 1
 set as-path prepend 100
!
route-map upstream-out permit 20
 match community 2
 set as-path prepend 100 100
!
route-map upstream-out permit 30
 match community 3
 set as-path prepend 100 100 100
!
route-map upstream-out permit 40
! etc., etc.
  

In summary, what can the customer do with such a configuration on his ISP side?

• Change local preference on the ISP side, by using communities 100:80 or 100:120, which can change which path the ISP uses to send the traffic to the customer. This is useful for maintenance - the customer changes his primary line himself without any outage caused by waiting for BGP timers to time out.
• Change ISP prepend behavior towards his upstream provider with community 100:1, 100:2, and 100:3 - can be useful in multi-homed scenarios to influence the prefix behavior in the internet (i.e., we want the traffic to go via the other ISP we have).
• Black-hole a prefix in case he is under DoS attack so that it is no longer reachable with RTBH triggered by the use of the 100:666 community.

Most Internet Service Providers publish their peering policy and list the communities you can use with descriptions of what it is doing. For example, here is the Verizon Business BGP policy as published in RIPE DB. We will cover most of these in this series.

VzBi uses the following communities with its customers: 
702:80 Set Local Pref 80 within AS702 
702:120 Set Local Pref 120 within AS702 
702:20 Announce only to VzBi AS'es and VzBi customers 
702:30 Keep within Europe, don't announce to other VzBi AS's 
702:1 Prepend AS702 once at edges of VzBi to Peers 
702:2 Prepend AS702 twice at edges of VzBi to Peers 
702:3 Prepend AS702 thrice at edges of VzBi to Peers 

Advanced communities for customers:
702:7020 Do not announce to AS702 peers with a scope of National but advertise to Global Peers, European Peers and VzBi customers. 
702:7001 Prepend AS702 once at edges of VzBi to AS702 peers with a scope of National. 
702:7002 Prepend AS702 twice at edges of VzBi to AS702 peers with a scope of National. 
702:7003 Prepend AS702 thrice at edges of VzBi to AS702 peers with a scope of National. 
702:8020 Do not announce to AS702 peers with a scope of European but advertise to Global Peers, National Peers and VzBi customers. 
702:8001 Prepend AS702 once at edges of VzBi to AS702 peers with a scope of European. 
702:8002 Prepend AS702 twice at edges of VzBi to AS702 peers with a scope of European. 
702:8003 Prepend AS702 thrice at edges of VzBi to AS702 peers with a scope of European.

Summary

This was a brief introduction to a basic BGP peering policy with BGP communities. In the next posts, we will look at further possibilities of using communities for internal traffic engineering and external service design and scaling.