Making it ping: Setting up Your First (OpenBSD) Router (Part 3)
OpenBSD version: 7.0
Arch: Any
NSFP: Mostly
After i talked about why one might want to run their own AS, and how one might end up with their own ASN and network(s), we will go over actually making it ping today: Setting up our first router. To make that happen, we first have to do some housekeeping. We need somebody that allows us to announce our network (an ‘upstream’), and as soon as soon as we know who our upstream will be, we also need to set some objects in the IRR (Internet Routing Registry) responsible for our network(s) and ASN. So, let’s get to it.
Finding a Place to Announce
Finding an upstream is not difficult. Quite quickly after you got your ASN, or maybe even became a LIR, you will most likely get a call (remember the earlier note on privacy) from a +44 phone number, with a nice chatty person on the other end of the line up for a nice conversation about your infrastructure and your upstream needs. While this will be, quite obviously, your first contact with an NSP’s (Network Service Provider) acquisition team, they will most likely be all to happy to provide upstream services to you,… for a fee. The same holds for nearly every other Tier1/2 network out there, even though they may not directly give you a call.
The thing is: No matter if you talk to chatty people with a +44 number, or anyone else, they will most likely think about upstream in multiples of 10gE (at least), for a monthly fee that usually exceeds even the budget of very ambitious hobby projects. They are, after all, providing a B2B (busines-to-business) service for, well, business needs.
Luckily, there are also many smaller options, in the form of various hosters which offer you to speak BGP with them, even when just renting a small VM from them. There are even some operators that offer you BGP sessions via various forms of IP tunnels.
In addition, if you are doing ‘not for profit’ things, which may be considered ‘for the good of the Internet’ (and you learning may just be part of that) there is also a lot of community driven options out there. You may also find the overall community of operators rather receptive (or not, see opinions i discussed before) of helping out smaller netizens. I personally rented a virtual machine from IN-BERLIN, an NGO enabling participation on the Internet for not-for-profit activities to setup my first router (and will use them as the example from here on on). Similarly, you may know netops at a (larger) provider who might slip you a (tunnel-based) upstream, or similar initiatives, where a bit of additional traffic does not really blow the bill. In the end, what all these community-based options have in common is that there are a lot of open doors, which will close surprisingly quickly if you start to exploit good will, i.e., start overusing these options or try to commercially exploit them. Network operators will usually not hold a grudge against you for an honest mistake (especially if owned up to it), but they will cultivate a longstanding grudge if you purposefully exploit their good will. So, if you are planning on making your AS a business, you better do pick up on that number starting with +44.
Whomever you will ultimately find to provide you upstream services, you will need a (somewhat) direct link between your first router and one of their BGP speaking routers. If you find someone to provide you upstream via a tunnnel, this means setting up a tunnel (we will get into the specifics of that in another post). Alternatively, you will usually either colocate/rent hardware in their datacenter (or a carrier-neutral datacenter where they are present), or rent a virtual machine from them.
Setting Route Objects
As mentioned in Part 1, ASes announcing things they should not announce is a major problem. To counter that, people record who announces what (to whom) in the IRRs. Of course there is also RPKI, but we will go about that in its own post. For now, setting up route objects, and recording your upstreams in your AS object should be sufficient to get things pinging. Please keep in mind that many operators generate filters based on IRR databases, and only do so in intervals of up to a day; So you should consider some lag until your published route objects become effective.
For my networks, I started out with the following two route objects:
IPv4:
route: 195.191.196.0/23
origin: AS59645
mnt-by: WYBT-MNT
created: 2022-01-31T16:40:28Z
last-modified: 2022-01-31T16:40:28Z
source: RIPE
IPv6:
route6: 2a06:d1c0::/32
origin: AS59645
mnt-by: WYBT-MNT
created: 2022-01-31T14:57:45Z
last-modified: 2022-01-31T14:57:45Z
source: RIPE
For both objects, you see the network to which it pertains (195.191.196.0/23
or 2a06:d1c0::/32) and the AS that is allowed to announce the network
(AS59645 in both cases). Furthermore, you see the maintainer object–the
object which is allowed to make changes to this object–as well es when the
objects were first created and last modified. Finally, the source field tells
us that these two objects come from the RIPE NCC’s IRR.
Telling the World About Your Upstreams
In addition to setting route objects, you should tell the rest of the world about your routing policy. You do so via import and export statements in your AS object. For AS59645, the most simple version, given IN-BERLIN with AS29670 as out upstream, looked like this:
aut-num: AS59645
as-name: wybt-net
org: ORG-TF84-RIPE
remarks: -- UPSTREAMS --
import: from AS29670 accept ANY
export: to AS29670 announce AS59645
admin-c: WA2412-RIPE
tech-c: WA2412-RIPE
status: ASSIGNED
mnt-by: RIPE-NCC-END-MNT
mnt-by: WYBT-MNT
created: 2022-01-31T13:59:01Z
last-modified: 2022-02-06T16:04:42Z
source: RIPE
With the import statement, we are essentially saying ‘we accept all routes from AS29670, while we only announce routes for which there is a route-object pointing to us back to them. This is the most basic configuration for a stub-AS like ours.
There is a lot more you can do about publishing your routing policy. However, for now, we just have one upstream and not a lot going on. Hence, we will come back to what you can do with your routing policy as soon as we set up more upstreams and peerings.
Router Sizing
On a small router only handling static routes, memory is usually not an issue. However, if you are speaking BGP, you have to be able to hold a ‘full table’ (a full view on all routes on the Internet) in memory, possibly multiple if you have multiple different upstreams connected to that router. Running OpenBGPD on OpenBSD, my systems usually use around 2GB of memory. Hence, i’d suggest to size your first router accordingly, and get at least 3GB of memory in your system. Apart from that–if you are not running that much traffic–other characteristics do not really matter for now.
OpenBGPD Basic Configuration Walk-Through
Let’s get started with configuring OpenBGPD. It actually comes with a rather
excellent default configuration, which you can find in
/etc/examples/bgpd.conf. Naturally, i built my configuration
based on this file. Hence, here we will only go over those parts where i had
to make changes.
Let’s start out with setting our ASN and router-id:
# define our own ASN as a macro
ASN="59645"
# global configuration
AS $ASN
router-id 195.191.197.249
#log updates
The ASN is obviously ‘our’ ASN, which we applied for in Part 2. The router-id should be a unique identifier of the router within your AS (at least since 2011). Ideally, you would use your router’s loopback IPv4 address for it (not 127.0.0.1, but the public one you might configure). However, especially if you are running an IPv6 only AS and actually lack IPv4 addresses to put on your loopback interface, going for something unique (within your AS) from RFC1918 ranges.
Next, we have the networks our AS is originating, i.e., the networks for which we would like to receive packets from the Internet. For me, this is a /23 IPv4 network, and a /32 IPv6 network:
# list of networks that may be originated by our ASN
prefix-set mynetworks {
195.191.196.0/23
2a06:d1c0::/32
}
Note that the default configuration makes sure that mynetworks gets a BGP
community, and only prefixes with that community are allowed to be announced to
neighbors. If you are not using OpenBGPD for this, you must make sure to
similarly ensure that you do not announce prefixes you learned from any other
upstream to your neighbors.
Apart from that, we now only have to configure our sessions to our upstream.
# upstream providers
group "upstreams" {
neighbor 217.197.83.130 {
remote-as 29670
descr "217.197.83.130"
tcp md5sig password SECRET
}
neighbor 217.197.83.131 {
remote-as 29670
descr "217.197.83.131"
tcp md5sig password SECRET
}
neighbor 2001:67c:1400::1 {
remote-as 29670
multihop 2
tcp md5sig password SECRET
descr "2001:67c:1400::1"
}
neighbor 2001:67c:1400::2 {
remote-as 29670
multihop 2
tcp md5sig password SECRET
descr "2001:67c:1400::2"
}
}
We group all upstream into a, well, group. We have a block for each neighbor,
starting with neighbor $neighborip; This is the IP address of our neighbor’s
router. Next, we set the remote-as, which again is simply the ASN of our
upstream. I personally explicitly add our neighbor’s IP address as a
description (to make things easier in bgpctl show).
Two things are somewhat special here: First, we are using multihop 2 for
our IPv6 neighbors. This is due to our upstream running their bgpd on the
routers loopback address, and not directly in the network segment in which our
VM resides. Furthermore, our upstream uses tcp md5sigs to ‘secure’ the
connections. Given that bgp usually communicates in plain-text, this is of
somewhat limited use, but would certainly help against others hijacking your
session (especially if you bgpd is reachable via the Internet, and not only for
your neighbors).
And, that’s it. We might still want to enable IP forwarding, depending on what
we do with these IPs, but apart from that we can just go for an rcctl enable
bgpd ; rcctl start bgpd and OpenBGPD should be up and running.
Accepting a default route
In some cases, your upstream may not provide you with a full routing table, but
instead only announce you a default route. As long as our AS is only connected
to a single upstream that would be, essentially, fine. To make OpenBGPD accept
a default route, you just have to uncomment two lines in your bgpd.conf, and
run bgpctl reload afterwards:
allow from group upstreams prefix 0.0.0.0/0
allow from group upstreams prefix ::/0
Testing if things work
As soon as we started our bgpd, we can run bgpctl show to see how things are
going. Ideally, at this point, sessions should be up (or start to establish
themselves), and you should see something like the following:
wybt.in-berlin.de ~ # bgpctl show
Neighbor AS MsgRcvd MsgSent OutQ Up/Down State/PrfRcvd
217.197.83.130 29670 43537081 159058 0 00:00:14 885438
217.197.83.131 29670 43578718 159051 0 00:00:13 885438
2001:67c:1400::1 29670 29483289 159050 0 00:00:14 144423
2001:67c:1400::2 29670 30328676 159042 0 00:00:14 144423
Your prefixes should now propagate in the GRT (Global Routing Table), and depending on the wind this may take some time. You can also explicitly verify what you are announcing to each neighbor using:
wybt.in-berlin.de ~ # bgpctl show ip bgp neighbor 217.197.83.130 out
flags: * = Valid, > = Selected, I = via IBGP, A = Announced,
S = Stale, E = Error
origin validation state: N = not-found, V = valid, ! = invalid
origin: i = IGP, e = EGP, ? = Incomplete
flags ovs destination gateway lpref med aspath origin
* V 195.191.196.0/23 217.197.83.197 100 0 59645 i
Here, ideally, you will only see your own prefixes. If not, you did something
horribly wrong, and react with an immediated rcctl stop bgpd, followed by a
thorough audit of your configuration. ;-)
After your networks propagated, you should also be able to see them on the Internet(tm). Hence, you should be able to see it, for example in RIPE’s BGPlay, on bgp.he.net (even though that usually takes some time), or other lookingglass services on the Internet. (Lookingglasses are essentially interfaces where AS operators let you look into their routing table for debugging purposes.)
Ultimately, you should now also be able to ping IP addresses in your network
from the Internet, if you assign them to your lo.
Outbound Firewalling
The last important step of this process is outbound firewalling (or technically inbound firewalling, because you should filter what goes into your AS before eventually leaving it). We do this, because otherwise people (well, so far it is only us… ) that have a host in our AS could send IP packets with arbitrary source addresses from our AS. That’s spoofing, and does not only violate BCP38, but is also not very well manrd. As my AS is small, i prefer doing filtering outbound. Apart from not allowing source addresses that are not part of our netblocks, we also want to prevent traffic to all IP blocks that are reserved for non-routed use from exiting our AS. This specifically includes all RFC1918 addresses, but also documentation ranges for IPv4 and IPv6, as well as other ranges considered [bogons][bogons].
Summary
So… where are we at? We now have our first BGP router connected to the global Internet. If we did things right, we should now be able to reach our very own IP addresses via our upstream. Before we will make actual use of our addresses two posts down from here, we will first talk about RPKI in the next one.