How to build an ISP… bootstrapper style

What type of ISP?

Decide what type of ISP you want to build. You could build a VISP, that’s a virtual ISP. You then need to just focus on customer acquisition, management and support allowing you to skip the whole set-up piece. This way the network is managed for you. There is another stage here where you could expand and ask you provider to allow you to do a L2TP inter-connect. You can then authenticate your own users using your own RADIUS and even provide your own IP addresses.

In this article I’m going to focus on a full blown traditional ISP and briefly give you a push in the right direction.

We can start from either end of the connection, so we can start from the users home/office/handset or from the core PoP (Point of Presence). In this case I will be just covering the core aspect. So firstly you need backhaul or IP Transit (a big internet pipe) from a Tier1 provider such as Level3, Cogent, BT, Verizon etc. This is a core internet connection and the supplier will be known as your ‘upstream’. So you get set-up in a data center which becomes your PoP with your upstream connection terminated on fibre optic cable. If you’re really lucky they might provision this on a router for you but this usually is not the case.

OK so you have your IP Transit in place, you now need to terminate this on to some equipment. This is where the real costs can start to shoot through the roof. Most would say you now need a Cisco AS, so about £50k or so. However here is where you can start to use some initiative. You could use Cisco L3 switches as long as you want to provide plain flat Ethernet. This can work and can be secured depending on the type of deployment. However I would suggest going for a Cisco L3 switch to terminate the fibre on an SFP port and then creating a VLAN off to a separate core router. Forget Cisco and buy some tin from HP or Dell and install RouterOS from Mikrotik. Configure this as a PPPoE server. Buy a second box and do the same and use it for fail-over. Buy a third box and configure this as a RADIUS server and double that up to. Buy two more boxes, install Ubuntu and configure them as DNS servers. Register the DNS servers with RiPE. Configure your inetnum on the core routers, place them both in the same VLAN with a bit more config and you have a fully working core ISP.

Lets break that down a little:

Core L3 Switches – The core switch could be a Cisco 3560 or the modern equivalent, I think a 3760. Anyway you need to terminate the fibre from your upstream provider on to one of the SFP ports. You will need a module that slots in to the SFP port. Try to tap up your upstream provider for these as you will need spares and they are expensive. The switches can be acquired for a few hundred from German resale houses on ebay, still come with full warranty too. You will need to create two VLANS on this switch as well as the management VLAN. One for the upstream provider to their head end and one for your customers. So lets call them VLAN 10 and 20. VLAN 10 will have SFP port 1 in its vlan and say ethernet port 1 which will connect to your router ethernet port 1. Vlan 20 will have an outbound fibre link to your customers in it on SFP port 2 and ethernet port 2 going to ethernet port 2 on your router. So now the traffic is flowing from the switch to the router. We can now forget about the switch in our thinking which makes everything a little easier to understand. The reason for the switch is so we can configure bigger and better networks as our ISP grows without having to disrupt service. We can also do lots of testing and introduce new services too.

The Router – The home brew router – RouterOS can be installed on any x86 machine, Dell Poweredge servers are best supported however Mikrotik do manufacture beefy hardware for this but I’m going to talk about Dell. So install ROS (RouterOS) on the dell, it runs in its own Kernel so treat it as an OS. It installs easy enough, install most packages as you’ll need them later on. Basically at this point you need to follow the Wiki on the MIkrotik site to learn how to configure the routes to your upstream provider and then configure PPPoE. It is pretty simple and their Wiki includes screen shots so its not worth me re-writing it all out here. This is also supposed to be a generic guide to help people that may be stuck not to go in to too much detail. Their Wiki can be found here. You’ll need to set-up the router as per the instructions. I would get it working first with their built in RADIUS called Usermanager and then change this to FREERADIUS running on another box.

So you should now have all your traffic running through your router and authenticating against an external database known as a RADIUS server.

IP Addressing – IP Addressing is administered by different organizations around the world, in the UK and Europe its RiPE, in the states its ARIN. You will need to apply to them to become a LIR (local internet registrar), it’s not cheap and now that IPV4 has run out you will only be accepted for IPV6. It can get very expensive so to do this bootstrapper style you may need to ask your upstream provider to lend you some subnets. Once you have your IP’s you can configure your network and create your DHCP pools etc.

So the core is complete but what about the end users. Here you will need a way to connect them using licensed or unlicensed wireless or fixed PSTN DSL services. Once you have decided this then you introduce that part of the network in to you core via SFP 2 fibre link. The end users will need to configure PPPoE on their routers or PC’s and thats it.

A very brief bootstrappers guide to building an ISP. It can be done as I’ve been there and done it. My ISP supported national health and police services, local government and an entire region of businesses and consumers with no unplanned downtime. At the time of building my ISP I wish I could have found a blog post like this but I don’t think anyone had ever bootstrapped an ISP before.

Please feel free to contact me with any questions.