ECMP – Equal Cost Multi Path – Mikrotik

Hey everyone.

I was recently asked to implement a basic 2x WAN Setup on a Mikrotik whereby the client wanted to use 2 x LTE devices in both a failover as well as load balancing scenario. I immediately thought that ECMP would most likely be the way to go here. Now I have not implemented or tested ECMP on many RouterOS devices so this was an opportunity for me to test this functionality.

Now before I discuss how this was implemented lets talk a little about ECMP. What exactly is ECMP and what does it do?

In tech talk, Equal-cost multi-path routing (ECMP) is a routing strategy where next-hop packet forwarding to a single destination can occur over multiple “best paths” which tie for top place in routing metric calculations.

ECMP can be used in a sort of Round Robin setup but it can also simply be used to create a dual Gateway route out onto the internet. In this exercise I had the below Items and configured them in the below setup.

• 2 x Huawei B618 LTE Routers with Cell C Sims.
• 1 x Mikrotik RB2011 (core router).
• LAN.

My task was to make this work seamlessly. Well here goes… PS this is a simple exercise.

Quick Start for Impatient

Configuration export from the gateway router:

/ ip address
add address=192.168.0.1/24 network=192.168.0.0 broadcast=192.168.0.255 interface=Local 
add address=10.111.0.2/24 network=10.111.0.0 broadcast=10.111.0.255 interface=wlan1
add address=10.112.0.2/24 network=10.112.0.0 broadcast=10.112.0.255 interface=wlan2

/ ip route
add dst-address=0.0.0.0/0 gateway=10.111.0.1,10.112.0.1 check-gateway=ping 

/ ip firewall nat 
add chain=srcnat out-interface=wlan1 action=masquerade
add chain=srcnat out-interface=wlan2 action=masquerade

/ ip firewall mangle
add chain=input in-interface=wlan1 action=mark-connection new-connection-mark=wlan1_conn
add chain=input in-interface=wlan2 action=mark-connection new-connection-mark=wlan2_conn   
add chain=output connection-mark=wlan1_conn action=mark-routing new-routing-mark=to_wlan1     
add chain=output connection-mark=wlan2_conn action=mark-routing new-routing-mark=to_wlan2     

/ ip route
add dst-address=0.0.0.0/0 gateway=10.111.0.1 routing-mark=to_wlan1 
add dst-address=0.0.0.0/0 gateway=10.112.0.1 routing-mark=to_wlan2

Explanation

First we give a code snippet and then explain what it actually does.

IP Addresses

/ ip address 
add address=192.168.0.1/24 network=192.168.0.0 broadcast=192.168.0.255 interface=Local
add address=10.111.0.2/24 network=10.111.0.0 broadcast=10.111.0.255 interface=wlan1
add address=10.112.0.2/24 network=10.112.0.0 broadcast=10.112.0.255 interface=wlan2 

The router has two upstream (WAN) interfaces with the addresses of 10.111.0.2/24 and 10.112.0.2/24. The LAN interface has the name “Local” and IP address of 192.168.0.1/24.

NAT

/ ip firewall nat 
add chain=srcnat out-interface=wlan1 action=masquerade
add chain=srcnat out-interface=wlan2 action=masquerade

As routing decision is already made we just need rules that will fix src-addresses for all outgoing packets. if this packet will leave via wlan1 it will be NATed to 10.111.0.2/24, if via wlan2 then NATed to 10.112.0.2/24

Routing

/ ip route 
add dst-address=0.0.0.0/0 gateway=10.111.0.1,10.112.0.1 check-gateway=ping 

This is typical ECMP (Equal Cost Multi-Path) gateway with check-gateway. ECMP is “persistent per-connection load balancing” or “per-src-dst-address combination load balancing”. As soon as one of the gateway will not be reachable, check-gateway will remove it from gateway list. And you will have a “failover” effect.

You can use asymmetric bandwidth links also – for example one link is 2Mbps other 10Mbps. Just use this command to make load balancing 1:5

/ ip route 
add dst-address=0.0.0.0/0 gateway=10.111.0.1,10.112.0.1,10.112.0.1,10.112.0.1,10.112.0.1,10.112.0.1 check-gateway=ping 

Connections to the router itself

/ ip firewall mangle
add chain=input in-interface=wlan1 action=mark-connection new-connection-mark=wlan1_conn
add chain=input in-interface=wlan2 action=mark-connection new-connection-mark=wlan2_conn
add chain=output connection-mark=wlan1_conn action=mark-routing new-routing-mark=to_wlan1     
add chain=output connection-mark=wlan2_conn action=mark-routing new-routing-mark=to_wlan2     

/ ip route
add dst-address=0.0.0.0/0 gateway=10.111.0.1 routing-mark=to_wlan1 
add dst-address=0.0.0.0/0 gateway=10.112.0.1 routing-mark=to_wlan2

With all multi-gateway situations there is a usual problem to reach router from public network via one, other or both gateways. Explanations is very simple – Outgoing packets uses same routing decision as packets that are going trough the router. So reply to a packet that was received via wlan1 might be send out and masqueraded via wlan2.

To avoid that we need to policy routing those connections.

Above taken from various sources. Hope this helps you.