I've just completed a certain unusual setup that involved NATing packets before they are sent to an IPsec tunnel, so I thought I'll write about this topic. Even in perfectly ordinary setups, the interaction between the two often catches people off guard, me included.
No, this is not a premature Friday post. The Friday post will be a continuation of the little known featured of the VyOS CLI.
Most routers these days have some NAT configured, so if you setup an IPsec tunnel, you need to understand the interaction between the two. Luckily, it's pretty simple.
Every network OS has a fixed packet processing order, and for a good reason. For example, source NAT has to be performed after routing because otherwise the OS will not know which outgoing interface must be used for the packet, and will not be able to determine which SNAT rule must be applied to that packet. Likewise, destination NAT must happen before routing if we want to be able to send incoming packets to the intended host — the routing decision depends on the new destination address.
Sometimes the order is less critical but reversing it would create inconvenience for network admins. For example, in Linux (and thus in VyOS), inbound firewall rules are processed after DNAT, so the destination address the firewall will see is the internal address, and you can easily setup a firewall that mentions private addresses on your WAN interface. If it was the other way around, then if you wanted to setup firewall rules for your private addresses, you would have to assign the firewall to the out direction of the LAN interface — not quite as logical or convenient, even if the end result is the same.
Where's IPsec in that processing flow and what are the implications of its position in it?
Let's revisit the complete diagram (image by Jan Engelhardt, CC-BY-SA):
If posthaven can't handle images properly, here's a direct link to the larger version:
The box you are looking for is "XFRM". In Linux, IPsec is not a special component, but a part of the XFRM framework that can do encryption amond other things (it also does compression and header modification).
From the diagram we can see that XFRM decode step (thus IPsec encryption) is before DNAT (NAT prerouting), and IPsec decryption is after SNAT (NAT postrouting). The implications of it are twofold: first you need to be careful when setting up SNAT and IPsec on the same machine, second, you can apply NAT rules to traffic that will go to the tunnel if you really have to.
Avoiding adverse interaction
Suppose you have this config:
vyos@vyos# show vpn ipsec site-to-site
peer 192.0.2.150 {
[SNIP]
tunnel 1 {
local {
prefix 192.168.10.0/24
}
remote {
prefix 10.10.10.0/24
}
}
}
vyos@vyos# show nat source
rule 10 {
outbound-interface eth0
source {
address 192.168.10.0/24
}
translation {
address 203.0.113.134
}
}
What will happen to a packet sent by host 192.168.10.100 to host 10.10.10.200? Since SNAT is performed before IPsec, and the 192.168.10.100 source address matches the rule 10, the rule will be applied and the packet will go down the packet processing pipeline with source address 203.0.113.134, which does not match the IPsec policy from tunnel 1. The packet will be sent out of the eth0 interface, unencrypted, and destined to be dropped by the ISP due to its private destination address (or it will be sent to a wrong host, which is not any better).
In this case this order of packet processing seems to be a real hassle. There's a very easy workaround though: exclude packets with destination address 10.10.10.0/24 from SNAT, like this:
vyos@VyOS-AMI# show nat source
rule 5 {
outbound-interface eth0
destination {
address 10.10.10.0/24
}
exclude
}
rule 10 {
outbound-interface eth0
source {
address 192.168.10.0/24
}
translation {
address 203.0.113.134
}
}
If you've setup IPsec, the SA is up, but for some reason packets don't get through, make sure that you didn't forget to exclude traffic to the remote network from NAT. It's easy to see with tcpdump whether packets are sent the wrong way or not.
Exploiting the interaction
So far we've only seen how this particular processing order can be bad for our setup. Can it be good for anything then? Sometimes it seems like the Linux network stack was optimized to allow doing crazy things. Just a few days ago I've run into a case when this turned beneficial.
Suppose you setup an IPsec tunnel to your partner, and it turns out you both are using 192.168.10.0/24 subnet internally. None of you is willing to renumber your own network to solve the problem cleanly, but some compromise must be made. The solution is to NAT packets before they are encrypted, which works as expected precisely because IPsec happens after SNAT.
For simplicity let's assume only a single host from our network (internal address 192.168.10.45) needs to interact with a single host from the remote network (10.10.10.55). We will make up an intermediate 172.16.17.45 address and NAT the tunnel traffic to and from 10.10.10.55 host to actually be sent to the 192.168.10.45 host.
The config looks like this:
vyos@vyos# show vpn ipsec site-to-site
peer 192.0.2.150 {
[SNIP]
tunnel 1 {
local {
prefix 172.16.17.45/32
}
remote {
prefix 10.10.10.55/32
}
}
}
vyos@vyos# show nat source
rule 10 {
outbound-interface any
destination {
address 10.10.10.55
}
source {
address 192.168.10.45
}
translation {
address 172.16.17.45
}
}
vyos@vyos# show nat destination
rule 10 {
destination {
address 172.16.17.45
}
inbound-interface any
translation {
address 192.168.10.45
}
}
If IPsec was performed before source NAT, this kind of setup would be impossible.