Build a VXLAN tunnel from a Linux client for Remote Transit IP

Introduction

VXLAN (Virtual Extensible LAN, RFC 7348) is an L2-over-L3 encapsulation that carries Ethernet frames inside UDP packets. Unlike GRE (L3 only), VXLAN can extend a broadcast domain, but very often we use it as a simple L2/L3 transport for transit.

Major advantage over GRE: VXLAN runs over UDP/4789, so it passes everywhere — including behind residential boxes that filter IP proto 47. If Bouygues, Orange, SFR block your GRE, VXLAN is plan B before WireGuard.

Prerequisites

• A Linux server with a public IP (ideally fixed, otherwise DDNS)
• VeryCloud's technical email: PUB_VC, PUB_CLIENT, tunnel IPs (169.254.X.0/30, 2a0e:XXXX:XXXX::/127), VNI (Virtual Network Identifier, e.g. 198825)
• Root / sudo
• Recent kernel (4.x+ for VXLAN dual-stack)

Step 1: Verify VXLAN support

sudo modprobe vxlan
lsmod | grep vxlan

Present by default on recent Debian, Ubuntu, RHEL.

Step 2: Create the VXLAN interface

VXLAN config uses:

• VNI (Virtual Network Identifier): unique tunnel ID (24 bits)
• remote: VeryCloud endpoint
• local: your public IP
• dstport: 4789 (standard IANA port)

sudo ip link add vxlan-vc type vxlan \
    id 198825 \
    remote PUB_VC \
    local PUB_CLIENT \
    dstport 4789 \
    ttl 64

sudo ip link set vxlan-vc up
sudo ip addr add 169.254.X.2/30 dev vxlan-vc
sudo ip -6 addr add 2a0e:XXXX:XXXX::1/127 dev vxlan-vc

💡 id 198825 is an example — use the VNI provided by VeryCloud.

Step 3: VXLAN MTU

VXLAN adds 50 bytes of headers (8 VXLAN + 8 UDP + 20 IPv4 + 14 Ethernet inner). On Ethernet MTU=1500, your in-tunnel effective MTU is 1450.

sudo ip link set vxlan-vc mtu 1450

MSS clamping:

sudo iptables -t mangle -A FORWARD -o vxlan-vc -p tcp --tcp-flags SYN,RST SYN \
    -j TCPMSS --clamp-mss-to-pmtu

sudo ip6tables -t mangle -A FORWARD -o vxlan-vc -p tcp --tcp-flags SYN,RST SYN \
    -j TCPMSS --clamp-mss-to-pmtu

Step 4: Static FDB (peer learning)

By default VXLAN tries to learn peer MACs via multicast — doesn't work over the Internet. Static FDB entry:

sudo bridge fdb append 00:00:00:00:00:00 dev vxlan-vc dst PUB_VC

The MAC 00:00:00:00:00:00 means "any unknown MAC, send here".

Step 5: Test connectivity

ping 169.254.X.1
ping6 2a0e:XXXX:XXXX::0

Step 6: systemd-networkd persistence

/etc/systemd/network/10-vxlan-vc.netdev:

[NetDev]
Name=vxlan-vc
Kind=vxlan

[VXLAN]
VNI=198825
Remote=PUB_VC
Local=PUB_CLIENT
DestinationPort=4789
TTL=64
MacLearning=no

/etc/systemd/network/10-vxlan-vc.network:

[Match]
Name=vxlan-vc

[Network]
Address=169.254.X.2/30
Address=2a0e:XXXX:XXXX::1/127
LinkLocalAddressing=no

[Link]
MTUBytes=1450

Step 7: Netplan persistence

network:
  version: 2
  tunnels:
    vxlan-vc:
      mode: vxlan
      local: PUB_CLIENT
      remote: PUB_VC
      id: 198825
      port: 4789
      ttl: 64
      mtu: 1450
      mac-learning: false
      addresses:
        - 169.254.X.2/30
        - 2a0e:XXXX:XXXX::1/127

Step 8: Prepare for BGP

Same as GRE: open TCP/179 between tunnel IPs.

Step 9: VXLAN vs GRE — summary

For pure transit, GRE remains simpler if your ISP allows it. VXLAN wins when proto 47 is blocked.

Troubleshooting

Ping doesn't work

• UDP/4789 firewall (iptables, cloud provider)
• VNI mismatch — must be identical both sides
• tcpdump -i any udp port 4789 to see packets

Packets arrive but tunnel doesn't react

• Wrong remote or local endpoint
• NAT between you and VeryCloud changes source IP

Performance degraded

• UDP checksum overhead on low-end CPUs
• Card offload disabled

FDB growing

• You have learning yes (default) — switch to nolearning:

sudo ip link set vxlan-vc type vxlan nolearning

Useful commands

ip -d link show vxlan-vc
bridge fdb show dev vxlan-vc
sudo tcpdump -i eth0 udp port 4789 -nn
ip -s link show vxlan-vc
sudo ip link del vxlan-vc

Conclusion

VXLAN on Linux = your plan B over GRE when your ISP plays games with IP proto 47. A few commands, a VNI, and you have a stable transport for your BGP session. MTU and MSS clamping like GRE, and you're good.

Going further: WireGuard for encryption, BIRD 2 or FRR for the BGP session.

Resources

• RFC 7348 — VXLAN
• Linux Kernel — VXLAN documentation
• systemd.netdev — VXLAN section
• Netplan — VXLAN tunnel